A
Concept Mapping Project
Project conducted for
The International
Association of Psychosocial Rehabilitation Services
Albuquerque, New Mexico
November 11-12, 1993
William
M.K. Trochim
Cornell University
Judith
Cook
Thresholds National Research and Training Center on Rehabilitation and
Mental Illness
| Contents |
The International Association
of Psychosocial Rehabilitation Services (IAPSRS) has as one of its primary
missions the task of developing Psychosocial Rehabilitation (PSR) as
a professional discipline. To that end, they have for several years
been working towards the development of a comprehensive set of workforce
competencies that could be utilized as standards in the certification
of PSR workers. This task has become even more pressing in view of the
national efforts to develop comprehensive health insurance coverage
in the United States (The White House Domestic Policy Council, 1993).
It is essential that professional standards for PSR be clearly delineated
if PSR is to be included as a service that is covered under national
health insurance.
In recent years, there
have been a several efforts to elucidate PSR workforce competencies
or competencies for related endeavors that might be relevant (Curtis,
1993; Friday and McPheeters, 1985; Jonikas, 1993; IAPSRS Ontario Chapter,
1992). To move the process along, IAPSRS contracted with the Thresholds
Research and Training Center on Rehabilitation and Mental Illness to:
a) review the literature on PSR competencies and develop a paper that
integrated that literature; and b) conduct a concept mapping project
with a selected national group of PSR experts designed to elucidate
a comprehensive framework of competencies. The Jonikas (1993) document
constituted the literature review. This report describes the concept
mapping project that was undertaken.
Concept mapping is
a process that can be used to help a group describe its ideas on any
topic of interest (Trochim, 1989a). The process typically requires the
participants to brainstorm a large set of statements relevant to the
topic of interest, individually sort these statements into piles of
similar ones and rate each statement on some scale, and interpret the
maps that result from the data analyses. The analyses typically include
a two-dimensional multidimensional scaling (MDS) of the unstructured
sort data, a hierarchical cluster analysis of the MDS coordinates, and
the computation of average ratings for each statement and cluster of
statements. The maps that result show the individual statements in two-dimensional
(x,y) space with more similar statements located nearer each other,
and show how the statements are grouped into clusters that partition
the space on the map. Participants are led through a structured interpretation
session designed to help them understand the maps and label them in
a substantively meaningful way.
The concept mapping
process as conducted here was first described by Trochim and Linton
(1986). Trochim (1989a) delineates the process in detail and Trochim
(1989b) presents a wide range of example projects. Concept mapping has
received considerable use and appears to be growing in popularity. It
has been used to address substantive issues in the social services (Galvin,
1989; Mannes, 1989), mental health (Cook, 1992; Kane, 1992; Lassegard,
1993; Marquart, 1988; Marquart, 1992; Marquart et al, 1993; Penney,
1992; Ryan and Pursley, 1992; Shern, 1992; Trochim, 1989a; Trochim and
Cook, 1992; Trochim et al, in press; Valentine, 1992), health care (Valentine,
1989), education (Grayson, 1993; Kohler, 1992; Kohler, 1993), educational
administration (Gurowitz et al, 1988), and theory development (Linton,
1989). Considerable methodological work on the concept mapping process
and its potential utility has also been accomplished (Bragg and Grayson,
1993; Caracelli, 1989; Cooksy, 1989; Davis, 1989; Dumont, 1989; Grayson,
1992; Keith, 1989; Lassegard, 1992; Marquart, 1989; Mead and Bowers,
1992; Mercer, 1992; SenGupta, 1993; Trochim, 1985 , 1989c, 1990).
The concept mapping process involves six major steps:
1 Preparation
2 Generation
3 Structuring
4 Representation
5 Interpretation
6 Utilization
This report presents the results of the project in sequential order according to the six steps in the process.
The preparation step
involves three major tasks. First, the focus for the concept mapping
project must be stated operationally. Second, the participants must
be selected. And, third, the schedule for the project must be set.
The
Focus for the Concept Mapping
In concept mapping,
the focus for the project is stated in the form of the instruction to
the brainstorming participant group. For this project this instruction
was operationalized as:
Generate statements
(short phrases or sentences) that describe specific workforce competencies
for psychosocial rehabilitation practitioners.
In most projects there
is a secondary focus that relates to the ratings of the brainstormed
statements. This focus is also stated in its operational form and, for
this project, was:
Using the following
scale, rate each competency for its relative importance for high-quality
service delivery.
|
1
relatively
less important |
2
somewhat |
3
moderately |
4
very |
5
extremely |
The
Participants
Twenty-one people participated
in the concept mapping process. They were purposively selected to represent
a broad range of PSR experiences and schools of thought. They included
the Director of IAPSRS, the Chair of the committee responsible for developing
competencies and several members of the IAPSRS Board of Directors. Several
participants were affiliated with the leading national centers for PSR.
There were several consumers of PSR services.
The
Schedule
The concept mapping project was scheduled for two consecutive days. It began on Thursday, November 11th at 2pm. Between 2 and 6 pm the generation and structuring steps were accomplished. The representation step (i.e., the data entry, analysis and production of materials for interpretation) was completed by the co-facilitators (Trochim and Cook) during the evening of November 11th. The Interpretation step was accomplished from 9 to 12 am on Friday, November 12th. Participants were given a two-hour lunch during which they could skim four documents that attempted to delineate competencies in PSR or related areas (Curtis, 1993; Friday and McPheeters, 1985; Jonikas, 1993; IAPSRS Ontario Chapter, 1992). The Utilization step was accomplished on Friday afternoon from 2 to 5 pm.
The generation step
essentially consists of a structured brainstorming session (Osborn,
1948) guided by a specific focus prompt that limits the types of statements
that are acceptable. The focus statement or criterion for generating
statements was operationalized in the form of the instruction to the
participants given above. The general rules of brainstorming applied.
Participants were encouraged to generate as many statements as possible
(with an upper limit of 100); no criticism or discussion of other's
statements was allowed (except for purposes of clarification); and all
participants were encouraged to take part. The group brainstormed ninety-six
statements in approximately a forty-five minutes.
The complete listing of brainstormed statements is given in Table 1. Participants were given a short break while the statements were printed and duplicated for use in the structuring stage.
Structuring involved
two distinct tasks, the sorting and rating of the brainstormed statements.
For the sorting (Rosenberg and Kim, 1975; Weller and Romney, 1988),
each participant was given a listing of the statements laid out in mailing
label format with twelve to a page and asked to cut the listing into
slips with one statement (and its identifying number) on each slip.
They were instructed to group the ninety-six statement slips into piles
"in a way that makes sense to you." The only restrictions in this sorting
task were that there could not be: (a) N piles (in this case 96 piles
of one item each); (b) one pile consisting of all 96 items; or (c) a
"miscellaneous" pile (any item thought to be unique was to be put in
its own separate pile). Weller and Romney (1988) point out why unstructured
sorting (in their terms, the pile sort method) is appropriate in this
context:
The outstanding strength
of the pile sort task is the fact that it can accommodate a large number
of items. We know of no other data collection method that will allow
the collection of judged similarity data among over 100 items. This
makes it the method of choice when large numbers are necessary. Other
methods that might be used to collect similarity data, such as triads
and paired comparison ratings, become impractical with a large number
of items (p. 25).
After sorting the statements,
each participant recorded the contents of each pile by listing a short
pile label and the statement identifying numbers on a sheet that was
provided. For the rating task, the brainstormed statements were listed
in questionnaire form and each participant was asked to rate each statement
on a 5-point Likert-type response scale in terms of the relative importance
of each competency as stated above. Because participants were unlikely
to brainstorm statements that were totally unimportant with respect
to PSR, it was stressed that the rating should be considered a relative
judgment of the importance of each item to all the other items brainstormed.
This concluded the structuring session.
In the representation
step, the sorting and rating data were entered into the computer, the
MDS and cluster analysis were conducted, and materials were produced
for the interpretation step.
The concept mapping analysis begins with construction from the sort information of an NxN binary, symmetric matrix of similarities, Xij. For any two items i and j, a 1 was placed in Xij if the two items were placed in the same pile by the participant, otherwise a 0 was entered (Weller and Romney, 1988, p. 22). The total NxN similarity matrix, Tij was obtained by summing across the individual Xij matrices. Thus, any cell in this matrix could take integer values between 0 and 11 (i.e., the 11 people who sorted the statements); the value indicates the number of people who placed the i,j pair in the same pile.
The total similarity
matrix Tij was analyzed using nonmetric multidimensional
scaling (MDS) analysis with a two-dimensional solution. The solution
was limited to two dimensions because, as Kruskal and Wish (1978) point
out:
Since it is generally
easier to work with two-dimensional configurations than with those involving
more dimensions, ease of use considerations are also important for decisions
about dimensionality. For example, when an MDS configuration is desired
primarily as the foundation on which to display clustering results,
then a two-dimensional configuration is far more useful than one involving
three or more dimensions (p. 58).
The analysis yielded
a two-dimensional (x,y) configuration of the set of statements based
on the criterion that statements piled together most often are located
more proximately in two-dimensional space while those piled together
less frequently are further apart.
This configuration
was the input for the hierarchical cluster analysis utilizing Ward's
algorithm (Everitt, 1980) as the basis for defining a cluster. Using
the MDS configuration as input to the cluster analysis in effect forces
the cluster analysis to partition the MDS configuration into non-overlapping
clusters in two-dimensional space. There is no simple mathematical criterion
by which a final number of clusters can be selected. The procedure followed
here was to examine an initial cluster solution that on average placed
five statements in each cluster. Then, successively lower and higher
cluster solutions were examined, with a judgment made at each level
about whether the merger/split seemed substantively reasonable. The
pattern of judgments of the suitability of different cluster solutions
was examined and resulted in acceptance of the fifteen cluster solution
as the one that preserved the most detail and yielded substantively
interpretable clusters of statements.
The MDS configuration
of the ninety-six points was graphed in two dimensions and is shown
in Figure 1. This "point map" displayed the location of all the brainstormed
statements with statements closer to each other generally expected to
be more similar in meaning. A "cluster map" was also generated and is
shown in Figure 2. It displayed the original ninety-six points enclosed
by boundaries for the eighteen clusters.
The 1-to-5 rating data
was averaged across persons for each item and each cluster. This rating
information was depicted graphically in a "point rating map" (Figure
3) showing the original point map with average rating per item displayed
as vertical columns in the third dimension, and in a "cluster rating
map" which showed the cluster average rating using the third dimension.
The following materials were prepared for use in the second session:
(1) the list of the brainstormed statements grouped by cluster
(2) the point map showing the MDS placement of the brainstormed statements and their identifying numbers (Figure 1)
(3) the cluster map showing the eighteen cluster solution (Figure 2)
(4) the point rating map showing the MDS placement of the brainstormed statements and their identifying numbers, with average statement ratings overlaid (Figure 3)
(5) the cluster rating
map showing the eighteen cluster solution, with average cluster ratings
overlaid
Representation
Results
The final stress value
for the multidimensional scaling analysis was .2980101.
Methods for estimating the reliability of concept maps are described in detail in Trochim (1993). Here, six reliability coefficients were estimated. The first is analogous to an average item-to-item reliability. The second and third are analogous to the average item-to-total reliability (correlation between each participant's sort and the total matrix and map distances respectively). The fourth and fifth are analogous to the traditional split-half reliability. The sixth is the only reliability that examines the ratings, and is analogous to an inter-rater reliability. All average correlations were corrected using the Spearman-Brown Prophesy Formula (Weller and Romney, 1988) to yield final reliability estimates. The results are given in Table 2.
The interpretation
session convened on Friday morning to interpret the results of the concept
mapping analysis. This session followed a structured process described
in detail in Trochim (1989a). The facilitator began the session by giving
the participants the listing of clustered statements and reminding them
of the brainstorming, sorting and rating tasks performed the previous
evening. The participants were asked to read through the set of statements
in each cluster and generate a short phrase or word to describe or label
the set of statements as a cluster. The facilitator led the group in
a discussion where they worked cluster-by-cluster to achieve group consensus
on an acceptable label for each cluster. In most cases, when persons
suggested labels for a specific cluster, the group readily came to a
consensus. Where the group had difficulty achieving a consensus, the
facilitator suggested they use a hybrid name, combining key terms or
phrases from several individuals' labels.
Once the clusters were labeled, the group was given the point map and told that the analysis placed the statements on the map so that statements frequently piled together are generally closer to each other on the map than statements infrequently piled together. To reinforce the notion that the analysis placed the statements sensibly, participants were given a few minutes to identify statements close together on the map and examine the contents of those statements. After becoming familiar with the numbered point map, they were told that the analysis also organized the points (i.e., statements) into groups as shown on the list of clustered statements they had already labeled.
Participants then examined
the labeled cluster map to see whether it made sense to them. The facilitator
reminded participants that in general, clusters closer together on the
map should be conceptually more similar than clusters farther apart
and asked them to assess whether this seemed to be true or not. Participants
were asked to think of a geographic map, and "take a trip" across the
map reading each cluster in turn to see whether or not the visual structure
seemed sensible. They were then asked to identify any interpretable
groups of clusters or "regions." These were discussed and partitions
drawn on the map to indicate the different regions. Just as in labeling
the clusters, the group then arrived at a consensus label for each of
the identified regions. Five regions were identified and are shown in
capital letters. No boundaries were drawn to distinguish these five
regions.
The facilitator noted
that all of the material presented to this point used only the sorting
data. The results of the rating task were then presented through the
point rating (Figure 3) and cluster rating (Figure 5) maps. It was explained
that the height of a point or cluster represented the average importance
rating for that statement or cluster of statements. Again, participants
were encouraged to examine these maps to determine whether they made
intuitive sense and to discuss what the maps might imply about the ideas
that underlie their conceptualization.
Table 3 shows the complete cluster listing with the cluster
labels the participants assigned and the average importance rating for
each statement and cluster.
Discussion
of Skills versus Values
The pattern of ratings
on the map suggested that participants attached more importance to the
clusters that had "value" statements than to those made up of skills.
This can perhaps be seen most clearly in Table 4 which shows the ninety-six competency statements
sorted from highest to lowest average importance rating. It is clear
from the table that the statements near the top of the table tend to
be more general in nature and more related to values while the statements
near the bottom of the table tend to be more specific, operationalized,
skill or knowledge-based ones. Some of the participants felt that the
value statements can't be considered competencies per se because they
are not sufficiently operationalized. Others felt that the value statements
have actually been holding IAPSRS back in their development of competencies
because they place too much importance on these generic values and not
on a more specific skill base. Still others felt that the value statements
are at the heart of what PSR represents and that they can and should
be operationalized as competencies. The facilitator characterized the
discussion as a choice between two alternatives:
A) Pull the value statements
out of the competencies, perhaps putting them in a section up front
describing the kinds of values and characteristics expected of psychosocial
rehabilitation workers.
B) Operationalize the
value statements so they can be included as formal competencies.
The consensus of the group was that option B was preferable. As a result, the group decided that a major portion of the afternoon utilization session would involve taking the value-oriented clusters (Clusters 1-5) and attempting to draft operationalized competency statements for the statements in these clusters.
Discussion
of What was Missing on the Map
The group also discussed what concepts seemed to be missing (primarily at the cluster level) from the map. The following potentially missing labels were generated:
1 Advocacy
2 Systems Change
3 Vocational-Employment
4 Spiritual
5 Housing
6 Education
7 Health
8 Social/Recreational
9 Outcome Evaluation
10 Client Budgeting/Finances
11 Program Management
12 Health and Safety
The group then discussed whether the eventual competencies should have subject-specific categories (such as housing, education, employment) or whether competencies related to such areas should be spread across the types of headings already on the map (for instance, consumer outcomes related to employment). The consensus of the group was that the competencies should not be grouped by subject.
The utilization step
took place on Friday afternoon from 2-5pm. The following schedule was
explained to the participants when they returned from lunch.
| Time | Activity | Facilitator |
| 2-3 | Review progress and where we stand | BT |
| Review and Feedback on the map's clusters and regions | BT | |
| Discuss the competency documents | JC | |
| Present the two small group tasks and have participants select their group/task | JC | |
| 3-4 | Small group sessions | |
| 4-4:50 | Presentation of results of small groups | |
| Summary of map revisions | BT | |
| Summary of operationalizing of the five clusters | Group Leaders | |
| 4:50-5 | Discussion of next steps and wrap-up | Anita Pernell-Arnold |
Review
and Feedback on the Map's Clusters and Regions
The first part of the
utilization discussion involved suggestions from participants regarding
changes that could be made to the final map in order to make it more
interpretable, cohesive and usable. The discussion which took place
raised the following points.
Reactions to the Five Regions
1 Doesn't matter which five labels we use.
2 Change the name "Techniques."
3 What is the meaning of "consumer" (consumer involvement issues).
4 "Practitioner" is very broad.
5 Change titles by adding "competencies" to the labels.
6 Some consumer competencies are knowledge-based, others are techniques, others are system issues.
7 View (regions) as
"key ingredients."
Reactions to Clusters
1 People did some categories according to the specific words in titles (e.g., "ability to...", or "knowledge of..."). Was this wise?
2 Family relationships is lacking key intervention skills--want to add more?
3 Reconsider the two consumer clusters -- are labels OK?
4 Take another look at Friday and McPheeters broad classification -- better than ours? (Some said they lose the values; do they exclude the consumers?).
5 Rename cluster 9 (Assessment) or think of dividing it up.
6 Revisit the cluster name "Personality Characteristics."
7 Consider combining "Interpersonal Social Skills" and "Supportive Behaviors."
Discussion
of Other Competency Documents
The group then discussed
the four competency statement documents (Curtis, 1993; Friday and McPheeters,
1985; Jonikas, 1993; IAPSRS Ontario Chapter, 1992) that they skimmed
over lunch and compared these to the map. The following comments were
made:
1. Current group has defined a set of competencies that is impressive. Need to be clear that we shouldn't come up with competencies that are unrealistic, over-skilled, characterize a broad range of competencies.
2. Curtis (1993) was not intended to specify competencies limited to PSR.
3. Curtis (1993) is good in its specificity.
4. Jonikas (1993) document has a totality that will be useful in deciding what to put where.
5. Eighty percent of all documents (including the concept map) were similar.
6. Friday and McPheeters (1985) shows earlier development of the field.
7. There is more in the literature of competencies than we thought.
8. Competencies related to knowledge of principles may not capture the centrality of safety, spirituality, work, decent place to live, social life, education, and physical health in PSR. Don't want to lose the essentials. Also want to emphasize high quality outcomes in these areas.
9. IAPSRS Ontario Chapter (1992) is impressive in its succinctness and specificity. Could help guide us in our document. Action verbs were good in this document.
10.Curtis (1993) document emphasizes the importance of creation of environments, social situations. Not just changing the individual, but creating contexts. Good use of respect as a concept/process.
Small
Group Sessions
In the middle of the afternoon utilization step, the participants were divided into small groups in order to accomplish some more detailed work. Five groups of 2-3 participants each took one of the first five clusters and attempted to operationalize the statements in the cluster into ones that better approximated competency statements. One small group of six participants discussed and made slight revisions to the final concept map. The results of these two types of small group exercises are described in separate sections below.
Small
Group Operationalizations of Five Clusters
Based on the interpretation
discussion in the morning session, it was clear that the participants
thought that many of the statements in the first five clusters were
better described as "values" than as operationalized competency statements.
The group thought that these value statements could be operationalized
and that this would be a central task for IAPSRS to accomplish as it
developed competencies. The central utilization task of the afternoon
therefore was to have small groups of participants, each assigned one
of the first five clusters, take the statements in the clusters and
develop draft operational competency statements. The summaries of these
discussions (taken from the newsprint sheets used at the presentation
of the results) are reproduced below.
Cluster 1: Interpersonal
Skills
This group took each
statement in the cluster and generated several more operationalized
statements. Where appropriate, they chose statements from several of
the other competency documents and these are cited. This listing shows
each brainstormed statement in Cluster 1 and the draft competency statements
that the small group generated.
1. ability to listen to consumers
ï not interrupt the consumer
ï able to repeat back what was said with the consumer affirming the correctness
ï not imposing your
agenda on them
10. ability to motivate clients to change behavior
ï to be able to identify reasons for changing the behavior
ï to be able to help them identify consequences
ï willingness to serve as role model for desired change
ï willingness to reinforce
behavior that has been changed
36. ability to use the helping relationship to facilitate change
ï use one's own experiences to encourage and guide the consumer
ï ability to demonstrate
approval and pride in their accomplishments
87. ability to interact and provide support in a non-judgmental fashion
ï do not demean or patronize consumers
ï give feedback on behavior and not the person (Friday and McPheeters, 1985)
ï use language and
behavior which reflects and perpetuates the dignity of the individual
(Curtis, 1993)
5. ability to offer hope to others
ï truly believe that there is hope and verbalize it to the consumer
ï share examples of change that was possible in a seemingly hopeless situation
ï have a healthy sense of humor and minimize the adversity (Friday and McPheeters, 1985)
ï focus on consumer
successes and help consumer see their own personal growth
6. belief in the recovery process
ï the worker has to demonstrate that he/she believes in the recovery process
ï to express the belief to the consumers that it's possible for them to live productive satisfying lives in the community (Jonikas, 1993)
ï help the consumer
believe in his/her inherent capacity to improve or grow, given the opportunity
and resources, as it's true for all persons (adapted from Jonikas, 1993)
39. ability to build on successes and minimize failures
ï point out and celebrate their successes
ï help them to see their failure as a learning experience
ï supporting risk-taking behaviors to move one step beyond
ï ability to have the
consumer feel good and acknowledge own success no matter how small (adapted
from Friday and McPheeters, 1985)
31. connecting (interpersonal) skills
ï demonstrate behaviors that accept the consumer where he/she is at
ï ability to establish a caring but not a consuming or possessive relationship
ï demonstrate behaviors
that show interest in the consumer and his/her interpretation of needs
78. ability to work with consumer colleagues
ï to show sensitivity to the difficulties that they may encounter in their dual role
ï avoid labeling persons (either consumers or consumer colleagues) with stereotypes or derogatory terms (Friday and McPheeters, 1985)
ï be straight with consumer colleagues
ï have the same expectations
as you do for all other colleagues
89. ability to normalize interactions and program practices
ï ability to generalize program experiences to activities in the broader community
ï have expectations within the program that are consistent with community expectations (with leeway in terms of enforcement)
ï set reasonable limits
on bizarre behavior with explanations as to why you are doing it
Cluster 2: Supportive
Behaviors
This group generated
the following draft competency statements to cover the material listed
in Cluster 2.
ï ability to maintain ongoing productive relationship based on client satisfaction
ï demonstrate high level of interaction (i.e., amount of time, interests, excitement, energy level)
ï communicates belief in growth potential
ï communicates understanding of thoughts/feelings of others in a non-judgmental manner
ï demonstrates holistic understanding of the individual
ï able to focus on the consumer's here and now needs/desires (there was some disagreement on the wording of this one)
ï ability to respond
in a normalizing manner to the individual's diverse needs and strengths
The following were
suggestions from the group about what statements might be "borrowed"
from existing lists:
from Curtis (1993):
4. Demonstrates basic communication and support skills
A1. Exhibits supportive interpersonal skills (i.e., ...)
A2.Establishes and maintains productive relationships with service recipients
ï All of 4A--some areas to "negotiate"
1. especially A and
B (language, behavior and holistic understanding)
from Friday and McPheeters (1985):
ï III. Interpersonal
- especially 2, 4, 6, 7, 8
Their group also listed
some ways to measure competencies in this area:
ï amount of time spent with client
ï client satisfaction with the relationship (amount of support perceived)
ï peer feedback/input
ï share and use own life experience
ï reciprocity of relationship
ï genuineness
Cluster 3: Professional
Role
For each statement
in Cluster 3, the group generated one or two potential competency statements.
14. ability to negotiate
ï to demonstrate communication
skills between stakeholders for the purpose of goal attainment which
is satisfactory to all parties
58. ability to set limits
ï to identify personal
skills and resources, and expectations held by stakeholders in order
to achieve realistic/attainable goals
17. willingness to have fun
ï to actively participate
in "activities"
82. ability to use self as a role model
ï to mutually share experiences and ideas
ï to achieve goals
through partnership
47. ability to ask
for help and receive constructive feedback from colleagues and consumers
51. ability to let go
ï to assist consumers
to identify their skills/resources and promote a belief in efficacy
of their skills in order for consumers to take charge
88. ability to overcome personal prejudices when providing services
ï to identify personal
values/beliefs and evaluate their potential impact on all interactions
Cluster 4: Personality
Characteristics
For each statement
in Cluster 4, the group generated one or two potential competency statements.
16. self awareness
ï be able to describe
and explain one's own actions
56. good personal stability but not ego-centric
ï respond consistently
and congruently to social and environmental demands
50. ability to handle personal stress
ï separate personal
needs and behaviors from job performance needs and behaviors
18. flexibility
ï be able to change
behaviors when situations, expectations and requirements are different
25. patience
ï to calmly wait until
the objective is reached
28. sense of humor
ï to laugh at what
is funny, to laugh at oneself, and to laugh with others
93. ability to know own limits
ï to be able to stop
when necessary; to be able to ask for help; to be able to ask for information
Cluster 5: Self
Management
24. ability to read and write
1. person must meet high school equivalency level of reading and writing
2. must include accommodations for disabilities like blindness
3. ability to write in behavioral language
4. ability to write with clarity
5. reading comprehension
skills must include ability to look up words in the dictionary, comprehend
language(s) used in service settings
29. ability to partialize tasks
41. ability to handle multiple tasks
69. ability to prioritize and manage time
ï recognition of total number of tasks inherent in responsibilities
ï identify critical tasks by applying an agreed-upon standard for what is most important
ï ability to gauge the level of effort and amount of time necessary to complete discrete tasks
ï ability to use organizational tools (calendars, to-do lists, tickler file) to keep track of tasks
ï ability to engage consumers in assisting with provider's task and time management
ï ability to recognize
and deal effectively with personal stress resulting from multiple tasks
33. tolerance for ambiguity
and enjoying diversity
Tolerating Ambiguity
1. Ability to problem-solve ambiguous situations through involvement of others in identification of problem, generation of a number of potential solutions, evaluating candidate solutions, seeking staff/consumer/family/network feedback re: viability of solutions, selection of solutions, implementation and evaluation of solutions.
2. Ability to recognize and accept unresolvable ambiguities through letting-go, acceptance, humor and other strategies.
3. Ability to distinguish
between truly ambiguous situations and situations based on lack of:
info, training, feedback from others. Also, ability to address lacking
areas by obtaining info, furthering education/training, seeking feedback.
Enjoying Diversity
1. Ability to identify
the opportunities presented by diversity and to incorporate them positively
into the rehabilitation process through providing alternatives for behavior,
problem solution, identification of opportunities.
91. willingness to take risks
1. demonstration of creative approaches
2. allowing/assisting consumers to exercise options not endorsed by practitioner, after applying standards of reasonable judgment (safety, etc.)
3. demonstration of
willingness to try new or untested approaches and interventions
45. ability to be pragmatic and do hands-on sorts of work
1. Recognition that PSR rehabilitation involves the doing of hands-on tasks for role modeling, relationship building, etc.
2. Willingness to accept and perform well on hands-on, practical tasks.
3. Ability to develop
and implement rehabilitation situations in which behavior or doing leads
to insight rather than vice versa.
94. never-ending willingness to develop oneself
1. NOTE: The group suggested that this item be moved to the Professional Development cluster. This suggestion was adopted.
2. Development of one's personal growth through hobbies, therapy, education, and to share that growth with consumers/peers for role modeling and motivation.
3. Willingness to seek help appropriately with one's own problems.
Small
Group Map Revision
The small group that
considered the revisions to the map began by working with the suggestions
generated earlier by the entire participant group. The following shows
these suggestions along with the actions taken, if any, by the small
group:
|
|
|
| Reactions to the Five Regions | |
| 1. Doesn't matter which five labels we use. | Two changes were made to the original five labels. The label "Techniques" was changed to "Rehabilitation Methodology Competencies" and the original label "Consumer" was changed to "Consumer-Centered Competencies". In addition, all five labels had the term "Competencies" appended to the end. |
| 2. Change the name "Techniques." | The label "Techniques" was changed to "Rehabilitation Methodology Competencies". |
| 3. What is the meaning of "consumer" (consumer involvement issues). | The original label "Consumer" was changed to "Consumer-Centered Competencies". |
| 4. "Practitioner" is very broad. | The group decided that the term "Practitioner" would be left as is because it was an appropriately broad label for a region name. |
| 5. Change titles by adding "competencies" to the labels. | This was done for all region and cluster labels. |
| 6. Some consumer competencies are knowledge-based, others are techniques, others are system issues. | The small group agreed but made no changes to the map in response to this. |
| 7. View (regions) as "key ingredients." | The small group agreed but made no changes to the map in response to this. |
| Reactions to Clusters | |
| 1. People did some categories according to the specific words in titles (e.g., "ability to...", or "knowledge of..."). Was this wise? | The small group agreed but made no changes to the map in response to this. |
| 2. Family relationships is lacking key intervention skills--want to add more? | The cluster label "Family Relationships" was changed to "Family-Focused." No intervention items were added. |
| 3. Reconsider the two consumer clusters -- are labels OK? | Changed the original cluster label "Consumer Goal Attainment" to "Consumer Outcome Competencies." |
| 4. Take another look at Friday and McPheeters broad classification -- better than ours? (Some said they lose the values; do they exclude the consumers?). | The small group felt that there was considerable cross-classifiability across the different competency documents and the map. No changes were made to the map in response to this. |
| 5. Rename cluster 9 (Assessment) or think of dividing it up. | The group retained the name for the cluster, only changing it to "Assessment Competencies." See table below for specific statements moved into and out of this cluster. |
| 6. Revisit the cluster name "Personality Characteristics." | The group changed the original cluster label "Personality Characteristics" to "Intrapersonal Competencies." |
| 7. Consider combining "Interpersonal Social Skills" and "Supportive Behaviors." | These clusters (original clusters 1 and 2) were combined into one cluster labeled "Interpersonal Competencies." |
| The original cluster label "Cultural Competence" was changed to "Multicultural Competencies." | |
| The positions of the original clusters "Family Relationships" and "Mental Health Knowledge Base" were switched on the map. |
In addition to the
above changes, several specific statements were shifted from one cluster
to another. These changes are shown in Figure 6 and listed in the table
below:
|
|
|
|
| 43. knowledge of a wide variety of approaches to mental health services | Family Relationships | Mental Health Knowledge Base Competencies |
| 40. ability to establish alliances with providers, professionals, families, consumers (partnership model) | Family Relationships | Community Resources Competencies |
| 12. skills in advocacy | Assessment | Community Resources Competencies |
| 15. strong crisis intervention skills | Assessment | Intervention Skills Competencies |
| 85. early identification and intervention skills to deal with relapse | Assessment | Intervention Skills Competencies |
| 94. never-ending willingness to develop oneself | Personality Characteristics | Professional Development Competencies |
| 53. ability to assess behavior in specific environments | Intervention Skills | Assessment Competencies |
| 55. functional assessment | Intervention Skills | Assessment Competencies |
| 64. ability to assess active addiction and co-dependency | Intervention Skills | Assessment Competencies |
In all of the nine
statement shifts described above, the shift was from one cluster into
an adjacent one on the map. The revised cluster listing showing the
new cluster labels and the average importance ratings is given in Table 5.
The small group also
drew explicit lines dividing the five regions. These are shown in Figure
7. They felt that several of the clusters actually overlapped multiple
regions and, consequently, the region lines cut through a cluster shape
rather than only going between clusters. For instance, The felt that
the cluster "Interpersonal Competencies" should fall simultaneously
and partially into the three regions of "Consumer-Centered Competencies",
"Practitioner Competencies" and Rehabilitation Methodology Competencies."
Similarly, they felt that the cluster "Professional Development Competencies"
should fall into both the "practitioner Competencies" and "Knowledge
Base Competencies" regions. The regional lines were drawn on the final
map to show these multi-regional clusters.
Figure 8 constitutes the final map for this project. It shows the clusters and regions and includes the average importance ratings for each cluster. There was considerable consensus across the participant group that it was a good and fair representation of their ideas regarding competencies for psychosocial rehabilitation workers.
Next
Steps
The final discussion
of the project involved consideration of the next steps in the competency
development process. The following points were made:
1. Print up list of competencies and survey PSR workers.
2. Review and comment on Trochim concept mapping report.
3. Circulate regions, clusters and individual competencies to various constituencies: consumers, families, PSR workers, other stakeholders.
4. Further operationalize remaining competencies.
5. Distinguish between entry-level and second-level competencies.
6. Edit and make language consistent on materials sent out for review.
7. Clarify the intent of the present process re: the use to which the final product will be put.
8. Inform a wide range of stakeholders of IAPSRS's intentions in this area.
9. Bring in an expert in credentialing to clarify legal risks, probable results, etc.
10. Involve Training and Certification Committee in this process.
11. Don't send document for review prematurely. Use simple format that helps potential reviewers. Perhaps include a glossary to aid potential reviewers.
12. Be aware of other lists of competencies so review process doesn't become confused.
13. Include feedback from IAPSRS chapter presidents.
14. Certification conference.
15. Further literature review.
16. Hire someone to draft standards from competencies.
17. Develop an ethics statement based on already-held ethics forum.
18. Requirements of an "arms length" certification organization.
19. Need to consider the voluntary nature of CARF accreditation for organizations parallel to possible implementation of standards for practitioners.
20. Conduct a cost/benefit analysis of certification.
Table 1. Complete listing of the ninety-six brainstormed statements for the IAPSRS Project.
| 1 | ability to listen to consumers |
| 2 | ability to relate to others |
| 3 | knowledge of mental illness |
| 4 | knowledge of side effects of medications and alternatives |
| 5 | ability to offer hope to others |
| 6 | belief in the recovery process |
| 7 | ability to emphasize client choices and strengths |
| 8 | knowledge of human services network in community |
| 9 | knowledge of community resources beyond human services |
| 10 | ability to motivate clients to change behavior |
| 11 | knowledge of family networks |
| 12 | skills in advocacy |
| 13 | view consumer as the director of the process |
| 14 | ability to negotiate |
| 15 | strong crisis intervention skills |
| 16 | self awareness |
| 17 | willingness to have fun with others |
| 18 | flexibility |
| 19 | knowledge of appropriate or applicable mental health acts (legislation) |
| 20 | knowledge of eligibility benefits |
| 21 | social group-work skills |
| 22 | ability to see consumers as equal partners |
| 23 | teaching ability |
| 24 | ability to read and write |
| 25 | patience |
| 26 | ability to empathize |
| 27 | ability to develop structured learning experiences |
| 28 | sense of humor |
| 29 | ability to partialize tasks |
| 30 | demonstration of respect and understanding for family members |
| 31 | connecting (interpersonal) skills |
| 32 | cultural competence and ability to deliver culturally relevant services |
| 33 | tolerance for ambiguity and enjoying diversity |
| 34 | value consumer's ability to seek and sustain employment opportunities |
| 35 | value consumer's ability to pursue educational goals |
| 36 | ability to use the helping relationship to facilitate change |
| 37 | ability to develop alliances/partnerships with family members |
| 38 | knowledge of ethnic-based familial role definitions |
| 39 | ability to build on successes and minimize failures |
| 40 | ability to establish alliances with providers, professionals, families, consumers (partnership model) |
| 41 | ability to handle multiple tasks |
| 42 | ability to replace self with naturally-occuring resources |
| 43 | knowledge of a wide variety of approaches to mental health services |
| 44 | knowledge of the community you serve and its environment |
| 45 | ability to be pragmatic and do hands-on sorts of work |
| 46 | ability to set goals |
| 47 | ability to ask for help and receive constructive feedback from consumers, peers, stakeholders |
| 48 | ability to work with employers |
| 49 | ability to generate enthusiasm |
| 50 | ability to handle personal stress |
| 51 | ability to let go |
| 52 | ability to understand the impact of culture and ethnicity on mental illness |
| 53 | ability to assess behavior in specific environments |
| 54 | knowledge of legal issues (e.g., civil commitment, guardianship) and the ethical context |
| 55 | functional assessment |
| 56 | good personal stability but not ego-centric |
| 57 | knowledge of relationship between health status and mental illness |
| 58 | ability to set limits |
| 59 | being able to help client set measureable goals |
| 60 | able to nurture |
| 61 | ability to assess resources |
| 62 | ability to encourage |
| 63 | ability to assess role of peer support |
| 64 | ability to assess active addiction and co-dependency |
| 65 | ability to assess and access decent housing |
| 66 | routinely solicits and incorporates consumer preferences |
| 67 | ability to explain illness to consumer |
| 68 | commitment to ongoing education and training |
| 69 | ability to prioritize and manage time |
| 70 | knowledge of history of psychosocial rehabilitation |
| 71 | knowledge of principles and values of psychosocial rehabilitation |
| 72 | ability to use and develop innovative approaches |
| 73 | knowledge of and respect for multi-lingual skills |
| 74 | ability to foster inter-dependence |
| 75 | belief in the value of self-help |
| 76 | ability to help consumers choose, get, keep jobs |
| 77 | understand the availability of alternatives |
| 78 | ability to work with consumer colleagues |
| 79 | ability to help consumer learn to manage own mental illness |
| 80 | ability to help consumers develop cohesive groups |
| 81 | ability and comfort in helping consumers in recreational pursuits |
| 82 | ability to use self as a role model |
| 83 | ability to design, deliver and ensure highly-individualized services and supports |
| 84 | ability to maintain consumer records |
| 85 | early identification and intervention skills to deal with relapse |
| 86 | ability to conduct skills training in a manner to help overcome cognitive deficits |
| 87 | ability to interact and provide support in a non-judgemental fashion |
| 88 | ability to overcome personal prejudices when providing services |
| 89 | ability to normalize interactions and program practices |
| 90 | commitment to furthering the methods and technologies in PSR through research and sharing of best practices |
| 91 | willingness to take risks |
| 92 | belief in the effectiveness of psychosocial methods |
| 93 | ability to know own limits |
| 94 | never-ending willingness to develop oneself |
| 95 | ability or willingness to consider alternative paradigms |
| 96 | ability to empower consumers |
Table
2. Reliability Estimates for IAPSRS Concept Mapping Project
| Reliability Estimator | Reliability |
| Average Sort-to-Sort Reliability | .9124 |
| Average Sort-to-Total Matrix Reliability | .9607 |
| Average Sort-to-Map Reliability | .9117 |
| Split-Half Total Matrix Reliability | .9332 |
| Split-Half Map Reliability | .8882 |
| Average Rating-to-Rating Reliability | .8446 |
Table
3. Cluster listing for original map interpretation showing cluster labels,
and statement and cluster average importance ratings.
Cluster 1: Interpersonal Skills
| 1 | ability to listen to consumers | 4.71 |
| 10 | ability to motivate clients to change behavior | 3.62 |
| 36 | ability to use the helping relationship to facilitate change | 3.76 |
| 87 | ability to interact and provide support in a non-judgemental fashion | 4.33 |
| 5 | ability to offer hope to others | 4.52 |
| 6 | belief in the recovery process | 4.33 |
| 39 | ability to build on successes and minimize failures | 4.10 |
| 31 | connecting (interpersonal) skills | 3.76 |
| 78 | ability to work with consumer colleagues | 3.52 |
| 89 | ability to normalize interactions and program practices | 3.71 |
Cluster 2: Supportive Behaviors
| 2 | ability to relate to others | 4.33 |
| 49 | ability to generate enthusiasm | 3.48 |
| 62 | ability to encourage | 4.14 |
| 60 | able to nurture | 3.43 |
| 26 | ability to empathize | 4.14 |
Cluster 2 Average =
3.90
Cluster 3: Professional Role
| 14 | ability to negotiate | 3.14 |
| 58 | ability to set limits | 3.14 |
| 17 | willingness to have fun with others | 3.00 |
| 82 | ability to use self as a role model | 3.48 |
| 47 | ability to ask for help and receive constructive feedback from consumers, peers, stakeholders | 3.86 |
| 51 | ability to let go | 2.95 |
| 88 | ability to overcome personal prejudices when providing services | 4.48 |
Cluster 4: Personality Charasterics
| 16 | self awareness | 4.00 |
| 56 | good personal stability but not ego-centric | 3.43 |
| 50 | ability to handle personal stress | 3.52 |
| 18 | flexibility | 4.10 |
| 25 | patience | 3.62 |
| 28 | sense of humor | 3.48 |
| 93 | ability to know own limits | 3.57 |
Cluster 5: Self Management
| 24 | ability to read and write | 3.52 |
| 29 | ability to partialize tasks | 3.14 |
| 45 | ability to be pragmatic and do hands-on sorts of work | 4.24 |
| 33 | tolerance for ambiguity and enjoying diversity | 3.71 |
| 91 | willingness to take risks | 3.57 |
| 41 | ability to handle multiple tasks | 3.05 |
| 69 | ability to prioritize and manage time | 3.29 |
| 94 | never-ending willingness to develop oneself | 3.57 |
Cluster 6: Mental Health Knowledge Base
| 3 | knowledge of mental illness | 3.76 |
| 57 | knowledge of relationship between health status and mental illness | 2.86 |
| 4 | knowledge of side effects of medications and alternatives | 3.43 |
| 19 | knowledge of appropriate or applicable mental health acts (legislation) | 2.05 |
| 54 | knowledge of legal issues (e.g., civil commitment, guardianship) and the ethical context | 2.43 |
Cluster 7: Family Relationships
| 11 | knowledge of family networks | 2.76 |
| 30 | demonstration of respect and understanding for family members | 3.38 |
| 37 | ability to develop alliances/partnerships with family members | 3.10 |
| 40 | ability to establish alliances with providers, professionals, families, consumers (partnership model) | 3.71 |
| 43 | knowledge of a wide variety of approaches to mental health services | 2.86 |
Cluster 8: Community Resources
| 8 | knowledge of human services network in community | 3.33 |
| 20 | knowledge of eligibility benefits | 2.81 |
| 9 | knowledge of community resources beyond human services | 2.76 |
| 44 | knowledge of the community you serve and its environment | 3.14 |
| 48 | ability to work with employers | 3.24 |
Cluster 9: Assessment
| 12 | skills in advocacy | 3.38 |
| 63 | ability to assess role of peer support | 2.95 |
| 61 | ability to assess resources | 3.29 |
| 65 | ability to assess and access decent housing | 3.48 |
| 15 | strong crisis intervention skills | 3.29 |
| 85 | early identification and intervention skills to deal with relapse | 3.81 |
Cluster 9 Average =
3.37
Cluster 10: Cultural Competence
| 32 | cultural competence and ability to deliver culturally relevant services | 3.71 |
| 38 | knowledge of ethnic-based familial role definitions | 3.10 |
| 52 | ability to understand the impact of culture and ethnicity on mental illness | 3.76 |
| 73 | knowledge of and respect for multi-lingual skills | 3.05 |
Cluster 11: Professional Development
| 68 | commitment to ongoing education and training | 3.10 |
| 72 | ability to use and develop innovative approaches | 3.76 |
| 95 | ability or willingness to consider alternative paradigms | 3.43 |
Cluster 12: Psychosocial Rehabilitation Knowledge Base
| 70 | knowledge of history of psychosocial rehabilitation | 2.76 |
| 71 | knowledge of principles and values of psychosocial rehabilitation | 4.14 |
| 77 | understand the availability of alternatives | 2.95 |
| 90 | commitment to furthering the methods and technologies in PSR through research and sharing of best practices | 3.00 |
| 92 | belief in the effectiveness of psychosocial methods | 4.14 |
Cluster 13: Consumer Empowerment
| 7 | ability to emphasize client choices and strengths | 4.48 |
| 96 | ability to empower consumers | 4.62 |
| 13 | view consumer as the director of the process | 4.05 |
| 22 | ability to see consumers as equal partners | 4.00 |
| 66 | routinely solicits and incorporates consumer preferences | 4.24 |
| 42 | ability to replace self with naturally-occuring resources | 3.19 |
| 74 | ability to foster inter-dependence | 3.24 |
Cluster 14: Consumer Goal Attainment
| 34 | value consumer's ability to seek and sustain employment opportunities | 4.24 |
| 76 | ability to help consumers choose, get, keep jobs | 4.10 |
| 35 | value consumer's ability to pursue educational goals | 3.71 |
| 80 | ability to help consumers develop cohesive groups | 2.90 |
| 75 | belief in the value of self-help | 3.76 |
| 59 | being able to help client set measureable goals | 3.86 |
| 79 | ability to help consumer learn to manage own mental illness | 4.24 |
| 67 | ability to explain illness to consumer | 3.00 |
| 81 | ability and comfort in helping consumers in recreational pursuits | 2.86 |
Cluster 15: Intervention Skills
| 21 | social group-work skills | 2.52 |
| 27 | ability to develop structured learning experiences | 2.62 |
| 86 | ability to conduct skills training in a manner to help overcome cognitive deficits | 3.00 |
| 46 | ability to set goals | 3.76 |
| 23 | teaching ability | 3.24 |
| 83 | ability to design, deliver and ensure highly-individualized services and supports | 3.62 |
| 84 | ability to maintain consumer records | 2.95 |
| 53 | ability to assess behavior in specific environments | 3.19 |
| 55 | functional assessment | 3.05 |
| 64 | ability to assess active addiction and co-dependency | 3.29 |
Table
4. Listing of brainstormed statements sorted from highest to lowest
average importance rating.
| 1 | ability to listen to consumers | 4.71 |
| 96 | ability to empower consumers | 4.62 |
| 5 | ability to offer hope to others | 4.52 |
| 7 | ability to emphasize client choices and strengths | 4.48 |
| 88 | ability to overcome personal prejudices when providing services | 4.48 |
| 2 | ability to relate to others | 4.33 |
| 6 | belief in the recovery process | 4.33 |
| 87 | ability to interact and provide support in a non-judgemental fashion | 4.33 |
| 34 | value consumer's ability to seek and sustain employment opportunities | 4.24 |
| 45 | ability to be pragmatic and do hands-on sorts of work | 4.24 |
| 66 | routinely solicits and incorporates consumer preferences | 4.24 |
| 79 | ability to help consumer learn to manage own mental illness | 4.24 |
| 26 | ability to empathize | 4.14 |
| 62 | ability to encourage | 4.14 |
| 71 | knowledge of principles and values of psychosocial rehabilitation | 4.14 |
| 92 | belief in the effectiveness of psychosocial methods | 4.14 |
| 18 | flexibility | 4.10 |
| 39 | ability to build on successes and minimize failures | 4.10 |
| 76 | ability to help consumers choose, get, keep jobs | 4.10 |
| 13 | view consumer as the director of the process | 4.05 |
| 16 | self awareness | 4.00 |
| 22 | ability to see consumers as equal partners | 4.00 |
| 47 | ability to ask for help and receive constructive feedback from consumers, peers, stakeholders | 3.86 |
| 59 | being able to help client set measureable goals | 3.86 |
| 85 | early identification and intervention skills to deal with relapse | 3.81 |
| 3 | knowledge of mental illness | 3.76 |
| 31 | connecting (interpersonal) skills | 3.76 |
| 36 | ability to use the helping relationship to facilitate change | 3.76 |
| 46 | ability to set goals | 3.76 |
| 52 | ability to understand the impact of culture and ethnicity on mental illness | 3.76 |
| 72 | ability to use and develop innovative approaches | 3.76 |
| 75 | belief in the value of self-help | 3.76 |
| 32 | cultural competence and ability to deliver culturally relevant services | 3.71 |
| 33 | tolerance for ambiguity and enjoying diversity | 3.71 |
| 35 | value consumer's ability to pursue educational goals | 3.71 |
| 40 | ability to establish alliances with providers, professionals, families, consumers (partnership model) | 3.71 |
| 89 | ability to normalize interactions and program practices | 3.71 |
| 10 | ability to motivate clients to change behavior | 3.62 |
| 25 | patience | 3.62 |
| 83 | ability to design, deliver and ensure highly-individualized services and supports | 3.62 |
| 91 | willingness to take risks | 3.57 |
| 93 | ability to know own limits | 3.57 |
| 94 | never-ending willingness to develop oneself | 3.57 |
| 24 | ability to read and write | 3.52 |
| 50 | ability to handle personal stress | 3.52 |
| 78 | ability to work with consumer colleagues | 3.52 |
| 28 | sense of humor | 3.48 |
| 49 | ability to generate enthusiasm | 3.48 |
| 65 | ability to assess and access decent housing | 3.48 |
| 82 | ability to use self as a role model | 3.48 |
| 4 | knowledge of side effects of medications and alternatives | 3.43 |
| 56 | good personal stability but not ego-centric | 3.43 |
| 60 | able to nurture | 3.43 |
| 95 | ability or willingness to consider alternative paradigms | 3.43 |
| 12 | skills in advocacy | 3.38 |
| 30 | demonstration of respect and understanding for family members | 3.38 |
| 8 | knowledge of human services network in community | 3.33 |
| 15 | strong crisis intervention skills | 3.29 |
| 61 | ability to assess resources | 3.29 |
| 64 | ability to assess active addiction and co-dependency | 3.29 |
| 69 | ability to prioritize and manage time | 3.29 |
| 23 | teaching ability | 3.24 |
| 48 | ability to work with employers | 3.24 |
| 74 | ability to foster inter-dependence | 3.24 |
| 42 | ability to replace self with naturally-occuring resources | 3.19 |
| 53 | ability to assess behavior in specific environments | 3.19 |
| 14 | ability to negotiate | 3.14 |
| 29 | ability to partialize tasks | 3.14 |
| 44 | knowledge of the community you serve and its environment | 3.14 |
| 58 | ability to set limits | 3.14 |
| 37 | ability to develop alliances/partnerships with family members | 3.10 |
| 38 | knowledge of ethnic-based familial role definitions | 3.10 |
| 68 | commitment to ongoing education and training | 3.10 |
| 41 |