Accounts of mental health services in the United States indicate a history of system ineffectiveness in addressing the needs of children of color and their families. Recognizing such disparities, children’s mental health systems of care incorporate cultural competence as a guiding principle for developing responsive service systems. A challenge for mental health researchers is identifying methods for assessing this complex construct. Moreover, there is a lack of empirical research describing definitions and practice models of culturally responsive mental health care.

This study examined cultural competence within a state legislated mental health initiative that included two urban and two rural systems of care communities. The study was part of a larger evaluation effort conducted by the Center for Social Work Research at The University of Texas at Austin. Practical objectives included generating baseline assessments for monitoring cultural competence development and identifying information to assist agencies in developing related training plans and policy. The research also aimed to examine the contextual validity of several established models of culturally competent practice within a theory of systems change framework. The research sought to determine (1) the differences and similarities in conceptualizations of cultural competence among multiple participants and systems of care communities; (2) whether the community conceptualizations supported assumptions and theoretical conceptualizations of culturally competent practice; and (3) the viability of Concept Mapping for conceptualizing and assessing cultural competence in specified systems of care.

In collaboration with local community representatives a diverse research team conducted cultural competence assessments in each of the four systems of care communities. Communities recruited the participants for their individual assessments, resulting in a combined sample of 188 adults. Participants included family members, staff at multiple levels, advisory board members, and mental health care providers. Demographic information gathered included variables such as individual’s role in the system of care, race/ethnicity, gender, age, recent cultural competence training, and household income. Each community’s assessment was conducted over a three-day period.

Data Collection. On day one, statements were generated by two separate groups in each community: a family member group and a professional provider group. In an effort to make the assessment process more culturally relevant, three interchangeable focus prompts were used to brainstorm the statements. The following prompts were also translated into Spanish for Spanish-speaking participants.

1. I know services to families are culturally competent when
2. I know services to families are respectful when
3. I know services to families are culturally responsive when

Participants from day one returned for day two to complete the sorting and rating activities. Some family members and providers who were unable to attend the brainstorming session chose to join the process on day two. All participants were asked to rate the ideas on how important the examples were for meeting the unique needs of families and how often the examples were demonstrated in the community’s system of care. The professional providers additionally rated the examples on the extent to which the providers believed the statements were covered under their respective agency’s policies.

Analysis and Interpretation. The research team conducted preliminary data analyses prior to meeting on day three with a smaller group of participants selected by the hosting project director. The researchers used the final cluster and labeling determination processes to provide a structure for conducting the interpretation sessions. This process resulted in rich dialogue around the meaning of the mapped information and the group comparisons made through pattern matching. The information gathered over the three-day period was used to provide contextualized written reports to individual communities.

Upon completion of the individual community assessments, the 303 statements generated from across all four communities were combined, reduced to one unduplicated list of 117 statements, and mailed to a representative sample of the original participants. Additional sorts and ratings were collected for an aggregate examination of cultural competence. Data analysis and map construction for the aggregate study were completed by the researchers based on the combined individual community and aggregate assessment information and experiences.

Following the Concept Mapping phase of the study, community maps were compared for common and unique concepts using statements within clusters as a method of clarifying overall thematic cluster meanings across communities. A structured theoretical framework was then developed from the literature to conduct a secondary analysis of the data. Practice model components were first embedded into the framework to ensure theoretical fit. The analyses concluded by examining the community conceptualizations for congruence with the theoretical framework and with established models of culturally diverse social work practice.

Results. Each community chose the cluster map solution that best represented the community’s ideas of cultural competence. One community selected a seven-cluster solution, two communities selected eight-cluster solutions, and one community selected a nine-cluster solution. A 15-cluster solution was chosen for the aggregate map. Differences between grouped participant ratings were found on each criterion; however, the extent of the differences varied across communities. Pattern match comparisons between ratings of importance and demonstration offered communities an overall baseline from which to monitor their cultural competence development. Comparisons with practice models identified a number of elements from each practice model embedded in the systems of care community conceptualizations. The extent to which each model’s elements was included varied by community. In contrast, a number of individual practice model elements were not reflected in the individual community conceptualizations, including some critical assumptions of the models.

Summary. The study allowed individual community participants to contextually define cultural competence for their own systems of care. Concept Mapping offered a unique way of gathering and analyzing perspectives from many individuals across multiple levels of the systems. By examining discrepancies between participant ratings, the study results helped participating systems of care develop cultural competence technical assistance and training plans tailored to their unique needs. Moreover, the assessments established a baseline for communities to use in monitoring their progress toward cultural competence development. Results from the aggregate data provided additional assistance to the State team in planning for resource development. The use of a theoretical framework provided a useful structure within which to make comparisons between community and practice model conceptualizations. Implications for professional education and practice were raised with the finding that no one model of culturally diverse practice wholly accounted for all of the concepts generated by the systems of care communities. On a broader scale, Concept Mapping demonstrated great promise as an alternative method to conceptualizing and assessing the complex value of cultural competence in children's mental health systems of care.

The Chronic Disease Conference, held in Washington DC from February 18 – 20, 2004, proved to be an excellent opportunity for Concept Systems to learn more about our clients, hear about the concept mapping work our research licensees have done, and share some of the exciting work we have been doing in projects addressing several different areas within chronic disease.

Mary Kane joined Dr. Darwin LaBarthe, Associate Director for Cardiovascular Health at the Centers for Disease Control and Prevention, in giving a talk about the work they have been doing on the Cardiovascular Health Action Plan implementation. Nancy Watkins of CDC’s Division of Adult and Community Health described the federal-state partnership perspective of the Action Plan, and Mark Schoeberl, Vice-President for Advocacy at the American Heart Association, helped to anchor the presentation from the advocacy and service point of view. The presentation provided an opportunity for Mary and Dr. LaBarthe to talk about the process that was used to create an action plan, and the priorities and focus of the plan itself.

Jeanine Draut enjoyed talking with both CDC employees and partners about Project Officer of the Future, the professional development program for CDC’s National Center for Chronic Disease Prevention and Health Promotion program consultants. One of these discussions was a formal roundtable discussion, but many of the conversations occurred while meeting people during the conference. Mary and Jeanine were also able to meet with our Project Officer of the Future planning team and prepare for the exciting months ahead. The first pilot course will be conducted in April.

Jeanine also attended a roundtable discussion about the Health Disparities Roadmap project Concept Systems has been working on with Dr. Vijaya Hogan from the University of North Carolina and Dr. Bill Jenkins of Morehouse College. Though the roundtable discussion was scheduled for 40 minutes, the participants had a lively discussion that lasted for over an hour, about the topic and the concept mapping methodology. Jeanine had the pleasure of meeting Dr. Jenkins, who presented the Roadmap Planning Team enthusiastically and knowledgeably.

A valued past client, the CDC’s Prevention Research Centers Office, presented details on their progress with Project DEFINE, the evaluation framework design project for the Prevention Research Centers. We had the pleasure of seeing the high quality evaluation product that has come from this project, which started by using concept mapping to explore and define the important factors to consider for evaluation. The project team - Lynda Anderson from CDC, the Collaborative Evaluation Design Team (CEDT), and the Cosmos Corporation team have almost completed the planning required to implement the evaluation design.

Both Mary and Jeanine were able to attend various workshops and presentations to learn more about the world our clients inhabit. Concept Systems was grateful for this opportunity to support our clients, talk about our recent concept mapping projects in chronic disease, and learn more about the latest scientific, programmatic, and strategic advances in addressing chronic disease from a public health perspective.

“A specific issue that is relevant to the mental health of girls and women is ..… .”
That is the question that the Office on Women’s Health, an office within the Department of Health and Human Services and the Surgeon General’s Office, wanted to ask a broad spectrum of relevant stakeholders and knowledge sources. Exploring sex/gender differences, personal environment, resilience and other topics, and then summing them up to enable a common framework, was made easy by involving CSI and the Concept System approach.

The primary desired outcome for this effort is to craft a supplemental publication to the 1999 document: Mental Health: A Report of the Surgeon General. An OWH staff team led the project, and collaborated with Concept Systems, Inc. (CSI) to gather input, facilitate discussions and analyze the results. Stakeholders included participants from academia, advocacy groups and public health organizations.

The Concept Mapping methodology was uniquely suited to building awareness and consensus on this important topic, as it provided a quantitative basis for discussion and planning within a group setting. Equally importantly, it enables subgroups knowledge- or interest-based subgroups to examine specific clusters of thought in detail, as a basis for action planning.

Work on this project continues, as we present the findings to the large cross-disciplinary group of Advisors in early April. In the meantime, project sponsor and lead client Dr. Wanda Jones of OWH presented some preliminary information and a model conceptual framework at the 2nd World Congress on Women’s Health. The OWH staff are currently working with CSI to create additional products using the concept mapping results. In addition to the supplemental publication, OWH is considering leadership interviews of sister agencies and hosting a Surgeon General’s Workshop. The results of this project will form the basis for developing a literature review and a crosswalk with previous reports.

Working through this intricate and timely issue with CSI’s assistance has enabled the development of an agreed-upon framework that is timely, rich in detail, and readily accepted by many stakeholders.