Background: Health research dissemination is essential to optimize the sustained use of evidence-based practices. Gaps exist to prioritize, package, and disseminate findings with end-users following study completion. The Behavioral Health Interdisciplinary Program—Collaborative Chronic Care Model (CCM) is an evidence-based practice implemented in the U.S. Department of Veterans Affairs (VA) to improve outcomes for patients with complex mental health conditions. This project describes a method used to identify CCM findings for active dissemination with mental health teams (i.e., staff and providers) and patients. Methods: In July 2024, VA researchers with CCM expertise independently used a prioritization matrix to rate CCM findings from 13 peer-reviewed publications for dissemination. Prioritization criteria were relevance (of interest), actionable (useful for care provision or to shape behavior), and prime for dissemination (for their immediate added value to CCM implementation and sustainment). CCM findings were rated on a 3-point Likert scale (1 = not, 2 = somewhat, 3 = very) for each criterion. Participants met to discuss discordant scores and reached consensus using nominal group voting. Results: Five VA researchers engaged in the prioritization process. Mean CCM finding ratings varied from 4 to 8 (range = 3-9). Of 13 CCM findings, 3 had mixed ratings, and 10 had consensus ratings: 7 were inadequate, and 3 were prioritized for dissemination by end-user group. For mental health teams, the sustainability of CCM, effectiveness of CCM and CCM impact on one-year all-cause mortality were prioritized for dissemination. For patients, the effectiveness of CCM was prioritized for dissemination. Conclusions: CCM findings published in peer-reviewed journals have limited reach and are not always easily understood by non-research audiences (e.g., mental health teams treating patients with CCM) intended to benefit from this new evidence. Prioritizing CCM findings for dissemination is an important steppingstone for knowledge transfer to end-users, which can optimize the sustained use of CCM and other evidence-based practices.

• Mental Health

• Knowledge Transfer

• Diffusion of Innovation

Information Dissemination

• Consumer Health Information

• Evidence-Based Practice

• End-User

Adjognon-Bancolé OL, Greece J, Lipson S, Beard J, Treichler E, et al. (2026) Expert Prioritization of Evidence-Based Mental Health Re search Findings for End-User Dissemination. Ann Public Health Res 13(1): 1142.

BHIP: Behavioral Health Interdisciplinary Program; BHIP-CCM: Behavioral Health Interdisciplinary Program – Collaborative Chronic Care Model; CCM: Collaborative Chronic Care Model; D4DS: Designing for Dissemination and Sustainability; NGT: Nominal Group Technique; VA: U.S. Department of Veterans Affairs; U.S.: United States

• While the science of dissemination and implementation has made great strides to bridge the health evidence-to-practice gap, these efforts have focused on implementation while dissemination has been neglected.
•  Dissemination of health evidence through scientific publications does not reach the end-users who need and can readily use the health evidence. However, health researchers are ill-equipped to engage in more effective dissemination, including knowing what piece(s) of evidence to share with end-users.
•  This manuscript provides health researchers with an initial “how to” guidance for selecting critical health evidence for dissemination with the end users who are poised to translate that knowledge into practice.

On average, it takes 17 years for health research evidence to reach practice and benefit patients or end users [1-3]. Dissemination and implementation research has introduced participatory (e.g., action research) and system-based (e.g., randomized studies) methods to better align and apply evidence in policy and practice [4]. Educating implementation partners remains the most common strategy for sustaining evidence-based practices [5]. Still, efficiently translating research into practice remains a challenge [4]. Researchers often rely on passive dissemination methods (e.g., reports, presentations, peer reviewed articles) that have limited reach [6,7], and are “largely ineffective in influencing practice” [8]. In contrast, active dissemination meets the needs and preferences of end-users and is a best practice for widespread adoption of evidence into routine practice [8,9]. As a result, valuable  findings remain unknown to end-users delivering or receiving care—healthcare teams, patients, caregivers, and communities [6,7]. Barriers to knowledge translation include researchers’ limited dissemination skills, lack of time, and insufficient organizational support. Many U.S. health researchers are not trained in active dissemination to study participants after study completion [7], and spend less than 10% of their time on dissemination efforts [9]. They also report a need for institutional capacity, including dedicated health communication expertise [9]. These challenges hinder the transfer of evidence to participants and end-users, despite its importance for successful implementation, scalability, and sustainability [4,6]. Designing for Dissemination and Sustainability (D4DS) is a 4-phase process model created to address these dissemination challenges [10]. To select health research evidence for dissemination (D4DS Conceptualization), health researchers can prioritize the knowledge they perceive most impactful for policy or practice among research findings, along with the audience likely to quickly use or apply this knowledge (i.e., end-users). Health researchers can then engage these end-users in active dissemination of this knowledge for policy or practice change (D4DS Design, D4DS Dissemination). Last, an evaluation can ascertain this dissemination impact on the implementation, scalability, and sustainability of the evidence-based policy or practice (D4DS Impact) [10]. One example of a project ready for dissemination to end users is the Behavioral Health Interdisciplinary Program using the Collaborative Chronic Care Model (BHIP-CCM) in the U.S. Department of Veterans Affairs (VA). In 2013, VA launched BHIP to deliver interdisciplinary, team-based care in outpatient mental health clinics [11]. By 2015, BHIP teams adopted the evidence-based CCM, aligning care practices with its core elements [11]. BHIP-CCM is a team-based approach to improve outcomes for Veterans with complex mental health needs, particularly those at high risk for suicide [11,12]. Since implementation, CCM has been evaluated in 13 peer-reviewed studies. However, these findings have not been formally disseminated to BHIP-CCM end-users—mental health teams and patients receiving CCM-based care. Methodologically, there is an initial need to select among all published CCM findings the ones most appropriate for active dissemination (D4DS Conceptualization) [10]. This project’s goal is to have CCM experts identify and prioritize CCM findings as the first of a four-step method for active dissemination of CCM evidence with end-users, toward adoption into practice.

Design

This project is aligned with the D4DS Conceptualization phase [10]. The project combines two consensus techniques, a prioritization matrix [13] and nominal group voting [14], to select CCM findings (i.e., knowledge) for dissemination. CCM findings are summary results taken verbatim from the abstracts of all 13 peer-reviewed publications available on evidence-based behavioral health CCM. This project is the first step toward active dissemination, where researchers select among all CCM findings those appropriate for two groups of end-users: mental health teams and patients. This project was deemed non-research by the local VA Research and Development Committee.

Use of a Prioritization Matrix Technique: Originally designed for setting priorities among health problems [13], this prioritization matrix was adapted to offer three rating criteria—relevant, actionable, and prime for dissemination—on a 3-point Likert scale (1 = not, 2 = somewhat, 3 = very) for CCM finding prioritization. Relevant is the degree to which a rater perceives that a CCM finding can be of interest to or resonate with the end-user receiving it. Actionable is the rater’s perception of the end-user’s ability to act on that knowledge, either for care provision or to shape or inform behavior change. Prime for dissemination refers to the rater’s perception of the urgency for disseminating this knowledge to the end user, based on its added value to CCM implementation and sustainment, and/or mental health decision-making. The prioritization score (Sj ) for each rater (j) is the sum of the three criteria (Ci ,) such that ij i S C = = prioritization score across raters is , and the mean 1 n j j S S n = = , the number of raters. The prioritization matrix for this project was created using Microsoft Excel. Information on peer-reviewed publications was organized chronologically across rows, while preferred end-users, rating criteria, and rater’s comments were in separate columns [Figure 1]. The project lead and senior author agreed on a priori preferred end-user group(s) assigned to each publication in the matrix. The Nominal Group Technique: The Nominal Group Technique (NGT) is a flexible consensus method used in research to problem-solve, generate ideas, and set priorities [14]. In this project, the NGT was used in combination with the prioritization matrix to select CCM findings for dissemination. First, raters provided their individual ratings using the matrix. Next, they met as a small group to discuss CCM findings with discordant ratings across raters (i.e., NGT clarification stage). Last, raters synchronously voted on discussed findings (i.e., NGT voting/ranking stage) to reach consensus [14]. Participants A panel of VA researchers with deep collaborations and prolonged experience with the multisite Behavioral Health Interdisciplinary Program using CCM (BHIP-CCM) was recruited via email to be Prioritization and NGT raters. Some of these VA researchers have provided facilitation for the implementation of BHIP-CCM at early adopting sites, and are aware of end-users’ needs. Raters were guided by a neutral project lead (not previously embedded in BHIP CCM) through all the steps in the prioritization process. Data Collection and Analysis Integrated data collection and analyses occurred in five steps.

Step 1: Independent rating: Raters used the matrix to rate each CCM finding for 13 peer-reviewed publications. They assessed 3 criteria: how relevant, actionable, and prime for dissemination CCM findings were perceived to be, on a 3-point Likert scale (1= not; 2 = somewhat; 3 = very). Total summary ranking by rater for each CCM finding auto populated in the rating total column (range: 3-9) (See Table 1). where n is

Step 2: Inter-rater comparison: This step was completed by the project lead. For each rater’s rating total column, the top two CCM findings with the highest scores were shaded green, and the bottom two CCM findings with the lowest scores were shaded red. CCM findings that consistently received high or low scores saw their row’s rating totals completely green or red across raters on this expanded matrix, while CCM findings with inconsistent ratings presented a mix of colors: green (high), white (medium), and red (low).

Step 3: Member checking: The project lead and senior author with previous experience with the BHIP-CCM project met to review prioritization matrix findings. They agreed that consensus discussion should focus on CCM findings with discordant ratings.

Step 4: Consensus discussion: Raters met virtually for one-hour facilitated group discussion to review CCM findings that received inconsistent or discordant ratings across raters. For these findings, each rater shared rationale for their independent ratings. The group then discussed each discordant finding to come to consensus.

Table 1: Prioritization Matrix Template.

CCM: Collaborative Chronic Care Model

Note: The Prioritization Matrix used in this project had 13 rows: one row for each of the 13 peer-reviewed publications. Select rows are presented here for illustration.

on the relevant, actionable, and prime for dissemination criteria for preferred end-users (i.e., mental health teams, patients). Thereafter, NGT voting was used, asking each rater to decide if each discordant CCM finding should be disseminated and was rated as “yes”, “maybe”, or “no” [14].

Step 5: Final analysis: The project lead and senior author met to review both the decisions from Step 2, and the results from the consensus discussion in Step 4. All ratings and voting outcomes were combined in this step to complete the prioritization process and identify CCM findings that met the prioritization criteria, (i.e., mostly relevant, actionable, and prime for dissemination).

Participant Characteristics

Five raters representing diverse mental health and research expertise (clinical psychology, implementation science, qualitative methods) engaged in the project (Table 2). One rater did not participate in Step 4: Consensus discussion but met with the project lead separately to complete that step. The project lead used the consensus discussion notes to convey what was discussed with that rater, who then proceeded to NGT voting asynchronously.

Prioritization of CCM Findings

Step 1: Independent rating: Raters rated CCM findings from 13 peer-reviewed publications with pre-assigned preferred end-users. Mean ratings ( S ) across raters varied from 4.4 to 7.6 (range = 3-9, lowest to highest). Table 3 S provides a summary of mean ratings ( decisions by analytic step.

Step 2: Inter-rater comparison: Raters reached consensus on the dissemination status of eight CCM findings. Six were labeled “do not disseminate” due to low scores (mean rating S ≤6.0, with at least 4 of 5 raters scoring them low), while two were labeled “disseminate” based on high scores (mean rating S ≥6.0, with at least 3 high and no low individual ratings). The first prioritized finding for dissemination to mental health teams is CCM’s sustainability [15], which showed that evidence-based psychotherapies and measurement-based care were maintained or expanded three years post implementation. The second is CCM’s effectiveness [16], initially identified for dissemination to patients receiving team-based outpatient care. This finding includes reduced hospitalization rates and improved health among patients with complex mental health conditions (i.e., ≥3 diagnoses). Three raters also recommended sharing this evidence with mental health teams.

Table 2: Participant Characteristics.

*Participants have more than one expertise; totals do not add to 100%.

Step 3: Member checking: The project lead and senior author reviewed the 13 CCM findings together. They confirmed the decision thresholds, i.e., “do not disseminate” for CCM studies with S ≤6.0 and at least 4 low ratings, and “disseminate” for CCM studies with ≥6.0 and at least 3 high and no low ratings. Upon review, 8 CCM S S studies had concordant ratings, with initial decisions: “do not disseminate” or “disseminate.” CCM studies with mean ratings that did not meet threshold were categorized as “discordant” (see Table 3). The team agreed that the consensus discussion should focus on 5 CCM studies with discordant ratings, and end-user review for one concordant CCM finding per raters’ comments (e.g., only patients vs. adding mental health teams).

Step 4: Consensus discussions addressed the 5 remaining CCM findings with discordant ratings. Using NGT voting, two findings received unanimous votes—either 5 “yes” (disseminate) for a specific end-user group or 5 “no”(do not disseminate). The remaining three received mixed votes (“yes,” “maybe,” or “no”) for dissemination to a given group. The unanimously endorsed finding was CCM’s impact on one-year all-cause mortality [17], which showed reduced mortality among patients treated in outpatient mental health clinics using CCM compared to those treated without. Raters also revisited the effectiveness of CCM finding from Step 2, previously prioritized for patients. With 5 “yes” votes, they expanded the recommended end-user group to include both patients and mental health teams. This finding includes reduced hospitalization and improved health for patients with complex mental health conditions [16]. An additional element—improved care team functioning (e.g., clearer roles, team-first goals)—was also prioritized for dissemination to mental health teams only [16].

Table 3: Summary Ratings and Decisions by Analytic Step.

*CCM: Collaborative Chronic Care Model

Step 5: Final analysis consolidated previous steps to complete the prioritization process from 13 CCM findings. CCM findings with consistent agreement across raters (Step 2) and those with at least 4 “yes” responses following NGT voting (Step 4) determined knowledge prioritized for dissemination. Overall, there was agreement for 10 CCM findings, including 3 labeled “disseminate” (Table 3), and 7 “do not disseminate.” Three CCM findings with discordant ratings and mixed NGT voting were labeled as “medium priority” for dissemination. Three CCM findings were equally prioritized by end-user group (Table 4). The effectiveness of CCM [16] was selected for dissemination with both patients and mental health teams. The sustainability of CCM [15] and CCM impact on one-year all-cause mortality [17] were prioritized for dissemination with mental health teams only.

This project aimed to share a method to select evidence-based CCM findings for active dissemination with mental health teams and patients. The method combined a prioritization matrix technique [13] with NGT voting [14] to prioritize 3 CCM findings from 13 peer-reviewed studies. Final selections include the effectiveness of CCM [16] for dissemination with mental health teams and patients; the sustainability of CCM [15], and the impact of CCM on one- year all-cause mortality [17] for dissemination with mental health teams only.

All CCM findings selected for dissemination in this project inform end-users that CCM is effective, including improved clinical outcomes and impact on mortality. In contrast, many findings pertaining to CCM implementation processes, lessons, and cost savings were not selected for dissemination. Future work may assess if information on effectiveness vs. implementation is consistently prioritized for dissemination across evidence-based programs.

As health research knowledge translation aims to narrow the knowledge-to-practice gap, transferring the knowledge that CCM works is a primer for getting end- user buy-in, but helping end-users understand how to best implement or receive CCM-aligned care is also needed. For instance, knowledge that CCM works can be paired with CCM implementation support and resources for mental health teams. Patients can also be guided in ways to access and benefit from CCM care. Additional future work could be done with end-users to support ways to share information on CCM implementation approaches.

This project proposed a method that addresses some of the common barriers to knowledge translation—limited time, skills, and organizational support for dissemination to study participants and end-users [7,9]. VA researchers/ raters found the matrix prioritization paired with nominal group voting simple and easy to complete, suggesting strong feasibility. Raters unanimously voted on two of five CCM findings with initially discordant ratings and added mental health teams as an end-user group for disseminating the effectiveness of CCM. A key strength of this project is its engagement process to guide dissemination with end-users. Pairing the D4DS model with two consensus techniques, the prioritization matrix and nominal group voting, offers a low-burden, structured approach for researchers to select key findings and engage experts in decision-making. Holding the consensus meeting virtually also supported feasibility.

Health research dissemination with study participants and end-users is essential to optimize the implementation, scalability, and sustainability of evidence-based policies and practices [4,18]. Selecting research evidence to disseminate with end-users following study completion is possible. This project provides health researchers with a feasible process to review and prioritize findings to disseminate with end-users of evidence-based practices.

Limitations include tradeoffs from predefined selection criteria, such as focusing only on mental health teams and patients. For example, two CCM findings rated unanimously as “do not disseminate” were considered valuable for another excluded group—local leadership. Prioritization efforts should carefully define target end-users while recognizing the opportunity cost of excluding others. Local 

leaders, who influence implementation decisions, should be considered as key end-users. Finally, this process did not yet include input from end-users.

In the next D4DS phases, mental health teams and patients co-developed tools to disseminate the prioritized CCM findings. In these next phases, the project has leveraged collaborations with select end-users to design and package prioritized mental health evidence in end-users preferred formats, language, and media. Ultimately, active dissemination using end-users preferred communication channels can expand the reach of BHIP- CCM, as well as health research evidence more generally, and these efforts can optimize health policy and practice.

Ethical Approval and Consent to Participate

This work was deemed non-research by the VA Boston Research and Development Committee (#1588376). No consent statements were taken from VA researchers who agreed to participate in this work.

Availability of Data and Materials

The datasets generated and/or analyzed during the current project are not publicly available due to privacy or ethical restrictions.

Funding

This work was supported by Merit Review Award Number QUE 20-026 from the United States (U.S.) Department of Veterans Affairs Health Services R&D (HSRD/QUERI) Service. The funder had no role in the analyses or content of this manuscript.

Authors’ contributions

O.L.A.: Funding Acquisition, Conceptualization, Methodology, Project Administration, Data Curation, Analysis Interpretation, Writing-Original Draft Preparation;

J.G.: Writing-Review and Editing; S.L.: Writing-Review and Editing; J.B.: Writing-Review and Editing; E.T.: Writing-Review and Editing;

B.K.: Methodology, Writing-Review and Editing; C.J.M.: Methodology, Writing-Review and Editing;

J.L.S.: Funding Acquisition, Conceptualization, Methodology, Data Curation, Analysis Interpretation, Writing-Review and Editing.

The authors acknowledge VA researchers who participated in the prioritization process and shared their time and expertise.

1. Balas EA, Boren SA. Managing clinical knowledge for health careimprovement. Yearb Med Inform 2000; 9: 65-70.
2. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011; 104: 510-20.
3. Rubin R. It takes an average of 17 years for evidence to change practice—the burgeoning field of implementation science seeks to speed things up. JAMA. 2023; 329: 1333-1336.
4. Estabrooks PA, Brownson RC, Pronk NP. Dissemination and Implementation Science for Public Health Professionals: An Overview and Call to Action. Prev Chronic Dis. 2018; 15: E162.
5. Flynn R, Cassidy C, Dobson L, Al-Rassi J, Langley J, Swindle J, et al. Knowledge translation strategies to support the sustainability of evidence-based interventions in healthcare: a scoping review. Implementation Science. 2023; 18: 69.
6. Brownson RC, Eyler AA, Harris JK, Moore JB, Tabak RG. Getting the word out: new approaches for disseminating public health science. J Public Health Manag Pract. 2018; 24: 102-111.
7. Melvin CL, Harvey J, Pittman T, Gentilin S, Burshell D, Kelechi T. Communicating and disseminating research findings to study participants: Formative assessment of participant and researcher expectations and preferences. J Clin Transl Sci. 2020; 4: 233-242.
8. Kerner J, Rimer B, Emmons K. Introduction to the special section on dissemination: dissemination research and research dissemination: how can we close the gap? Health Psychol. 2005; 24: 443.
9. Brownson RC, Jacobs JA, Tabak RG, Hoehner CM, Stamatakis KA. Designing for dissemination among public health researchers: findings from a national survey in the United States. Am J Public Health. 2013; 103: 1693-1699.
10. Kwan BM, Brownson RC, Glasgow RE, Morrato EH, Luke DA. Designing for dissemination and sustainability to promote equitable impacts on health. Annu Rev Public Health. 2022; 43: 331-353.
11. Bauer MS, Weaver K, Kim B, Miller C, Lew R, Stolzmann K, et al. The Collaborative Chronic Care Model for Mental Health Conditions: From Evidence Synthesis to Policy Impact to Scale-up and Spread. Med Care. 2019; 57: S221-s227.
12. Miller C, Sullivan J, Kim B. A Rani E, Karen LD, Samantha LC, et al. Implementing Collaborative Care for Outpatient Mental Health Teams: The BHIP Enhancement Project. Implement Sci Commun. 2021; 2: 33.
13. Peoples-Sheps M, Farel A, Rogers M. Assessment of health statusproblems. Maternal and Child Health Bureau, HRSA. 1996.
14. McMillan SS, King M, Tully MP. How to use the nominal group andDelphi techniques. Int J Clin Pharm. 2016; 38: 655-662.
15. Miller CJ, Kim B, Connolly SL, Spitzer EG, Brown M, Bailey HM, et al. Sustainability of the Collaborative Chronic Care Model in Outpatient Mental Health Teams Three Years Post-Implementation: A Qualitative Analysis. Adm Policy Ment Health. 2023; 50: 151-159.
16. Bauer MS, Miller CJ, Kim B, Lew R, Stolzmann K, Sullivan J, et al. Effectiveness of Implementing a Collaborative Chronic Care Modelfor Clinician Teams on Patient Outcomes and Health Status in Mental Health: A Randomized Clinical Trial. JAMA Netw Open. 2019; 2: e190230.
17. Ruderman MA, Byers AL, Bauer MS, Stolzmann K, Miller CJ, Connolly SL, et al. One-Year All-Cause Mortality and Delivery of the Collaborative Chronic Care Model in General Mental Health Clinics. Psychiatr Serv. 2023; 74: 1077-1080.
18. Brownson RC, Fielding JE, Green LW. Building Capacity for Evidence- Based Public Health: Reconciling the Pulls of Practice and the Push of Research. Annu Rev Public Health. 2018; 39: 27-53.