MPH 520 Week 6 Evaluating a Program for Behavior Change Example

Did It Reach the People Who Needed It? Evaluating a Year of a Multilevel Diabetes Prevention Program With the RE-AIM Framework

[Student Name]

University of Phoenix

MPH/520: Social and Behavioral Aspects of Public Health

Week 6 Assignment

[Instructor Name]

[Date]

The county health department, its program and first-year results are composites written for a model paper; research findings come from the sources cited.

After the first full year of the composite southern Colorado health department's multilevel diabetes prevention program, the director asked the evaluation team a direct question: did it work? The team answered that it depended on what working meant, and used a framework designed to answer that question from several angles. This paper presents the evaluation.

Why a Framework

Evaluations of behavior change programs often focus on whether participants changed. That is necessary but not enough: a program can produce excellent results for a small, motivated group while missing most of the people at risk.

RE-AIM

Glasgow and colleagues built RE-AIM to judge how much good a health promotion effort does across a population, scoring it on five dimensions: reach, meaning how many of the people meant to benefit actually take part and how well they mirror that group; efficacy or effectiveness; adoption by settings and organizations; implementation, the extent to which the program is delivered as intended; and maintenance at both individual and organizational levels (Glasgow et al., 1999). It warns that failure to evaluate all five dimensions can waste resources and fail to improve public health as much as possible.

Reach

Of 1,480 adults referred with prediabetes during the year, 402 enrolled, 27%, up from 9% before the changes. Enrollment among Hispanic referrals rose from 6% to 31%, reflecting the promotoras, adapted recipes and Spanish-language groups. Enrollment among shift workers remained low at 14%. Tripling enrollment mattered more for the county's health than any change in weight loss per participant.

Reach by Neighborhood

Enrollment was highest among referrals from the neighborhoods where the program met in churches and lowest in the eastern farming towns, where only the online option was available.

Effectiveness: The Benchmark Question

Choosing the right benchmark matters. The original randomized trial, run under close research supervision, found that participants in the intensive lifestyle arm developed diabetes at well under half the rate of those taking placebo (Diabetes Prevention Program Research Group, 2002). Real-world programs rarely match trial conditions. A meta-analysis of programs modeled on the trial and delivered in real-world settings found average weight loss of about 4% at twelve months, similar whether delivered by clinically trained professionals or lay educators, with weight loss increasing by 0.26 percentage points for each additional session attended (Ali et al., 2012).

Effectiveness: The Department's Results

Among participants who completed at least four sessions, average weight loss at twelve months was 3.8%, and 33% reached the 5% goal. Weight loss did not differ significantly between Hispanic and non-Hispanic participants. As in the meta-analysis, attendance tracked closely with results.

Effectiveness by Attendance

The team examined weight loss by number of sessions attended. Participants attending 16 or more sessions lost an average of 5.1%, while those attending fewer than 9 lost 1.9%. The pattern matched the meta-analysis and pointed to attendance as the lever: improving attendance would improve results more than changing the curriculum.

Effectiveness for Mothers

Thirty-one women with prior gestational diabetes enrolled after the postpartum outreach began. Their weight loss was similar to other participants, and several brought partners to family sessions. The numbers are small, but the group is at such high risk that even modest effects matter.

Adoption

Of 14 primary care clinics invited to change referral practices, 11 adopted the new approach. Four churches and a community center hosted groups. One of two large employers, the hospital, launched a worksite program; the steel mill declined, citing the difficulty of scheduling sessions around rotating shifts.

Implementation

Implementation was reviewed through session observations, attendance logs and interviews. Most groups covered the core curriculum as intended. Adaptations, such as family recipes and family sessions, were consistent with the program's goals. Online groups had lower attendance, with a median of 9 sessions compared with 15 in person.

Maintenance: Individuals

Among participants who finished the program at least six months earlier, 64% maintained at least half of their weight loss at follow-up. Longer follow-up, using clinic records of glucose tests, will show whether diabetes is prevented.

Maintenance: The Organization

The program's promotoras and manager are funded by a grant ending in two years. Medicaid reimbursement for the program was approved during the year, covering about 30% of costs. Sustaining the program requires additional payers.

Participant Voices

Exit interviews with 40 participants added depth. Many valued the promotoras and the adapted recipes most. Several shift workers said they dropped out when schedules changed. Online participants said they felt less connected. These accounts explained patterns in the numbers and shaped the recommendations.

Costs

The program cost about $412 per enrolled participant in its first year, including staff, materials and outreach. Cost per participant fell during the year as enrollment grew, and the department will track cost alongside results to judge value.

Equity Findings

The program narrowed the enrollment gap for Hispanic residents but not for shift workers or rural residents. These groups will be priorities in year two.

What the Evaluation Found Overall

The program reached far more people than before, produced results close to real-world benchmarks, was adopted by most clinics and was delivered as intended in person. Weaknesses were reach among shift workers and rural residents, online attendance and long-term funding.

Recommendation One: Shift Workers

Add groups scheduled around shift rotations at the hospital and approach the steel mill again with a proposal designed with its union.

Recommendation Two: Rural Reach

Offer in-person groups in two eastern towns through the county extension office and train local promotoras.

Recommendation Three: Online Attendance

Add weekly text check-ins and a promotora for online groups to improve attendance.

A Stronger Design Next Year

For year two, the team will compare clinics that adopt the new referral approach later with those that adopted it first, giving a stronger basis for judging the program's effect on enrollment than a simple before-and-after comparison.

Recommendation Four: Sustainability

Seek reimbursement from commercial insurers and the hospital's employee health plan.

Sharing the Results

The team presented the evaluation to the board of health, the community advisory group and partner clinics, and posted a summary online. Clinics received their own referral-to-enrollment rates, which prompted two lagging clinics to ask for help.

Limits

The evaluation lacks a comparison group, so improvements cannot be attributed solely to the program; enrollment may also have risen partly because clinics were paying more attention after the analysis began, and the team will watch whether gains hold in year two. Weight data come from program records, and long-term diabetes outcomes are not yet available.

Conclusion

Asking whether the program worked led to five answers. RE-AIM showed that reach tripled, effectiveness was near real-world benchmarks, most clinics adopted the approach, implementation held in person and maintenance depends on funding. Evidence from the original trial and real-world translations set realistic expectations, and each finding became a specific change for the next year.

References

Ali, M. K., Echouffo-Tcheugui, J. B., & Williamson, D. F. (2012). How effective were lifestyle interventions in real-world settings that were modeled on the Diabetes Prevention Program? Health Affairs, 31(1), 67-75. https://doi.org/10.1377/hlthaff.2011.1009

Diabetes Prevention Program Research Group. (2002). Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. New England Journal of Medicine, 346(6), 393-403. https://doi.org/10.1056/NEJMoa012512

Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89(9), 1322-1327. https://doi.org/10.2105/AJPH.89.9.1322