CHL 640 Week 1 Evaluation Purposes and Stakeholders Example

Who Needs to Know What, and Why: Defining the Purposes and Stakeholders of the Walk and Eat Well Evaluation

[Student Name]

University of Phoenix

CHL/640: Evaluating Community Health Initiatives

Week 1 Assignment

[Instructor Name]

[Date]

The initiative, its stakeholders, their questions and the evaluation team are composites written for a model paper modeled on Yakima, Washington; research findings and federal guidance come from the sources cited.

Six months into Walk and Eat Well, the regional health foundation asked the coalition for its evaluation plan. At the next initiative committee meeting, it became clear that people wanted different things from an evaluation. The foundation wanted to know whether its money produced results. The health center wanted to know whether to fund promotoras after the grant. Store owners wanted to know whether produce sold. Residents wanted to know whether their neighbors were healthier and whether the program treated them well. This paper defines the evaluation's purposes and stakeholders before any data are collected.

Why Evaluate

Evaluation serves several purposes. It improves a program by showing what works and what does not, early enough to adjust. It provides accountability to funders, partners and the community. It builds knowledge for others planning similar work. And it supports advocacy, giving the coalition evidence to seek lasting funding and policy change.

Types of Evaluation

Different questions call for different types. Formative evaluation, done early, tests whether materials and activities fit the community. Process evaluation examines whether activities are delivered as planned and reach the intended people. Outcome evaluation examines short-term and intermediate changes, such as confidence, activity and weight. Impact evaluation examines long-term change, such as fewer new cases of diabetes, and usually requires more time and stronger designs.

CDC's Framework

The team adopted CDC's 1999 evaluation framework. The framework runs in six linked steps. The first two concern people and the program: involve those with a stake, then describe what the program does. The middle two concern method: narrow the evaluation's focus, then gather evidence people will find credible. The last two concern meaning: reach conclusions the evidence supports, then see that findings are used and lessons shared. It also sets four groups of standards, utility, feasibility, propriety and accuracy, to judge whether an evaluation will be effective (Centers for Disease Control and Prevention [CDC], 1999). The first step, engaging stakeholders, is the focus of this paper.

Stakeholder Group One: Those Involved in Operations

This group includes the coordinator, lead promotora and promotoras, walk leaders, store owners, the health center as fiscal agent, the health district's evaluator and the parks department. Their questions are practical: Are sessions well attended? Which recruitment channels work? Do stores sell the produce they stock? What is not working?

Stakeholder Group Two: Those Served

This group includes lifestyle group participants, walkers, shoppers at partner stores and their families. Their questions are about benefit and respect: Is the program helping people like me? Is it offered at times and places I can manage? Am I treated with respect? Is my information kept private?

Stakeholder Group Three: Those Who Use Results

This group includes the foundation, the hospital's community benefit office, the Medicaid managed care plan, the coalition's steering committee, city council members and other communities considering similar work. Their questions concern results and value: Does the initiative reduce weight and increase activity? What does it cost per participant? Should it continue, grow or change?

Mapping Questions to Stakeholders

The team built a simple table with stakeholder groups in rows and their top questions, the decisions those questions inform and when answers are needed in columns. The table showed that operational questions need answers monthly, while funding questions need answers before specific budget dates. Timing, not only content, shapes the evaluation plan.

What Stakeholders Worried About

Conversations surfaced concerns. Promotoras worried the evaluation would be used to judge them individually; the team agreed to report fidelity by group, not by name. Store owners worried sales data would be shared with competitors; data will be combined across stores in public reports. Residents worried about being studied without seeing results; every data collection will be followed by a report back.

Decisions at Stake

Tying questions to decisions sharpens the evaluation. The foundation will decide whether to renew in year three. The Medicaid plan will decide whether to pay for promotora services. The coalition will decide whether to add stores or groups. The city will decide whether to fund more path improvements.

Why Involvement Matters for Use

Evaluations that stakeholders help shape are more likely to be used. A review of 41 empirical studies of evaluation use from 1986 to 2005 added stakeholder involvement as a new category to an established framework and found that engagement, interaction and communication between evaluators and those who use evaluations are critical to meaningful use (Johnson et al., 2009). An evaluation nobody asked for is an evaluation nobody reads.

Residents as Evaluation Partners

Residents will not only answer surveys. Two residents on the initiative committee will join the evaluation team, help write questions, review instruments, collect some data and interpret findings. A participant advisory group will review results before they are released.

Evaluating Participation Itself

Participation is itself something to evaluate. Butterfoss's review of participation research offers the evaluation a set of questions: which residents take part and what draws them, what they gain and what gets in their way, and which blend of counts and conversations captures participation best (Butterfoss, 2006). The evaluation will track resident participation in the initiative and in the evaluation.

Internal or External Evaluator

The team considered hiring an outside evaluator. An external evaluator brings independence, which the foundation values, but costs more and knows the community less. The health district's evaluator, internal to the partnership but separate from program staff, offers local knowledge and continuity. The team chose the district's evaluator, with a university faculty member reviewing the design and final report for independence.

Applying the Standards

Utility: the evaluation answers the questions stakeholders named. Feasibility: data collection fits staff time and budget. Propriety: participants consent, data are private and results are reported honestly, including disappointing ones. Accuracy: measures are valid and methods are described transparently.

Ethics

Participants will give informed consent, names will not appear in reports, weights and health information will be stored securely and immigration status will never be recorded. Findings about specific stores will be shared with owners before public release.

Statement of Purposes

The evaluation has three purposes: to improve Walk and Eat Well during implementation; to assess whether it changes participants' weight, activity and eating and at what cost, for accountability and funding decisions; and to share lessons with other communities.

The Evaluation Team and Next Steps

The team includes the health district's evaluator as lead, the coordinator, two resident members, the lead promotora and a university reviewer. Next steps are to write evaluation questions from the logic model and to review them with each stakeholder group.

Conclusion

Different stakeholders want different answers, and a useful evaluation starts by naming them and the decisions they face. CDC's framework puts engaging stakeholders first and sets standards for quality, research on evaluation use shows involvement drives use and research on participation guides residents' roles. Three purposes and a team with resident members now anchor the Walk and Eat Well evaluation.

References

Butterfoss, F. D. (2006). Process evaluation for community participation. Annual Review of Public Health, 27, 323-340. https://doi.org/10.1146/annurev.publhealth.27.021405.102207

Centers for Disease Control and Prevention. (1999). Framework for program evaluation in public health. MMWR Recommendations and Reports, 48(RR-11), 1-40. https://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm

Johnson, K., Greenseid, L. O., Toal, S. A., King, J. A., Lawrenz, F., & Volkov, B. (2009). Research on evaluation use: A review of the empirical literature from 1986 to 2005. American Journal of Evaluation, 30(3), 377-410. https://doi.org/10.1177/1098214009341660