DHA 731 Week 3 Evaluating Study Designs and Evidence Example

Before We Fund It: Weighing Trials, Reviews and a Local Pilot on Community Health Workers for Rural Chronic Disease

[Student Name]

University of Phoenix

DHA/731: Population Health and Epidemiology

Week 3 Assignment

[Instructor Name]

[Date]

The health network, its pilot, pilot results and funding decision are composites written for a model paper; research findings come from the sources cited.

The population health director brought the regional vice president a proposal to hire eight community health workers to support adults with multiple chronic conditions in the four rural counties. Attached were results from a six-month pilot: among 40 patients with poorly controlled diabetes, average blood sugar had fallen sharply, and emergency visits had dropped by half. The director called the results proof and wanted to hire immediately. Before recommending $640,000 a year, the vice president asked how strong the evidence really was. This paper answers that question.

Study Designs

Designs differ in how well they support causal conclusions. Cross-sectional studies measure conditions at one time and are good for prevalence. Cohort studies follow people over time and can identify risks. Case-control studies compare people with and without an outcome. Before-and-after studies measure a group before and after an intervention. Randomized trials let chance decide who receives the intervention, so that differences between groups, measured or not, even out. Systematic reviews combine the results of many studies, and their strength depends on the quality of the studies they include. Each design answers some questions well and others poorly.

The Local Pilot

The pilot was a before-and-after study without a comparison group. Its results were encouraging, but its design is weak for judging effects. Patients were chosen because their blood sugar and emergency use were unusually high at the start.

Regression to the Mean

That choice creates a well-known problem. People selected for extreme values tend to move toward the average when measured again, even with no intervention, because part of what made their values extreme was temporary: a bad month, an illness or a missed prescription. Some of the pilot's improvement would likely have occurred anyway. A simple check makes the point: the quality team pulled records for 40 similar patients from the prior year who had equally high blood sugar but received no support, and their average also fell over the following six months, by about half as much as the pilot group's. The pilot's true effect, if any, is the difference, not the whole change. Patients picked at their worst will look better later, whether or not anyone helps them.

Other Threats in the Pilot

The pilot had other weaknesses. Forty patients is a small sample, so a few patients' changes can move the average a great deal. Patients who dropped out were not counted. And staff who knew the program's goals measured the results, which can introduce bias.

A Randomized Trial

Stronger evidence comes from a trial. Kangovi and colleagues randomly assigned 302 residents of high-poverty Philadelphia neighborhoods who were uninsured or publicly insured and had two or more chronic conditions, such as diabetes, obesity, smoking and hypertension, to goal setting alone or goal setting plus six months of support from community health workers (Kangovi et al., 2017).

Reading the Trial's Results

The results require careful reading. Support from community health workers led to improvements in several conditions, including a 0.4-point drop in glycated hemoglobin compared with no change and fewer cigarettes per day, but the combined measure of disease control had a P value of .08, just above the usual threshold; self-rated mental health and quality of care improved significantly; and hospitalization fell 28% at one year, a result with a P value of .11 (Kangovi et al., 2017). Systolic blood pressure fell more in the comparison group, an unexpected result.

What the Trial Shows

The trial suggests real benefits, particularly for mental health and patients' experience of care, and likely benefits for diabetes and smoking. The hospitalization finding is promising but uncertain. The trial was conducted in a city, so its application to rural counties with long travel distances is indirect. Community health workers in Philadelphia could walk or take a bus to patients' homes; in the rural counties, a single visit may require an hour of driving, which reduces how many patients each worker can support and raises the cost per patient. The trial's six-month support period may also be too short for patients whose needs include housing, transportation and food.

A Systematic Review

A review adds breadth. Kim and colleagues reviewed 61 studies of community-based health worker interventions for chronic disease among vulnerable populations, most in the United States, and found that roles commonly included health education, counseling and navigation, that training and supervision were often underreported and that interventions appeared effective compared with alternatives and cost-effective for certain conditions, particularly with low-income and minority communities (Kim et al., 2016).

Rating Certainty With GRADE

GRADE offers a shared scale for judging how much confidence a set of studies deserves. Guyatt and colleagues describe how evidence from randomized trials begins as high certainty and can be rated down for limitations in study conduct, inconsistency, indirectness, imprecision and publication bias, while recommendations are graded as strong or weak based on the balance of benefits and harms, certainty and values (Guyatt et al., 2008).

Applying GRADE

For the region's question, evidence begins high because of randomized trials, but it is rated down for imprecision, since key results like hospitalization are uncertain, and for indirectness, since most trials were urban. Overall certainty is moderate for improvements in patient experience and mental health and low to moderate for reductions in hospital use.

What the Evidence Means for the Decision

Moderate evidence of benefit, a vulnerable population and modest costs support acting, but not with the confidence the pilot suggested. Waiting for perfect evidence would leave patients without help that is likely to benefit them, while committing permanently without evaluation would leave the region unable to tell whether the money is well spent. The right response is to fund the program with a design that will produce better local evidence.

Recommendation

The vice president recommended funding eight community health workers, with patients assigned by lottery to begin support immediately or six months later. The waitlist design is fair, since everyone eventually receives support, and it creates a comparison group to measure effects in rural settings. The region's research partner at the academic center will analyze the results, and the design will be registered in advance so that outcomes cannot be chosen after the fact.

Measures

The evaluation will measure glycated hemoglobin, blood pressure, smoking, hospitalizations, emergency visits, mental health and patient experience, collected by staff not involved in the program. Costs per patient, including travel time, will be tracked so that a cost-effectiveness analysis can follow. Results will be reported to the board after one year, whatever they show.

Conclusion

The local pilot looked like proof but was vulnerable to regression to the mean. A randomized trial shows benefits for mental health, quality of care and some disease measures, with uncertain effects on hospitalization, and a review supports effectiveness in underserved communities. Evidence of moderate certainty justifies funding the program with a rigorous rural evaluation.

References

Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ, 336(7650), 924-926. https://doi.org/10.1136/bmj.39489.470347.AD

Kangovi, S., Mitra, N., Grande, D., Huo, H., Smith, R. A., & Long, J. A. (2017). Community health worker support for disadvantaged patients with multiple chronic diseases: A randomized clinical trial. American Journal of Public Health, 107(10), 1660-1667. https://doi.org/10.2105/AJPH.2017.303985

Kim, K., Choi, J. S., Choi, E., Nieman, C. L., Joo, J. H., Lin, F. R., Gitlin, L. N., & Han, H. (2016). Effects of community-based health worker interventions to improve chronic disease management and care among vulnerable populations: A systematic review. American Journal of Public Health, 106(4), e3-e28. https://doi.org/10.2105/AJPH.2015.302987