DNP 752 Week 2 Data Analysis Example

From Weekly Counts to a Confidence Interval: Cleaning, Checking and Analyzing the Process, Distress and Staff Data From the Diabetes Distress Pilot

[Student Name]

University of Phoenix

DNP/752: DNP Applied Project III

Week 2 Assignment: Data Analysis

[Instructor Name]

[Date]

The health center, staff and figures are composites written for a model paper.

Data Preparation

Four data sets were analyzed: the 12 weekly process reports, the full-scale scores, the care manager encounter records and the staff surveys. Weekly reports were merged by study number into one file of 231 eligible pilot visits by 219 patients and 214 baseline visits. Eleven visits appeared twice because a patient had been rescheduled on the same day; the second record was removed in each case. Every item on both distress instruments was checked for values outside 1 to 6; none were found. Of 57 patients with a first full-scale score, 44 had a repeat at 12 weeks and 13 did not. No values were imputed, as the proposal specified.

Screening Rate on the Run Chart

Because no distress screening existed before the pilot, the baseline rate was zero in every week, and the center line for the pilot run chart was set at the median of the first four pilot weeks, 67%. Perla et al. (2011) describe four rules for reading a run chart, and two signals appeared. From pilot week 6 through week 12, seven consecutive weekly points fell above the median, meeting the rule for a shift. Weeks 5 through 9 showed five consecutive increases, meeting the rule for a trend. Both signals began after the cycle two changes, the shortened introduction and the earlier position of the screen in rooming, and were sustained after cycle three added phone completion. Across all twelve weeks the screen was completed at 81.4% of eligible visits (188 of 231), and in the six weeks of cycle three the rate was 93%, above the proposal's target of 80% for four consecutive weeks.

Timely Follow-Up on the Run Chart

Of 61 positive screens, 52 (85.2%) had a care manager call or visit on record no later than day 14. The weekly chart, plotted by week of screen, showed a shift of six points above its median of 78% beginning in week 5, the week care managers moved to next-day telephone contact. Median time to first contact fell from nine days in weeks 1 to 4 to three days in weeks 5 to 12.

Distress Scores: Assumptions

For the 44 patients with both scores, paired differences in the full-scale mean item score were plotted in a histogram and tested for normality. The distribution was roughly symmetric, and the Shapiro-Wilk test did not indicate a departure from normality (W = 0.97, p = .31), so the paired t test planned in the proposal was used. A Wilcoxon signed-rank test was run as a check and gave the same conclusion.

Distress Scores: Paired Comparison

The mean item score fell from 3.02 (SD = 0.61) at the first assessment to 2.48 (SD = 0.66) at 12 weeks, a mean change of -0.54 points (95% CI [-0.71, -0.37]), t(43) = 6.40, p < .001. Using the standard deviation of the paired differences, 0.56, the effect size was Cohen's d = 0.96. Among the 44 patients, 20 scored in the high range of 3.0 or above at entry, following the cut points of Fisher et al. (2012); at 12 weeks, 9 did, a relative reduction of 55%.

Subscale Analysis

Patients were grouped by the subscale with the highest score at entry. Among the 19 patients whose distress was mainly regimen-related, that subscale fell from 3.31 to 2.52. Among the 14 whose distress was mainly emotional burden, that subscale fell from 3.46 to 2.94. Groups with physician-related (n = 6) and interpersonal (n = 5) distress were too small for meaningful summaries and are reported as counts only.

Subgroups

Screening rates were 82% for English-speaking and 76% for Spanish-speaking patients, a gap of six points, below the 10-point threshold the proposal set for a review of the Spanish workflow. Among patients using insulin, 44% screened positive, compared with 29% of those not using insulin. Mean distress change was similar for patients under and over 60.

Staff Implementation Measures

Fourteen of 15 staff returned surveys after cycle three. Mean scores on the five-point scales developed by Weiner et al. (2017) were 4.3 for acceptability, 4.4 for appropriateness and 3.9 for feasibility. Feasibility was lowest among medical assistants, whose written comments mentioned time pressure on busy afternoons.

Exploratory A1C

Thirty-eight patients had A1C values before their first positive screen and at least 12 weeks later. The median fell from 9.4% to 9.0%. No test was run, as planned, because the time frame and design cannot support a claim about A1C.

Patients With and Without Repeat Scores

The 13 patients without a repeat score had a slightly higher mean first score (3.21 compared with 3.02), were more often Spanish-speaking (4 of 13) and were younger on average. Their absence may make the observed improvement look somewhat larger than it would have been with complete follow-up.

Sensitivity Check

Because the 13 patients without repeat scores started slightly higher, a sensitivity check assumed that none of them improved at all, assigning each a change of zero. Averaged over all 57 patients with a first score, the mean change under that assumption is about -0.42 points, smaller than the completers' -0.54 but still a meaningful drop on a 1 to 6 scale. The conclusion that distress fell does not depend on who was lost, although its size does.

Decisions Made During Analysis

Two decisions were not spelled out in the proposal and are recorded here. First, the center line for the screening run chart could not come from baseline weeks, since the baseline rate was zero; the median of the first four pilot weeks was used instead, which is a common choice when a process is new. Second, three patients had a repeat full scale at 11 rather than 12 weeks because their next visit fell early; these were kept, since excluding them would have lost data without changing the meaning of the measure. Both decisions were made before the paired test was run and will be stated in the manuscript.

What the Analysis Cannot Show

The analysis describes change at one clinic over one season. Without a comparison group it cannot separate the pathway's effect from regression toward the mean, seasonal influences or other changes at the site. The run chart signals link process change to specific cycles in time, which is stronger evidence for the process results than for the distress results.

Summary

The analysis followed the approved plan. Process measures met their targets with clear run chart signals after specific changes, distress fell by a large amount among patients followed up, and staff rated the pathway acceptable and appropriate. The results will be presented with tables and figures next week.

References

Fisher, L., Hessler, D. M., Polonsky, W. H., & Mullan, J. (2012). When is diabetes distress clinically meaningful? Establishing cut points for the Diabetes Distress Scale. Diabetes Care, 35(2), 259-264. https://doi.org/10.2337/dc11-1572

Perla, R. J., Provost, L. P., & Murray, S. K. (2011). The run chart: A simple analytical tool for learning from variation in healthcare processes. BMJ Quality & Safety, 20(1), 46-51. https://doi.org/10.1136/bmjqs.2009.037895

Weiner, B. J., Lewis, C. C., Stanick, C., Powell, B. J., Dorsey, C. N., Clary, A. S., Boynton, M. H., & Halko, H. (2017). Psychometric assessment of three newly developed implementation outcome measures. Implementation Science, 12, Article 108. https://doi.org/10.1186/s13012-017-0635-3