RES 720 Week 7 Interpreting Complex Results Example

Making Sense of Many Results: Interpreting a Set of Advanced Analyses Together

[Student Name]

University of Phoenix

RES/720: Statistical Research Methods and Design II

Week 7 Assignment

[Instructor Name]

[Date]

The learner, the company, its associates and all data are composites written for a model paper.

Over five weeks, the invented HR director at a nine-site Kentucky logistics company ran one-way and factorial ANOVA, multiple and logistic regression, chi-square and rank-based tests on the company's records. Each analysis answered one question. Leaders, however, need one account of what drives early turnover and what to do next. Here the results are read together.

The Findings in One Place

Shift and productivity: rotating associates trailed day associates by 14 units per hour (Week 2) and by 9.8 after controls (Week 4).

Onboarding and productivity: the new program added 3 units on days, 8 on nights and 11 on rotating shifts, an interaction (Week 3).

Quitting: rotating shifts, longer commutes, lower pay and less experience raised the odds of leaving within 90 days; the program lowered them about 30 percent across shifts (Week 5).

Safety: rotating associates had more incidents per hour worked, and incidents clustered in the first month (Week 6).

Readiness: supervisors rated program associates higher at 30 days (Week 6).

Three patterns run through the list. The rotating schedule is associated with worse outcomes on every measure. Experience protects on every measure. And the onboarding program helps on every measure examined.

How Many Tests Is Too Many?

The course plan named twelve confirmatory tests in advance. Running twelve tests at alpha .05 makes at least one false positive fairly likely. Benjamini and Hochberg (1995) proposed controlling the false discovery rate, the expected share of significant findings that are false, by ranking p values and comparing each with a threshold that rises with its rank; the method keeps more power than corrections that control the chance of any false positive.

Applying the procedure at a false discovery rate of .05: eight tests had p < .001; the commute coefficient in Week 4, p = .002, ranked ninth against a threshold of .0375; the shift-by-program interaction, p = .021, ranked tenth against .042. Both survive. The night-shift odds ratio for quitting, p = .06, and age, p = .31, do not. Ten of twelve planned findings stand after correction.

A Result That Seems to Disagree

The program's benefit for productivity depended on shift, but its benefit for quitting did not. This is not a contradiction. Productivity depends on skill and support during the shift, which the buddy system supplies most where supervisors are scarce. Quitting depends on many things, including whether a new associate feels welcomed and whether the first week is manageable, which the program's reduced first-week pick rate addresses on every shift. Different outcomes can respond to different parts of the same program.

The program works through two doors: support on the floor, which matters most at night, and a gentler first week, which matters everywhere.

A Reversal Across Warehouses

Looking at all 3,600 hires together, associates at higher-paying warehouses quit more often. Within each warehouse, associates with higher pay quit less often, as Week 5 found. Bickel et al. (1975) described a famous case of this kind in graduate admissions at a large university, where an apparent bias against women in overall admissions disappeared within departments because women applied more often to departments with low admission rates. Here, the highest-paying warehouses are in the city's tightest labor market, near other employers competing for the same workers, so they pay more and still lose more people. Pay helps within a site, but site-level competition swamps it in the pooled data.

Planned and Post Hoc Findings

Kerr (1998) defined HARKing as presenting a hypothesis formed after seeing results as if it had been stated beforehand and argued that the practice misleads readers about the strength of evidence and hides how a theory was actually developed. Two ideas in this study arose from the data: that the program's buddy substitutes for missing supervision at night, and that first-month safety coaching would reduce incidents. The learner reports both as hypotheses generated by the analysis, to be tested with new data, rather than as findings.

Practical Significance

Effect sizes convert into costs. The company's $4.1 million annual turnover cost works out to about $6,700 per early quit. The rotating shift's higher quit probability, about 12 percentage points for a typical associate, implies roughly 65 extra early quits a year among the 540 annual rotating hires, about $440,000. The program's drop in the quit rate from 37 to 29 percent implies about 145 fewer early quits a year across 1,800 hires, worth about $970,000, against a program cost of about $180,000. The productivity gap and incident rates add to the case. By contrast, the commute effect, though statistically solid, translates into smaller savings unless transportation support is cheap.

Do the Effects Add Up?

A natural question is whether the program could offset the rotating schedule's costs. For a typical rotating associate, the program's odds ratio of 0.70 would bring the quit probability from 39 percent to about 31 percent, close to but still above the 27 percent of a day associate under the old onboarding. The program narrows the gap without closing it, which is why the learner also recommends piloting schedule changes rather than relying on onboarding alone.

Checking the Story Against Alternatives

One alternative explanation is that weaker candidates are steered to rotating shifts because those openings are hardest to fill. If so, rotating associates would look worse on every outcome for reasons unrelated to the schedule. Controls for experience and pay reduce but cannot remove this concern. The learner checked hiring records and found that rotating hires had slightly lower assessment scores at hiring, a difference that explains a small share of the productivity gap when added to the Week 4 model.

How Certain Is Each Conclusion?

Firm: experience protects, rotating schedules carry costs on several outcomes and the onboarding program reduces early quitting, the last from a randomized rollout. Moderately firm: the program helps productivity most on night and rotating shifts, given the interaction's small effect and p value near the threshold. Tentative: the pay findings, because pay is entangled with site and labor market. Untested: the supervision mechanism and the safety coaching idea.

What Interpretation Cannot Fix

All findings except the program's come from observational records, and unmeasured factors such as family obligations may explain part of the rotating shift's costs. Results come from one company in one city. And outcomes stop at 90 days; longer-term retention is unknown.

Communicating to Leaders

For the executive team, the learner will present four conclusions with their certainty, the cost estimates and three recommended actions: extend the program to all sites, prioritize it for night and rotating shifts and pilot changes to the rotating schedule. Technical detail will sit in an appendix.

Conclusion

Interpreted together, the analyses tell a consistent story: a demanding rotating schedule and inexperience drive early losses, and a structured onboarding program reduces them. Correcting for multiple tests, reconciling an apparent contradiction, explaining a pay reversal and labeling post hoc ideas honestly make that story trustworthy. Week 8 turns it into a quantitative analysis plan for the dissertation.

References

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289-300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x

Bickel, P. J., Hammel, E. A., & O'Connell, J. W. (1975). Sex bias in graduate admissions: Data from Berkeley. Science, 187(4175), 398-404. https://doi.org/10.1126/science.187.4175.398

Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196-217. https://doi.org/10.1207/s15327957pspr0203_4