RES 720 Week 1 Advanced Design Choices Example

Choosing the Right Design and Procedure: Planning Advanced Analyses of Warehouse Retention

[Student Name]

University of Phoenix

RES/720: Statistical Research Methods and Design II

Week 1 Assignment

[Instructor Name]

[Date]

The learner, the company, its warehouses and all data are composites written for a model paper.

The learner is a composite HR director at a third-party logistics company in Louisville, Kentucky, operating nine warehouses with about 2,800 hourly associates. In the past year, 34 percent of new associates left within 90 days, and the company spent an estimated $4.1 million replacing them. Site managers disagree about the cause: some blame the rotating 4-on, 3-off shift pattern; others blame thin onboarding. The company is about to launch a structured onboarding program with a peer buddy and a reduced first-week pick rate. The director wants her DBA study to tell leaders what is driving early quits and whether the new program works.

Research Questions

RQ1: Does 90-day productivity, measured in units picked per hour, differ across the three shift patterns: fixed days, fixed nights and rotating?

RQ2: Do shift pattern and the new onboarding program jointly influence productivity, and does the program's effect depend on shift?

RQ3: Which factors, among pay rate, commute distance, prior warehouse experience, shift and onboarding, predict productivity when considered together?

RQ4: Which of those factors predict whether an associate quits within 90 days?

RQ5: Are safety incidents distributed differently across shifts?

Design Options

Shadish et al. (2002) describe how experiments, quasi-experiments and nonexperimental designs differ in their ability to support causal inference, and how threats to internal validity such as selection, history and maturation can mimic treatment effects when groups are not randomly formed. Three designs are possible here.

A randomized experiment would assign new associates at random to the new program or the current onboarding. It offers the strongest evidence for RQ2, but site managers resist running two onboarding processes in one building, and associates who talk to each other could contaminate the comparison.

A quasi-experiment would use the company's plan to roll out the program across warehouses over six months. Comparing cohorts before and after the rollout at each site, with sites that have not yet adopted it as comparisons, controls for some history and selection threats. This stepped approach is feasible and fairly strong.

A correlational design would analyze existing records of associates hired over the past two years. It can address RQ1, RQ3, RQ4 and RQ5 quickly but cannot isolate the program's effect, since the program is new.

The Chosen Design

The study combines two parts. Historical records of about 3,600 associates hired in the past two years supply data for RQ1, RQ3, RQ4 and RQ5. A stepped rollout of the onboarding program across the nine warehouses, three at a time at two-month intervals, supplies data for RQ2, with randomization of the order in which warehouses adopt. Randomizing the order is acceptable to leadership because every site eventually gets the program, and it strengthens causal inference.

Factors, Levels and Covariates

In RQ2, shift is a between-subjects factor with three levels, and onboarding is a between-subjects factor with two levels, giving a 3 x 2 factorial design. The interaction term tests whether the program works differently by shift; night-shift associates, for example, have fewer supervisors available and may benefit more from a buddy. Prior warehouse experience is a covariate, since experienced associates start faster regardless of onboarding. Warehouse is a clustering variable that later models must account for.

Validity Threats

Selection is the main threat in the historical data: associates choose shifts, so shift differences may reflect who chooses each shift. Controlling for pay, experience and commute helps but cannot rule out unmeasured differences, such as family obligations. In the rollout, history is a threat if a regional wage increase coincides with a step; staggering the steps helps separate such events from the program. Instrumentation is a lesser concern, since productivity comes from scanners that measure the same way across sites.

Matching Questions to Procedures

Tabachnick and Fidell (2019) organize statistical procedures by the type of question, the number and level of variables and the role of covariates, and they stress testing assumptions such as normality, homogeneity of variance and absence of multicollinearity before interpreting results. Following that logic:

RQ1, one continuous outcome and one factor with three levels: one-way ANOVA with post hoc comparisons (Week 2).

RQ2, one continuous outcome, two factors and a covariate: factorial ANOVA, extended to ANCOVA for experience (Week 3).

RQ3, one continuous outcome and several predictors of mixed types: multiple regression (Week 4).

RQ4, a binary outcome and several predictors: logistic regression (Week 5).

RQ5, two categorical variables: chi-square test of independence, with a nonparametric fallback for skewed outcomes (Week 6).

The question decides the test: two factors call for a factorial design, and a yes-or-no outcome calls for logistic regression.

Power Analysis

Faul et al. (2007) describe G*Power, a program that computes required sample sizes for many tests given alpha, power and an expected effect size. For RQ2's 3 x 2 interaction, with alpha .05, power .80 and an effect size f of 0.15, drawn from a published onboarding study's small-to-moderate effects, G*Power gives a total sample of about 432, or 72 per cell. The rollout will produce about 900 new hires over six months, enough even if some cells are uneven. For RQ4, the historical data contain about 1,220 early quitters among 3,600 hires, comfortably above common rules of thumb for the number of outcome events per predictor in logistic regression.

Why Not Simpler Tests

A director with limited statistics might compare shifts two at a time with t tests, look at onboarding and shift separately and correlate each predictor with productivity one by one. Each shortcut causes problems. Three t tests inflate the chance of a false positive; separate looks at shift and onboarding miss their interaction; and one-at-a-time correlations ignore that pay, shift and experience overlap. The advanced procedures exist precisely to handle several groups, several factors and correlated predictors at once.

Data Sources

Productivity comes from warehouse scanners; quits and pay from HR records; commute distance from home ZIP codes; shift and experience from hiring records; and incidents from the safety log. All are existing records, so the main ethical issue is confidentiality. The learner will receive de-identified files with codes replacing names and will not report results for any warehouse with fewer than 20 associates in a group.

Practical Constraints

Leaders want results within a year; the stepped rollout takes six months, and 90-day outcomes for the last cohort arrive three months later. Site managers must apply the program consistently; a fidelity checklist completed by each site's trainer will document that. And because the director is a senior HR leader, site managers may present onboarding favorably; checklists will be verified by short spot audits.

Conclusion

Starting from five precise questions, the plan combines historical records with a stepped, randomized rollout to balance feasibility and causal strength. Each question is matched to a procedure suited to its outcome and factors, with covariates, interactions and clustering identified and sample sizes justified. Week 2 will begin with one-way ANOVA across shift patterns.

References

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175-191. https://doi.org/10.3758/BF03193146

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.

Tabachnick, B. G., & Fidell, L. S. (2019). Using multivariate statistics (7th ed.). Pearson.