RES 709 Week 5 Quantitative Research Designs Example

Measuring Onboarding and Retention: Choosing a Quantitative Design for a Study of Hybrid Newcomers

[Student Name]

University of Phoenix

RES/709: Research Conceptualization and Design

Week 5 Assignment

[Instructor Name]

[Date]

The learner, the firms, the planned study and all figures are composites written for a model paper.

Week 4 framed the partner's study as mixed methods: a quantitative test of how onboarding, adjustment and embeddedness relate to staying two years, and a qualitative look at how hybrid-era associates experience acceptance. This paper designs the quantitative strand, which addresses RQ1 on practices and retention, and explains each choice of design, sample, measure and test.

Hypotheses

H1: Onboarding practices are positively associated with two-year retention. H2: Newcomer adjustment mediates the association between onboarding practices and job embeddedness. H3: Job embeddedness mediates the association between adjustment and two-year retention. Null hypotheses state no association or no indirect effect.

Comparing Designs

Shadish et al. (2002) explain that experiments, with random assignment, provide the strongest basis for causal inference because they rule out many alternative explanations, while quasi-experiments and correlational designs require researchers to address threats to validity through design features and analysis. An experiment, randomly assigning new associates to different onboarding programs, would be ideal for causal claims but is not feasible: firms will not randomize onboarding, and the learner cannot control their practices. A quasi-experiment comparing firms that adopt a new program with those that do not would be possible only if firms changed practices during the study, which cannot be planned. A causal-comparative design would compare existing groups but would oversimplify practices into categories. A correlational design measuring practices, mediators and outcomes as they occur fits the questions and is feasible, with the limit that it cannot establish cause.

Population and Sampling

The population is associates hired into hybrid work in the past three years at accounting firms in Ohio with 50 to 500 employees, about 140 firms. The learner will recruit firms through the state society of CPAs, aiming for 15 to 20 participating firms. All associates hired within the window at participating firms will be invited, a census within firms, to reduce selection bias inside each firm. Firm participation itself is voluntary, so firms with strong onboarding may be more willing to join; the learner will compare participating and declining firms on size and region.

Sample Size

Faul et al. (2007) developed G*Power, software that calculates required sample sizes for common tests. For logistic regression predicting retention with five predictors, assuming a retention rate of 70 percent, an odds ratio of 1.6 for the main predictor, alpha of .05 and power of .80, the estimate is about 260 associates. Allowing for a 60 percent response rate within firms, the learner will need about 430 eligible associates, achievable with about 18 firms.

Variables and Measures

Onboarding practices: an inventory listing practices such as structured training weeks, assigned mentor, in-person cohort days, partner check-ins and buddy assignments, completed by firms and confirmed by associates.

Newcomer adjustment: established scales for role clarity, self-efficacy and social acceptance.

Job embeddedness: a validated short measure of links, fit and sacrifice.

Two-year retention: from firm records, coded retained or departed, avoiding reliance on intentions.

Controls: average weekly hours in busy season, starting salary relative to market, firm size and percentage of days in office.

Measuring actual departures from firm records answers the weakness the Week 3 critique found in studies that relied on intentions.

Reliability and Validity of the Measures

Each scale will be checked in the study sample. Internal consistency will be reported for multi-item scales, with values of .70 or higher treated as acceptable. Because the onboarding inventory is new, it will be reviewed by three human resources leaders and two former associates for content validity and pilot tested with 20 associates at a firm outside the study, with items revised where respondents found them unclear. Confirmatory factor analysis will test whether adjustment and embeddedness items load as expected, providing evidence of construct validity before hypotheses are tested.

Ethical Protections

Associates will consent to linking their survey responses with retention records, and firms will never see individual responses. The learner will not recruit associates at her own firm, removing pressure on people who report to her. Data will be stored on secure university servers with codes rather than names, and findings will appear only as group totals, never broken out by firm.

Data Collection

Associates will complete an online survey during their first year, and firms will provide retention status 24 months after each associate's start date, linked by a confidential code. Because the study spans two years from first survey to final retention for the latest cohort, the learner will also use earlier cohorts with retrospective survey items, accepting some recall bias, to complete the dissertation on time.

Analysis Plan

Hayes (2018) describes regression-based approaches to mediation using bootstrapped confidence intervals for indirect effects, which do not assume normal sampling distributions. For H1, a logistic regression will predict retained versus departed from the onboarding measures plus the controls. H2 and H3 will both be tested with a serial mediation model, onboarding to adjustment to embeddedness to retention, with bootstrapped indirect effects. Because associates are nested within their employing firms, firm-level clustering will be addressed with cluster-robust standard errors.

Handling Missing Data

Some associates will skip items or leave before completing all surveys. Missing data will be examined for patterns; if data are missing at random, multiple imputation will be used rather than dropping cases, which could bias results toward associates who stayed and were more engaged. The analysis will report how much data were missing and compare results with and without imputation.

Preparing the Data

Before testing hypotheses, the learner will screen for outliers, check the distribution of continuous predictors and test for multicollinearity among onboarding practices, which may be correlated, since firms that offer mentors often also offer structured training. If two practices are very highly correlated, they will be combined into an index to keep estimates stable.

Threats to Validity

Self-selection: associates who chose firms with strong onboarding may differ; firm-level controls and census sampling help. Common method bias: adjustment and embeddedness are self-reported at the same time; separating survey timing and using records for retention reduce the risk. Retrospective recall for earlier cohorts may be inaccurate; analyses will compare cohorts. External validity is limited to mid-size Ohio firms; findings may differ in large national firms with formal programs or in very small practices where partners onboard every associate personally.

Reach and Limits of the Design

This design will reveal whether onboarding practices are associated with retention and whether that association runs through adjustment and embeddedness. It cannot prove that changing onboarding would change retention. The qualitative strand in Week 6 will help explain the mechanisms behind any relationships found, and the integration in Week 7 will test whether the two strands agree.

Conclusion

A correlational design with validated measures, records-based retention, power-based sample size and mediation analysis fits RQ1 and is feasible for a practitioner researcher. Its main limit, the inability to establish cause, is acknowledged and partly offset by the mixed methods design.

References

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175-191. https://doi.org/10.3758/BF03193146

Hayes, A. F. (2018). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach (2nd ed.). Guilford Press.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.