| Course | DOC 715 Doctoral Seminar I (DOC/715) |
|---|---|
| Week | 5 |
| Paper type | Doctoral research design defense |
| Length | about 1,155 words, 4 double-spaced pages plus title page and references |
| Format | APA 7 student paper |
| School | University of Phoenix |
| Program | DBA |
| Updated | October 2026 |
Free sample paper for DOC 715 Week 5
Building a Fair Comparison: Defending a Propensity Score Design for an Onboarding Study
[Student Name]
University of Phoenix
DOC/715: Doctoral Seminar I
Week 5 Assignment
[Instructor Name]
[Date]
The learner, the company, its customers and all figures are composites written for a model paper.
Week 4 made the case for a quantitative methodology over qualitative and mixed alternatives. Within that methodology, the design determines how fairly onboarded and non-onboarded customers can be compared with each other. This paper defends the chosen design against the alternatives and explains how it will be carried out.
The Central Threat
Between 2023 and 2025, about 3,900 of 12,000 new small service business customers received proactive onboarding from a customer success manager; the rest received automated onboarding. Assignment was not random. Customers who signed up through a sales representative, chose higher plan tiers or were in regions with more managers were more likely to be onboarded. These same characteristics may also relate to retention. If onboarded customers stay longer, the difference could reflect who they were, not what they received. Selection bias is therefore the central threat the design must address.
Designs Considered
A randomized experiment would remove selection bias by design. The learner proposed randomizing onboarding for new customers for six months; leadership declined the proposal, because sales representatives promise onboarding to some customers and the company would not withhold it from those who ask.
A correlational regression would estimate the association while adjusting for covariates. It is simple, but it relies on the model's form being correct and does not show whether onboarded and non-onboarded customers overlap on their characteristics; if some onboarded customers have no comparable counterparts, the model extrapolates.
A difference-in-differences design would use a change in onboarding over time in some regions but not others. In early 2024 the company added customer success managers in its Southwest region, sharply increasing onboarding there, while other regions stayed the same. Comparing retention before and after in the Southwest with other regions could estimate the effect, if trends would otherwise have been parallel.
Propensity score matching would pair each onboarded customer with a similar automated-only customer, based on the probability of onboarding given observed characteristics, and compare outcomes in matched pairs.
Why Propensity Score Matching
Rosenbaum and Rubin (1983) showed that the propensity score, the probability of treatment given observed covariates, is a balancing score: among units with the same score, treatment assignment is independent of those covariates, so comparing treated and untreated units with similar scores removes bias from the observed covariates, provided there are no unmeasured confounders.
Austin (2011) introduced propensity score methods for applied researchers, describing matching, stratification, weighting and covariate adjustment, recommending matching on the logit of the propensity score with a caliper of 0.2 of its standard deviation and stressing that balance must be assessed after matching using standardized differences rather than significance tests.
Stuart (2010) reviewed matching methods and emphasized that matching is a design step, separating the construction of comparable groups from the analysis of outcomes, so that the comparison can be checked before outcomes are examined.
That last point is decisive. Matching lets the learner, and her committee, see whether comparable groups exist and how close they are before any retention data are analyzed, which protects against choosing models that produce a desired result.
Matching shows the committee the comparison before anyone sees the outcome, which is the strongest guard against wishful analysis.
Procedures
Estimate each customer's propensity score with logistic regression, using plan tier, sales channel, region, industry, business size, signup month and prior use of scheduling software.
Match each onboarded customer to one automated-only customer within a caliper of 0.2 standard deviations of the logit of the score, without replacement.
Check balance: standardized mean differences below 0.1 for every covariate, and similar distributions in plots.
Analyze outcomes in the matched sample: logistic regression for retention, count regression for feature adoption and an interaction with size for RQ3, with standard errors clustered by customer success manager.
The estimand is the average effect of onboarding among customers who received it, the question the company needs to answer about its current program.
Sensitivity to Unmeasured Confounding
Matching addresses only measured characteristics, a limit worth stating plainly. An owner's motivation, unmeasured, could lead both to requesting onboarding and to staying. A sensitivity analysis will estimate the size of hidden bias needed to erase the result, so readers can judge the finding's robustness.
A Difference-in-Differences Check
The Southwest expansion supports a second analysis with different assumptions. Difference-in-differences does not require that all confounders be measured, but it requires parallel trends, which can be partly checked using quarters before 2024. If both designs point the same way, the conclusion is stronger; if they diverge, the difference is itself informative.
Overlap and Who Is Left Out
Matching works only where onboarded and automated-only customers overlap. Customers on the highest plan tier who signed through sales representatives were almost always onboarded, so few comparable automated-only customers exist for them. The learner will examine the distribution of propensity scores in both groups before matching and report how many onboarded customers could not be matched. If a substantial group is unmatched, the findings will be described as applying to the customers for whom a fair comparison exists, and that group will be described clearly.
Why Not Weighting Instead?
Inverse probability weighting uses all customers rather than discarding unmatched ones, which can increase precision. But extreme weights for customers with very low or very high scores can make estimates unstable. The learner will run a weighted analysis as a secondary check and report whether it agrees with matching; matching remains primary because its comparison is easier for the committee and company leaders to inspect.
Validity
Internal validity: addressed by matching, balance checks, sensitivity analysis and the second design. External validity: results describe small service businesses at one company; transfer to other firms depends on similarity in product, price and customers. Construct validity: onboarding counts only when a manager met the customer during the first month, a clear but deliberately narrow definition.
Timing and Measurement
Covariates are measured at signup, before onboarding begins, so they cannot be affected by the program. Outcomes are measured afterward: feature adoption at ninety days and retention at twelve months. This order matters, since matching on anything measured after onboarding started, such as early logins, could remove part of the program's effect. The design specifies which fields qualify as pre-treatment and excludes all others from the propensity model.
What the Design Can Support
If matched onboarded customers retain at higher rates, balance is good and sensitivity analysis shows the result would require a strong unmeasured confounder to erase, the study can conclude that proactive onboarding is associated with higher retention among comparable customers, with a reasonable case for a causal interpretation, stated with caveats.
Conclusion
Selection into onboarding is the central threat. With randomization unavailable, propensity score matching, grounded in foundational and applied methods literature, offers a transparent way to build comparable groups and check them before outcomes are analyzed. Sensitivity analysis and a difference-in-differences check address what matching cannot. Week 6 plans sampling and access.
References
Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46(3), 399-424. https://doi.org/10.1080/00273171.2011.568786
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41-55. https://doi.org/10.1093/biomet/70.1.41
Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25(1), 1-21. https://doi.org/10.1214/09-STS313
What the DOC 715 Week 5 instructions ask
The fifth DOC 715 paper asks doctoral learners to defend their research design. Learners usually compare the designs available within their methodology, explain the threats to internal and external validity each faces, justify the chosen design with methodological sources, describe its main procedures in enough detail to evaluate and state what conclusions it can support. Some versions ask learners to diagram the design or to defend it against a specific alternative. Apply the comparison to the learner's own study and data, cite design literature in APA, address how the design handles the most serious alternative explanations and explain any secondary analyses that test the robustness of the main results.
How this DOC 715 Week 5 example is built
Our worked paper explains that customers were not randomly assigned to onboarding: assignment depended on manager availability, region and sales channel. A randomized trial was rejected because the company would not withhold onboarding on request. Simple regression would adjust for covariates but cannot show whether the groups overlap. Following the foundational paper on propensity scores, a widely cited introduction to propensity score methods and a review of matching methods, the design matches onboarded customers to similar automated-only customers, checks balance with standardized differences and tests sensitivity to unmeasured confounding. A regional expansion of the customer success team in 2024 supports a difference-in-differences check that rests on different assumptions, so agreement between the two would strengthen the conclusion.
DOC 715 Week 5 grading rubric: where the points go
Doctoral graders reward design defenses that are specific about threats and procedures. Strong papers compare realistic designs, explain why the chosen one best handles the study's main threats and describe key procedures, such as how groups are formed and compared, in enough detail to evaluate. Credit goes to acknowledging assumptions the design cannot test, to sensitivity and robustness checks and to clear statements of what the results can mean. Graders also value diagrams or step lists, a stated estimand and a plan for reporting balance so readers can judge the comparison themselves. Design literature, cited in APA, anchors the argument.
DOC 715 Week 5 help: mistakes to avoid
Design defenses often name a design without explaining how it handles the study's biggest threat. Name the threat first, then show step by step how the design addresses it. Another frequent gap is presenting propensity score matching as if it removed all bias; it addresses only measured differences. Learners also skip balance checks, so readers cannot tell whether matching actually produced comparable groups. Some papers add robustness analyses without explaining which different assumptions they test. Finally, be clear and explicit about the estimand, such as the effect among those who received the program. Report what the matched sample looks like before reporting outcomes. A tutor can help you plan balance checks and sensitivity analysis.
Related DOC 715 sample papers
Other DOC 715 week samples
- DOC 715 Week 1: Research Problem
- DOC 715 Week 2: Purpose and Research Questions
- DOC 715 Week 3: Alignment
- DOC 715 Week 4: Defending the Methodology
- DOC 715 Week 6: Sampling and Access
- DOC 715 Week 7: Feasibility and Scope
- DOC 715 Week 8: Phase 2 Foundation
More DBA sample papers
- BUS 721 Week 5: Process Improvement
- BUS 733 Week 5: Sustainability in Transformation
- FIN 711 Week 5: Angel and Venture Capital Investment
- LDR 711A Week 5: Charismatic Leadership Critique
DOC 715 Week 5 questions, answered
What does DOC 715 Week 5 usually cover?
It usually covers defending the research design: comparing designs, addressing validity threats, explaining procedures and stating what conclusions the design supports.
Where can I find a free DOC 715 Week 5 sample paper?
The DOC 715 Week 5 design defense using propensity score matching is above, complete and free.
What is a propensity score?
The probability that a unit receives a treatment given its observed characteristics, used to match or weight treated and untreated units so they are comparable.
What is covariate balance?
Similarity between treated and comparison groups on measured characteristics after matching or weighting, often checked with standardized mean differences.
What is difference-in-differences?
A design that compares changes over time in a group that received a program with changes in a group that did not, assuming the groups would have followed parallel trends.
Write yours, or have the desk draft it
This paper is an original model document written by our desk, not a submitted student paper and not an official University of Phoenix document. Read it for the moves, then write your own to the instructions in your classroom. If you want one built to your exact prompt and rubric, the first custom sample is free and arrives in 24 to 48 hours.
Request this one custom, free · All DOC 715 week samples · All courses