PSYCH 664 Week 2 Research Designs and Validity Example

Random Assignment in a Retirement Community: Choosing a Design for the Phone-Call Study and Guarding Against Threats to Validity

[Student Name]

University of Phoenix

PSYCH/664: Research Methods and Statistics in Psychology

Week 2 Assignment

[Instructor Name]

[Date]

The senior community, its residents and the study design are composites written for a model paper; methodological sources and research findings come from the sources listed.

The research question from Week 1 asks whether weekly phone calls from college student volunteers reduce loneliness among residents of our senior community. A design determines whether the answer can be trusted. This paper compares designs for the question and plans safeguards for the threats that remain.

Validity and Its Threats

Campbell (1957) described factors that threaten the validity of experiments in social settings. Internal validity asks whether the experimental treatment actually made a difference in the specific instance; external validity asks to what populations, settings and variables the effect can be generalized. Campbell identified threats to internal validity, including history, events occurring between measurements; maturation, changes within participants over time; testing, effects of taking a pretest; instrumentation, changes in measurement; regression, movement of extreme scores toward the mean; selection, differences between groups; and experimental mortality, loss of participants. He also noted that pretesting could interact with the treatment, limiting generalization.

Shadish et al. (2002) expanded this framework in their standard text, adding construct validity, whether the study measures and manipulates what it claims, and statistical conclusion validity, whether the analysis supports the inference, and described designs and design features that address each threat.

Design One: One Group, Before and After

The simplest design would offer calls to every lonely resident and measure loneliness before and after twelve weeks. It is easy and ethical, since everyone receives the program. But many threats remain. History: a new activities director, a holiday season or a heat wave could change loneliness. Maturation: some residents may grow lonelier with declining health. Regression to the mean is especially serious: residents selected because they scored high on loneliness will tend to score lower on retest even without any intervention. A decrease could not be attributed to the calls.

Design Two: Comparing Two Buildings

A second design would offer calls to lonely residents in the assisted living building and compare them with lonely residents in the independent living building. This adds a comparison group, which helps with history and maturation, but introduces selection: residents of the two buildings differ in health, age and daily routines. Differences in outcome could reflect those differences rather than the calls.

Design Three: A Randomized Waitlist Trial

The third design would randomly assign eligible residents to receive calls immediately or after a twelve-week wait, measuring loneliness in both groups at baseline, twelve weeks and twenty-four weeks. Random assignment makes the groups equivalent on average in known and unknown characteristics, addressing selection, and the comparison group experiences the same history, maturation and regression. Because waitlisted residents receive calls after twelve weeks, no one is denied the program, which matters ethically for a low-risk intervention.

Kahlon et al. (2021) used a similar approach in their trial of empathy-focused telephone calls, randomly assigning participants to calls or to a control condition, measuring loneliness, depression and anxiety before and after and finding reduced loneliness in the call group. Their design shows that randomization is feasible for such programs.

Randomizing who gets calls first turns twelve weeks of waiting into the comparison that makes the study believable.

Why Not Randomize Buildings Instead?

Another option would assign whole buildings or floors to calls or waitlist, a cluster design, which would reduce contamination because neighbors would share the same condition. With only three residential buildings, however, a cluster design would have too few units to make the groups comparable, and building differences would again resemble selection. Individual randomization with safeguards against contamination is the better compromise for a community of this size.

Construct Validity of the Intervention

Construct validity asks whether the study delivers and measures what it claims. If some student callers chat briefly and others listen with care, the "intervention" is really several different interventions. To keep it consistent, callers will complete a two-hour training in empathetic listening, follow a short guide with suggested topics and log each call's length. A coordinator will listen, with residents' permission, to a sample of calls to check that the approach is delivered as intended. These steps make it possible to say what, exactly, produced any effect.

Threats That Remain

Randomization does not solve everything. Contamination is a risk in a community where residents eat together and talk: waitlisted residents might hear about the calls and feel left out, which could raise their loneliness and inflate the apparent effect, or ask friends to share calls. Attrition is another risk: some residents may move, become ill or die, and losses may differ between groups. Assessor expectations could also shape results if staff who administer questionnaires know who received calls.

Safeguards

To reduce contamination, I will explain the waitlist openly to all participants, emphasizing that everyone receives calls within three months, and avoid publicizing who is in which group. To track attrition, I will record reasons for dropout and compare dropouts by group. Questionnaires will be administered by a staff member who does not know group assignments, and residents will be asked not to mention the calls during assessments. Analysis will include all randomized residents, following an intention-to-treat approach.

Statistical Conclusion Validity

A design can be sound and still fail to detect a real effect if the sample is too small or the measure too noisy. This kind of validity asks whether the numbers, as analyzed, can bear the weight of the conclusion. With roughly sixty eligible residents expected to enroll, the study may detect only moderate effects. Planning the sample size and the analysis in advance, which later weeks will address, protects against both missing a real effect and overinterpreting a chance one.

Ethics of the Design

The waitlist design also answers an ethical question. Withholding a beneficial program from a control group indefinitely would be hard to justify for lonely older adults, but delaying a low-risk program by twelve weeks, with usual activities continuing, is a reasonable trade for the knowledge gained. Residents in the waitlist group can still join every community activity, and anyone whose loneliness or mood worsens sharply will be referred to the community's social worker regardless of group.

External Validity

Our community is relatively affluent and mostly white, and residents chose to live in senior housing with on-site activities. Results may not generalize to isolated older adults living alone in their own homes or to more diverse communities. I will describe the sample carefully so readers can judge applicability.

Reporting

Schulz et al. (2010) presented the CONSORT 2010 statement, a checklist and flow diagram for reporting randomized trials, covering items such as how participants were recruited and randomized, how allocation was concealed, who was blinded, how many participants were assigned, received the intervention and were analyzed and what harms occurred. Planning to report according to CONSORT from the start shapes procedures, such as recording reasons for exclusion and dropout.

Conclusion

A one-group design is simple but cannot separate the calls from regression and history; comparing buildings introduces selection. A randomized waitlist trial controls most threats, treats all residents fairly and, with safeguards against contamination and attrition and blinded assessment, offers the best chance of a trustworthy answer.

References

Campbell, D. T. (1957). Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 54(4), 297-312. https://doi.org/10.1037/h0040950

Kahlon, M. K., Aksan, N., Aubrey, R., Clark, N., Cowley-Morillo, M., Jacobs, E. A., Mundhenk, R., Sebastian, K. R., & Tomlinson, S. (2021). Effect of layperson-delivered, empathy-focused program of telephone calls on loneliness, depression, and anxiety among adults during the COVID-19 pandemic: A randomized clinical trial. JAMA Psychiatry, 78(6), 616-622. https://doi.org/10.1001/jamapsychiatry.2021.0113

Schulz, K. F., Altman, D. G., & Moher, D. (2010). CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. BMJ, 340, Article c332. https://doi.org/10.1136/bmj.c332

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.