RES 710 Week 3 Sampling and Probability Example

Who Answered and What It Means: Sampling and Probability in a Credit Union Survey

[Student Name]

University of Phoenix

RES/710: Statistical Research Methods and Design I

Week 3 Assignment

[Instructor Name]

[Date]

The learner, the credit union, its members and all data are composites written for a model paper.

Week 2 described 1,240 survey responses from members of the invented western Michigan credit union, which serves people through branches, a phone center and an app. Before the operations director can say anything about members in general, she must explain how those 1,240 people came to be in the data and what probability theory allows her to infer from them. This paper works through the population, the sampling design, the probability ideas behind inference and the risk that people who answered differ from people who did not.

Population, Frame and Sample

The target population is every member who completed a branch, phone or app transaction during the six-week study window, about 61,500 people out of 180,000 members. The sampling frame is the transaction log, which lists each transacting member with branch, channel, date and time. The frame covers the population well, since every transaction is logged, but it excludes members who did not transact in those weeks, so results describe active members, not the full membership.

From the frame, 4,100 members were drawn and invited; 1,240 completed the survey. Each level is smaller than the last, and each step can introduce error: coverage error from frame to population, sampling error from sample to frame and nonresponse error from respondents to sample.

Probability and Nonprobability Sampling

In probability sampling, every unit in the frame has a known, nonzero chance of selection, which allows sampling error to be estimated. Simple random sampling gives every unit an equal chance; stratified sampling divides the frame into groups and samples within each; cluster sampling selects groups, such as branches, and then members within them. Lohr (2019) explains that stratification can improve precision and guarantees that small groups appear in the sample, while clustering usually reduces cost but increases variance.

Nonprobability samples, such as members who answer a survey link posted in the app, are cheaper but give no basis for estimating sampling error, and their results may reflect whoever chose to respond.

The Stratified Design

The learner stratified the frame by branch and channel, giving 26 strata: one for each branch's in-person transactions, one for phone and one for app. Within each stratum, members were selected at random. Small branches were sampled at higher rates so that each had at least 100 invitations, which allows branch comparisons. Because selection rates differ across strata, members in large strata represent more of the population, which later weighting must reflect.

Contact followed the tailored design method's advice to make responding easy and to send several personalized contacts (Dillman et al., 2014): an email invitation from the operations director, a reminder three days later and a final text message after a week, each with a short, mobile-friendly link.

Basic Probability with the Survey Data

Among respondents with a recorded wait, 12 percent waited more than 15 minutes. That is a marginal probability: P(long wait) = 0.12. Restricting attention to Friday transactions between 11:30 a.m. and 1:30 p.m., 31 percent waited more than 15 minutes, so the conditional probability P(long wait | Friday lunch) = 0.31, about 2.6 times the overall rate. The two events are therefore not independent; if they were, the conditional probability would equal the marginal one.

For members with a recorded wait, P(phone) = 347 / 992, or 0.35. The probability of a branch transaction with a long wait combines both: the share of branch transactions multiplied by the conditional probability of a long wait at a branch, a step the learner shows explicitly in her appendix.

A long wait is not random bad luck: on a Friday at lunch, it is almost a one-in-three event.

The Normal Distribution and Z Scores

Satisfaction is roughly symmetric, with a mean of 5.4 and a standard deviation of 1.2. A z score tells how many SDs a value sits above or below the mean: subtract the mean from the value, then divide by the SD. A member who scored 3.0 has z = (3.0 minus 5.4) / 1.2 = -2.0, placing that member among roughly the lowest 2 to 3 percent if the distribution were normal. In the actual data, 4.1 percent scored 3.0 or lower, slightly more than the normal model predicts, which matches the mild left skew Week 2 reported.

Sampling Distributions and Standard Error

A single sample mean is one draw from a distribution of means that different samples would produce. The standard error is the spread of that distribution of means; it is found by taking the SD of the data and dividing it by the square root of n. For satisfaction, SE = 1.2 / sqrt(1,240) = 0.034, so the 95 percent confidence interval is 5.4 plus or minus 1.96 times 0.034, or 5.33 to 5.47.

Wait time is strongly skewed, but the central limit theorem holds that means of large samples are approximately normally distributed whatever the shape of the data. With 992 waits and a standard deviation of 7.1 minutes, SE = 7.1 / sqrt(992) = 0.23, and the 95 percent interval for the mean wait is 7.9 plus or minus 0.44, or 7.46 to 8.34 minutes. A bootstrap check, resampling the waits 5,000 times, gave nearly the same interval, confirming that the approximation works here.

Margin of Error for a Proportion

Of the 1,222 members who answered the net promoter item, 41 percent scored 9 or 10. The standard error of a proportion is the square root of p(1 minus p) over the sample size = sqrt(0.41 x 0.59 / 1,222) = 0.014, so the margin of error at 95 percent is about plus or minus 2.8 percentage points. These calculations assume simple random sampling; the stratified design and weighting will change them slightly, and the final estimates will come from software built for stratified, weighted samples.

Nonresponse

About 30 percent of invited members responded. Groves (2006) reviewed studies that measured nonresponse bias directly and found that response rates alone were weak predictors of bias; what matters is whether the tendency to respond is related to the variables of interest. The learner therefore compared respondents with all invited members on variables in the frame. Respondents were older, with a median age of 46 against 41, and more often branch users, 52 percent against 47 percent. Satisfaction cannot be compared, since it is known only for respondents, but if older branch users are more or less satisfied than others, unadjusted results will be biased.

Weighting

To correct for unequal selection rates and the age and channel differences, the learner built weights in two steps: a base weight equal to the inverse of each stratum's selection rate, then an adjustment so the weighted sample matches the frame's distribution of age group and channel. Weighted mean satisfaction was 5.45, close to the unweighted 5.4, suggesting that nonresponse on these characteristics did little to distort the estimate. Weights cannot correct for differences on characteristics the frame does not record, such as general attitudes toward the credit union, a limit she will state.

What the Sample Can and Cannot Support

The sample supports estimates for active members across the credit union and comparisons among channels and branches. It does not support claims about inactive members, and branch-level estimates for small branches carry wide intervals. Because members were sampled within branches and branches share staff and layouts, later analyses should treat members as clustered.

Conclusion

A clear chain from population to frame to stratified sample to respondents, combined with probability rules, z scores, standard errors and the central limit theorem, lets the director estimate member satisfaction and wait times with known precision. Checking nonresponse against the frame and weighting the results make those estimates more credible. Week 4 will use these foundations to test hypotheses.

References

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Wiley.

Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646-675. https://doi.org/10.1093/poq/nfl033

Lohr, S. L. (2019). Sampling: Design and analysis (2nd ed.). CRC Press.