IOP 480 Week 5 Interpreting and Using Assessment Results Example

Scores, Gut Feelings and Missing Voices: Using a Year of Assessment Results at a Resort Company

[Student Name]

University of Phoenix

IOP/480: Assessment Tools for Organizations

Week 5 Assignment

[Instructor Name]

[Date]

The resort company, staff, data and decisions are composites written for a model paper; research findings come from the sources listed.

Collecting assessment data is only the start. Organizations must interpret results honestly, combine information wisely and act on findings. This final paper reviews a year of assessment data at Saguaro Springs Resorts, drawing together the tools built in earlier weeks.

What the Data Show

A year after launching its front desk battery, the company has data on 410 hires. Battery scores correlate .31 with supervisor ratings on a structured form and predict ninety-day retention: hires in the top third of scores stayed at a rate of eighty-four percent, those in the bottom third at sixty-one percent. Group differences in pass rates fall within federal guidelines. However, hiring managers overrode the battery's ranking for seventy-eight hires, nineteen percent, usually choosing a lower-scoring applicant who "seemed like a great fit." Those override hires stayed at a rate of fifty-eight percent and received lower supervisor ratings on average.

The engagement and climate survey reached a seventy-two percent response rate overall. Among night-shift housekeeping staff, the rate was forty percent.

Interpreting the Validity Evidence

A correlation of .31 is meaningful for hiring: it means battery scores carry real information about future performance, though they leave much unexplained. Because the company hired only applicants who passed, the correlation is likely an underestimate; applicants with the lowest scores were never observed on the job. The retention difference, twenty-three percentage points between top and bottom thirds, is easy for managers to grasp and worth highlighting.

Statistics Versus Intuition

Dawes et al. (1989) reviewed research comparing clinical judgment, in which experts combine information in their heads, with actuarial judgment, in which information is combined by fixed rules or statistical formulas. Over many studies in medicine, psychology and hiring, a fixed formula predicted as well as or better than expert judgment, even when the experts held extra facts. Experts struggled to weight information consistently and were influenced by vivid but irrelevant details.

The override data fit this pattern. Managers' sense of "fit" led them to choose applicants the battery ranked lower, and those hires performed worse and left sooner.

Why Managers Trust Their Gut

Highhouse (2008) examined why employers continue to prefer unstructured interviews and intuition despite evidence favoring structured methods. Highhouse argued that people believe human behavior is too complex to predict with formulas, overestimate their own ability to read others, value the sense of control that intuition provides and see standardized tools as missing the whole person. These beliefs persist even when people know the research.

Recognizing these beliefs helps Jonah respond respectfully. Managers' concern about fit is legitimate; the problem is that unstructured impressions measure it poorly.

Managers chose seventy-eight applicants who felt right; the retention data suggest that feeling was wrong more often than the scores were.

Who Did Not Answer the Survey

Rogelberg and Stanton (2007) discussed survey nonresponse in organizations and noted that low response rates matter mainly when nonrespondents differ from respondents in ways related to the survey's topics. They recommended comparing respondents and nonrespondents on known characteristics, following up with groups that responded less, examining reasons for nonresponse and interpreting results with appropriate caution rather than either ignoring or overreacting to low rates.

Night housekeeping staff may differ systematically from others: they work alone more, see managers less and may have different concerns about safety and recognition. Their low response rate means the survey may understate problems in that group.

What the Other Tools Showed

The assessment center from Week 3 promoted six managers; after nine months, five are rated effective by their general managers and their departments' turnover has fallen, while one struggled with budgets, an area the in-basket exercise had flagged as her weakest. The 360 program reached forty managers; those who met with a coach and set goals improved their staff ratings on recognition more than those who did not, consistent with research on when feedback leads to change. These results are encouraging but based on small numbers, so Jonah presents them as early signals rather than proof.

Presenting Uncertainty Honestly

Decision makers often want certainty that data cannot provide. Jonah's reports state each finding with its limits: the battery's retention difference is based on 410 hires, the assessment center on six, and night-shift survey results on a minority of staff. Clear statements of how confident the company can be help leaders avoid both ignoring the data and overreacting to it.

Recommendations

Limit overrides. Managers may choose among the top-scoring third of applicants, preserving their judgment within a strong pool. Any choice outside that range requires a written job-related reason reviewed by HR, and override outcomes will be tracked.

Measure fit better. Add two structured interview questions about teamwork and guest service at the specific resort, so fit is assessed systematically.

Reach missing voices. Hold short in-person sessions with night housekeeping staff, led by a bilingual facilitator during paid time, and compare their concerns with day-shift results.

Communicate clearly. Give each department a one-page report with three key findings, one strength and agreed actions; give executives a summary linking results to retention, guest scores and injuries.

Keep checking. Recompute validity and group differences annually, since jobs and applicant pools change.

Ethical Use of Data

Results must be used for the purposes employees were told about. Survey data remain confidential at the individual level, and assessment scores are stored securely and used only for hiring decisions and validation.

Reflection on the Course

The course moved from judging whether a tool is any good, to selecting, assessing leaders and measuring climate and finally to using results. The common thread is evidence: tools must be validated, results interpreted with care and decisions made by rules that are fair and transparent.

Conclusion

A year of data shows that Saguaro Springs' hiring battery predicts performance and retention and that managers' overrides weaken it, a pattern research on statistical versus intuitive judgment predicts. Low survey response among night staff calls for targeted follow-up rather than confident conclusions. Limiting overrides, measuring fit systematically, reaching missing voices and communicating plainly turn assessment results into better decisions.

References

Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243(4899), 1668-1674. https://doi.org/10.1126/science.2648573

Highhouse, S. (2008). Stubborn reliance on intuition and subjectivity in employee selection. Industrial and Organizational Psychology, 1(3), 333-342. https://doi.org/10.1111/j.1754-9434.2008.00058.x

Rogelberg, S. G., & Stanton, J. M. (2007). Understanding and dealing with organizational survey nonresponse. Organizational Research Methods, 10(2), 195-209. https://doi.org/10.1177/1094428106294693