PSYCH 678 Week 3 Data Gathering and Diagnosis Example

What Twenty-Two Interviews, a Survey and Two Years of Ratings Say About an Engineering Firm's Performance System

[Student Name]

University of Phoenix

PSYCH/678: Consulting and Business Skills

Week 3 Assignment

[Instructor Name]

[Date]

The engineering firm, its employees, the survey and all figures are composites written for a model paper; research findings come from the sources listed.

Four weeks into a six-week engagement, the data are in. This paper describes how they were collected, what each source showed and the diagnosis I will present to the partners of a civil engineering firm in Chandler whose performance system has become the main source of employee complaints.

The Data Plan

The contract called for three sources. Interviews offer depth and explanation; the survey shows how widespread views are; records show what actually happened to ratings and people. Each source checks the others.

I interviewed both partners, eight of the firm's eleven managers and twelve engineers chosen to cover all three offices, junior and senior levels and every rating category. Interviews lasted forty-five minutes, followed a guide and were confidential. The anonymous online survey, twenty-two items plus two open questions, went to all 158 employees; 131 responded, an 83 percent rate. HR provided two years of ratings, including the ratings managers submitted before calibration meetings and the final ratings after, along with all voluntary departures.

What the Records Showed

Managers' submitted ratings and final ratings differed for 22 percent of employees, almost always downward, because calibration meetings enforced the distribution rule across the whole firm. In the firm's smallest office, with twelve engineers, three were moved down from the top category to meet the quota. Of the twenty-four voluntary departures over two years, fourteen had been rated in the middle category or lower after calibration, and nine of those fourteen had been moved down from the rating their manager submitted. The three senior project managers who left had all been moved from the top category to the middle.

What the Survey Showed

Seventy-one percent disagreed that their rating reflected their contribution. Only twelve percent mentioned the form's wording in open comments, while fifty-eight percent mentioned the quota, calibration or "being compared with people I don't work with." Sixty-four percent said they received feedback from their manager once a year or less. Among managers, seven of eleven reported no training at all in giving feedback.

What the Interviews Showed

Ten of twelve engineers described their rating as decided in a room they never entered. Six managers said they had argued for an engineer in calibration and lost, then had to deliver a rating they did not believe. Both partners believed the system was working as intended: before it, ratings were inflated and bonuses spread evenly. Ted, the managing partner, said, "Now we know who our best people are." Several managers disagreed: "Now we know who had the best advocate in calibration."

The form was never the problem; the quota, the calibration room and the silence between reviews were.

Testing the Client's Theory

Lauren's initial theory, that the form was poorly worded, finds little support. Few employees mentioned it, and managers' submitted ratings, made on the same form, were broadly accepted by the engineers interviewed. Ted's theory, that the system identifies the best people, is partly supported: ratings are now more spread out. But the records suggest that spread was achieved by moving people down in calibration, not by better judgments, and some of those moved down were the senior people the firm can least afford to lose.

What Research Says

Pulakos and O'Leary (2011) argued that performance management is broken because organizations keep redesigning formal procedures, forms and rating scales while neglecting what actually drives performance: the everyday behavior of managers and employees in setting expectations, giving ongoing feedback and solving problems together. They noted that formal systems are asked to serve too many purposes at once, such as pay, development and documentation, and that ratings tied to money invite gaming and distrust.

Kluger and DeNisi (1996) pooled hundreds of comparisons of feedback interventions. Performance improved overall, but more than one feedback intervention in three left people doing worse. When feedback pulled attention toward the self and off the task, such as comparisons with others or threats to self-image, tended to weaken or reverse effects.

Scullen et al. (2005) simulated the effects of forced distribution systems that remove low-rated employees each year. The simulation showed sizable improvements in workforce potential in the first few years that shrank quickly over time, and gains depended heavily on how accurate ratings were and on how many good performers left voluntarily.

The Diagnosis

The firm's performance problems stem from three linked causes. First, forced distribution applied firm-wide through calibration meetings, in small teams where strong performers cluster, overrides managers' judgments and produces ratings employees see as arbitrary. Second, feedback happens once a year, centered on a number, with managers untrained to discuss performance, so the rating becomes the whole conversation, the pattern the feedback meta-analysis warns against. Third, the same rating drives bonuses, promotions and development, which raises the stakes and, as the critique of performance management described, invites gaming in calibration.

The likely result is that the firm is losing some of its strongest people. The firm does not intend to remove anyone through the system, so the main benefit the simulation identified does not even apply; the firm bears the costs of forced distribution without the claimed gains.

Where the Sources Disagreed

Not every finding lined up. The survey suggested that engineers in the largest office were somewhat less unhappy than others, yet two of the three senior departures came from that office. Interviews explained the gap: the largest office had a manager who held monthly one-on-one meetings and openly told engineers when calibration had changed their ratings, which softened the blow for most of her team but not for the senior people moved down. That manager's practice is itself a finding, a local example of the everyday feedback the research recommends.

What Managers Need

Managers were the group most squeezed by the system. They are asked to deliver ratings they did not choose, to discuss performance once a year with little training and to defend outcomes in calibration without clear criteria. Several said they had stopped giving honest feedback during the year because any praise might be contradicted by a calibrated rating later. Any redesign that ignores managers' position will fail.

Limits

Departing employees may have left for reasons unrelated to ratings. Interviewed engineers may not represent everyone. The analysis cannot show what would have happened without the system. But three sources agree, which supports the diagnosis.

How I Will Present It

The partners hold different views, and one chose the system. I will present the records first, because numbers about their own departures are hard to dismiss, then survey and interview findings, then the research. I will frame the question not as whether Ted was wrong but as whether the system is delivering what he wanted from it.

Conclusion

The data redirect the engagement. The firm does not need a better form; it needs to reconsider how ratings are distributed, how often managers talk with engineers about performance and how many purposes one rating must serve.

References

Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254-284. https://doi.org/10.1037/0033-2909.119.2.254

Pulakos, E. D., & O'Leary, R. S. (2011). Why is performance management broken? Industrial and Organizational Psychology, 4(2), 146-164. https://doi.org/10.1111/j.1754-9434.2011.01315.x

Scullen, S. E., Bergey, P. K., & Aiman-Smith, L. (2005). Forced distribution rating systems and the improvement of workforce potential: A baseline simulation. Personnel Psychology, 58(1), 1-32. https://doi.org/10.1111/j.1744-6570.2005.00361.x