PM 591 Week 6 Measuring Value Delivery Example

Velocity Is Not Value: Measuring What an Agile Game Studio Actually Delivers to Players and to the Business

[Student Name]

University of Phoenix

PM/591: Agile Project Management

Week 6 Assignment

[Instructor Name]

[Date]

Bluebonnet Forge Games, its measures, results and figures are composites written for a model paper.

A year ago, Bluebonnet Forge Games, the composite independent game studio in Austin followed through PM 591, began moving from milestone-driven production to agile teams. It now has sixteen teams, a light program layer and a portfolio funded by value stream. At the studio's annual review, the chief executive asked a simple question: is it working? This paper designs the measures that answer that question and reports what they show.

A Measure the Studio Rejected

Several executives asked for velocity charts to compare teams. The studio declined. Velocity, the number of story points a team completes per sprint, is useful to the team for planning, but points are estimates each team calibrates differently, so comparing them across teams is meaningless. Under pressure, teams inflate estimates and velocity rises without any change in what players receive. Hauser and Katz (1998) warned that organizations become what they measure and that metrics which are easy to influence but only loosely linked to long-term goals can push people toward the wrong behavior. Velocity is such a metric when used for comparison.

Outputs, Outcomes and What Lies Between

Outputs are what teams produce: features, content, releases. Outcomes are the changes those outputs cause: players staying longer, spending more or rating the game higher. Between them sit flow and quality, the conditions that let teams deliver outputs quickly and reliably. A useful system measures all three, with outcomes at the top. Kupiainen et al. (2015) reviewed industrial studies of metrics in agile and lean software development and found that teams used metrics mainly for sprint and project planning, progress tracking, quality measurement and identifying process problems, and that the metrics teams found most useful were those tied to their own decisions, with velocity and effort estimates featuring prominently despite their known limits.

Outcome Measures

For the live games: 30-day player retention, average player rating in app stores and platforms, revenue per active player and the share of seasonal content released on schedule. For the new title: player test scores at each playable checkpoint, the share of features passing the fun review on first attempt and publisher acceptance of checkpoints. For the business: revenue against plan, the share of spending on new products and voluntary staff turnover.

Flow Measures

Lead time from an approved idea to players' hands, for live game changes; throughput of completed backlog items per team; work in progress per team and per value stream; and the share of time items spend waiting rather than being worked on. Forsgren et al. (2018) drew on several years of surveys of software organizations and reported that four delivery measures, how long a change takes to reach users, how often teams release, how quickly they recover from failures and how often releases fail, tracked with wider organizational performance. The studio adapts these to games, where deployment means a patch or content release.

A team that moves faster without moving players is busy, not productive.

Quality Measures

Crashes per thousand play hours on each platform; defects reported by players after release; nightly build success rate for the new title; and the number of live game builds broken by engine changes, which Week 4's integration rules targeted.

Measures by Level

Team: sprint goals met, throughput, work in progress, escaped defects and the quarterly team health survey. Used by teams for their own improvement.

Program: cross-team dependencies resolved before sprints start, build stability, lead time for live game changes and playable checkpoint progress. Used by the production director and program board.

Portfolio: player outcomes for each game, revenue against plan, spending split between new and existing products and prototype gate results. Used by the portfolio board for funding decisions.

First-Year Results

Flow improved: lead time for live game changes fell from about 31 days to about 12, and the share of seasonal content released on schedule rose from 60 to 92 percent. Quality improved: crashes per thousand play hours on the live games fell by about 40 percent, and no live game build has been broken by an engine change since the integration rules took effect. The new title passed its vertical slice checkpoint on time, with player test scores above the publisher's threshold. Average overtime fell from about 9 hours a week per person to about 3, and voluntary turnover fell from 26 to 14 percent.

Not everything improved. Live Game B's 30-day retention did not change, despite faster and more reliable releases. The team concluded that its content, not its delivery, was the problem, and the portfolio board funded a redesign of its progression system. One feature team's work in progress stayed high all year, traced to a product owner who could not say no to requests; coaching followed.

How the Measures Are Collected

Most measures come from systems the studio already runs. Retention, ratings and revenue come from the platforms' analytics and the studio's own telemetry; lead time, throughput and work in progress come from the teams' boards, which timestamp every item's movement; crash rates come from automatic crash reporting built into the games; and build stability comes from the nightly build system. Only the team health survey and playtest scores need new effort. Collecting measures automatically matters: when people must type numbers into a spreadsheet, the numbers drift toward what people hope they will be.

Rules for Using Measures

Measures are used for learning and decisions, never for ranking teams or individuals. Team-level measures stay with teams unless they ask for help. Speed measures are always shown with quality measures. Each measure names the person who answers for it and when it will next be questioned, and the whole set is reviewed yearly, retiring measures that no longer drive decisions. The current standard emphasizes that measures should be chosen for their value in decision-making and that teams should beware of measures that encourage the wrong behavior (Project Management Institute [PMI], 2021).

Reflection on the Studio's Agile Change

Looking back across the course, the biggest gains came not from any single framework but from fitting practices to the work: short cycles to find fun, flow for content and live operations, a light program layer for the shared engine and funding that follows evidence. The measures confirm that delivery became faster, more reliable and more humane. They also show that agile improves the ability to deliver, not the quality of ideas: Live Game B's retention problem needed a better design, which only players' behavior could reveal.

Conclusion

Measuring value at Bluebonnet Forge means rejecting velocity as a comparison, putting player and business outcomes first and supporting them with flow and quality measures assigned to the level that acts on them. The first year shows faster, steadier delivery, better quality and less overtime, with one live game's outcomes still lagging. Clear rules for using measures keep them a tool for learning rather than a target to be gamed.

References

Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The science of lean software and DevOps: Building and scaling high performing technology organizations. IT Revolution Press.

Hauser, J., & Katz, G. (1998). Metrics: You are what you measure! European Management Journal, 16(5), 517-528. https://doi.org/10.1016/S0263-2373(98)00029-2

Kupiainen, E., Mäntylä, M. V., & Itkonen, J. (2015). Using metrics in agile and lean software development: A systematic literature review of industrial studies. Information and Software Technology, 62, 143-163. https://doi.org/10.1016/j.infsof.2015.02.005

Project Management Institute. (2021). A guide to the project management body of knowledge (PMBOK guide) (7th ed.). Project Management Institute.