HCS 335 Week 5 Emerging Ethical Issues and Social Responsibility Example

An Algorithm Picks Who Gets Extra Care: The Emerging Ethics of Predictive Tools and a Nonprofit Health System's Responsibility to Its Community

[Student Name]

University of Phoenix

HCS/335: Health Care Ethics and Social Responsibility

Week 5 Assignment

[Instructor Name]

[Date]

The health system, its tool and its figures are composites written for a model paper; research findings and guidance come from the sources listed.

The care management director of a composite nonprofit health system noticed something odd three months after the system began using a vendor's risk score. The program, which assigns nurses and social workers to patients with complex chronic illness, had enrolled 1,400 people chosen by the score. Few came from the two ZIP codes where the system's patients were poorest and most often Black. This paper examines the emerging ethical issue behind her observation and what the health system owes its community.

Why the Issue Is Emerging

Predictive tools now help decide who gets outreach, who is flagged for sepsis, who receives care management and which claims are reviewed. They spread faster than the rules for using them. Most codes of ethics for health care managers were written before such tools existed, so organizations must work out their responsibilities as they go.

What Research Has Shown

Obermeyer et al. (2019) studied a widely used commercial algorithm that helped health systems identify patients for care management. The algorithm predicted future health care costs as a stand-in for health needs. Because health systems had spent less on Black patients than on equally ill White patients, the model scored Black patients as healthier than they were. At a given risk score, Black patients had more uncontrolled chronic illnesses. The authors estimated that correcting the bias would substantially increase the share of Black patients selected for extra help, and they showed that changing the prediction target reduced the bias.

Checking the System's Own Tool

The director asked the analytics team to compare patients selected by the score with those who had the most active chronic conditions. The pattern matched the published finding: the tool, trained on claims costs, favored patients who already used a lot of care. The algorithm was not designed to exclude anyone; it was designed to predict spending, and in a system where some groups have long received less care, predicting spending quietly reproduces that gap.

Stakeholders

Stakeholders include patients who needed care management and did not receive it, patients who were enrolled, the nurses and social workers whose caseloads the tool shaped, the vendor, the system's leaders and board and the communities the system serves.

Principles

Justice is the principle most directly at stake: a benefit was distributed in a way that disadvantaged a group already facing barriers. Beneficence and nonmaleficence apply because patients with unmet needs may have been harmed. Respect for autonomy raises the question of whether patients know an algorithm helps decide their care.

International Guidance

International guidance on the ethics and governance of artificial intelligence for health rests on six principles (World Health Organization, 2021): protecting human autonomy; promoting human well-being, safety and the public interest; ensuring transparency, explainability and intelligibility; fostering responsibility and accountability; ensuring inclusiveness and equity; and promoting AI that is responsive and sustainable. The system's tool failed the equity principle and, because no one had asked the vendor how it worked, the transparency principle as well.

Social Responsibility

As a nonprofit, the system is exempt from many taxes in return for benefiting its community. Rozier (2020), reviewing a decade of research on nonprofit hospital community benefit, found wide variation in how much hospitals spend and how well spending matches community needs. A care management program is exactly the kind of service that should reach the neighborhoods with the greatest need. Using a tool that steered help away from those neighborhoods would undercut the very bargain that justifies the system's tax exemption.

Options

Stop using the tool. Keep it unchanged. Correct it by changing its target from cost to measures of health need and adding direct clinician referral.

Recommendation

Correct the tool rather than abandon it. The analytics team, working with the vendor, will train the model to predict active chronic conditions and avoidable hospital use instead of cost, and clinicians will be able to refer patients directly. The system will review the 1,400 enrolled patients and invite those missed from the two ZIP codes.

Governance for Future Tools

The system will create an algorithm review committee including clinicians, an ethicist, a data scientist and two community members. No predictive tool will be deployed without a bias test across race, ethnicity, sex, age and payer, a plain-language description for clinicians and a plan for monitoring results every six months. Vendors must disclose what their models predict and on what data.

Costs and Tradeoffs

Correcting the tool is not free. Retraining the model and validating it will take the analytics team about three months, and inviting the missed patients will add roughly 180 people to caseloads, requiring one more nurse care manager. Leaders weighed these costs against the alternative of continuing to spend program money on patients with less need. The added nurse is itself a community benefit expense the system can report.

What Staff Should Know

Nurses and social workers in the program will be told why the selection changed and shown the bias findings. They will be encouraged to refer patients they believe were missed, since clinicians often see needs that data do not capture.

Transparency With Patients

Patient-facing materials for care management will state that the system uses data tools to help identify people who may benefit, and that clinicians make the final decision. Patients may ask how they were selected, and staff will answer in plain terms. Community members on the review committee will help write these materials so that the explanation makes sense to the people it describes, and the committee will publish a short annual summary of the tools in use and their test results.

Conclusion

Predictive algorithms are an emerging ethical issue because they allocate care at scale while hiding their choices in design details. The health system's own tool repeated a documented bias. Its responsibility as a nonprofit community institution requires correcting the tool, reaching the patients it missed and governing every future tool for fairness and transparency.

References

Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447-453. https://doi.org/10.1126/science.aax2342

Rozier, M. D. (2020). Nonprofit hospital community benefit in the U.S.: A scoping review from 2010 to 2019. Frontiers in Public Health, 8, Article 72. https://doi.org/10.3389/fpubh.2020.00072

World Health Organization. (2021). Ethics and governance of artificial intelligence for health: WHO guidance. https://www.who.int/publications/i/item/9789240029200