DNP/715 Week 8: Evaluating a Technology for Adoption, sample paper

Should Our Nurse Practitioners Use an Ambient AI Scribe? Evaluating a Documentation Technology for Adoption With the Technology Acceptance Model, an Information Systems Success Model and the NASSS Framework

[Student Name]

University of Phoenix

DNP/715: Information Systems and Health Care Delivery Technology

Week 8 Assignment

[Instructor Name]

[Date]

The practice and pilot are composites written for a model paper.

I direct advanced practice for a primary care network of 12 clinics with 48 nurse practitioners. In our last engagement survey, documentation was the leading cause of burnout, with nurse practitioners reporting an average of 1.4 hours of after-hours charting each workday. A vendor has proposed an ambient AI scribe that listens to visits and drafts notes. This paper evaluates whether we should adopt it.

Early Experience Elsewhere

Tierney et al. (2024) described the rapid rollout of ambient AI scribes in a large integrated medical group, where thousands of physicians used the tool across a large number of patient encounters within weeks. Users generally reported that it reduced documentation burden and improved their ability to focus on patients, while the authors emphasized that clinicians must review and edit every note, since the tool can omit or misstate information. This experience suggests benefit but was not a controlled study.

Will Clinicians Accept It?

Holden and Karsh (2010) reviewed applications of the technology acceptance model to health care, which proposes that perceived usefulness and perceived ease of use shape intention to use a technology and actual use. Across 16 data sets, some relationships were consistently significant, and the model predicted a substantial portion of acceptance, but the authors recommended adapting it to health care contexts. For our nurse practitioners, perceived usefulness is likely high, given the burden of after-hours charting. Ease of use depends on how easily drafts can be reviewed and edited in our record.

What Counts as Success?

DeLone and McLean (2003) updated their information systems success model to include six dimensions: system quality, information quality, service quality, use or intention to use, user satisfaction and net benefits, with feedback between them. For a scribe, system quality includes reliability and integration with our record; information quality is note accuracy and completeness; service quality is vendor support; use and satisfaction are measured directly; and net benefits include time saved, burnout and patient experience.

A tool that saves an hour of charting and inserts one wrong allergy has not saved anything; information quality has to be measured, not assumed.

Why Adoption Might Fail

Greenhalgh et al. (2017) developed the NASSS framework from a review of 28 frameworks and ethnographic case studies, identifying seven domains that explain nonadoption, abandonment and failure to scale: the condition, the technology, the value proposition, the adopter system including staff, patients and caregivers, the organization, the wider context and the interaction and adaptation among these over time. They classify each domain as simple, complicated or complex, with complexity across domains predicting failure.

Applying NASSS

Condition: primary care visits are varied, often covering several problems, which is complicated for note generation. Technology: requires a smartphone app, reliable connectivity and record integration, complicated. Value proposition: clear for clinicians, less clear for the network if the subscription cost is not offset. Adopter system: clinicians must learn to review drafts; patients must consent to recording, and some may decline, especially those with limited English or concerns about privacy. Organization: requires training, policy on recording and consent and a process for errors. Wider context: regulations on recording, privacy and the use of AI are evolving. Several complicated domains suggest a cautious, staged approach.

Clinician Perspectives

I held two listening sessions with nurse practitioners. Most were eager, but several raised concerns: whether patients would speak freely if recorded, whether reviewing drafts would take as long as writing notes and whether the tool would handle visits conducted partly in Spanish. These concerns map onto NASSS domains and shaped the pilot design.

Patient Perspectives

Our patient advisory council reviewed a draft consent script. Members supported the tool if it meant more eye contact from clinicians, but wanted assurance that recordings would not be kept or shared. Two members said they would decline for visits about mental health or sexual health, which the consent process must accommodate easily.

Equity Considerations

If the tool performs worse in Spanish-language visits, Spanish-speaking patients could receive less accurate notes or less clinician attention. The pilot will compare accuracy by visit language and will not expand unless performance is comparable.

Risks

Risks include inaccurate or fabricated content in notes, privacy concerns with audio recordings, reduced clinician attention to documentation, uneven performance with accents or Spanish-language visits and cost. Each requires a safeguard.

Measuring Burden Honestly

Self-reported time savings may overstate benefit. The pilot will use record audit logs to measure actual after-hours documentation time before and during use, alongside clinicians' own reports, so that the decision rests on objective as well as subjective data.

Safeguards

Clinicians review and sign every note. Audio is deleted after the note is finalized, per contract. Patients receive a verbal and written consent explanation in their language and can decline without penalty. A sample of notes is audited monthly for accuracy against the recording before deletion.

Training

Clinicians in the pilot will complete a one-hour session on starting and stopping recording, obtaining consent, reviewing drafts efficiently and reporting errors. They will also receive a short checklist of high-risk note elements to verify every time: medications, allergies, doses and follow-up plans.

Pilot Design

A 12-week pilot with 10 volunteer nurse practitioners across four clinics, including two clinics with many Spanish-speaking patients, will measure after-hours documentation time from record logs, burnout on a validated single-item measure, note accuracy on audit, patient consent rates and patient experience. Pilot data will be compared with 10 matched nonusers.

Alternatives Considered

Alternatives include human medical scribes, which our network cannot staff at scale, templates and dictation, which reduce some burden but still require after-hours work, and team documentation with medical assistants, which we have partly implemented. The AI scribe may complement rather than replace these approaches.

Cost

The subscription costs about $150 per clinician per month. If each user saves 45 minutes of after-hours work daily and burnout-related turnover falls, the value likely exceeds cost, but the pilot must confirm this.

Governance of AI Tools

Because AI tools change through vendor updates, the network needs a standing process to review new versions, monitor accuracy and receive error reports. I recommend that the existing clinical informatics committee take on this role, with a nurse practitioner member.

Recommendation

Adopt conditionally: proceed with the 12-week pilot, with defined success criteria of at least 30 minutes daily reduction in after-hours charting, accuracy errors below a set threshold and patient consent above 85%, before any network-wide rollout.

Conclusion

An ambient AI scribe promises relief from documentation burden. The technology acceptance model suggests clinicians will accept it if it is easy to use, the DeLone and McLean model defines success across quality, use and net benefits, and the NASSS framework reveals several complicated domains that could cause failure. A conditional adoption with a measured pilot and safeguards balances promise and risk.

References

DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9-30. https://doi.org/10.1080/07421222.2003.11045748

Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017). Beyond adoption: A new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. Journal of Medical Internet Research, 19(11), Article e367. https://doi.org/10.2196/jmir.8775

Holden, R. J., & Karsh, B.-T. (2010). The technology acceptance model: Its past and its future in health care. Journal of Biomedical Informatics, 43(1), 159-172. https://doi.org/10.1016/j.jbi.2009.07.002

Tierney, A. A., Gayre, G., Hoberman, B., Mattern, B., Ballesca, M., Kipnis, P., Liu, V., & Lee, K. (2024). Ambient artificial intelligence scribes to alleviate the burden of clinical documentation. NEJM Catalyst Innovations in Care Delivery, 5(3). https://doi.org/10.1056/CAT.23.0404