HCS 487 Week 5 New Technology and System Evaluation Example

Did the E-Consult Platform Deliver? A Twelve-Month Evaluation of a New System at a Community Health Center Network, Measured on Six Dimensions of Success

[Student Name]

University of Phoenix

HCS/487: Technology and Systems Approach for Health Care Managers

Week 5 Assignment

[Instructor Name]

[Date]

The health center network, its measures and its results are composites written for a model paper; the evaluation model and published findings come from the sources listed.

A year after a composite network of community health centers launched an electronic consultation platform at its 12 sites, the executive team asked a direct question: should we expand it, fix it or stop paying for it? The platform had cost $168,000 in its first year, and the network was paying specialists $30 for each answered consult while it negotiated with Medicaid plans. This paper presents a structured evaluation to answer the question.

The Evaluation Model

DeLone and McLean (2003), reviewing a decade of research on their model of information systems success, described six linked tests of a system: how well it runs, how good its information is, whether the vendor and staff back it up, how widely people actually use it, whether the people using it are satisfied and the net good it does. The model's strength for this evaluation is its breadth: it asks not only whether the technology works and whether people like it but whether it produced benefits worth its cost.

Data Sources

The evaluation drew on platform reports, the electronic health record, the referral tracking list, vendor support tickets, a survey of 64 primary care clinicians and 11 specialists and cost data from finance. Baseline data came from the year before go-live.

System Quality

The platform was available 99.7% of the time, above the 99.5% contract standard. Median time for a clinician to send a consult was four minutes, matching the selection demonstration. Interface errors fell from 22 in the first month to two a month by month six.

Information Quality

Consults returned by specialists as incomplete fell from 11% in the first quarter to 4% in the fourth, after templates were revised. In a sample of 100 answers reviewed by two primary care physicians, 91 were rated clear and actionable. Dermatology answers depended on photograph quality, which improved after the network bought new cameras.

Service Quality

Vendor support met the four-hour urgent response standard in 94% of tickets. Clinicians rated site champions more helpful than vendor support, a finding that argues for keeping the champion role funded.

Use

The network sent 4,380 e-consults in the year, slightly above the expected 4,000. Cardiology and dermatology use met targets, with 86% and 88% of eligible referrals sent as e-consults. Endocrinology lagged at 41%, as clinicians reported that diabetes questions often required ongoing management rather than a one-time answer.

User Satisfaction

Of surveyed primary care clinicians, 81% said the platform improved their ability to care for patients and 72% said it was easy to use; 19% reported that templates took too long. Specialists were more positive: 10 of 11 said consults were well prepared.

Net Benefits

Median time to specialist advice fell from 94 days to 2.4 business days for cardiology and from 112 days to 2.1 days for dermatology. Across specialties, 58% of consults were resolved without a visit. For patients who still needed a visit, the median wait fell from 94 to 41 days in cardiology, because slots once filled by questions that did not need a visit were freed for patients who did. The share of referrals completed within six months rose from 48% to 79%. The platform's largest benefit was not the consults it answered, but the appointments it freed for the patients who truly needed them.

Comparison With Published Results

Across the e-consult systems summarized by Liddy et al. (2016), specialists typically answered within hours to a few days, and the network's median fits inside that span. Olayiwola et al. (2016) reported, in a randomized trial with underserved patients, that roughly seven in ten cardiology questions were settled by the e-consult alone; the network's cardiology rate was 63%, somewhat lower, possibly because its templates encouraged referral when in doubt.

What Staff Said

Open comments in the survey added detail the numbers could not. Coordinators said the single list of open consults had replaced dozens of phone calls a week to specialty offices. Several clinicians said they learned from specialists' answers and now handled some questions themselves, a benefit the model captures only indirectly. Two clinicians said the platform made it harder to refer patients who simply wanted to see a specialist in person, a concern the network will address by keeping a direct referral option for such cases.

Costs

Total first-year cost, including licenses, specialist payments, staff time and cameras, was about $312,000, or roughly $71 per consult. Against this, the network avoided about 2,500 specialist visits that would otherwise have required transportation assistance, interpreter time and coordinator follow-up, though a full cost-benefit analysis was beyond the evaluation's scope.

Limits

Because there was no comparison network, only a before-and-after view, other changes, such as a new cardiology practice opening in the region during the year, may account for part of the improvement. Survey response was 72% among clinicians.

Recommendations

First, continue and expand the platform: add pulmonology and gastroenterology, the next longest waits. Second, fix endocrinology by adding an ongoing co-management option rather than a one-time consult. Third, shorten templates flagged as slow, with champions leading the revision. Fourth, press Medicaid plans to pay for e-consults, using the evaluation's results on access and completed referrals, and set a date: if no plan pays within 18 months, the network will review the specialist payment model.

Equity Check

The evaluation also looked at who benefited. E-consult use and response times were similar for patients who preferred English and those who preferred other languages, because the exchange happens between clinicians. But patients who still needed a specialist visit and preferred Spanish or Haitian Creole waited nine days longer on average than English speakers, reflecting fewer interpreter-supported slots at specialty offices. The network will raise the issue with its specialty partners.

Next Evaluation

The steering group will repeat the evaluation at 24 months, adding a comparison with a similar network that has not adopted e-consults, if one agrees to share data.

Conclusion

Measured on six dimensions, the platform works technically, produces useful information, is widely used in two specialties and satisfies most users. Its net benefits are substantial: advice in days instead of months and more completed referrals. Endocrinology use and payment remain problems. The evidence supports expanding the platform while fixing what the evaluation revealed.

References

DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9-30. https://doi.org/10.1080/07421222.2003.11045748

Liddy, C., Drosinis, P., & Keely, E. (2016). Electronic consultation systems: Worldwide prevalence and their impact on patient care: A systematic review. Family Practice, 33(3), 274-285. https://doi.org/10.1093/fampra/cmw024

Olayiwola, J. N., Anderson, D., Jepeal, N., Aseltine, R., Pickett, C., Yan, J., & Zlateva, I. (2016). Electronic consultations to improve the primary care-specialty care interface for cardiology in the medically underserved: A cluster-randomized controlled trial. Annals of Family Medicine, 14(2), 133-140. https://doi.org/10.1370/afm.1869