DNP/710 Week 6: Selecting Outcome Measures and Examining Validity and Reliability, sample paper

Thirst on a Line, Recovery in Fifteen Items: Choosing and Defending the Outcome Measures for a Preoperative Carbohydrate Drink Project, With Validity and Reliability Checked Against COSMIN

[Student Name]

University of Phoenix

DNP/710: Evidence-Based Practice Measurement and Clinical Inquiry

Week 6 Assignment

[Instructor Name]

[Date]

The project is a composite written for a model paper.

Last week's synthesis ended in a conditional recommendation to let eligible patients drink until shortly before surgery, including a carbohydrate drink. To know whether the change helps our patients, I need outcome measures that match my PICOT outcomes, measure what they claim to, give consistent results and are practical for preoperative nurses. This paper selects and evaluates those measures.

A Framework for Measurement Properties

Mokkink et al. (2010) developed the COSMIN checklist through an international Delphi study to assess the methodological quality of studies on measurement properties of health status instruments. The resulting taxonomy groups measurement properties into three domains: reliability, including internal consistency, test-retest reliability and measurement error; validity, including content validity, structural validity, hypothesis testing and criterion validity; and responsiveness, the ability to detect change over time, with interpretability as a related consideration. I use these domains to evaluate each measure.

Primary Outcomes: Thirst and Hunger

Hausel et al. (2001) measured thirst, hunger, anxiety and other discomforts with 100-millimeter visual analog scales and concluded that such scales provide useful information about preoperative discomfort. They detected differences among groups, which supports responsiveness. For our project, I will use a 0-to-10 numeric rating scale for thirst and hunger, which is easier to administer verbally and in electronic records.

Evaluating the Numeric Scales

Content validity: asking "How thirsty are you right now, from 0, not at all, to 10, the worst thirst imaginable?" directly reflects the construct. Reliability: single-item scales cannot have internal consistency, and test-retest reliability is hard to establish for a state that changes by the hour, so consistency depends on standardized wording and timing. Responsiveness: similar scales detected differences in the trial. The main risk is inconsistent administration, which training will address.

Secondary Outcome: Quality of Recovery

Stark et al. (2013) developed the QoR-15 and found good convergent validity with a global recovery scale, correlation 0.68; construct validity shown by negative correlations with surgery duration, recovery room time and hospital stay; excellent internal consistency, 0.85; split-half reliability, 0.78; test-retest reliability, 0.99; and excellent responsiveness, with an effect size of 1.35. The mean completion time was 2.4 minutes.

A measure can be perfectly valid in a published study and useless on a unit where no one has two minutes to read it aloud.

Evaluating the QoR-15

By the COSMIN domains, the QoR-15 has strong evidence for internal consistency, test-retest reliability, construct validity and responsiveness (Stark et al., 2013). Its short completion time makes it feasible for our next-day phone call, although it was developed for in-person administration. Phone administration is a change in mode that could affect scores, so I will pilot it with 10 patients to confirm that they understand the items and that completion time is acceptable.

Process Outcome: Fasting Duration

Fasting duration from liquids and from solids will be calculated from the time of last intake, asked by the preoperative nurse, and the time of anesthesia induction, taken from the anesthesia record. Content validity is strong, since this is exactly what the recommendation aims to change. Reliability depends on patient recall; asking for a specific time rather than an estimate, and asking whether the patient drank the carbohydrate drink and when, improves accuracy.

Safety Measures

Aspiration events will be identified from anesthesia records and incident reports. Postponements and cancellations linked to what a patient ate or drank will be pulled from the scheduling system. These are rare events with clear definitions, so their measurement is reliable, but they will be too few to compare statistically; they serve as balancing measures to detect harm.

Validity Threat: Expectation

Patients who know they received a new drink may report less thirst because they expect to. This is a threat to validity that the before-and-after design cannot remove. Comparing thirst with objective process measures, such as time since last fluid, helps judge whether reported changes track real changes in intake.

Adherence Measure

The proportion of eligible patients who drank the carbohydrate drink within the target window measures whether the intervention was actually delivered. Without it, a null result could mean the drink does not work or that patients did not receive it.

Standardizing Collection

All preoperative nurses will complete a 15-minute training on the scripts, practice with a partner and have a laminated card at each station. I will audit 10% of records in the first month to check that scales are recorded at the correct time, in the half hour before the patient leaves the preoperative area.

Timing the Follow-Up Call

The QoR-15 asks about the previous 24 hours. Calls will be made between 9 a.m. and noon on the first postoperative day so that the recall window is consistent. Patients not reached after two attempts will be recorded as missing, and I will compare their characteristics with responders to check for bias.

Interpretability

For thirst and hunger, a change of 2 points on a 0-to-10 scale is often considered meaningful for patients. For the QoR-15, published literature suggests that a change of about 8 points is clinically important. I will use these thresholds to interpret whether changes matter, not only whether they are statistically significant.

Why Not Measure Length of Stay

Length of stay is the most common outcome in the carbohydrate literature, but nearly all our patients leave the same day. A measure that barely varies cannot show change. Time from arrival in recovery to discharge might be more useful and can be extracted from the record without extra work, so I will track it as an exploratory measure.

Anxiety as an Additional Outcome

The trial that informed my question found reduced anxiety with a carbohydrate drink (Hausel et al., 2001). Adding a single anxiety item using the same 0-to-10 format costs a few seconds and captures an outcome patients care about. I will add it and treat it as secondary.

Burden on Patients and Staff

The full set of measures adds about three minutes to the preoperative assessment and three minutes to the follow-up call. Staff agreed this was acceptable when shown the time estimates, provided the new fields were placed where they already document preoperative vital signs.

Language and Literacy

About a quarter of our patients prefer Spanish. The numeric scales and the QoR-15 will be administered in Spanish by bilingual nurses or through our interpreter line, using a validated translation of the QoR-15 where available. Measures that work only in English would bias the results toward one group.

Conclusion

Each outcome measure maps to a PICOT outcome and has been evaluated using the COSMIN domains. Numeric rating scales for thirst and hunger have strong content validity and demonstrated responsiveness, the QoR-15 has strong published evidence for reliability, validity and responsiveness, and record-based fasting duration and safety events are clearly defined. Standardized scripts, training and auditing will protect reliability in practice.

References

Hausel, J., Nygren, J., Lagerkranser, M., Hellström, P. M., Hammarqvist, F., Almström, C., Lindh, A., Thorell, A., & Ljungqvist, O. (2001). A carbohydrate-rich drink reduces preoperative discomfort in elective surgery patients. Anesthesia and Analgesia, 93(5), 1344-1350. https://doi.org/10.1097/00000539-200111000-00063

Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2010). The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: An international Delphi study. Quality of Life Research, 19(4), 539-549. https://doi.org/10.1007/s11136-010-9606-8

Stark, P. A., Myles, P. S., & Burke, J. A. (2013). Development and psychometric evaluation of a postoperative quality of recovery score: The QoR-15. Anesthesiology, 118(6), 1332-1340. https://doi.org/10.1097/ALN.0b013e318289b84b