Did Students Keep Talking to Mr. Alvarez? Formative and Summative Evaluation of a Supported Conversation Module From Pilot to the Clinical Unit
[Student Name]
University of Phoenix
NSG/534: Facilitating Engaged Learning
Week 6 Assignment
[Instructor Name]
[Date]
The school, the students and all data are a composite written for a model paper.
Over five weeks, I carried a module on communicating with adults who have aphasia through analysis, design, development and implementation for junior nursing students. The module aimed to change a behavior seen repeatedly on our stroke rehabilitation unit: students who stopped talking to patients with aphasia and spoke to families or finished care quickly instead. This paper reports the formative evaluation that shaped the module and plans and reports the summative evaluation of its first cohort.
Evaluation in the ADDIE Model
Branch (2009) places evaluation throughout the model, with formative evaluation during each phase and summative evaluation after implementation. Formative evaluation asks how to improve the product; summative evaluation asks whether it worked. Both matter, because a module that is only judged at the end may carry fixable flaws into its first full use.
Formative Evaluation During Development
Three formative steps shaped the module. The speech-language pathologist reviewed the script and found that two of Mr. Alvarez's responses were more fluent than a person with his type of aphasia would produce; they were rewritten. The alpha test with two faculty found a navigation error that sent students who chose a poor response back to the start rather than to the decision. Beta testing by four students who had finished the rotation found that the scenario took 22 minutes rather than the planned 15; one decision point was shortened.
Formative Evaluation During the Pilot
Eight students completed the whole module two weeks before the cohort. Results and comments led to four changes. The quiz item on fluent aphasia confused five of eight students because the scenario's patient description was ambiguous; it was clarified. One standardized patient answered yes-or-no questions more clearly than the case intended, making closed questions seem to work; she received additional coaching. Raters found the checklist item "allows time" hard to judge; it was redefined as "waits at least five seconds after a question before rephrasing." Students asked for a printable summary card of techniques to carry on the unit, which was added. The pilot's most useful finding was that a standardized patient's inconsistency could teach students the wrong lesson, which no amount of design review would have revealed.
Summative Evaluation: Framework
The summative evaluation uses Kirkpatrick's four levels: reaction, learning, behavior and results (Kirkpatrick & Kirkpatrick, 2016).
Level 1: Reaction
Of 48 students, 45 completed the end-of-module survey. Ninety-one percent agreed that the module prepared them for patients with aphasia, and 87% rated the standardized patient encounter as the most useful part. The empathy exercise was the most frequently mentioned in comments, with students writing that it "changed how I thought about it." The most common criticism was that the online module and simulation were a week apart, and some students forgot techniques in between.
Level 2: Learning
All 48 students passed the quiz, 41 on the first attempt. On the video analysis, students identified a median of five of six barriers and facilitators. In the simulation, raters recorded a median of five of seven supported conversation techniques used; 44 of 48 students used at least five, meeting objective 3. For objective 4, 43 students assessed pain using a method the patient could answer. Two raters scored 12 encounters independently and agreed on 88% of checklist items. Confidence ratings, collected before and after, rose from a median of 2 to 4 on a five-point scale.
Level 3: Behavior
Behavior on the unit is the measure that matters most. During the rehabilitation rotation, clinical instructors used a short observation form during one care episode with a patient with aphasia for each student, recording whether the student addressed the patient directly, used at least three techniques and assessed pain with an accessible method. In the first cohort, 38 of 44 observed students met all three criteria. Instructors' written comments mentioning difficulty communicating with patients with aphasia fell to 4 of 48 students, compared with 31 of 142 over the previous three semesters, a drop from about 22% to 8%.
Level 4: Results
Results for patients are harder to measure. The unit's nurse manager reported no family complaints about student communication during the rotation, compared with two in the previous year. Speech-language pathologists noted that several students asked how to adapt techniques for specific patients. These are encouraging signals rather than proof, since one rotation is a short period and other factors, such as the newly visible communication boards, may have contributed. Communication partner training has been associated with improved partner skill in many studies (Simmons-Mackie et al., 2016), and these findings fit that pattern.
Limits
The evaluation has limits. Behavior was observed once per student, instructors knew about the module and may have looked for its techniques and there was no comparison group in the same semester. The previous-semester comparison is suggestive but not controlled.
Unexpected Findings
Two findings were not anticipated. Students who scored lowest on the quiz did as well as others in the simulation, which suggests that the practice activities, not prior knowledge, drove performance. And several students reported using the techniques with patients who had dementia or hearing loss, a transfer the module did not aim for but which clinical instructors confirmed.
What Students Said They Would Change
In the reflections, the most common improvement students named for themselves was waiting longer after a question; many wrote that silence felt uncomfortable but that the standardized patient needed it to answer. This matches the checklist item raters found hardest to judge in the pilot and suggests that waiting deserves more practice time in the next version.
Cost of the Evaluation
Evaluation added modest work: a survey, a quiz already built into the module, rater time for twelve double-scored encounters and a one-page observation form for clinical instructors. Keeping measures simple made it possible to collect behavior data on the unit, which is the evidence that matters most and the kind most often skipped.
Sharing the Results
Results will be shared with the course coordinator, clinical instructors, the rehabilitation unit's staff and the curriculum committee, which may consider similar modules for other communication challenges.
Decisions
Keep the module's structure and activities. Move the online module closer to the simulation, completing it in the same week. Add a two-minute review of techniques at the start of the simulation. Add the observation form to the rotation permanently, so behavior is measured each semester. Repeat the evaluation with the next two cohorts before making further changes.
Conclusion
Formative evaluation found and fixed problems a design review alone would have missed, including an inconsistent standardized patient. Summative evaluation found that students valued the module, learned the techniques, used them in simulation and, in most cases, used them on the unit, where concerns about communication with patients with aphasia fell sharply. Evaluation completes the ADDIE cycle and begins the next one.
References
Branch, R. M. (2009). Instructional design: The ADDIE approach. Springer. https://doi.org/10.1007/978-0-387-09506-6
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick's four levels of training evaluation. ATD Press.
Simmons-Mackie, N., Raymer, A., & Cherney, L. R. (2016). Communication partner training in aphasia: An updated systematic review. Archives of Physical Medicine and Rehabilitation, 97(12), 2202-2221. https://doi.org/10.1016/j.apmr.2016.03.023
How this NSG 534 Week 6 example is structured
The NSG/534 description ends with the evaluation phase of the ADDIE model. This paper separates formative evaluation, which improved the product while it was built and piloted, from summative evaluation, which judges whether it achieved its objectives, and uses Kirkpatrick's four levels to organize the summative measures. Each finding leads to a decision. Students search this week as NSG 534 Week 6, NSG534 Wk 6 or NSG/534 Wk 6; all three are the same assignment.
NSG/534 Week 6 questions, answered
What does NSG/534 Week 6 usually ask for?
Many sections end with the evaluation phase of the ADDIE model: planning or reporting formative and summative evaluation of the instructional product and deciding on revisions.
What is the difference between formative and summative evaluation in instructional design?
Formative evaluation happens during design, development and piloting to improve the product. Summative evaluation happens after implementation to judge whether it achieved its goals.
Can evaluation show a change in clinical behavior?
Yes, if it includes measures taken in the clinical setting, such as structured observation or instructor ratings, though such measures are harder to standardize than tests.
Write yours, or have the desk draft it
This paper is an original model document written by our desk, not a submitted student paper and not an official University of Phoenix document. Read it for the moves, then write your own to the instructions in your classroom. If you want one built to your exact prompt and rubric, the first custom sample is free and arrives in 24 to 48 hours.