Last week, I completed the annual report for our department’s American Nurses Credentialing Commission’s (ANCC) provider unit. Our team of nurse planners and I assessed, implemented and evaluated seventy-seven continuing education programs that were presented across the Commonwealth of Massachusetts.
The exercise of completing this report allowed me to reflect upon our methods for evaluating the effectiveness of these programs.
Our accreditation status requires that we
evaluate the effectiveness of programs on based upon changes in the participants’ level of competence, performance or patient outcomes. But, the key question to be asked is
HOW to measure and interpret changes in these areas.
I believe the answer lies in building a process of program evaluation that is based upon
appreciative inquiry.
Five Key Principles and Assumptions of Appreciative Inquiry
Program evaluation involves the systematic collection of information about the
activities,
characteristics, and
outcomes of programs to make evidence-based judgements about the program, its effectiveness, and/or assist in the planning and implementation about future programs (Preskill & Catsambas, 2006).
An
appreciative inquiry approach can be used to construct:
- An anonymous paper or online evaluation tool;
- A focus group to look back 3-6 months following a program to assess if the program content resulted in a change in practice or policy; or
- A psychomotor evaluation to assess changes in competency following a program that focused on skills or new technologies.
Principle #1: The Constructivist Principle
In the United States, there are several educational programs that our various accrediting bodies mandate on an annual basis in order to demonstrate both currency and competency of practice.
However, there are endless amounts of other topics that influence nursing practice. For example, additional topics of interest may include:
- Compassion fatigue;
- Using Smart Technology to support behavioural management for challenging populations; and
- Meeting the holistic needs of dual diagnosed patients.
In order to meet the needs of your nursing staff, I recommend surveying the staff at two different times - the first survey can be completed during staff meetings and follow-up surveys can be sent via an anonymous link. This two-step process will allow for all nurses to have an opportunity to participate in the evaluation process regardless of their ability to attend a staff meeting.
The opportunity for all nurses to have a voice in the evaluation process will strengthen the bonds of your staff and generate new knowledge that will enhance patient care.
Principle #2: The Principle of Simultaneity
Prior to the conclusion of each program, create a visual of all of the questions asked during the presentation. This can be accomplished by utilising a free “word cloud” program available via the web, or by simply writing the questions on a white Board or flip chart that is visible to the participants.
Questions are never nonaligned, they are significant, and people move towards those questions that they most persistently and passionately discuss (Kessler, 2013).
I recommend that you allow your audience to view those questions while they complete the written program evaluation. That visual may spark ideas for new or expanded programs.
Principle #3: The Poetic Principle
Throughout our history, stories have provided the vehicles from which we make sense of our past and plan for our futures.
The stories of your patient care units are constantly changing and thus being reauthored. However, are we cognisant of sharing those stories?
How can we as educators expect to assess a change in nursing practice if we are not utilising the art and science of storytelling?
I have made it my practice to weave at least three stories into every educational program that I present.
I have educated nurses on a variety of topics related to dementia care, behavioural management, health care policy, and ethics. In the written program evaluations that I distribute to participants following these programs, one of the questions asks the nurse to relate one or more of the most memorable concepts that they learned during the program. The participants universally relay the concepts learned through one or more of the stories that were presented.
Principle #4: The Anticipatory Principle
It is the nature of human beings to constantly be looking toward the future.
I encourage you to ask your audience their recommendations for how to reshape existing programs through the use of different media (e.g., online programs, podcasts). Imagine the possibilities of being able to access a CE program while working the night shift via a workstation, or smartphone technology.
I would recommend “piloting” a program using a new strategy. Reflect upon the following questions:
- Did the participants feel that the program objectives were met?
- Did the participants express the ability to apply the concepts presented to their nursing practice?
- Did the participants express an interest in participating in future programs using this method of presentation?
Principle #5: The Positive Principle
True, lasting change whether it is practice, skill, or behaviour requires positive affect and role modelling behaviour. Sentiments such as empathy, camaraderie and hope increase the likelihood that change will be sustainable.
I recommend that you plan one-month post-program to observe the behaviours/skills of the participants on all three shifts to accurately evaluate for a positive change.
Did the program positively effect the change that you were anticipating? Ask yourself if there were barriers to that change such as unforeseen budgetary limitations or public health issues that may have caused a shift in priorities. Accept the limitations and make adjustment based upon the opportunities for enhancement that were apparent during your observations.
Concluding Thoughts
Historically, the process of programmatic evaluation has been thought of as cumbersome and often difficult to measure.
The art and science of appreciative inquiry strives to interpret the positives that emerge from the process of program evaluation so that
nurse educators can build upon the strengths and methodologies, we use to create these programs.
The end result of this approach is process built upon an asset-based and strengths-based approach to addressing the impact and influence of continuing education for nurses.
References
- Kessler, E.H. (ed.). (2013). Encyclopedia of management theory, Sage Publications. Retrieved from: http://www.gervasebushe.ca/the_AI_model.pdf
- Preskill, H. & Catsambas, T.T. (2006). Reframing evaluation through appreciative inquiry. Thousand Oaks, CA. Sage Publications.
New 
