Competency 18

Evaluate and reflect on results of actions and decisions.

Effective educational leaders must have the ability to:

• Analyze data and evidence.
• Draw valid inferences and conclusions from data.
• Involve participants and key constituencies in reviewing and discussing data for accuracy and further inquiry.
• Reflect on the results in ways that challenge their own assumptions and encouraging others to do the same.
• Model open-mindedness and inquiry skills.
• Communicate results clearly, accurately, and in ways that demonstrate a balance between advocacy and inquiry that is appropriate to the situation.

Competency 18 is the third part of a three-part action research cycle that includes competencies 16, 17, and 18. Together, the competencies form a cycle of inquiry that can be applied to site-based applied research and problem solving:

• Plan for research and problem solving (including identification and framing of problems, generation of alternatives or solutions, and development of action plans and metrics).
• Implement actions and measure results (including data collection and analysis, and dialogue with participants).
• Evaluate and reflect on results of actions and decisions (including collective reflection and discussion with participants, communication of results, and decisions about new actions). 

When the first two parts of the inquiry cycle are complete, an evaluation and reflection phase ensues. That is, a problem was first defined and framed, a plan for research and/or the implementation of a solution was developed, and the plan was implemented and carefully monitored against pre-established criteria. Data and evidence were collected and analyzed, as appropriate, during the implementation itself.  Ongoing assessment and potential adjustments to the implementation, conversation and dialogue with participants and stakeholders, and reflection on action as it occurs may be ongoing during the implementation. But a summative analysis of all data and an evaluation of the results takes place at the end of the timeline for implementation. Reflection on the implementation and its outcomes is undertaken by the leader individually, and in dialogue with others. Questions such as these are addressed:

• What are the key findings? What happened?
• How do we know if we succeeded? Did we meet key criteria that were established for success?
• What do the data say about the actions that were implemented? About the problem?
• What unexpected elements or outcomes were identified?
• What did we learn? 
• What are the political, economic, social, and ethical implications?
• Were the results promising?
• How do the results relate to organizational goals and values?
• Based on the reflection cycle (individually and collaboratively), what are the next steps?
• How does the next action step reflect the previous learning?
• What is the best way to communicate the results and next steps? (Kuhne & Quigley, 1997).

In a continuous improvement process, the evidence-based results of a solution to a problem, an intervention intended to improve an educational or organizational outcome, or the introduction of major change hold implications for further action. The status of the problem or issue after implementation is different in relation to its status before the implementation. The cycle of inquiry begins again by framing the problem or issue anew in light of the new knowledge provided by the actions already taken, and "revealing further possibilities for action" (Stringer, 2004, p. 10).

To adequately design and evaluate action research (or site-based applied research)-or on a smaller scale, to implement solutions to complex problems-leaders need to understand basic principles of research logic (for example., developing research questions that address the problem). Leaders also need a working knowledge of quantitative and qualitative data collection and analysis methods, concepts of statistical reasoning (including fundamental concepts of probability, scaling, reliability and validity, variables, statistical significance, correlation, and regression), and concepts of qualitative method (such as authenticity, validity, scope of application, and the presence of hidden assumptions).  

Evaluation and decision making, however, involve more than knowledge of research methods. To make decisions about further actions and next steps, leaders must be excellent reflective thinkers. How aware is a leader of the reasoning that is the basis for action? In the words of Senge (1994), "decision-making processes could be transformed if people become more able to surface and discuss productively their different ways of looking at the world" (p. 182). While this sounds simple enough, mental models are often quite difficult to examine because they are "deeply held internal images of how the world works, images that limit us to familiar ways of thinking and acting" (Senge, p. 174). The images and beliefs that comprise a leader's mental models can constrain actions and decisions, even in the face of evidence to the contrary. Behavior is often not congruent with a leader's espoused theories (what he or she says), but with his or her "theories-in-use" (her mental models) (Argyris, 1977; Argyris & Schön, 1974; Bushe, 2001). 

When a leader reflects on the results of actions and decisions at the level of mental models, he or she is examining and questioning his or her own assumptions and beliefs about what is real. If beliefs are suspended through reflection, it becomes possible to imagine and understand changes to mental models that enable changes to other systems. Leaders can help themselves uncover their views as well as communicate and advocate for their decisions by explicitly, step by step, examining how evidence informed the decision and the assumptions upon which it is based. (Kim, 1999; Senge, 1994 ).

Before acting on next steps, leaders need communicate results and decisions. Leaders will likely have discussed findings with participants at various times during and after implementation as part of the applied research process. But at the end of an applied initiative, summative findings and decisions for further action are important messages to finalize and convey. Consistent communication to all who are involved and affected by the changes and decisions is part of this competency as well. Effective communication strategies are similar to those for communication of other important. Kotter (1996) recommends these principles of communication: simplicity (that is, no jargon), use of metaphor and analogy, multiple forums and types of communication, repetition, example (that is, leadership behavior consistent with the message), transparency about inconsistencies, and two way communication when possible (p. 90). 

It may seem obvious that evaluation, reflection, and communication of results and next steps (new plans) are important leadership skills and behaviors that have been researched and emphasized in leadership literature for many years. Evidently, however, they are not all that easy to apply or implement. The literature of researchers like Argyris, Schön, and Senge would not otherwise have remained so current. A recent book, The Game Changer: How You Can Drive Revenue and Profit Growth With Innovation (Lafly & Charan, 2008), comments on the same principles when discussing the elements and expectations of an innovation culture. Among other things, the authors call out the need to be "open-minded to new ideas-from anyone, anywhere, anytime" and "open to learn with [the] assumption that others' ideas will ultimately make a product or service better" (p. 245);  to be "curious" by looking "for unobvious patterns," discovering "new possibilities," and looking for "analogies and metaphors" (p. 244); and to "facilitate connections and expect collaboration" (p. 244).  

Competencies 16, 17, and 18 encompass the action research and inquiry cycle and a set of knowledge, skills, and dispositions that develop as habits of mind. These habits of mind are critical not only to applied site-based research and problem solving, but to decision making and a wide spectrum of other leadership behaviors as well.

References

Argyris, C. (1977). Double loop learning in organizations. Harvard Business Review,115-125.

Argyris, C., & Schön, D. A. (1974). Theory in practice: Increasing professional effectiveness. San Francisco: Jossey-Bass.

Busche, G. R. (2001). Clear leadership: How outstanding leaders make themselves understood, cut through mush, and help everyone get real at work. Mountain View, CA: Davies-Black Publishing.

Kim, D. H. (1999). Introduction to systems thinking. Waltham, MA: Pegasus Communications.

Kotter, J. P. (1996). Leading change. Boston, MA: Harvard Business School Press.

Kuhne, G., & Quigley, B. A. (1997, Spring). Understanding and using action research in practice settings. New Directions for Adult and Continuing Education, 73, 23-40.

Lafly, A.G., & Charan, R. (2008). The game-changer: How you can drive revenue and profit growth with innovation. New York: Crown Publishing Group.

Senge, P. (1994). The fifth discipline: The art and practice of the learning organization. New York: Doubleday.

Stringer, E. (2004). Action research in education. Upper Saddle River, NJ: Pearson Education, Inc.