The Best Evaluation and Assessment Practices

The Best Evaluation and Assessment Practices For Non-formal Pastor-Trainers

Background

Leadership Resources International and MasterWorks Foundation hosted a 24-hour, invitation-only summit in November 2009 in Chicago on the subject, “Best Practices” for Non-formal Pastoral Training Agencies. The overall goal was to help practitioners, through dialogue with informed and experienced peers, identify ways (best practices) to more effectively evaluate their training ministries as they seek to equip shepherds for the Church of our Lord Jesus Christ.

The 16 participants were (and are) leaders in the field—either principals from non-formal pastoral training agencies or representatives from foundations that have a particular passion for leadership development and pastoral training. The training agencies represented included BILD International, Entrust, Russian Ministries, Africa Ministries Network (AFMIN), Leader Source, Development Associates International (DAI) and Leadership Resources International (LRI). The foundations represented were: James C. Blankenmeyer Foundation, Cornerstone Trust, First Fruits Inc., Harry J. Lloyd Trust, Maclellan Family Foundations, and MasterWorks Foundation.

In addition, TOPIC (Trainers of Pastors International Coalition) was represented. TOPIC, as a global coalition of several hundred non-formal pastoral trainers and their organizations, has a keen interest in helping establish and promote “industry” best practices [www.TOPIC.US]. Overseas Council, an organization that works with Bible institutes and seminaries worldwide, also participated.

Focus of the Conversation

Through a collaborative process, the summit participants agreed to focus the conversation on pastoral training evaluation—identifying the best assessment and improvement practices. (The focus of this roundtable was on developing pastoral leaders rather than the broader topic of leadership development.)

Questions and issues raised on the subject of evaluation and assessment included…

  • What is a biblical basis for evaluation? How did Jesus evaluate?
  • Integrating evaluation into the training methodology (vs. as an after-thought).
  • Inculcating a continuous improvement mindset—“How are we doing? And, how can we get better?”
  • Maintaining the quality of training, especially to the 2nd/3rd generation.
  • Appropriate ways to measure different types of outcomes, such as…
    • Knowledge/theology/wisdom/worldview
    • Skills/competencies
    • Character/integrity/humility/“on-going brokenness”
    • Love/heart/passion/will
    • Spiritual formation/habits
  • What can we rightfully “take credit” for? How certain are we that our training is (solely) responsible for the changes that we observe? (The problem of multiple explanatory variables.)
  • Obtaining “real” numbers (vs. “evangelistically-speaking”).
  • Appropriate use of anecdotes.
  • Cost/benefit—are donations being “leveraged?”
  • How do donor goals mesh with agency goals? What role should donors play in establishing outcomes and measurement systems?

The day of dialogue was marked by attentive listening with lots of give and take, vulnerability and collegiality. Often, there were more questions than answers. Nevertheless, we were able to identify 15 “best practices” for pastor trainers and pastoral training organizations as we evaluate our training efforts.

Summary of the Findings

1. Evaluation as part of the organizational DNA

All agreed that evaluation and assessment were critical values that training organizations need to embrace.

“Best practice” pastoral training organizations will…

  • Create and maintain a “culture of reflection,” a community of self-reflecting leaders.
  • Develop a “holy discontent” that results in a commitment to continuous improvement.
  • Self-assess their ministry from start to finish, including strategy, design, process, and implementation.
  • Weave evaluation throughout the teaching and training times, and not simply as a brief “add on” at the end.

2. Establishing clear goals and outcomes

All agreed that effective evaluation is dependent on establishing clear goals. However, differences emerged regarding the appropriate scope of the goals. Namely, should the goals we pursue be…

Comprehensive or narrow?

The changes that training organizations seek to stimulate might be visualized as concentric circles: the more immediate changes we seek (the inner circles) often relate to the learner’s knowledge, competency, and Christ-likeness. Broader, more holistic spheres of change might include expanded organizational capacity, multiplying churches and transformed communities. On one level, of course, we want our training to ultimately result in these larger changes. On the other hand, the broader the changes, the greater the difficulty we face directly connecting our training to those changes. Big, overarching outcomes often have multiple explanatory variables. By way of example: to what extent was the significant church growth that we observed the direct result of our training? And/or, to what extent was it the result of many other influences in church and society (e.g. a migration influx, a widely-known miraculous healing, the arrival of an anointed evangelist, improved formal training institution programs, etc.)?

Two different organizations participating in the roundtable shared different approaches that they have taken on this question of comprehensive vs. narrow scope. LeaderSource has developed a comprehensive survey-based leader evaluation system called 5C CheckPoint which assesses: Christ, Character, Community, Calling, and Competency. They have created 18 indicators for each “C” with questions behind each indicator. The model includes both self-evaluation as well as evaluation by others, enabling an in-depth, holistic evaluation of the leader. [www.5ccheckpoint.com]

Leadership Resources International (LRI) embraces a narrower approach to evaluation tied to a set of tightly defined goals. LRI training goals are to teach pastors how to study, teach and preach God’s Word with God’s heart, and then multiply that training to other pastors and leaders. A group of 10-15 pastors form a given training cohort which meets with an LRI team multiple times over a 4-year period. In this highly-relational setting, evaluation actually becomes a part of the teaching and mentoring process. The combination of practice, interaction, oral and written reports, intentional observations, statistical tracking, and self-evaluation enables LRI to encourage the pastors while, at the same time, track their progress over several years. LRI evaluation focuses on those competencies and heart issues which are the goals of the training as well as on the multiplication of the training (to the 2nd & 3rd generations) and resulting church growth.

Longer vs. shorter timeframes?

All pastoral training organizations want to achieve long-term results that far outlive the actual training sessions. But at what point do we make a definitive assessment of our training program–after 1 year, 5 years, 1 generation, or ??? The longer the timeframe, the greater is the confounding effect of multiple explanatory variables.

The twin issues of scope and timeframe merged in a discussion of outputs and outcomes. “Outputs” were defined as the quantifiable results that we seek. Outputs tend to be more immediate and more observable—knowledge gained and competencies demonstrated. But should we be satisfied with outputs? Don’t these outputs actually serve a higher cause, namely, “outcomes?” Outcomes are often more qualitative and anecdotal (e.g. heart changes, healed relationships, spiritual growth, character transformation, organizational impact, healthy multiplying churches, transformed communities), yet more significant in that they point to deeper and longer-lasting impacts. Outcomes are the profound, transformative changes which emerge over a period of years or even generations.

Evaluating broadly-defined and long-term outcomes can lead to a highly complex evaluation design. Yet, for an evaluation plan to be sustainable, especially at the field level where the information is collected, the data collection tools and procedures need to be kept simple. The challenges of geography, security, lack of technology, cultural and language barriers, and literacy challenges all argue for a “keep it simple” approach to evaluation. Clearly a tension exists between the twin desires for comprehensiveness and simplicity.

How then do we evaluate outcomes and the relationship of our training to those outcomes?

“Best practice” pastoral training organizations will…

  • Clearly define their goals in terms of both outputs (i.e., near-term learning including skill development and utilization) and outcomes (long-term, broad transformations sought).
  • Not be content to only evaluate inputs (i.e., what is taught,) but outputs (do they get it, can they do it?) and outcomes (how is the organization, church and community impacted?).
  • Stay focused on evaluating long-term outcomes, recognizing that while long-term outcomes are more difficult to assess accurately, they are, in fact, the ultimate goal and justification for the training effort.
  • Conduct longitudinal research where appropriate, obtaining multiple measurements along a timeline. When appropriate, begin with baseline/pre- measures.
  • Keep the evaluation plan as simple as possible, taking into account data gathering challenges in the field.
  • Evaluate with humility and honesty, keenly aware that God is using many other people, organizations, and situations to accomplish His work.

3. Maintaining relational integrity

Who determines what is to be learned? Who defines the teaching goals and corresponding changes sought—the teacher or the learner? Pastoral training organizations have a core set of competencies and a particular ministry calling. At the same time, learners have an understanding of their own training needs and desires.

“Best practice” pastoral training organizations will…

  • dialogue with indigenous learners and leaders with “appreciative inquiry” to align the learners’ training needs with their own organizational training capacities and calling.

Who determines what should be evaluated? Who should do the evaluating? Evaluation may be viewed as a threat by learners, especially in shame-based cultures or oral-oriented societies. In worst case scenarios, evaluation will be seen as a tool of power, control or manipulation. The design of an evaluation plan should carefully reflect relational and “power” dynamics among the major stakeholders. Effective evaluation can and should be undertaken in a highly-relational, non-threatening way. Some of the best evaluation flows out of deep relationships. As one participant noted, “paper flow decreases as relationships increase.”

Evaluation and accountability should cut both ways—both teachers and learners will have valid opinions about the effectiveness of the training. Evaluation is a way for learners to discover what they learned (“how do they know they know?”) and a way for teachers to become better teachers—to discover what about their teaching and methodology was effective and helpful, and what was less so.

“Best practice” pastoral training organizations will…

  • Dialogue with indigenous learners and leaders about the value of evaluation in order to encourage a culture of reflection and build ownership for an appropriate evaluation plan.
  • Design an evaluation plan that includes a learner self-assessment component and other appropriate tools such as “growth portfolios” which document on-going growth and demonstrated competencies.
  • Invite indigenous learners and leaders to evaluate the pastor trainers and the relevancy and effectiveness of the training that they bring.
  • Approach evaluation in a relationally healthy way, sensitive to both cross-cultural realities and relational implications.

Conclusions

Participants identified 15 evaluation and assessment best practices for pastoral trainers and pastoral training agencies. The group clearly recognized the need for more learning in this area and that non-western trainers and training organizations need to be invited into the conversation.

Other pastoral training arenas deserving a best practices forum might include:

  • Best Program Design Practices (e.g. selection of trainees, scalability and replicability)
  • Best Cross-cultural Partnership Practices (creating and maintaining ownership and alignment)
  • Best Curriculum Practices
  • Best Training Practices for Oral-learners
  • Best Pedagogy Practices (training methodologies)
  • Best Administrative Practices

A final area of exploration that generated much interest was the fostering of healthy synergies between the worlds of formal and non-formal training.

Craig Parro
President
Leadership Resources International

Leadership Resources

Leadership Resources exists to equip and encourage pastors around the world to teach God’s word with God’s heart. Learn more about our work, our approach, and see how your church can make a worldwide difference with God's Word.