{"id":5049,"date":"2011-10-14T14:07:16","date_gmt":"2011-10-14T18:07:16","guid":{"rendered":"\/news_blogs\/john\/?p=5049"},"modified":"2022-08-17T13:39:15","modified_gmt":"2022-08-17T17:39:15","slug":"program-evaluation-1-selecting-the-program","status":"publish","type":"post","link":"https:\/\/www.cccc.org\/news_blogs\/john\/2011\/10\/14\/program-evaluation-1-selecting-the-program\/","title":{"rendered":"Program Evaluation 1 – Selecting the Program"},"content":{"rendered":"
This entry is part 1 of 7 in the series Program Evaluation<\/a><\/div><\/div><\/div>\n

It happens. A new person takes a job and improves everything. I’d rather not have my successor have such an easy time! I’ll make improvements myself! One way to do that is to do program evaluations<\/strong>, and we\u2019re starting one at CCCC that you can follow and learn from as it unfolds.<\/p>\n\n\n\n

If you aren’t yet doing program reviews<\/strong> in your ministry, this series in real time should help you get started. The goal of doing a review is to decide whether to continue, discontinue or modify a program. Some people may question the value of doing program evaluations because you can just ‘eye-ball’ a program and get an idea of its performance. Maybe. But you are only looking at what is and not what could be. I don’t want to lead a ministry that is only doing ‘okay;’ I want to lead a ministry that is doing the very best possible! I want to get the whole job done, not just part of it. A formal program review will force you to get outside of your own observations and take an objective look at the program as it is and as it could be.<\/p>\n\n\n\n

This first post will document the organizational context for the review, the rationale for the selection of the program to be reviewed and the high-level research questions that will drive the evaluation.<\/p>\n\n\n\n

Organizational Context<\/h2>\n\n\n\n

Not every organization is ready to do a program evaluation. Employees might be suspicious of ulterior motives. Perhaps previous evaluations were badly done and no one wants to go through that exercise again! There may be little credibility in evaluations if program staff were not involved in designing and conducting the review. If the focus is on assigning blame for a failing program, you have bigger problems than the evaluation to address. Make sure your team knows that taking a hard look at performance is something that is rewarded. No one who is fearful of being blamed and penalized will take an honest look at their own work.<\/p>\n\n\n\n

CCCC has had a good experience with program reviews and there are no special concerns or considerations to be addressed prior to launching this evaluation. CCCC  began doing formal program evaluations when we added a new standard for our Certified Members<\/a> that requires program evaluations to be done. After all, what’s good for the goose is good for the gander! If CCCC says it should be done, then we’d better be able to show how to do it.<\/p>\n\n\n\n

Having no experience with formal <\/em>program evaluations, we started with a small scale review. “Start small and grow with experience,” is what I advise first-time program evaluators. In my opinion, some <\/em>evaluation is always better than no<\/em> evaluation! The result of the first formal evaluation at CCCC was to terminate the Certified Stewardship Counselor<\/em> program. The program was muddling along but not going anywhere, and the research question was simply “Do we kill the program or invest in growing it significantly?” Stopping that program gave us the time to develop the highly successful three day Advancing Stewardship I<\/a><\/em> course.<\/p>\n\n\n\n

We next did an operational evaluation of the Community Trust Fund<\/em>, which was a labour-intensive program. Not only was it not scalable, we wondered if the program was even covering its costs. The result of this evaluation is that the Community Trust Fund<\/em> today is highly automated, donors can open an account to manage their gifts, and the program is paying for itself. (I still manually sign the cheques that go out, and it gives me great pleasure because while signing I think of the ministries who are receiving these donations.) If you have donors who want to give you publicly-traded securities and you aren’t set up for that, send them to the entry page of the Community Trust Fund’s website<\/a>. We’ve done other reviews since these first two.<\/p>\n\n\n\n

Our positive experiences with program evaluations mean that our employees know that doing a review is risk-free for them, so they need not be afraid of them. Every review, including the one that cancelled a program, led to a positive outcome for all staff because we are more effective and efficient and we have made room for new, more promising programs. The point of finding out why something did or did not work is to help us make better use of our resources.<\/p>\n\n\n\n

So given this context, I think we are ready to tackle a major program and ask the hard question, “Is this program still worth doing? Is it effective?”<\/p>\n\n\n\n

Program Selection<\/h2>\n\n\n\n

People generally evaluate their programs on a rotational basis over a period of years. Some might be reviewed every year or even a couple of times per year. There are a few factors that will help you decide which program to evaluate:<\/p>\n\n\n\n

  1. If a program is clearly not performing or is causing you trouble, evaluate it.<\/li>
  2. Programs that consume a lot of resources, whether money or time, should be evaluated.<\/li>
  3. If you simply have questions about a program, if you find yourself wondering if it is worth it, then evaluate it.<\/li>
  4. If a program hasn’t been evaluated in a long time, evaluate it.<\/li><\/ol>\n\n\n\n

    This year we are reviewing the Annual Conference. Here’s why:<\/p>\n\n\n\n