You are currently not Logged In
Seal of Accountability

Sample Program Evaluation

Example of a simplified evaluation

This is a report of how CCCC did a program review of a small, uncomplicated program. It is not a template, but is an illustration of how one organization evaluated a program using a simplified approach.

Since this was the first opportunity to do a formal program evaluation, staff decided to pick one that was uncomplicated and small. The Certified Stewardship Counselor program was selected because there was agreement that it was lagging and needed attention.

This program had run for a number of years but with limited results. Due to personnel changes and competing priorities, it was ‘orphaned’ for the three years prior to the review. The program had 13 people with the CSC designation and another 6 working towards it.

The evaluation committee included:

  • the CEO
  • the senior leadership team of three VP’s
  • one manager

The committee was assembled by including those who were responsible for the program from a policy standpoint, an administrative standpoint and an operational standpoint. The other VP and the manager were volunteers who wanted to be on the committee.

Three meetings were held.

  • Meeting 1
    • The meeting opened with a roundtable survey to get everyone’s thoughts about the program.
    • It was agreed the program was operating below the original expectations when it was founded. It was also agreed that it fell far short of any reasonable success criteria that might be established today.
    • It was decided not to do a full logic model because the evaluation would be a summative, not formative evaluation. A summative evaluation determines whether to continue or discontinue a program while a formative evaluation determines the improvements that could be made. The group would do a partial logic model instead (problem and assumptions).
    • The group discussed the problem the program addresses and agreed on a problem statement.
    • The problem was then analyzed to discover the major assumptions that had been made when defining the problem.
    • It was decided to test the major assumptions. Two people were tasked with contacting 20 individuals representing various stakeholders, including the graduates, churches, denominational offices and independent agencies. The other three people were tasked with doing a comparative analysis of all the solutions to the problem that are presently available. The purpose was to check the validity of our assumptions about both need and uniqueness of the program.
  • Meeting 2
    • This meeting took place three weeks after the first.
    • The comparative survey was reviewed. There were alternatives, both Christian and non-Christian, for the program that addressed all the needs that had been identified.
    • The interviews revealed that certification is not valued, but content is. People valued the Christian perspective that CCCC offers on the technical issues of fundraising. People identified a number of products and services they wanted from CCCC, but which could be delivered without the certification program.
    • The group discussed the CCCC Ends statements and the new mission statement the staff had developed a few months prior. This discussion brought out the fact that all other services of CCCC are directed to the ministry organization, rather than individual staff members.
    • The group discussed what it would take in terms of resources (particularly staff time) to move the program to where it could be considered successful.
    • After discussion, it was agreed that there was an educational component to the program which was in demand, but that the designation was not necessary.
    • It was agreed that people would think about the question, “What if the CSC program was discontinued” and its implications. A tentative decision was made to discontinue the program.
  • Meeting 3
    • This meeting took place about three weeks later.
    • A document was circulated that outlined numerous ideas for new products and services related to stewardship that could be done without the certification program.
    • Another document was presented which detailed all the action steps that would be needed to close the program.
    • After discussion, it was decided to cancel the CSC program.
    • A plan was developed to look after the existing graduates and applicants and ensure they were treated very fairly.
    • The group affirmed the need for the next person hired for the technical staff to be a stewardship specialist. The group agreed to a timeline for introducing new stewardship products and services.

The program was cancelled without a single complaint and CCCC is progressing with the ideas that had been suggested.

One of the Standards that CCCC’s certified members adhere to is that they do evaluate their programs. If CCCC was asked to verify to an outside party that this evaluation took place, the following records would be available as supporting evidence:

  • A final report (such as this one)
  • Minutes of the meetings
  • Research that was presented at the meetings
  • Action plans that came out of the meetings
  • The interview guide and notes from each interview

Return to Standard 8

 
 

"CCCC is uniquely positioned to raise awareness about integrity in ministry by giving or withholding the Seal of Accountability and holding members to the highest possible standards."

- Billy Graham