September 2008

<< back to issue

Evaluation's Role in Creating Better Programs

by Sharon Charnell Gherman, President/CEO of The Funding Exchange, Inc.

Evaluation doesn’t need to be a mysterious topic, but it’s certainly a topic that people are curious about. It seems there’s a little fear about “really finding out the nitty-gritty truth” about a project we’ve invested so much of ourselves into -- what will we do if an evaluation shows we haven’t done a good job? Over the next two Exchange issues, we’ll look at evaluation and do our best to de-bunk some of the mysteries surrounding evaluations!

Do I Need to Hire an Evaluator?

The correct answer is – maybe. Here are some criteria to help you decide:

You can probably complete the evaluation internally if:

• Your grantor is a foundation or other private source who doesn’t require a “formal” evaluation. (In this case, a final written report is still good protocol even if it’s not a requirement… It establishes that you appreciate the investment the grantor has made in your organization, and that you desire to maintain and cultivate that relationship.)
  
• Your evaluation is intended to simply determine whether or not you met the requirements of the grant; data collection procedures to document your accomplishments were designed at the beginning of the project; and these procedures were faithfully followed throughout the grant or project period.
  
• You want to simply step back with your project team, see if there are changes you can make to improve your program, and brainstorm solutions.

You probably need to hire an outside evaluator if:

• Your grantor requires an independent evaluation (this is most common in federal and some state grants)
• You need to present the evaluation information publicly (on your web site or delivered to the public or government officials in some way)
• Your project’s subject is highly charged emotionally and an internal evaluation won’t be acceptable to all parties, or
• You’re experiencing problems in the program or project and want an objective assessment of how to improve it or what to do next.

Components of a Good Evaluation

Contrary to what you may have been told, a useful evaluation usually doesn’t require a professional with a Ph.D. in statistics. It does, however, require common sense, organizational skills, and a “big picture” viewpoint.

A good evaluation, designed and applied correctly, will give you important information about the ‘temperature and pulse’ of that program. There are two components of a useful evaluation you’ll want to consider:

• the data, and
• what that data means about your program.

The Data

Your information will only be as good as your data, and data collection starts as you design your program.

The best time to hire a project evaluator is at the very beginning -- as you plan and write the grant proposal. Brought in as a member of the team during the planning stages, an independent evaluator can design targeted tools to provide meaningful program data and feedback. Using a professional evaluator from the beginning can save you money on the final evaluation and often on the overall program. (Yes, the Funding Exchange provides evaluation services, but many programs and grants don’t require an outside evaluator, as discussed above.)

When designing tools to gather data for an internal evaluation, ask yourself the same questions a professional evaluator would ask:

• If this project is highly successful, what will it look like? Think about how communication would flow; how programs would be delivered; how the program would create or attract the funds needed to sustain itself in the future; how your intended audience would respond; and how you would get the word out about your program. Once you have these pictures clearly defined in your head, ask yourself:
  
• If asked, how would we PROVE we’d been successful in these areas? The answers to this question will determine what “instruments” you’ll use to collect data, which simply means the tools, documents and processes with which you’ll collect information during your project.

Next month, we’ll consider some of the evaluation tools you can use to collect data, what collected data can mean for your project, and how to use it effectively.