Assignment 1

The program evaluation that I evaluated is titled "Youth Financial Education" and was conducted by the University of Wisconsin.

Citation: University of Wisconsin-Extension, Cooperative Extension (2008). Program evaluation report: Youth financial education. Madison, WI: UW-Extension, Program Development and Evaluation. http://www.uwex.edu/ces/pdande/evaluation/evalstudies.html

This evaluation is a summative, outcomes-based evaluation that aims to answer the following questions:

1) To what extent are desired outcomes occuring for youth participants related to changes in knowledge, skills, confidence, motivation, and behaviour in their financial practices?

2) What do youth view as the value of this type of reality/experiential program?

3) What suggestions and lessons can be learned to improve these reality programs?

The strengths of this evaluation include its transparency in regards to information and the authors diligence in following format protocol as proposed by Carter McNamara ( http://www.managementhelp.org/evaluatn/fnl_eval.htm).
Its logic model also clearly outline the input, output, and outcomes.

This evaluation has many strengths.

The evaluation was conducted by a third party, rather than by the organizatons (schools, program creators), thus giving the evaluation more credibility.

Furthermore, the authors of this evaluation have ensured that all 4 levels of evaluation (reactions and feelings, learning, change in skills, effectiveness) were explored and reported.

The selection of an outcomes based evaluation is appropriate because it evaluates whether the desired outcome ( if youth financial education does indeed make youth more knowledgeable about finances). The Y.F.E. evaluation is based on the increased knowledge, perceptions/attitudes, or skills of the participants.

The Y.F.E. evaluation uses questionnaire surveys with open and close ended items to elicit this information, which is an instrument that is used in many studies wishing to explore participant knowledge, perceptions, and known skills.

Additionally, the authors have included participants in the evaluation which some suggest is essential to the evaluation process ( Michael Scriven ).

The evaluators have used a variety of data collection methods, such as questionnaires with open and closed form questions. This strengthen the results. The evaluators also did a questionnaire-survey nine months after the initial questionnaire-surveys were given.

As there are strengths there are also weaknesses.First, the instrument used for data collection ( questionnaire survey) is not the most reliable instrument. There is a possibility that participants did not give careful feedback. This is more likely since participants were high school and middle school aged and the questions dealt with complex financial concepts (credit).

Second, the questionnaires were quite lengthy which may have caused participants to rush through it.
Third, the use of a questionnaire/survey may not allow participants to express their full opinions. Lastly and most importantly this evaluation does not include evidence of actual behaviour, rather it focuses on participants' perspectives. This leaves us with a crucial question: How are we to know if the program is actually effective?

While this was a very interesting evaluation to read through because I learned so much about how an evaluation should look and be conducted, I am left with the feeling that an additional evaluation must be undertaken to truly understand the outcomes of Youth Financial Education. Perhaps an evaluation where participants' finances are charted over the course of five years would offer hard evidence of Youth Financial Education outcomes.