What is Qualitative Program Evaluation?
What is program evaluation?
Program evaluation is the systematic assessment of a program to determine its efficiency and effectiveness and make recommendations to improve its outcomes. It involves the systematic collection and analysis of data about a specific program to determine whether it successfully achieve its intended objectives.
Qualitative program evaluation employs social science research methodologies to evaluate the processes and outcomes of a specific project. It is carried out ethically, and the processes followed are outlined to ensure the results are replicable by future researchers. Qualitative research program evaluation seeks to provide answers to important questions like:
What is the outcome of the program under evaluation?
Is the program worthwhile?
Did the program produce unintended consequences?
Are there set procedures for the program?
Were the procedures followed?
Are there better ways to achieve the desired outcomes?
What is a program?
A program can be described as a set of specific activities designed to achieve a specific purpose. Programs have quantifiable goals and objectives and usually follow preset procedures.
What are the procedures involved in program evaluation?
The following are the processes involved in program evaluation:
Determine the purpose of the evaluation
The first step in program evaluation is to clarify what you hope to achieve with the evaluation. You have to answer questions like:
Will the evaluation be conducted during the implementation stage of the program or after implementation?
Is the purpose of the evaluation to determine the worth of the program, account for outside stakeholders, or improve the program?
Will the evaluation be completed at once or in multiple phases?
Develop a logic model
A logic model, sometimes called program theory, provides a clear description of all the components of the programs and their expected outcomes. It explains how the program should work and why it should produce the stated outcomes.
Collect data
After clarifying the purpose of the evaluation and deciding the evaluation approach, the next step is to collect data about the program. The main methods for collecting data for qualitative program evaluation are:
Document analysis - Data collection in qualitative research program evaluation starts with collecting and reviewing documents about the program.
Interviews - You can collect data by conducting a one-on-one interview with people who have first-hand knowledge about the program.
Surveys - These are a set of closed-end questions that the researcher asks everyone in the target population.
Observations - You can also collect information by observing the target group to learn more about the program. Learn more about qualitative observation here.
Focus groups - This involves a structured discussion with a limited number of persons in your target groups.
Analyze the data
Data analysis in qualitative program evaluation involves the review of the data collected during and after the fieldwork. You start the analysis by grouping the collected data to identify recurring themes and patterns. After this, you interpret the data.
Better still, you can analyze and interpret your data faster, save yourself the stress of dealing with large amounts of qualitative data, and gain better insights into the program of interest by using Delve qualitative data analysis tool.
Present report
The last stage of qualitative research program evaluation is the presentation of your findings to the participants and other interested stakeholders with the hope that the program managers will implement the results and recommendations.
Program Evaluation Example
Here is a Program evaluation example to help you understand how researchers use it to assess projects.
In Minney, Garcia, Altobelli, Perez-Brena, and Blunk’s paper on social-emotional learning and evaluation in after-school care, the researchers evaluated the effectiveness of an after-school program that incorporated social-emotional learning (SEL) into its curriculum.
The aim of the evaluation was to gauge the effectiveness of the after-school program and determine if there is a need to make changes to it.
After clarifying the purpose of the evaluation, the researchers collected data by conducting a pre and post-program survey of 191 children aged 4 to 11 years old. Students aged eight years and above completed the survey themselves, while those below eight were individually interviewed.
The researchers analyzed the collected data, and the results revealed that after-school programs were effective in developing SEL skills in children.
Types of program evaluation research
The most common types of program evaluation research are:
Monitoring evaluation
This type of evaluation is conducted to determine the extent to which a program reaches its intended target group. It also evaluates the processes involved in the program to ascertain whether it was conducted according to the set procedures.
Impact evaluation
Researchers use impact evaluation to assess the extent to which a program produces the intended outcomes.
Formative evaluation
These are evaluations conducted with the aim of improving the program of interest. It is usually used when new programs are developed. The formative evaluation results are reported internally to enable managers to improve the program.
Summative evaluation
Summative evaluation is the assessment of the effectiveness and efficiency of a program. It is the most important type of program evaluation research. Researchers using mixed methods program evaluation usually combine summative evaluation with others to determine a program’s worth.
Benefits of program evaluation research
Some of the benefits of program evaluation include:
Program evaluation is used to measure the effectiveness of social programs and determine whether it is worth it or not.
The results from program evaluation can be used to improve a program and achieve better outcomes.
Program evaluation helps public and private organizations to justify the use of their limited resources.
Program evaluation provides insights into the project of interest. It helps stakeholders understand the strengths and weaknesses of a program.
Qualitative analysis doesn't have to be overwhelming
Take Delve's free online course to learn how to find themes and patterns in your qualitative data. Get started here.
Try Delve Qualitative Analysis Tool
The Delve qualitative analysis tool can streamline the code for your qualitative data and help you to practice peer debriefing. Start a free trial of Delve today!
References
Metz, Allison. (2007). Why conduct a program evaluation? Five reasons why evaluation can help out-of-school time program. Child Trends.
Hogan, R. (2007). The Historical Development of Program Evaluation: Exploring the Past and Present. Online Journal of Workforce Education and Development.
Minney, D; Garcia, J; Altobelli, J; Perez-Brena, N; Blunk, E. (2019). Social-Emotional Learning and Evaluation in After-School Care: A Working Model. Journal of Youth Development.
Research for Action. (2000). A guide for program evaluation.
