valuating Progress Toward Short-Term and Intermediate Objectives
Evaluating your program’s opportunity to be successful is the initial step toward determining your program’s success. The second category of evaluation questions will address how your program is working. That is, how do you know if your program is on track to meeting its long- term goals? Measuring progress toward short-term and intermediate objectives plays a significant role in determining how your program is working. By examining progress, you can
catch early problems with the program and remediate them before they become critical impediments to your program’s success. Program staff can use interim evaluation findings to plan, shape, and improve the program prior to the evaluation of final outcomes. It is much easier and more cost-effective to uncover problems or issues early in your program’s implementation. Your evaluation should strive to provide program staff with the necessary
information for them to be able to understand the degree to which the program is on course so that they can make midcourse adjustments and refinements as needed.
Your evaluation questions at this stage should focus on your program’s specific short-term and intermediate objectives. However, an overarching evaluation question at this stage might be: To what extent is the program on track to achieving long-term goals? Use your logic model to guide you as you create your evaluation questions pertaining to early and intermediate objectives, just as you used the strategies and activities from your logic model to create your first set of evaluation questions.
While you created your logic model right to left (starting with your long-term goals), it is often easier to craft your evaluation questions left to right. Begin with your early (short-term) objectives and work
your way toward your intermediate objectives, and then long-term goals. Some evaluation questions may address more than one objective, while some objectives may have more than one evaluation question. That is, there does not need to be a one-to-one correspondence between objectives on your logic model and evaluation questions. However, you should have at least one evaluation question that addresses each objective. Later, the evaluation team can prioritize evaluation questions. In doing this, it is possible that you will decide, based on your priorities and resource constraints, not to address certain questions and objectives in your evaluation.
While you created your logic model right to left (starting with your long-term goals), it is often easier to craft your evaluation questions left to right. Begin with your early (short-term) objectives and work your way toward your intermediate objectives, and then long-term goals.
There does not need to be a one-to-one correspondence between objectives and evaluation questions. Some evaluation questions may address more than one objective, while some objectives may have more than one evaluation question.
At this point in your evaluation design, it is important to brainstorm evaluation questions and not be hindered by resource concerns. Prioritizing your questions will come later. When prioritizing your evaluation questions, you will decide based on resource constraints and feasibility which questions your evaluation can adequately address.
27
Next, the E-Team worked with the READ oversight team to create several evaluation questions addressing READ early/short-term objectives and intermediate objectives:
Early/Short-Term and Intermediate Objectives
Evaluation Questions
Increased student use of READ at home (early/short-term).
How often did students receive READ homework assignments? To what extent did students complete READ homework assignments? **Note frequency and duration of use.
Increased teacher use of READ in the classroom (early/short-term).
In what ways and how often did teachers use READ in the classroom with students? **Note frequency, duration, and nature of use.
Increased student exposure to standards-based learning opportunities (early/short-term).
To what extent did students complete READ homework assignments?
How often did teachers use READ in the classroom with students?
Increased availability of standards- based, formative READ assessment data on student reading performance (early/short-term).
How often did teachers access READ student assessment data? **Note frequency and type of access.
Increased teacher use of standards- based READ assessment data (early/short-term).
In what ways did teachers use READ student assessment data?
Increased student interaction during learning (intermediate).
To what extent and how did students interact during classroom instruction when READ was used? **Note frequency and type of interaction.
Improved integration of READ into classroom instruction (intermediate).
In what ways and to what extent did teachers integrate READ into their classroom instruction? **Note the quality with which READ was integrated into classroom instruction by teachers.
Improved differentiation of instruction (intermediate).
In what ways and to what extent did teachers use READ assessment data to plan and differentiate instruction? **Note what data were used and how data were used in instructional planning.
28
Evaluating Progress Toward Long-Term Goals
Finally, a third set of evaluation questions should focus on the program’s long-term goals. While evaluation findings at this stage in your program’s operation can still be used to improve the program’s operation, assessment of long-term goals is typically used for summative decision- making. That is, results from the measurement of progress toward long-term goals are often used to make decisions about whether program funding should be extended and if a program should be continued, expanded, scaled down, or discontinued. Your questions will be specific to your program’s goals, though they should address the following: To what extent does the program work? For whom does the program work best? Under what conditions does the program work best?
Finally, the E-Team and the READ oversight team created evaluation questions addressing READ long-term goals:
Long-Term Goals Evaluation Questions
Increased student engagement in reading.
To what extent and in what ways did READ foster student engagement during reading lessons?
Improved student reading skills.
To what extent did READ improve student learning in reading?
•
•
•
•
•
•
•
•
