Logic Model Components
Evaluation Questions
Indicators Targets** Data Sources Data Collection Data Analysis
Increased student participation in technology-based ALOs
How often do students participate in technology-based ALOs?
Increased number and percent of students participating in technology-based ALOs (within and outside of the regular school day)
Using Year 1 survey data as a baseline –by the end of
Year 2 at least <>%,
Year 3 at least <>%,
Year 4 at least <>%, Year 5 at least <>%, and
Year 6 at least <>% of students will participate in technology-based ALOs.
Participation logs
Student surveys
Baseline student survey administered during Year 1
Student survey administered annually (and electronically) to all students
Survey data analyzed using frequency distributions and basic descriptive statistics
Changes over time analyzed using significance testing
Results disaggregated by type of ALO (e.g., home-based, outside of school day)
Increased student exposure to technology-based ALOs
To what extent and in what ways do students participate in technology-based ALOs?
Increased number and percentage of students who have increased their overall learning time through technology-based ALOs (Note: investigate the nature of use, i.e., replacing an ALO or adding a new ALO)
Using Year 1 survey data as a baseline – students will have increased their learning time through technology-based ALOs by the end of
Year 2 at least <>%,
Year 3 at least <>%,
Year 4 at least <>%, Year 5 at least <>%, and
Year 6 at least <>%.
Participation logs
Student surveys
Baseline student survey administered during Year 1
Student survey administered annually (and electronically) to all students
Survey data analyzed using frequency distributions and basic descriptive statistics
Changes over time analyzed using significance testing
Results disaggregated by type of ALO (e.g., home-based, outside of school day) and nature of the ALO (e.g., if the student replaced an ALO with a technology- based ALO)
140
Logic Model Components
Evaluation Questions
Indicators Targets** Data Sources Data Collection Data Analysis
Improved long- term hardware/ software acquisition planning
In what ways has hardware/software acquisition planning improved?
Increased number of teachers and technology staff who report improved acquisition planning
By the end of
Year 2 at least <>%,
Year 3 at least <>%,
Year 4 at least <>%, Year 5 at least <>%, and
Year 6 at least <>% of teachers/staff will report improved planning.
Interviews with technology staff
Teacher surveys
Interviews conducted annually
Teacher surveys administered annually
Interviews summarized and analyzed by theme
Survey data analyzed using frequency distributions and basic descriptive statistics
Increased availability of appropriate and necessary technology
To what extent has the availability of appropriate and necessary technology improved?
Increased number of teachers and technology staff who report improved availability of necessary technology
By the end of
Year 2 at least <>%,
Year 3 at least <>%,
Year 4 at least <>%, Year 5 at least <>%, and
Year 6 at least <>% of teachers/staff will report improved availability.
Interviews with technology staff
Teacher surveys
Interviews conducted annually
Teacher surveys administered annually
Interviews summarized and analyzed by theme
Survey data analyzed using frequency distributions and basic descriptive statistics
141
Increased student learning
To what extent has technology contributed to student learning as measured by local assessments?
To what extent did learning outcomes vary by school and classroom technology use?
(Not currently evaluated: In what ways have technology-based ALOs contributed to student learning?)
Increased scores on local assessments
Increased correlation between local assessment scores and technology implementation
Students in schools with high rubric scores will have higher gains on local assessments than students in schools with lower rubric scores, and the difference will be statistically significant.
Local assessments
NowPLAN-T rubric scores
Local assessment data collected quarterly
Baseline rubric data collected at start of Year 1
Rubric data collected quarterly (for each school) through teacher surveys (all classrooms) and classroom observations (case study classrooms)
Correlational analyses between local assessment scores and NowPLAN-T rubric scores
T-test of mean test scores pre-NowPLAN-T and each academic year post-NowPLAN-T
Results disaggregated by school, grade level, gender, race/ethnicity, special education status, and English language proficiency
**Targets will be updated once baselines are measured.
142
Table 18: NowPLAN-T Evaluation Matrix—Long-Term Goals
Logic Model Component
Evaluation Questions
Indicators Targets Data Source Data Collection Data Analysis
Improved student achievement
To what extent did the district’s technology plan contribute to student achievement?
→
→
To what extent did student learning improve after NowPLAN-T was implemented?
To what extent did learning outcomes vary by school and classroom technology use?
Increased scores on statewide standards-based achievement assessments
Increased correlation between achievement scores and NowPLAN-T implementation
Within 2 years, the correlation between improvement in student scores on the statewide standards-based achievement tests and scores on the NowPLAN-T technology rubrics will be statistically significant.
Students in schools (and classrooms) with high rubric scores will have higher achievement gains than students in schools (and classrooms) with lower rubric scores, and the difference will be statistically significant.
State test scores in reading and math (as well as science and writing)
NowPLAN-T rubric scores
