Table 16: NowPLAN-T Evaluation Matrix—Strategies and Activities/Initial Implementation
Logic Model Components
Evaluation Questions*
Indicators Targets Data Sources Data Collection Data Analysis
Districtwide technology curriculum/revised technology curriculum
Student technology orientation/revised student orientation plan
Technology professional development model/revised professional development model
Teacher technology orientation/revised teacher orientation plan
Technology-based communications/revi sed districtwide protocol for technology-based communication
Hardware/software acquisition plan/revised protocol for hardware/ software acquisitions
In what ways were the districtwide technology curriculum, student technology orientation plan, technology professional development model, teacher technology orientation plan, districtwide protocol for technology-based communication, and protocol for hardware/soft-ware acquisitions revised?
Creation of a revised districtwide technology curriculum, student technology orientation plan, technology professional development model, teacher technology orientation plan, districtwide protocol for technology-based communication, and protocol for hardware/software acquisitions
By the end of Year 1, a revised districtwide hardware/software acquisition plan and districtwide protocol for technology-based communication will have been created.
By the end of Year 2, a revised districtwide technology curriculum and technology professional development model will have been created.
By the end of Year 3, revised student and teacher technology orientation plans will have been developed.
Technology records
Document analysis
Meeting minutes
Interviews with technology personnel
Technology records and documents, as well as meeting minutes, reviewed monthly
Interviews with technology personnel conducted quarterly
Documents/minutes summarized for evidence of implementation
Interview data summarized, and if warranted, analyzed for themes
134
Logic Model Components
Evaluation Questions*
Indicators Targets Data Sources Data Collection Data Analysis
Technology-based additional learning opportunities (ALOs)/improved identification of ALOs
To what extent were technology-based ALOs identified?
Increasing number of technology-based ALOs identified
By the end of Year 2, a process to identify technology- based ALOs will be operational.
Interviews with technology personnel
Interviews with technology personnel conducted quarterly
Interview data summarized, and if warranted, analyzed for themes
*Note: Logic model components are combined in the evaluation questions but will be disaggregated in the data analysis.
Table 17: NowPLAN-T Evaluation Matrix—Early/Short-Term and Intermediate Objectives
Logic Model Components
Evaluation Questions
Indicators Targets** Data Sources Data Collection Data Analysis
Improved integration of technology into the core curriculum
How was technology integrated into the core curriculum?
Increased number of schools with improved classroom integration of technology
By the end of Year 4, at least <>% of schools will score a 3 or better on the NowPLAN-T rubrics.
By the end of Year 6, <>% will score a 4 out of 4 on the NowPLAN- T rubrics.
NowPLAN-T rubrics
Teacher surveys
Baseline rubric data collected at start of Year 1
Rubric data collected quarterly (for each school), through teacher surveys (all classrooms) and classroom observations (case study classrooms)
Rubric data analyzed by frequency distributions of rubric scores
Changes over time analyzed using significance testing
135
Logic Model Components
Evaluation Questions
Indicators Targets** Data Sources Data Collection Data Analysis
Improved classroom integration of technology to enhance instruction
In what ways and to what extent was technology integrated into classroom instruction?
Increased number of schools with improved curricular integration of technology
By the end of Year 4, at least <>% of schools will score a 3 or better on the NowPLAN-T rubrics.
By the end of Year 6, <>% will score a 4 out of 4 on the NowPLAN- T rubrics.
NowPLAN-T rubrics
Teacher surveys
Baseline rubric data collected at start of Year 1
Rubric data collected quarterly (for each school), through teacher surveys (all classrooms) and classroom observations (case study classrooms)
Rubric data analyzed by frequency distributions of rubric scores
Changes over time analyzed using significance testing
Improved student understanding of technology availability and appropriate use
To what extent do students understand the technology available to them, as well as its appropriate use?
Increased number of students who have an understanding of available technology
Using Year 1 survey data as a baseline –by the end of
Year 4 at least <>%, Year 5 at least <>%, and
Year 6 at least <>% of students will appropriately understand and use available technology to enhance learning.
Student surveys
Baseline student survey administered during Year 1
Student survey administered annually (and electronically) to all students
Survey data analyzed using frequency distributions and basic descriptive statistics
Changes over time analyzed using significance testing
136
Logic Model Components
Evaluation Questions
Indicators Targets** Data Sources Data Collection Data Analysis
Increased student use of technology to enhance learning
In what ways and how often do students use technology for learning (in the classroom and at home)?
Increased number of students who use technology in their learning activities
Using Year 1 survey data as a baseline –by the end of
Year 4 at least <>%, Year 5 at least <>%, and
Year 6 at least <>% of students will appropriately understand and use available technology to enhance learning.
Student surveys
Baseline student survey administered during Year 1
Student survey administered annually (and electronically) to all students
Survey data analyzed using frequency distributions and basic descriptive statistics
Changes over time analyzed using significance testing
Improved teacher understanding of technology availability
To what extent do teachers understand the technology available to them?
Increased number of schools with improved teacher understanding of the technology available to them
By the end of Year 4, at least <>% of schools will score a 3 or better on the NowPLAN-T rubrics, and will demonstrate an understanding of technology (as measured by the teacher survey).
By the end of Year 6, <>% will score a 4 out of 4 on the NowPLAN- T rubrics, and will demonstrate an understanding of technology (as measured by the teacher survey).
Teacher surveys
NowPLAN-T rubrics
Baseline teacher survey data and rubric data collected at start of Year 1
Rubric data collected quarterly (for each school), through teacher surveys (all classrooms) and classroom observations (case study classrooms)
Survey data analyzed through basic descriptive statistics and frequency distributions
Rubric data analyzed by frequency distributions of rubric scores
Changes over time analyzed using significance testing
137
Logic Model Components
Evaluation Questions
Indicators Targets** Data Sources Data Collection Data Analysis
Increased use of technology by teachers
To what extent do teachers use technology to improve student learning?
Increased number of schools with increased teacher use of technology to improve student learning
By the end of Year 4, at least <>% of schools will score a 3 or better on t he Now PLAN-T rubri cs, and will demonstrate an understandi ng of te chnology (as measured by the tea cher survey).
By the end of Year 6, <>% will score a 4 out of 4 on the NowPLAN-T rubrics, and will de monstrate an understa nding of technol ogy (as measure d by the teacher survey).
Teacher surveys
NowPLAN-T r ubrics
Baseline tea cher survey data and rubri c data collected at start of Y ear 1
Rubri c data colle cted quarterly (for each school ), through teacher surveys (all classrooms) a nd classroom observations (ca se study classrooms)
Rubri c data analyzed by frequency distributions of rubri c scores
Change s over time analyze d usi ng signifi cance testing
Improved teacher use of technology
In what ways do teachers use technology to improve student learning?
Increased number of schools with improved teacher use of technology to improve student learning
By the end of Year 4, at least <>% of schools will score a 3 or better on t he Now PLAN-T rubri cs, and will demonstrate an understandi ng of te chnology (as measured by the tea cher survey).
By the end of Year 6, <>% will score a 4 out of 4 on the NowPLAN-T rubrics, and will de monstrate an understa nding of technol ogy (as measure d by the teacher survey).
Teacher surveys
NowPLAN-T r ubrics
Baseline tea cher survey data and rubri c data collected at start of Y ear 1
Rubri c data colle cted quarterly (for each school ), through teacher surveys (all classrooms) a nd classroom observations (ca se study classrooms)
Rubri c data analyzed by frequency distributions of rubri c scores
Change s over time analyze d usi ng signifi cance testing
Improved communication with families
To what extent has communication with families improved?
Increased number of families who report improved communication
Using Year 1 survey data as a baseline –by the end of
Year 4 at least <>%,
Year 5 at least <>%, and
Year 6 at least <>% of parents will report improved communication.
Parent survey Baseline parent survey administered during Year 1
Parent survey administered annually (and electronically) to all families
Survey data analyzed using frequency distributions and basic descriptive statistics
Changes over time analyzed using significance testing
138
Logic Model Components
Evaluation Questions
Indicators Targets** Data Sources Data Collection Data Analysis
Increased parental involvement
In what ways has technology contributed to parental involvement?
Increased number of parents for whom communication has contributed to increased parental involvement
Using Year 1 survey data as a baseline –by the end of
Year 4 at least <>%,
Year 5 at least <>%, and
Year 6 at least <>% of parents will report that communication has contributed to their increased parent involvement.
Parent survey Baseline parent survey administered during Year 1
Parent survey administered annually (and electronically) to all families
Survey data analyzed using frequency distributions and basic descriptive statistics
Changes over time analyzed using significance testing
Increased availability of technology-based additional learning opportunities (ALOs)
To what extent are technology-based ALOs available to students?
Increased number of technology-based ALOs offered to students
Each year, the number of technology-based ALOs offered to students will increase by 20% (e.g., online courses, supplemental programs).
Technology records
Technology records and participation logs reviewed quarterly
Technology records and participation logs reviewed for evidence of ALOs availability
139
