A districtwide technology curriculum,
A technology orientation plan for students,
A technology professional development model,
A technology orientation plan for teachers,
Technology resources for communication,
Technology-based additional learning opportunities, and
A districtwide software and hardware acquisition plan,
Student achievement will be improved, AND
Student postsecondary success ultimately will be improved.
128
The external evaluator helped the district people develop a set of assumptions as to why they believed NowPLAN-T strategies would result in improved student achievement. They based these assumptions on research and developed them in consultation with educational technology professionals.
1. A districtwide technology curriculum will lead to a revised technology curriculum, which will lead to improved integration of technology into the core curriculum. This integration will lead to improved integration of technology in the classroom to enhance instruction.
2. A technology orientation plan for students will lead to a revised student orientation plan, which will lead to improved student understanding of technology availability and appropriate use. Improved understanding will lead to increased student use of technology to enhance learning and then improved integration of technology to enhance instruction.
3. A technology professional development model will lead to a revised professional development model, which will lead to improved teacher understanding of technology availability and then an increased use of technology by teachers. Increased use of technology will lead to improved teacher use of technology, which will in turn improve the integration of technology to enhance instruction.
4. A technology orientation plan for teachers will lead to a revised teacher orientation plan, which will lead to improved teacher understanding of technology availability and then to increased use of technology by teachers. Increased use of technology will lead to improved teacher use of technology. Improved teacher use will in turn improve the integration of technology to enhance instruction.
5. Technology-based communications will lead to a revised districtwide protocol for technology-based communications, which will lead to improved communication with families regarding events, assignments, and emergencies. Improved communication will lead to increased parental involvement in their child’s education.
6. Technology-based additional learning opportunities (ALOs) will improve the identification of technology-based ALOs. Improved identification will lead to increased availability of ALOs outside of the regular school day, which will lead to increased student participation in technology-based learning opportunities. Increased participation will lead to increased student exposure to learning opportunities.
7. A districtwide software and hardware acquisition plan will lead to a revised protocol for hardware and software acquisitions, which will lead to improved long-term acquisition planning. Improved acquisition planning will lead to increased availability of appropriate and necessary technology, then to increased availability of technology- based learning opportunities, and then to increased student participation, which will lead to increased student exposure to learning opportunities.
129
8. Improved use and integration of technology to enhance instruction, increased parental involvement in their child’s education, and increased student exposure to learning opportunities will lead to increased student learning and improved student achievement and ultimately improved postsecondary success.
130
Program Logic Model Figure 5 illustrates these assumptions in the NowPLAN-T logic model.
Figure 5: NowPLAN-T Logic Model
131
Resources Nowgarden School District worked with its external evaluator to identify several contextual conditions and resources that are necessary for the success of the district’s technology plan. These are listed in the first column of the NowPLAN-T logic model and include financial resources to implement the technology plan, administrative support throughout the NowPLAN- T implementation, technology infrastructure (or resources to build this infrastructure), and the necessary technology personnel.
Step 2: Plan the Evaluation
Evaluation Design Nowgarden School District is a public school district educating over 56,000 students. The district has 38 elementary schools (grades K-5), 13 middle schools (grades 6-8), and 11 high schools (grades 9-12). Nowgarden employs over 4,000 teachers across its 62 schools.
The district’s technology plan is implemented districtwide. The NowPLAN-T program and its evaluation will employ a single group evaluation design and will include both quantitative and qualitative data sources.
Design: Mixed-method, nonexperimental design
To improve the quality of the NowPLAN-T evaluation, Nowgarden’s external evaluator and the district worked to embed the evaluation into the technology plan. The evaluation is theory based and uses logic modeling to relate NowPLAN-T strategies and activities to long-term district goals. The evaluation design is longitudinal (6 years) and uses repeated measures. (Some data will be collected annually, while other data will be collected quarterly.)
Enriching the Evaluation Design: Logic modeling; longitudinal data with some repeated measures. In-depth case study of NowPLAN-T schools and sampling of classrooms within schools will be used to collect observational data.
Data Collection Methods Data will be collected through a variety of methods, including interviews, documents, surveys, rubrics, and assessments. All schools in the district will participate in certain components of the evaluation, while a purposefully selected group of schools will be chosen for a comprehensive case study, based on the quality of their implementation of the NowPLAN-T activities.
In the districtwide portion of the evaluation, all teachers and students will participate in surveys examining the use and integration of technology into the classroom. Teacher and student
132
surveys will be based on the NowPLAN-T rubrics. That is, along with questions regarding the program’s implementation, teachers and students will self-report on rubric components.
To provide a rich understanding of the classroom context of NowPLAN-T, the external evaluator will conduct a case study in a purposefully selected group of schools. Schools will be selected for the case study based upon the quality with which they have implemented NowPLAN-T activities. The case study will focus on four schools and provide linkages between classroom- level implementation and student learning that may not be fully understood by examining survey data only. A random sample of classrooms will be chosen at each case study school to participate in observations using the NowPLAN-T rubrics. The external evaluator and trained members of an evaluation team assembled by the external evaluator will observe these classrooms periodically throughout the school year. During observations, the NowPLAN-T rubrics will be completed and classroom context recorded. Case study teachers will also participate in in-depth interviews.
Student surveys (delivered electronically) will address classroom and home use of technology for primary learning activities, as well as additional learning opportunities offered as part of NowPLAN. The evaluation also will use participation logs from additional learning opportunities to determine opportunities that were offered and attendance. Additionally, a parent survey will be electronically administered to all parents in the district. The survey will be voluntary and anonymous. Information from the parent survey will examine communication and parent involvement related to NowPLAN components. The district would like but does not currently have the funds to conduct a graduate follow-up survey to address the second long-term goal of improved postsecondary success. The proposed survey is included in the evaluation design as a placeholder for future evaluation possibilities.
Step 3: Implement the Evaluation The NowPLAN-T evaluation matrix is provided in tables 16, 17, and 18. Evaluation questions, indicators, and targets, as well as data sources, collection, and analysis are addressed by logic model component.
Note that some very early (short-term) objectives are included in Table 16: NowPLAN-T Evaluation Matrix—Strategies and Activities/Initial Implementation. The remaining short- term and intermediate objectives are covered in Table 17: NowPLAN-T Evaluation Matrix— Early/Short-Term and Intermediate Objectives. Table 18: NowPLAN-T Evaluation Matrix— Long-Term Goals addresses long-term goals on improving student achievement and postsecondary success.
133
