MONEYBALL FOR HEAD START 15
SECTION 2: IMPROVING PROGRAM OVERSIGHT
At the federal oversight level, the Office of Head Start (OHS) needs a valid accountability and performance measurement system that allows federal officials to detect trends and patterns in performance; identify, learn from, and disseminate lessons from trends and patterns in performance; differentiate oversight, support, and interventions on the basis of grantee performance; and support continuous improvement across Head Start as a whole.
Head Start grantees are subject to multiple forms of accountability. Through the Aligned Monitoring System, OHS monitors grantees’ compliance with the 1,400 requirements in the Head Start Program Performance Standards. The Designation Renewal System (DRS), mandated by the Improving Head Start for School Readiness Act of 2007 (Head Start Act), also creates a form of accountability by requiring grantees that hit one of seven “triggers” to compete to renew their grants. The criteria that trigger competition are:
» deficiencies in compliance with Head Start Program Performance Standards, as identified through the Aligned Monitoring System;
» license revocations by state or local licensing agencies;
» suspensions by HHS;
» disbarment by any state or federal agencies;
» significant financial risk;
» failure to establish or track and analyze data on goals for improving children’s school readiness; and
» scores on the Classroom Assessment Scoring System (CLASS) — a tool that OHS adopted to observe the quality of teaching in Head Start classrooms — that fall below a certain threshold or within the lowest 10 percent of grantees on any of the three CLASS domains.
The problem isn’t a lack of accountability in Head Start, but rather what grantees are held accountable for. Head Start monitoring is largely compliance-based, using detailed checklists and requiring grantees to provide extensive paper documentation of meetings, plans, and other compliance requirements. Most of the seven factors that trigger designation renewal are related to grantees’ compliance with federal, state, local, or financial requirements. Only the last requirement on the trigger list, CLASS scores, emphasizes program quality beyond mere compliance, and it focuses on a very narrow domain of Head Start performance. Further, the DRS does not differentiate between deficiencies that indicate serious shortcomings in programs’ health and safety practices, finances, or quality and deficiencies that reflect minor compliance problems. As a result, designation renewal has actually intensified the culture of compliance in Head Start.
MONEYBALL FOR HEAD START 16
Designation renewal is grounded in sound principles — identify grantees that consistently fail to achieve desired results; take action to correct underperformance; and, where necessary, create opportunities for higher performers to replace the underperforming grantees. Data and reports from the field also suggest that designation renewal has led to the replacement of historically underperforming grantees — roughly 5 percent of all Head Start grantees have lost their grants in the past three years — and has stimulated improvement in underperformers. But the system’s reliance on imperfect measures has also resulted in identifying some high-performing grantees for competition, while failing to identify others that are low-performing.
Further, the DRS — and compliance-based monitoring more generally — identifies only those programs that fall short of standards, not exemplary performers. The Centers of Excellence program, authorized in the Head Start Act, was designed to identify, recognize, and reward high-performing Head Start grantees. It identified only 10 grantees a year, however — out of more than 1,400 — and has not been funded in recent years.
To really support continuous improvement, Head Start needs a fairer and more results-based and robust way of measuring grantee quality. This is a necessary precondition for both identifying and learning from programs that are producing exemplary results, and for achieving the intended goals of the DRS. While OHS should be responsible for putting in place and administering such a performance measurement system, it cannot do this work on its own. Identifying, developing, and implementing a robust set of performance measures for Head Start will require the engagement of a broader range of actors, including researchers, the philanthropic sector, state and local policymakers, and — crucially — Head Start grantees. Ultimately, the Head Start statute itself should incorporate references to the existence, purposes, and expected use of performance measures. But Congress should allow OHS to pilot and refine these measures with grantees before requiring them in a performance management system, in order to ensure that measures are valid, reliable, and useful to the field. Otherwise congressional action will be premature and potentially damaging. This paper seeks to outline an iterative process for developing and adopting more robust performance measures for Head Start programs, making information about grantee performance more transparent, and ultimately using this information to differentiate grantee performance in ways that support continuous improvement at both the individual grantee level and across Head Start nationally.
CURRENT CHALLENGES IN MEASURING HEAD START PERFORMANCE
Head Start stakeholders agree that federal oversight of Head Start grantees needs to shift from monitoring compliance to focusing more on results.19 The challenge is knowing what to measure and how to measure it. Head Start is a complex program that is delivered in a variety of contexts across the country and seeks to address multiple child and family outcomes. Therefore, it needs to track multiple measures of program performance, including measures of child and family outcomes and other indicators of program quality. Using multiple measures provides a more comprehensive picture of program performance and avoids placing too much weight on any single measure. A meaningful system of performance measures for Head Start programs should include:
» Child outcomes: Head Start’s primary purpose is to improve child outcomes. Current monitoring systems ask grantees to demonstrate a variety of things — but not how they’re improving children’s outcomes. A meaningful system for differentiating program performance should incorporate multiple measures of children’s outcomes across the domains of child development included in the Head Start early learning standards, using tools that are valid and reliable for this purpose and for the population of students that Head Start programs serve — including dual-language learners and children with developmental delays and disabilities.
MONEYBALL FOR HEAD START 17