New Report Outlines Recommendations for Improving Teacher Observation Practices

Today the Center for American Progress released a new report recommending that when it comes to evaluating teachers through observation, America's public K-12 schools have much to learn from the experiences accrued by their counterparts in the early childhood sector. While there are too few models of how to do observation well in the K-12 sector, the early childhood field provides more than two decades of widespread application of classroom observation from which to draw lessons.

The report, entitled "Implementing Observation Protocols," draws from decades of experience using observation in early childhood education, which has implications for administrative decisions, evaluation practices, and policymaking in K-12. Early childhood education has long embraced the value of observing classrooms and teacher-child interactions. In early childhood education the features of the settings in which children are served are the hallmarks of quality. These features can include health and safety considerations, the materials and physical layout of the space, and the interactions that take place between adults and children--such as conversations, emotional tone, or physical proximity. The observational assessment of teachers' classroom practices, along with estimates of teachers' contributions to student achievement gains and other measures, is the cornerstone of new assessments in K-12 education. Standardized observations of early childhood education features yield metrics that are used in state and federal policy, program improvement investments, and the credentialing of professionals--all uses that K-12 education is now considering.

The report examines lessons learned from observation in early childhood education that may be helpful as states and districts begin implementing more rigorous observation protocols for K-12 teachers. Lessons focus on the importance of standardization, trained observers, methods for ensuring the validity and reliability of the instruments, and the use of observational measures as a lever to produce effective teaching. These lessons form the basis for the following recommendations outlined in the report:

• Any measure must provide information in the form of metrics that clearly differentiate those being assessed. Observation is no exception--thus observation is a form of measurement and assessment consisting of codes and benchmarks that must be applied rigorously, just as they are in assessments of student performance.
  
• Observations used in systems of decision making and performance improvement must adhere to standardized procedures. There are three components of standardization that are key elements for evaluating any observation instrument and its implementation--training protocol, parameters around observation, and scoring directions.
  
• The technical properties of observational protocols and scoring systems are fundamental for their use. Reliability is one of these properties and pertains to the level of error or bias in the scores obtained. It is critical that users select tools that have documented reliability for use across observers, teachers, time, and situations. Effective training programs for observers help ensure raters are consistent with one another as they make ratings. Similarly, including periodic "drift" testing at predetermined intervals will help to improve the degree to which raters remain consistent with scoring protocols and with each other.
  
• Any observation of teacher performance must show empirical relations with student learning and development if the use of observation is expected to drive improvement in student outcomes. Selecting an observation system that includes validity information cannot be overstated.
  
• Pragmatically, observation takes time and different systems of observation require different time commitments. The amount of observer time available can be an important practical consideration when selecting an observational system. In general the more ratings a school or district is able to obtain and aggregate, the more stable an estimate of typical teacher practices will result.
  
• Observations can identify teacher classroom behaviors that matter for students, can describe typical teacher practices, can show how a given classroom or teacher compares with a national or district average, can forecast the likely contribution of a teacher to children's learning, or can document improvement in teachers' practices in response to professional development. Users, however, must be cautious to not overstep the appropriate use of observational instruments in their enthusiasm to apply them in any and all circumstances.
  
• Observations can be used in both accountability and program-improvement applications. Importantly, policy and program investments over time can change the typical distribution of scores as teachers, classrooms, and programs improve, and as a consequence it can be necessary to periodically "raise the bar" on performance standards or cutoff scores.
  
• Feedback to teachers is most effective when it is individualized and highly specific, focused on increasing teachers' own observation skills, promotes self-evaluation, and helps teachers see and understand the impact of their behaviors more clearly.

Read the report: Implementing Observation Protocols: Lessons for K-12 Education from the Field of Early Childhood by Robert C. Pianta

To speak with a CAP expert on this topic, please contact Katie Peters at 202.741.6285 or kpeters@americanprogress.org.