School Inspector Decision-Making: Fact or Bias?

How do Ofsted inspectors arrive at their decisions?

Despite its wide use around the world, our understanding of how school inspection is carried out remains limited.

Understanding school inspectors

In a new 23-page paper, School Inspectors’ Decision-making: Evidence from a Comparative Perspective (Bezem et al., 2022), researchers highlight the era of high-stakes accountability and how school inspection can offer a nuanced understanding of why schools fail.

The crux: How do school inspectors arrive at their decisions?

A comparative case study is offered, “understanding inspectors’ thinking and how personal perspectives are used” to explain the process. The research analyses inspection in the United States, the Netherlands, and Argentina. I am offering a summary of this paper to prompt reflection on English education’s preferred method – Ofsted.

The research uses sensemaking theory – a way of “understanding how individual actors comprehend an unknown or ambiguous situation, make meaning of it, and then act based on this interpretation” (Weick, 1995). Sensemaking literature suggests that “individual reasoning about a complex judgment tends to be biased toward interpretations that are consistent with their beliefs and values” (Spillane et al., 2002).

Something Ofsted have ‘tried’ to quash and defend as part of the methodology they use in England. new research suggests Ofsted grades are weak predictors of students’ long-term outcomes (Von Stumm et al., 2021).

Research objectives

Two questions are addressed:

1. What are the sources guiding inspectors’ thinking during inspections? and
2. How do the personal perspectives of inspectors influence school evaluations?

Elements of the literature review contain some of the following observations:

• Most previous research focuses narrowly on European systems
• There is limited empirical research on school inspectors
• The professional judgment of inspectors plays a key role in their evaluations.
• Inspectors’ professional background impacts their judgement.

… prior experience serving as a classroom teacher can increase empathy and a sense of collegiality with teachers (Baxter, 2013)

The paper highlights each of the three inspection systems which I have summarised below:

United States

• Local districts have historically held great freedom to manage their schools through local boards and superintendents
• The federal government, states, and school districts play a role in the accountability structure
• High-stakes accountability testing
• The inspection program, referred to as Quality Reviews, primarily targets low-performing schools
• A written report summarizing conclusions is provided to schools, which includes suggestions for priority areas, but not specific recommendations for improvement. (That’s helpful!)

Netherlands

• The Dutch Ministry of Education coordinates educational policy with municipalities.
• High-stakes testing plays an important role
• Inspection is a central instrument for monitoring standards.
• Yet, Dutch schools are among the most autonomous in the world
• All schools are inspected at least once every 4 years, the lowest performing receive more frequent / rigorous visits.

Argentina

• Each province in Argentina manages its educational system
• Standardized tests are low-stakes and only used for diagnostic purposes
• School inspection are also low-stakes and can only lead to sanctions when regulation about school safety or student well-being are at risk.

Conclusions and recommendations

The paper explores are the methods used and the structured interview process. I’ve skipped straight to conclusions.

“Most inspectors perceive the process of reaching a consensus to be straightforward” with “all interviewed inspectors [having] experience as classroom teachers.” There’s no doubt that personal bias is evidenct.

“Inspectors in the Argentinean case did not shy away from frequently explaining how their personal perspectives influence their thought process” and in the Netherlands, “several inspectors explained how they determine what the problems are, relying on their expertise and ‘gut feeling’ …”

The research sources “guiding inspectors’ thinking and how personal perspectives influence school evaluations” with “a tradeoff between rigid and flexible inspection approaches” but it is this sentence which condems Ofsted here in England:

“Protocols aim to avoid personal bias at the expense of detailed understanding of individual schools. Data collection emphasizes consistency by adhering to an evaluation rubric. Information collected during inspection forms the basis for determining overall school trends.”

The evaluation process relies heavily on inspector perspective, experience, and intuition, as well as local context information with the international comparison contrasting approaches to inspection, all influenced by the local culture and professional traditions, which are associated with inspectors’ views on school accountability.

These findings demonstrate that the human element and professional judgment remain central in the inspection process, regardless of efforts to standardize processes and procedures (Bezem et al., 2022).