What metrics are used to determine which school Ofsted will visit next?
On 22nd January 2019, I made a Freedom of Information request to Ofsted. I wanted to know what sources of data Ofsted uses to calculate/predict which schools to inspect next?”.
Freedom of Information
“Dear Ofsted, For a number of years, Ofsted has used statistical models to ensure proportionate inspection. Your new methodology has not changed and you believe will improve your capacity to identify concerns about performance. In your documentation, published March 2018, you share methodology notes which set out the risk assessment process that Ofsted uses for primary and secondary maintained schools and academies that are judged to be good and outstanding.
In light of your proposed education inspection framework for September 2019, published on 16th January 2019, please could you share the current (at the time of this FOI request) the risk assessment process for the two stages:
1) the assessment of each school based on analysis of published data.
2) the in-depth ‘desk-based’ review of a wider range of available information.
For schools and academies, I would like to know 1) what school data (published or not) do you consider for assessment? 2) What ‘desk-based’ information do you assess to evaluate which schools to visit?”
Ofsted responded …
Ofsted responded: “In this case, the process of how schools are selected for inspection is set out in our published guidance documents. Paragraph 11 of the school inspection handbook sets out the risk assessment process and provides further details of the information considered in paragraph 13. We also publish a methodology note which provides further in-depth detail of the risk assessment process concerning good and outstanding schools. In all cases, the data used as part of the risk assessment process is publicly available.
As we set out in our guidance documents we use:
Ofsted uses risk assessment to ensure that its approach to inspection is proportionate and so that it can focus its efforts where it can have the greatest impact.
Interestingly, Ofsted cite the risk assessment process for good and outstanding schools, 5 days after I had publicly written about Machine Learning concerns (following attendance to a NAHT Accountability Commission meeting) which I have been told (by Ofsted) that this risk assessment has 67 per cent accuracy of predicting which schools are at risk of decline. Therefore, if Ofsted can use metrics to gauge which schools to visit – and get the judgements right – it supports their long-term viability.
Ofsted uses a broad range of indicators to select providers for inspection. Well, according to the above, they cite just 3 sources. Parent views, workforce headcount – how on earth do these sources provide reliable information to determine how good a school is? – and examination data, including attendance.
Ofsted’s evidence base is often insufficient to support its sweeping judgements. I have already highlighted how dangerous Parent View is for schools. Ofsted surveys 1,500 parents to justify publishing reports to the public and consider this reliable research data. We have waited 26 years to explain that ‘grading schools is what parents want‘, with the results due next month.
The National Audit Office published a very useful report on sample size which recommended. “When reporting the results of a sample it is important to cover several key facts:
- the sample size;
- the sample selection methodology;
- the estimates resulting from the sample, and
- the precision and confidence intervals for the estimates.”
Based upon NAO’s recommendations, Ofsted’s sample sizes are too small to make any credible judgement. (NAO, May 2018)
As we know, Ofsted are currently consulting on a new framework. The jury is out to gauge opinion as to whether or not Ofsted will be able to evaluate a 5-year curriculum offer in just two days… You can download the full response here.