‘More data does not equal better data’ Arfan Ismail, PhD, Product Manager, Education Software Solutions July 2022
A recent report from Deloitte found that as of 2020, 97% of companies were either already using AI or had plans to use it in the short term. Importantly, the report bears out a key trend: the use of data and advanced machine learning has become ubiquitous in industry generally. Yet somehow in education, policy decisions are still predicated on meta-analysis and generalised data – the former essentially synthesizing numerous studies to find statistically significant relationships.
Amongst the most well known in education are the studies of John Hattie whose oft-cited work is used as justification at both systemic policy, individual school and class level. The problem with amalgamating data sets in this way is that what works in one context doesn’t necessarily work in another. Let me provide one example of this. In answer to the question ‘does the competence of the school principal have a significant impact on student outcomes?’, Robert Marzano, in his best-selling text School Leadership that Works explains how initial findings from their meta-analysis found the strength of correlation varied. They had to remove non-American studies from their cohort of studies for the relationship to become significant, perhaps due to the extra training and qualifications held by school principals in America. What’s the lesson here? Context matters! The more localized the data, the better it is. When researching schools and education, what works in a school in London may not necessarily work for a school in Newcastle. This is where the need for better data becomes paramount.
What then does effective use of data look like in an educational context, be that at policy, regional, local authority, multi academy trust or school level? This is a question that management information systems providers (should) understand better than most and are currently tackling head on. Indeed, from our initial research, it is clear that schools do a great job in collecting data and that there are yet more opportunities to use this data even more effectively. So, what would a data-infused education system look like and how can we work to make this a reality?
As someone with a long-standing interest in data what is clear to me is that more data does not equal better data. In fact, providing more data can lead to worse outcomes if it leads to attention being focused on the wrong thing. Not every statistical relationship between intervention and outcomes matters. We have finite resources and these need to be focused on doing those things that have the largest impact on student outcomes.
For example, research from Fazıla Duyan and Rengin Ünver found that classrooms with purple walls are better at holding student attention than red coloured walls. Does that mean we should head down to the nearest B&Q and buy shed loads of purple paint and start redecorating our classrooms? Probably not. We need to prioritise our focus on those interventions, the things we do to make education better, that have a high return on investment (RoI), where the effect on outcomes is more pronounced. In order to do that we really do need a better handle on the data and better ways of analysing that data.
The way industries generally tackle the problem of working out how spending relates to RoI is through the utilisation of artificial intelligence – and more specifically machine learning – to mine the data and look for those key factors, predictors, that cause one thing to lead to another. The interest is not in correlations here, interesting though they might be, but rather on causation, something more difficult to fathom. That’s why we need the help of machines to do this in education. It’s simply too complex for the human mind to do this at scale or at an individual school and pupil level. Human beings, custodians of consciousness and complex cognitive abilities, are not easy to understand. Artificial Intelligence can help us do just that.
Joseph Aoun, President of Northeastern University, in his book Robot-proof, suggests that the key to an effective use of AI is to marry the creative ability of humans to understand and identify problems with the ability of the machines to process, synthesis and analyse the data to help solve these problems. This is the approach we are taking at Education Software Solutions (ESS) as we take our first steps on a journey to transform the way schools use data to keep children safe and improve student outcomes. Our initial focus is on predictive and then prescriptive analytics for persistent absence. We have begun this journey by spending a considerable amount of time listening to our customers to understand what their key underlying challenges are when it comes to data and what problems they are trying to solve, so we can create data products that help solve these problems.
Educators are constantly placed under a significant cognitive load. Through leveraging data engineering, data-science and education experience, as well as expertise and passion to improve student outcomes, data reporting software that delivers intelligent insights can be built. This will add significant value to Multi-academy trusts, schools and other educational establishments. Great implementations of AI transform general transactions to idiosyncratic ones. Education is a perfect use-case for the implementation of machine learning so each child receives the specific, personal interventions and support that will help them remain safe and achieve their very best. This is the future we, at ESS, are committed to delivering. Our collective journey with our valued customers has begun.