Data Driven Decisions and the Seduction of Pragmatism

“Data, data everywhere – nor any spot to think”.

With apologies to S T Coleridge for the misappropriation of the famous lines of his Rime of the Ancient Mariner poem I think we are at risk of being swamped by data in a similar way as the Ancient Mariner was swamped by the sea  – surrounded by it but not being able to access it properly or to benefit from it.

Data – or talk of data - is everywhere, seemingly in every context.

Student working on MacBooks in class

Data-gathering, data-mining, big data, data analysis, data-driven planning, data-informed decisions. Data, data, data.

Don’t get me wrong. I am not advocating a complete rejection of this trend towards the measurement and accounting of those things that have to this point been left to anecdote or intuition. It is, I think, a very exciting time to be involved in education when there is not just a growing body of evidence as to what really effects changes in students’ learning outcomes but also a growing willingness to ask the questions and to trust the answers that the research brings.

Like no time before we teachers can confidently predict the effect our behaviours as teachers will have. Moreover, we can confidently predict the effect of our own attitudes to teaching on the effectiveness of our classroom practice.

This allows us to avoid “messing about in the dark” in regard to our professional practice, unsure of what works or why, and allows us to be more intentional, efficient and effective. The result of heeding the message of both the data from large scale educational research and the data from our own specific measures of student achievement in class will be more students showing more progress. That must be a good thing. Professionally it allows us to better master the rules of cause and effect in learning and, in understanding them, having capacity to “play” with them for even greater effect, as a master writer plays with the rules of grammar to increase the impact of their work.

The profession has come to know that data is essential for informing our practice. There will be sceptics of this development, those with a more romantic view of teaching, who will lament the implied movement of teaching away from it being appreciate as an art and toward it being prescribed as a science. And for good reason.

Amid the wash of data we should always be careful not to drive from our awareness the essential humanness of the enterprise of teaching. It is inescapably, undeniably personal.

There is, I fear, a risk that we will so standardise the practice of teaching that we will lose the colour and flavour of difference that so enriches a community or professional practice. In advocating for teaching standards and achievement standards we must not forsake the unique individualness of the teacher-learner relationship. In using data we must not let the forceful weight of repeatability obliterate the fragility of the instance.

But there is another risk – perhaps even more dangerous.

In a context where data drives decisions it is a small step to allowing the data to assume ultimate value to the point where any method is approved if it produces the right data. This is the ultimate “ends justifying the means” philosophy. It works – that’s what matters. Whatever it takes.

But we all know that the way something is done is as equally important, if not more important, than what gets done.

I remember visiting a school that had recently adopted a deconstructed class approach to learning, with over 100 students being taught simultaneously by one lead teacher supported by several other “rovers”. The lead instructor had a microphone and was teaching something about Maths to a noisy and rabble of distracted and disengaged students. To my sensibilities it was less than ideal.

Talking with the Principal later they proudly reported that since the adoption of the new pedagogy reports of behavioural incidents had plummeted. The data was in – the pedagogy was a success in terms of managing student behaviour. The reality was that the report data on misbehaviour was reduced because the staff were now not addressing it!

I wonder if with these new tools for measuring the effectiveness of teaching strategies and the priority of particular outcomes we might find ourselves adopting teaching practices that are defensible based on the data but questionable when scrutinised by our morals and our understanding of the humanness of each student.

I wonder if with the power of data pressing so forcefully on our decision making and planning we don’t take time to think through the values implicit in our approach to education. Without thinking through the basis for our practices, even our successful practices, we are at risk of simply being pragmatic. It works – don’t worry about anything else.