One thing this book makes clear, if you didn’t know it already, is that when it comes to research findings it’s very much a case of caveat emptor — buyer beware.

Very little is written here about educational research, but there’s no reason to believe that education is immune to the influences that the author describes. Indeed, I wrote about some of them in my article 7 Questions To Ask Regarding Whether Education Technology Improves Learning.

Hopefully, outright fraud is not very common, but I’d be surprised if some of the various kinds of bias described here would not be discovered. Some of the bias is unconscious, which arguably makes it harder to tackle in the context of education. For example, I don’t think it would be a trivial task to set up a double-blind experiment in which neither the practitioners nor the subjects knew who was receiving the “treatment”.

Fraud and bias are worrying enough, but then there is what the author refers to negligence. This can include accidentally omitting crucial data, or typos caused by, in one instance, Excel’s autocorrect feature converting a specialised term into a date.

And then there is hype, examples of which arrive in my inbox every day. In my experience, when someone has been taken in by some hype, it’s almost impossible to shift them. I recall in the 1990s when various kinds of automated learning systems were all the rage meeting a deputy headteacher who was going to set up a room of 30 dedicated computers and have kids learning English and Maths on them for whole lessons, while the teacher “facilitated” the session.

I tried to convince him that he would need to give the teacher(s) concerned an extra free lesson for every lesson on the machines, because the so-called automation only worked effectively if the teacher set up the learning goals for each pupil.

This wasn’t because there was anything wrong with the systems: they were designed for and intended to be used in remedial work, so that if a pupil in a class couldn’t understand, say, decimals, then they could spend some time catching up in a programmed learning environment that adjusted itself to their needs as they became apparent.

I pointed all this out to the deputy headteacher, along with two reports which showed that the best results from the systems were obtained if the pupil was using them for no more than 15 minutes at a time. He wasn’t having any of it.

This is an example of why hype can, in own way, be dangerous. It detracts time, energy and financial resources away from interventions that may be less exciting to look at but which actually work better.

It’s also concerning that at least some of “evidence-based practice” may be based on pretty flimsy evidence indeed. Thus when the Department for Education in England announces that it’s going to set up an institute of teaching there is not necessarily a cause for celebration. The comment by the Education Secretary:

simply caused me to wonder which particular “evidence” the DfE was going to cherry-pick (or, if I allow myself to be even more cynical), commission.

As I said in my quick look at this book, there’s an excellent chapter on how to read scientific reports. For suggestions specific to educational research, have a look at the slides embedded in the article Read All About It: What Does The Research REALLY Say?

The bottom line is that Science Fictions is very readable, and will be useful for both teachers and older students of science subjects, and teachers in general who would like some insight into why even the most correct-looking results from research may not be as good as they seem.