lørdag 11. februar 2012

Did Hattie get his statistics wrong II: Comments from John Hattie and Arne Kåre Topphol

Earlier this week, I wrote an entry here on my blog referring to and commenting upon an article written by Associate Professor Arne Kåre Topphol at the University College of Volda, concerning what seems to be an erroneous use of a specific statistical tool, the Common Language Effect size (CLE), in John Hattie's world-renowned and highly influential 2009 publication Visible learning.

Being as I am also a student at the teacher-training program at the University of Oslo, and, moreover, a member of the student association there – and further, being both curious to get to the bottom of this debacle, and concerned with the fact that references to Hattie's research bespatter our curriculum without (to my knowledge) ever having been questioned – myself and some of my fellow members of the student association contacted John Hattie himself, and presented him with the criticism advanced by Arne Kåre Topphol.

John Hattie
To our utter delight, Hattie took the time to answer our query; and when whereupon we presented Hattie's respons to Topphol, he kindly took the time to make a comment of his own.

We in the student association have been given permission by the two parties to publish their respective comments on our blog. Our request to Hattie, and his response, is published here. The follow-up comment from Topphol is published here.

I would like to emphasize that I am not on a mission to "dethrone" Hattie, or to try to find faults where none are to be found, or to exaggerate or blow out of proportion those that may be there. But I am in agreement with the main focal point of Topphol's initial criticism, which I also believe is the important element we should take away from this discussion:

That is, as a quantitative approach is becoming more and more dominant in research on schools and education, all of us depending on this research to make informed judgments on policy decisions or professional practices tend either to trust more or less blindly the results researchers present us, or else we do not possess the necessary statistical competency to assess the accuracy or validity of a result. I agree with Topphol that publishers, peer reviewers and researchers themselves have a responsibility to make sure that their use of statistical tools is trustworthy. However, I believe that this is a responsibility that has a wider currency, and that there is also a need for more methodological literacy in general – among students, teachers, educators, and, probably more pressing than anywhere else, among policy makers and politicians.

Hence, I would encourage everyone being a stakeholder in the discussion on education in general and best teacher practices in particular to (i) read (i.e. actually read) Visible learning, (ii) read Topphols article, (iii) read the aforementioned comments from the respective parties, and (iv) make up your own mind on this particular issue. Good luck!

See also Did Hattie get his statistics wrong?

17 kommentarer:

  1. Hei,
    og takk for innsatsen!
    Var det en tidsbegrensning på Hattie og Topphols innlegg, eller er det faktisk en feil(Error 404) på siden når de ikke kommer opp når man trykker på lenken?

    Camilla Stabel Jørgensen

    SvarSlett
  2. Hei Camilla,

    Det er en faktisk feil, dessverre. Bloggen vår (altså LPU - Lektorprogrammets programutvalg sin) var nede en stund, og noen av postene ser ut til å ha gått tapt.

    Jeg har ikke redigert dette innlegget enda, i håp om at vi enten skal finne tilbake til disse postene, eller skrive dem igjen, om vi kan finne tilbake til utkastene. Kan dermed ikke love noe, men svaret fra Topphol vet jeg at jeg har et sted, om du sender meg din mailadresse kan jeg sende deg det på mail.

    SvarSlett
  3. Hei igjen Camilla,

    Bare se bort fra det forrige svaret, jeg har fikset lenkene nå. Takk for at du gav meg sparket jeg trengte for å ordne opp i dette!

    SvarSlett
  4. Hello- the links to Hattie and Topphol's responses appear to be broken. Would you be able to email them to me directly at mjones@mulgrave.com

    Thanks!

    Martin

    SvarSlett
    Svar
    1. I would also like to read the responses. If you ever received them, I'd love to get them forwarded.

      Slett