Saturday, September 29, 2012

Numbers that count

Most of the education world is filled with numbers that shouldn't count. A book by John Hattie is making the rounds in BC and Canada. The book is used to develop a theme: that there is data-based evidence that class size and other working conditions don't matter that much, but teaching methods do. It is the usual message from the top - fix the teacher, not the conditions in which they work, or the conditions our children find themselves learning in.

Hattie's book is a meta-analysis - a complicated set of edubabble statistics to apparently show what is effective and not by reducing hundreds of studies on factors that influence student outcomes down to a single number. Finances: .12 (small). Peer tutoring: .5 (medium). Instructional quality: 1.0 (huge).

Just one small problem - what is he actually measuring, and should it be measured in the first place? What is a good student outcome? What did these studies measure? Our data-obsessed world produces an awful lot of questionable data, based on questionable assumptions. First and foremost in the education world is that "school quality" or good "outcomes" are measured by standardized test results.

This goes to the very heart of what schooling and education are about. Are they about doing an algebra problem correctly? Or are they about the broad range of knowledge, skills and abilities that enrich our lives and our society.

Professor Alfie Kohn has written on this subject at length and provides some healthy skepticism to the world of edu-statistics. In "Schooling Beyond Measure", he notes:

The habit of looking for numerical answers to just about any question can probably be traced back to overlapping academic traditions like behaviorism and scientism (the belief that all true knowledge is scientific), as well as the arrogance of economists or statisticians who think their methods can be applied to everything in life.  The resulting overreliance on numbers is, ironically, based more on faith than on reason.  And the results can be disturbing.
In education, the question “How do we assess (kids, teachers, schools)?” has morphed over the years into “How do we measure…?”  We’ve forgotten that assessment doesn’t require measurement -- and, moreover, that the most valuable forms of assessment are often qualitative (say, a narrative account of a child’s progress by an observant teacher who knows the child well) rather than quantitative (a standardized test score).  Yet the former may well be brushed aside in favor of the latter -- by people who don’t even bother to ask what was on the test.  It’s a number, so we sit up and pay attention.  Over time, the more data we accumulate, the less we really know.
All that said, I am of the belief there are a few numbers that we should pay attention to. Here are some.
1. The graduation rate in BC is 80%. This compares to 96% in Denmark. (see: Clearly we can do better at getting most students through high school and to a diploma. A little breakdown of BC's graduation rates is also instructive: for both special education students and aboriginal students the number is around 50%. Just this little bit of information is plenty for some policy decisions - we need to seriously re-invest in special and aboriginal education. And by re-invest I don't mean test more, for more data, but teach more, for more learning.
2. Class sizes. It seems to me it is the most basic and obvious thing that the amount of teacher time any student has access to will directly impact their learning. The same obvious fact holds true of group homes, foster homes and day cares, where adult/child ratios are mandated. At the extreme, hundreds of children with a few adults can result in serious physical and psychological damage (remember those Romanian orphanages ). But more back to our world, as every parent knows, a class of twenty is better than a class of thirty. More teacher time. More one-on-one instruction. More ability for the teacher to ensure pro-social behaviours. More time to address individual learning needs. It may be that a teacher can produce high standardized test scores with classes of thirty, forty or more, but is that what parents want when they send their children to school? A simple answer to this question came to my door last week. The Globe & Mail included in the paper a glossy magazine, "Our Kids: Canada's Private School Guide". For each school in the directory, they list basic information: Grades, Gender, Boarding options, Cost, Contact Information and just one other thing: Average Class Size. 
Here is a little "data" regarding class size averages in Victoria, where I live:
Glenlyon Norfork: 18 - 20
St. Margaret's School: 14 - 20
St. Michael's University School: 18 - 20
Queen Margaret's School: 18
Shawnigan Lake School: 14 -15
Public schools: 18 - 27


  1. Public school parents should be up in arms over the above stats. They should demand equal opportunities for their publically educated children and that public tax money go to their children's schools.

  2. Edubabble is a good way to present it. Bill Gates, Sir Ken Robinson et al all love to use this type of statistical loop dee loop to illustrate that teachers, unions and their pensions are the problem. Most "research based evidence" in education is not worth the paper it is written on and one could predict the outcome from the hypothesis. Kieran Egan's book "Getting it Wrong from the Beginning", is an excellent analysis and critique of the underlying philosophical thinking behind much of educational thinking, including the research. Education does provide our young with the knowledge, skills and abilities to operate in our society. It develops the cognitive skills to allow more deeper thinking, self confidence that help the individual develop. Vygotsky is right, "learning precedes development." It is through the group that we become individuals. The maxim is true, weighing the pig more often does not make the pig fatter.