Education Guide / Evidence
There is a wealth of research studies in the field of education. And as with data, journalists should proceed with care: Watch for weaknesses in research design and be clear about caveats with readers. In assessing those studies, you might ask: How big is the sample size? (1,000 is standard for a national poll with a 95 percent confidence interval, for example.) And is the study observational or experimental?
It’s important to consider the study’s research question. A good study will state a clearly defined purpose, research goals, and background research underlying the question at hand.
It’s also important to be aware of and to understand conflicts over research design and findings. One well- known example is the debate over how to measure the effectiveness of charter schools. Two major studies found significantly different results – one that compared only students who applied to oversubscribed charter schools in order to compare the achievement of those who got in versus those that didn’t, found that charters raised student achievement, while the other, which compared students in charters to similar students in regular public schools, found they mostly performed similarly or worse than regular public schools. The different methodologies employed by the researchers became part of the story. Similarly, broad-based studies, such as those conducted by the government to evaluate major programs like Head Start or federally-funded after-school services, often mask a great deal of variation across programs.
Especially if you’re reporting on the findings of a single study, try to get a second opinion or note previous, conflicting research. And always keep in mind possible biases:
“One of the great frustrations of this beat is that education studies always seem to conflict – kind of like the news that spinach is good for you. No it’s not. Yes it is... It seems like for every study that comes out on charters or vouchers or whatever, another study conflicts it. Consider the source and funder, as always.”