Stupid Data Tricks



How silly has our society become we when it comes to matters of importance, like education? Initial reaction to yesterday's release of the New Jersey School Report Card gives us some indication.

Let's start with the 21st Century's new Pennysaver: AOL's Patch Network. I took a quick tour around the state, and reaction runs like this:

Jefferson:
SAT scores went below the state average in 2010-2011 at Jefferson Township High School, according to the New Jersey Department of Education School Report Cards, released on Thursday.
A typical Jefferson student scored 1,453, while the state average was 1,508. This is the second year district seniors have been below average. Jefferson students averaged 1,462 in 2009-2010, below the state average for that year of 1,515.
Um, you do know that not everyone takes the SAT? And some students with means can afford to take it more than once? And get tutoring outside of school? So it's ridiculous to judge Jefferson's scores outside of this context.

Lacey:
For the second straight year, the school report card for Lacey Township High School (LTHS) reveals that the number of students taking advanced courses is significantly lower than the state's average and the graduation rate is down to 90.77 percent but state assessment figures are well above the state average.
First of all, of course the "number" taking AP courses is lower than many other high schools: Lacey High only has 1437.5 kids (did they split that last kid vertically or horizontally?).  The reporter was obviously referring to the percentage of kids taking APs. But I'd say there's another relvant question: what's the percentage of kids getting 3s or higher (out of 5) compared to the state average? Sorry, the report card doesn't say.

I love this one from Hopatcong:
Hopatcong High School's average SAT scores dropped below the state average for the second straight year, according to statistics released by the state Department of Education Thursday.
The typical Hopatcong student earned a total score of 1,430 in 2010-2011 while the average state pupil scored a 1,508. The school also saw a drop the previous year, when the typical 2009-2010 Hopatcong student scored a 1,478 while the average state pupil recorded a 1,515.
[...]
The high school did, however, see an increase in the percentage of students taking the SATs. Only 56 percent took the test, upon which many college admissions departments place heavy emphasis, in 2009-2010 compared to the 64 percent in 2010-2011, the report said. But Hopatcong was still below the state average in that category for the third straight year. In 2010-2011, 73 percent of students throughout the state took the exam. [emphasis mine]
So the scores went down, but the number of kids taking the exam went up. Gosh, think there's a connection?

You get the point; there's no context for judging any of this. And there's certainly no questioning of the tests themselves, what with their talking pineapples and invasions of privacy.

I'll get to some of the other problems with the Report Card this weekend, but let me point out one important thing right now:

Even though we have the data, the School Report Card does not give the percentage of students at each school who are living in poverty, as measured by qualification for Free or Reduced Price Lunches. We know the Christie administration is on a jihad against FRPL as a measure of student poverty; getting rid of it is the first step in gutting funding for districts with large numbers of kids in economic crisis. But, as Bruce Baker points out, FRPL is the best metric we've got we've got to measure the relative wealth of each school's population.

Why does this matter? Because we know the correlation between test scores and socio-economic status is nearly perfect.


SAT scores by income class
Shouldn't the Report Card put FRPL front-and-center in its school assessments? Isn't this critical for understanding whether a school is doing its job or not? Or are Christie and ACTING Commissioner Cerf afraid FRPL's inclusion into the Report Card would undermine their case for getting rid of the measure?

Christie and Cerf putting politics before meaningful data interpretation? I know, you're just shocked...

More to come, including my all-time favorite Bruce Baker cartoon: