[Population: One] <A HREF="http://popone.innocence.com/ar

[bryant]

Since I think this piece may make the rounds, some deflating is in order.

John Lott purports to have proven that the media is biased in favor of black quarterbacks. He claims that his research means that Rush Limbaugh was right. However, his research (whether or not it’s sound) is completely irrelevant when judging what Limbaugh had to say about Donovan McNabb. Limbaugh made a very specific claim about one quarterback in particular. Straw man fallacy.

Above and beyond that, his research is kind of shaky. Problem one: he only considered newspaper data. Justification? “[T]his is measurable and it is not clear why newspapers would be so different from the rest of the media.” That’s assuming the conclusion. Good research tests assumptions like that.

Problem two: the data on which he bases his report is flawed.

“We also collected data by week for each of the first four weeks of the season on a host of other factors that help explain the rate at which a player is praised: the quarterback’s rating for each game; whether his team won; the points scored for and against the team; ESPN’s weekly rank for the quarterback’s team and the opponent; and whether it was a Monday night game. In addition, I accounted for average differences in media coverage both in the quarterback’s city and the opponent’s city as well as differences across weeks of the season.”

Points scored against a team generally aren’t seen as the quarterback’s fault. A better metric would be the points scored off a QB turnover. Why is it important that it’s a Monday night game? Why are all these elements weighted equally? Are they weighted equally? Lott’s not saying.

Not atypical of the man.

Comments

[kodi]

Doesn't this sentence, all by itself, discredit him totally?

And [the 27 percent] difference was quite statistically significant — the chance of this result simply being random is the same odds as flipping a coin five times and getting heads each time.

Do people who understand statistics write sentences like that? There are no telltale signs that he's not a native speaker of English...

I'll have to pull that Excel file up when I get home; don't have it on this machine.

[kodi]

I pulled up that Excel file. Some interesting elementary views of the data:

Top ten positive comments, by week:

Name	Race	Positive	Negative
Jake Plummer	Caucasian	13	8
Vinny Testaverde	Caucasian	14	15
Daunte Culpepper	African American	14	0
Steve McNair	African American	14	0
Quincy Carter	African American	15	3
Kurt Warner	Caucasian	16	20
Matt Bulger	Caucasian	17	4
Gus Frerotte	Caucasian	19	3
Drew Bledsoe	Caucasian	19	2
Doug Johnson	Caucasian	25	3

Top 10 negative comments, by week:

Name	Race	Positive	Negative
Kelly Holcomb	Caucasian	4	14
Vinny Testaverde	Caucasian	14	15
Kyle Boller	Caucasian	7	15
Quincy Carter	African American	5	15
Kurt Warner	Caucasian	16	20
Kyle Boller	Caucasian	7	23
Donovan McNabb	African American	3	23
Kordell Stewart	African American	5	27
Donovan McNabb	African American	3	29
Jake Plummer	Caucasian	7	38

Caucasian quarterbacks make up 88% of the pool.

From looking at the data, though, the biggest conclusion I draw is that there's a lot of different quarterbacks in the league, and this is a very small sample of data about them. Why wasn't this study done with last year's data?

Beyond that... Why didn't it occur to Lott to mention that Donovan McNabb had two of the worst 5 weeks, and none of the best 10? I know it's not statistically significant, but it still seems relevant.

I've got more to say, but I'm going to drop it in a separate post, since I had to turn off autoformat to get those tables in there, and I'm tired of <p>

[kodi]

Ok. So I'm going to need someone to check my math on this bit, because it conflicts strongly with Lott's. I trust myself to hit the "sort" button, but apparently I'm not great with the sigma button. These are the numbers I got:

White Positive 574
Black Positive 178

White Negative 439
Black Negative 155

Total Positive 752 (Percent Black: 23.7%)
Total Negative 594 (Percent Black: 26.1%)

Total Black 333 (Percent Positive: 53.4%)
Total White 1013 (Percent Positive: 56.7%)

This is what Lott says:

Looking at just the averages, without trying to account for anything else, reveals a ten-percent difference in coverage (with 67 percent of stories on blacks being positive, 61 percent for whites).

I can't reconcile his statement with my numbers. Anybody see what I did wrong? Here's the Excel file.