More on "Dispute Over the Economics of File Sharing Intensifies"

David Glenn has a long discussion about the paper published in the Journal of Political Economy by Oberholzer-Gee and Koleman S. Strumpf on file sharing. Glenn has a useful and well-done discussion of the debate. For example, he notes:

Mr. Strumpf replies that most of Mr. Liebowitz's criticisms are trivial, even if correct. "Almost every point raised in Liebowitz's various pieces," he says, "involves incidental points which are not central to our conclusions."

Mr. Liebowitz, however, says his objections cast doubts over the entire study. Because he has not been able to scrutinize the OpenNap data at the heart of the study, his criticisms are largely aimed at a few non-OpenNap-based auxiliary tests that are presented at the end of the paper. But those tests are so weakly constructed, he says, that they call into question the validity of all the work.

One of the tests has to do with seasonal variations in record sales. "The number of file-sharing users in the United States drops 12 percent over the summer ... because college students are away from their high-speed Internet connections," Mr. Oberholzer-Gee and Mr. Strumpf write. Because of that seasonal dip, we might expect to see higher CD sales during the summer. But summer CD sales, as a proportion of the full year's sales, have not risen during the file-sharing era. Ergo, there doesn't seem to be much relationship between file sharing and CD sales.

That all sounds logical enough. But Mr. Liebowitz points out that one of the argument's premises—"the number of file-sharing users in the United States drops 12 percent over the summer"—is profoundly misleading.

The authors' citation is to a report on monthly file-sharing usage prepared by BigChampagne, a company that measures traffic on peer-to-peer networks. Mr. Liebowitz has a copy of the same report, which covers the period from August 2002 through May 2006. It's true that summer file sharing drops by an average of 12 percent (11.7 percent, to be precise) during the three summers—2003, 2004, 2005—covered in that report. But all of that effect comes from a severe drop in the summer of 2003, during a much-publicized wave of industry lawsuits against file sharers. During the summer of 2004, file sharing was flat, and during the summer of 2005, it actually rose slightly. So the test Mr. Oberholzer-Gee and Mr. Strumpf have set up—which is based on the ratio of summer-to-full-year CD sales—tells us nothing, Mr. Liebowitz says.

"If one my undergraduates did that, I would fail him," says Bruce D. McCullough, a professor of decision sciences at Drexel University who became interested in the dispute because he is a proponent of data transparency in economics publishing. "To take one decline, one flat, and one advance, and to suggest that it always goes down in the summer is just wrong." . . .

As too often seems to be the way of academic debate from people in part of the academy, this has a personal attack on Stan Liebowitz:

In an e-mail message to The Chronicle, Mr. Strumpf defends his work and suggests that Mr. Liebowitz's zeal stems from the fact that an academic center he directs, the Center for the Analysis of Property Rights and Innovation, receives grants from the Recording Industry Association of America and other commercial interests. "One might ask why Professor Liebowitz has remained so engrossed with our study," he writes.

I agree with Craig Newmark that Stan Liebowitz's concern that he couldn't redo their results and that they wouldn't share their data is enough to motivate many academics, including Liebowitz.

Other comments by CaveatBettor, Peter Klein, Josh Wright, and sivacracy.