Response to Evan DeFilippis and Devin Hughes' claims at "ArmedwithReaon" about my research

Recently Evan DeFilippis and Devin Hughes claimed that defensive gun use was a myth.  Gary Kleck wrote a response where he noted that these authors were merely repeating earlier criticisms and ignored the responses that he and others have made to those critiques.

Well, DeFilippis and Hughes use the same approach in discussing my research (a screen shot of their original post is saved here).  Let's try to go through these points in order that they are presented:

-- Tim Lambert as a source.  Professor Jim Purtilo at the University of Maryland put up a post in 2004 that he has updated over the years that shows that Lambert has been caught falsifying evidence on multiple occasions and has otherwise been dishonest.  See:

• Tim Lambert types are the reason nobody can trust WP presentations
• History of namecalling in WP talk sections
• Detailed history of edits on WP's Lott page
• Other points

-- Cherry picking surveys on gun ownership.

In an audacious display of cherry-picking, Lott argues that there were “more guns” between 1977 to 1992 by choosing to examine two seemingly arbitrary surveys on gun ownership, and then sloppily applying a formula he devised to correct for survey limitations. Since 1959, however, there have been at least 86 surveys examining gun ownership, and none of them show any clear trend establishing a rise in gun ownership. Differences between surveys appear to be dependent almost entirely on sampling errors, question wordings, and people’s willingness to answer questions honestly.

My paper with Mustard as well as my book looked at all the crime data available when those pieces were written and I updated that data with each successive updated edition of my book.  

-- Paper with David Mustard: crime data for all the counties and states in the US from 1977 to 1992.  

-- First edition of MGLC: crime data for all the counties and states in the US from 1977 to 1992 as well as up to 1994 for a comparison.  Literally hundreds of different factors that could impact crime rates were accounted for.

-- Second edition of MGLC: crime data for all the counties, cities, and states in the US from 1977 to 1996.  

-- Third edition of MGLC: crime data for all the counties and states in the US from 1977 to 2005.  

The regressions in those publications account for all the data available (all counties, all cities, all states for all the years the data is available), no cherry picking, and, following earlier work by William Alan Bartley and Mark Cohen, report all possible combination of these hundreds of control variables to show that the results are not sensitive to a particular specification.

The only survey discussion that I made in my first two editions of MGLC was for the 1988 and 1996 voter exit poll surveys.  Those two exit polls included a question on gun ownership.  The third edition of MGLC updates the data to include the 2004 exit poll survey.  The reason for using those large exit polls is that they can contain up to 32,000 people surveyed (though in other years it might only be about 3,600) and that allows one to breakdown the data on a state by state basis to see how gun ownership is changing across different states.  The GSS survey only has data for 600 to 800 observations at a time every two years.  Some other surveys may occasionally have up to 1,200 people, but those samples are just too small to make cross state comparisons.  So I wasn’t looking at these exit poll surveys to check general gun ownership rates for the whole US, but to look at the data for specific states.

However, we know this assertion is factually untenable, based on surveys showing that 5-11% of US adults already carried guns for self-protection before the implementation of concealed carry laws.

The surveys that DeFilippis and Hughes are referring to involve people carrying guns for any reason, including going hunting or simply moving guns between places (See the 

discussion in MGLC).  

It’s extremely unlikely, therefore, for the 1% of the population identified by Lott who obtained concealed carry permits after the passage of “shall-issue” laws to be responsible for all the crime decrease.

Again, I refer to the same discussion from MGLC as it shows that this 1% number is misleading and it also shows a simple numerical example regarding what would be required to get the expected reduction in crime.  This is part of a consistent pattern where DeFilippis and Hughes make no attempt to discuss the responses that I have already made on these issues.

On Hood and Neeley -- "zip codes with the highest violent crime before Texas passed its concealed carry law had the smallest number of new permits issued per capita."

I have a long discussion about why purely cross-sectional analysis is unreliable.  Regarding: "zip codes with the highest violent crime before Texas passed its concealed carry law had the smallest number of new permits issued per capita.”  Well, given that it cost $140 and 10 hours of training to get a permit, it isn’t very surprising to me that poor areas have both high crime rates and low permit rates.  As to cherry-picking, even if cross-sectional analysis was useful, somehow the authors have to explain why they picked one city in the entire US to look at.  In any case, I note this paper and respond to it in MGLC.

Note on the Dade county data.  

Dade county police records, which cataloged arrest and non-arrests incidents for permit holders in a five-year period, also disproves Lott’s point. This data showed unequivocally that defensive gun use by permit holders is extremely rare. In Dade county, for example, there were only 12 incidents of a concealed carry permit owner encountering a criminal, compared with 100,000 violent crimes occurring in that period. . . .

Anyone who has been following the debate on justifiable police homicides knows that the data is not very reliable.  The justifiable homicide data for civilians is even worse.

As Albert Alschuler explains in “Two Guns, Four Guns, Six Guns, More Guns: Does Arming the Public Reduce Crime,” Lott’s work is filled with bizarre results that are inconsistent with established facts in criminology. . . .

My responses to these claims can be found in MGLC (here and here), though DeFilippis and Hughes ignore my responses.

Dennis Hennigan writes, “the absence of an effect on robbery does much to destroy the theory that more law-abiding citizens carrying concealed guns in public deter crime.”

My response to this type of point is available here in MGLC.

Frank Zimring and Gordon Hawkins as well as Dan Black and Daniel Nagin are intertwined here.

Black and Nagin noticed that there were large variations in state-specific estimates for the effect of “shall-issue” laws on crime. For example, Lott’s findings indicated that right-to-carry laws caused “murders to decline in Florida, but increase in West Virginia. Assaults fall in Maine but increase in Pennsylvania.” In addition, “the magnitudes of the estimates are often implausibly large. The parameter estimates that RTC laws increased murders by 105 percent in West Virginia but reduced aggravated assaults by 67 percent in Maine. . . .

Again, DeFilippis and Hughes ignore that I have extensive discussions on this in both MGLC and a 1998 paper published in the Journal of Legal Studies.   

1) Note that even throwing out all counties with populations below 100,000 and Florida, still produced statistically significant drops in some violent crime categories.  They thus removed about 89 percent of the data in the study.  There are so many combinations of county sizes and states that could have been dropped from the sample -- for example, why not Georgia or Pennsylvania or Virginia or West Virginia or any of the other six states?  Why not drop counties with populations under 50,000?  Black and Nagin never really explain the combination that they pick.

2) More importantly, even when they drop out counties with fewer than 100,000 people as well as Florida, Black and Nagin still find statistically significant drops in aggravated assaults (significant at the 5% level) and robberies (significant at the 8% level) and no evidence that any type of violent crime increases.  Note that they also didn't report over all violent crime, and the reason that they don't report that is because even with their choices the drop in over all violent crime would have been statistically significant.

3) As to the increase in West Virginia, there was only one county in WV (Kanawha County) with more than 100,000 people in it.  What they showed is not that crime increased in WV (it fell over all), but that there was an increase in one type of violent crime in one county in WV.

4) DeFilippis and Hughes continually write about "Florida" being removed from the sample, but it is Florida as well as counties with fewer than 100,000 people.

5) If one is interested in my other responses, I suggest that people read both MGLC and the paper published in the Journal of Legal Studies.

Regarding  Ted Goertzel's comments, DeFilippis and Hughes plagiarize/copied his comments in their discussion of Dan Black and Nagin.  In general their approach is to copy, slightly rewrite other critiques, and then ignore what I have written in response.

DeFilippis and Hughes write: 

Within a year, two econometricians, Dan Black and Daniel Nagin validated this concern. By altering Lott’s statistical models with a couple of superficial modeling changes, or by re-running Lott’s own methods on a different grouping of the data, they were able to produce entirely different results.

Goertzel wrote:

Within a year, two determined econometricians, Dan Black and Daniel Nagin (1998) published a study showing that if they changed the statistical model a little bit, or applied it to different segments of the data, Lott and Mustard's findings disappeared. Black and Nagin found that when Florida was removed from the sample there was "no detectable impact of the right-to-carry laws on the rate of murder and rape." They concluded that "inference based on the Lott and Mustard model is inappropriate, and their results cannot be used responsibly to formulate public policy."

This is one time where DeFilippis and Hughes pretend that they are actually linking to what I wrote in response to Goertzel, but instead they misstate what I wrote and link back again to Goertzel.  My responses to Goertzel were similar to what I just note above in response to Black and Nagin.  

DeFilippis and Hughes claim "Lott’s response to Goetzl was to shrug him off, insisting that he had enough controls to account for the problem."  But that is not accurate.  I point out that I was also concerned that the sensitivity of specifications.  That is why I pointed to papers such as the one by Bartley and Cohen that provided tests of whether the results were indeed sensitive.

As to Ayres and Donohue's 2003 law review paper, DeFilippis and Hughes are just simply wrong about the facts.  They write:

"Fortunately, Lott’s data set ended in 1992, permitting researchers to test Lott’s own model with new data. Researchers Ian Ayres, from Yale Law School, and John Donohue, from Stanford Law School, did just this, and examined 14 additional jurisdictions between 1992 and 1996 that adopted concealed carry laws."

The 2nd edition of MGLC came out in 2000 and, as noted above, it had data through 1996.  I provided Ayres and Donohue with my data set and they added one year to the study, 1997.  That single year did not change the results.  While Ayres and Donohue also claimed that the my research had ended with 1992, anyone who checks the 2nd edition of the book or reads chapter 9 in the third edition will see that I had looked at data from 1977 to 1996.

The reply to Ayres and Donohue in the law review was by Florenz Plassmann and John Whitley.  I had helped them out and Whitley notes "We thank John Lott for his support, comments and discussion."  There were minor data errors in the additional years that they added from 1997 to 2000, but those errors didn't alter their main results that dealt with count data.  They had accidentally left 180 cell blank out of some 7 million cells.  Donohue has himself made much more serious data errors in his own work on this issue.  For example, he repeats the data for one county in Alaska 73 times, says that Kansas' right to carry law was passed in 1996 and not 2006, and made other errors.  I did co-author a corrected version of the Plassmann and Whitley paper that fixed the data errors and is available here.  But DeFilippis and Hughes can't even get it straight what paper I co-authored.

In any case, for those who want my response, you can read what I wrote in MGLC (the link only provides part of my discussion).

Again, talk about DeFilippis and Hughes cherry-picking, there are several ways of responding to the quotes by Kleck and Hemenway.

1) Note that Kleck has also said many positive things about my research. For example, see this quote: “John Lott has done the most extensive, thorough, and sophisticated study we have on the effects of loosening gun control laws. Regardless of whether one agrees with his conclusions, his work is mandatory reading for anyone who is open-minded and serious about the gun control issue. Especially fascinating is his account of the often unscrupulous reactions to his research by gun control advocates, academic critics, and the news media.” 
2) I have discussed Kleck's quote in MGLC (see attached file). 
3) The vast majority of peer reviewed research that looks at national data on crime rates supports my research (see table 2 here and also here).  
4)  There are a lot of prominent academics and people involved in law enforcement who have said positive things about my research.  I can list a few here, but I don't really see the point.

“John Lott documents how far ‘politically correct’ vested interests are willing to go to denigrate anyone who dares disagree with them.  Lott has done us all a service by his thorough thoughtful, scholarly approach to a highly controversial issue.”

— Milton Friedman, Nobel prize winning economist

“John Lott is a scholar’s scholar and a writer’s writer — and this book shows why.  That gun ownership might bring social benefits as well as costs is a story we do not often see in the press, and Lott here explains why.  With a blend of new data, evidence, and examples, he unpacks the bias against such stories in the media.”

— Mark Ramseyer, Harvard University

“For anyone with an open mind on either side of this subject this book will provide a thorough grounding. It is also likely to be the standard reference on the subject for years to come.”
—Stan Liebowitz, University of Texas at Dallas

“John Lott’s work to uncover the truth about the costs and benefits of guns in America is as valuable as it is provocative. Too much of today’s public debate over gun ownership and laws ignores the empirical evidence. Based on carefully proven facts, Professor Lott shatters the orthodox thinking about guns and debunks the most prominent myths about gun use that dominate the policy debate. For those who are convinced that the truth matters in formulating public policy and for anyone interested in the role of guns in our society, More Guns, Less Crime is must reading.” —Edwin Meese III, U.S. Attorney General, 1985–88

“More Guns, Less Crime is one of the most important books of our time. It provides thoroughly researched facts on a life-and-death subject that is too often discussed on the basis of unsubstantiated beliefs and hysterical emotions.”
—Thomas Sowell, Stanford University

“Armed with reams of statistics, John Lott has documented many surprising linkages between guns and crime. More Guns, Less Crime demonstrates
that what is at stake is not just the right to carry arms but rather our performance in controlling a diverse array of criminal behaviors. Perhaps most disturbing is Lott’s documentation of the role of the media and academic commentators in distorting research findings that they regard as politically incorrect.”

—W. Kip Viscusi, Cogan Professor of Law and Director of the Program on Empirical Legal Studies, Harvard Law School

“Until John Lott came along, the standard research paper on firearms and violence consisted of a longitudinal or cross-sectional study on a small and artfully selected data set with few meaningful statistical controls. Lott’s work, embracing all of the data that are relevant to his analysis, has created a new standard, which future scholarship in this area, in order to be credible, will have to live up to.”

—Dan Polsby, Kirkland & Ellis Professor of Law, Northwestern University

“His empirical analysis sets a standard that will be difficult to match. . . . This has got to be the most extensive empirical study of crime deterrence that has been done to date.”

—Public Choice

Up to this point in their list, I have tried to go through each of DeFilippis and Hughes' claims.  What should be clear is that I haven't skipped points and I have already answered these claims elsewhere and the same is true for their other assertions.  I would suggest that people get a copy of MGLC for issues up to 2010 and look at my later academic papers at the Social Science Research Network or the Crime Prevention Research Center website.  Regarding their attack on "The Vanishing Survey," they again completely ignore what I have already written on the issue.  Without any attempt to address any of the responses that I have already made to these points, DeFilippis and Hughes' is just a big waste of people's time.

For example, regarding Donohue's latest piece that DeFilippis and Hughes you can see discussions here, here, and here.

I will make one final point.  DeFilippis and Hughes incorrectly describe the National Research Council report.  Their report examined seemingly ever possible gun law that has been studied by academics, but the panel could not identify one single law that made a statistically significant difference.  They made the same response regarding right-to-carry, but unlike all the other laws studied the discussion on right-to-carry laws was the only one that drew a dissent by James Q. Wilson, who pointed out that all of the panel's own regressions found that right-to-carry regressions reduced murder rates.  In 15 years prior to that there had only been one other dissent.  Academics who don't sign on to a NRC report are not invited back to be on future panels.  That creates pressure for people not to dissent, but it also means that virtually all the reports indicate that they can't say anything matters.

Ann Coulter on "America's Most Feared Economist"

Ann's newest piece starts this way:

You can tell the conservatives liberals fear most because they start being automatically referred to as "discredited."  Ask Sen. Ted Cruz. But no one is called "discredited" by liberals more often than the inestimable economist John Lott, author of the groundbreaking book More Guns, Less Crime: Understanding Crime and Gun Control Laws.  

Lott's economic analysis of the effect of concealed-carry laws on violent crime is the most thoroughly vetted study in the history of economics, perhaps in the history of the world.  

Some nut Dutch professor produces dozens of gag studies purportedly finding that thinking about red meat makes people selfish and that litter leads to racism -- and no one bothers to see if he even administered questionnaires before drawing these grand conclusions about humanity.  

But Lott's decades-long studies of concealed-carry laws have been probed, poked and re-examined dozens of times. (Most of all by Lott himself, who has continuously re-run the numbers controlling for thousands of factors.) . . .

Read the rest available here.

Note on John Donohue's latest paper

Abhay Aneja, John J. Donohue III, and Alexandria Zhang have a new paper published in the ALER entitled: "The Impact of Right-to-Carry Laws and the NRC Report: Lessons for the Empirical Evaluation of Law and Policy." Below is just a small portion of the correspondence that I have had trying to get the data for this paper. Many of the issues discussed in this paper, such as an index for cocaine use, are already used in the third edition of MGLC. The paper continually makes false claims about issues such as whether I have accounted for law enforcement or incarceration. Finally, I will make a very simple point: by throwing out the arrest rate variable, these authors create a truncation problem. Crime can't fall below zero no matter how effective a crime fighting effort is. There are lots of ways to deal with the truncation issue (count data, Tobit, etc), but these authors introduce this problem and they bias their results towards zero.  For those interested, a copy of the response that I put together regarding Donohue's paper can be found here.

This is an email that I have sent off this evening.

Dear John:

It is my understanding that your paper has now appeared in the ALER. It is very disappointing that you have chosen to delay providing the necessary data for 3.5 months until after your paper has been published.

What makes your delay especially disappointing are your allegations at the beginning of your conclusion asserting that I have provided flawed data to others so that you can not replicate various results. You make this claim without specifically explaining where those mistakes in the original data sets are, obviously putting a burden on me to sort through your data set and respond.

In sharp contrast to your approach, when you and your co-authors have asked that I provide you with data before my papers have been published, I have provided it within days of the request, and my papers were not attacks on your work. Your claims that you simply didn't understand my multiple requests or kept forgetting them just seems too clever by far.

Your unwillingness to cite or acknowledge or let alone respond to the third edition of my book that came out in 2010, which already anticipates many of your attacks, goes to your integrity as a researcher.

It is amazing that your paper can go from submission to publication faster than you can provide the data for the paper.

John


From: John Donohue
Date: Friday, November 11, 2011 Friday, November 11, 2:20 PM
To: John Lott
Cc: John Donohue , shavell@law.harvard.edu, emorri@law.columbia.edu
Subject: RE: Response to Donohue

Sorry for the delay. I had asked my RA to put it up on my bepress web page and he had told me it was up. see email below. If not there by Monday, let me know.


All best,

John

John J. Donohue
C. Wendell and Edith M. Carlsmith Professor of Law
Stanford Law School
559 Nathan Abbott Way
Stanford, CA 94305
650 721 6339
650-723-4669 (FAX)
650 575-7166 (cell)
http://works.bepress.com/john_donohue/


all best,

John

. . .


From: John Lott [mailto:johnrlott@aol.com]
Sent: Thursday, November 10, 2011 5:14 PM
To: John Donohue
Cc: John Donohue; shavell@law.harvard.edu; emorri@law.columbia.edu
Subject: Re: Response to Donohue


John:

My emails are perfectly clear. Reread the October 21st email for a recent request. It is my understanding that Carl Moody has also had correspondence making the exact same requests.

This information is particularly crucial in this case since you claim that you can't replicate previous results. I have made repeated data requests since August 4th. Steve and Ed have now seen several of my requests themselves.

You ask someone to write a comment, but you delay for months the information that is necessary to write such a comment. Your delaying tactics are simply unacceptable. When you and your co-authors have asked for data, I have provided it within days, not well after several months.

John

On Thursday, November10, 2011, at Thursday, November 10, 9:27 AM, John Donohue wrote:


My apologies. I didn't realize. Can you re-send any outstanding questions?
John

Sent from my iPhone

On Nov 10, 2011, at 3:38 AM, John Lott wrote:


John:

It has been three weeks since my last email and I have yet to receive any response from you on getting the rest of the needed information. Are you planning on providing this information or are you going to wait until after your paper has been published? These delays are really inexcusable.

John

Begin forwarded message:


From: John Lott
Subject: Fwd: Response to Donohue
Date: Friday, October 21, 2011 Friday, October 21, 11:22 AM
To: John Donohue
Cc: shavell@law.harvard.edu, emorri@law.columbia.edu

John:

I have finally received part of the data for the paper (thank you), but I have not received everything. I still need the "do" file. It is my understanding that Carl Moody has also asked for this, but that he has not yet received it either. I look forward to receiving it soon. Thank you.

John

. . .

Begin forwarded message:

From: John Lott [mailto:johnrlott@aol.com]
Sent: Thursday, August 04, 2011 4:21 AM
To: John Donohue
Subject: Re:

Dear John:

Please provide me the data for your ALER paper or tell me where I can obtain it.

Sincerely,
John

"What Passes for Debate Among the Left in Academia"

It would be nice if there was some real condemnation for these types of personal attacks in academia.

Donohue wasn’t willing to let Wilson’s dissent go unanswered, and the attacks became quite personal. In a debate carried nationally on National Public Radio, Donohue attacked Wilson as letting his alleged employment by the NRA bias his academic findings:

The lone dissenter [Wilson] was someone who was not an econometrician, who admitted in his dissent that he wished he knew more econometrics, and who had previously testified as an expert witness on behalf of the execrable NRA.

When later called on this claim after the debate, Mr. Donohue did not offer proof, but instead called on Wilson to prove that he had never gotten paid by the NRA. When asked for evidence, . . . .

Responding to Jack D'Aurora's piece in the Columbus Dispatch

A little over a week ago, Jack D'Aurora had a pretty amazing piece in the Columbus Dispatch (available here), where Mr. Jack D'Aurora made the mistake of accepting what Ayres and Donohue wrote in their 2003 law review piece as being accurate. The mistakes listed below are basically mistakes in the Ayres and Donohue paper. My Letter was finally published on Saturday (available here).

Will letting law-abiding citizens carry concealed handguns deter criminals and offer protection? Or is Jack D'Aurora right (Forum column, July 29) when he states that “heated arguments can escalate into shootings” and that criminals will become “more inclined to pack heat”?

He alleged that my research, published in my book More Guns, Less Crime, ignored such concerns. But obviously D'Aurora did not trouble himself to actually read any edition of my book.

According to D'Aurora, my research found a benefit from concealed-handgun laws merely because it covered the time period from 1977 to 1992, and, supposedly, if I had studied the seven years up to 1999, the results would have been different.

This is absurd. Even the first edition of my book covered up to 1994. I updated the evidence with each new edition, and by the third edition in 2010, I covered county- and state-level data for all the years from 1977 to 2005. No matter the time period, the results remained very similar.

D'Aurora also conveniently ignored the massive amount of work by other researchers during the past 15 years. Among peer-reviewed studies in academic journals by criminologists and economists, 18 studies examining national data found that right-to-carry laws reduce violent crime, 10 indicated no discernible effect, and none — absolutely none — found a bad effect from the law.

D'Aurora further mischaracterized the law-review article he cites by Ian Ayres and John Donohue. That paper claimed to find a small temporary increase in crime, subsequently followed by a downward trend in crime rates. Other evidence of theirs shows no temporary increase.

His accusation that I “did not account for the large increase in crack cocaine use” is false: I did so in five different ways, such as studying county-level cocaine prices.

Questions about whether heated arguments can escalate into shootings were not ignored. If permit holders did these things, they would be put in jail and lose their permits. But permit holders are extremely law-abiding. In Ohio, about 178,000 had concealed-handgun permits last year, but just 206 — 0.1 percent — had their permits revoked. Most revocations involved people losing their permits because they moved out of state, died or decided not to hold their license anymore.

Other states provide more detailed data. Florida is typical. From Oct. 1, 1987, to June 30, 2011, more than 2 million people were issued permits, with the average person possessing one for more than a decade. One hundred sixty-eight (about 0.01%) had their permits revoked for any type of firearms-related violation, mostly for accidentally carrying a concealed handgun into a gun-free zone.

There is no research showing that criminals are more likely to “pack heat.” In fact, right-to-carry laws cause gun crimes to drop more than non-gun crimes.

Finally, the most telling fact may be this: No state that has adopted a right-to-carry law has ever even held hearings on rescinding it.

The Abortion Debate Gets Heated

I guess that it would have been helpful if Ms. Marjorie Signer had actually explained why my research showing that the liberalization of abortion rules increased crime contained "many unfounded, fallacious, racist, and confused assumptions."

"Lott offers so many unfounded, fallacious, racist, and confused assumptions that his overall opinions are useless," said Marjorie Signer, spokeswoman for the Religious Coalition for Reproductive Rights, in a statement sent to Cybercast News Service.

"To speculate that abortion is responsible for various and sundry social trends is irresponsible and misses critical points," she said.

"Abortion has always been an option that women at all socioeconomic levels have considered, and many abortions are performed for medical reasons. Trying to correlate abortion with social trends smacks of pseudo-science and shows little understanding of women's lives and decisions," Signer added. . . . .

John Donohue also refused to comment on the actual research, but was reduced to saying this: "I am a social scientist, however, so Lott's behavior has in my mind, put him outside the bounds of scientific discourse." That will certainly settle the debate, though I can understand why Donohue is reluctant to actually discuss the facts on this issue. Since John Whitley co-authored this paper with me, does Donohue's statement tar Whitley also?

Donohue also doesn't comment on recent issues regarding his co-author Levitt.

New Research that Abortion Increases Violent Crime Gets Attention

From today's Chicago Sun Times:

A high-profile economist is challenging the conclusion in the best-selling book Freakonomics by University of Chicago professor Steven D. Levitt that the legalization of abortion in the early 1970s led to a major drop in murder and other violent crimes a generation later.

John R. Lott Jr., a former U. of C. economist now teaching in New York, says the Supreme Court's 1973 Roe v. Wade decision actually caused violent crime to rise.

Lott and fellow researcher John Whitley plan to publish a paper in October in Economic Inquiry that questions Levitt's research on abortion and crime.

Lott and Levitt already were feuding over Lott's charge that Levitt had defamed him in Freakonomics.

In their new paper, Lott and Whitley say that legalization of abortion prompted a cultural change that increased the number of children born out of wedlock. Those children of unwed mothers caused murders to rise by more than 700 cases in 1998 alone, saddling the public with more than $3.3 billion in "victimization costs," the paper says.

On the other hand, Levitt's research found that Roe v. Wade resulted in a savings of $30 billion a year that crime would have cost the public.


More unwed mothers

His Freakonomics, co-authored by Stephen Dubner and published last year, says legalized abortion led to a large drop in murder and other violent crime in the late 1980s and early '90s, and continues to reduce crime.

The book suggests that if the aborted fetuses had instead been born, they would have become adults more likely to commit crimes because they were unwanted by their mothers.

To illustrate the point, the book says the five states that allowed abortion three years before Roe vs. Wade saw major declines in violent crime between 1988 and 1994 -- earlier than the other states.

But Lott says the Levitt study did not fully consider the increase of children born out of wedlock. His theory is that with the option of abortion, women became more likely to have premarital sex, but then had their babies and raised them as single parents.

Children born out of wedlock have had smaller investments in "human capital" by their parents and are more likely to get into trouble when they grow older, Lott says.

On average, his paper says, about 5 percent of whites were born out of wedlock from 1965 to 1969, rising two decades later to 16 percent. For blacks, the figure rose from about 35 percent to about 62 percent, the paper says.

Before legalized abortion, more than 70 percent of children born out of wedlock ended up in families with a father, but the fraction fell to 44 percent in 1984, according to the paper. . . . .

If you want to read the research, you can find it here. THe newspaper article says that I concede that abortion through its effect on "unwanted" births slightly reduces violent crime, but what I believe that I said is that it is possible. The net effect however is abortion increases violent crime.

Will Donohue and Levitt predict that a massive new crime wave is coming?

Guttmacher Institute, associated with Planned Parenthood, claims that: "The rate of unintended births — unintended pregnancies carried to term — rose by 44 percent among poor women from 1994 to 2001, but declined by 8 percent for wealthier women." If true, I assume that those few who still believe that abortion massively reduces violent crime rates after the errors were discovered in the original research will be predicting that we will soon see a big increase in violent crime rates. If up to 80 percent of the changes in murder rates can be explained by abortion as has been argued, those who believe this relationship must be worried we could be in for big increases in murder. (Of course, the abortion is more likely to slightly increase crime.) People born in 1994 will become teenagers starting next year. Presumably, proponents of this theory will start warning people very soon.

The Economist Magazine on Steve Levitt and Abortion

Of course, lots of people have always thought Mr Levitt was in the wrong. Even if abortion cuts crime, it is still immoral, they fulminate. But this is largely beside the point: Mr Levitt's research does not take a position on abortion's social virtues, but aims merely to uncover its societal effects. Besides, for someone of Mr Levitt's iconoclasm and ingenuity, technical ineptitude is a much graver charge than moral turpitude. To be politically incorrect is one thing; to be simply incorrect quite another.

But what is most amazing is that despite all the statistical significance being eliminated from their panel data set being eliminated when the results are done Donohue and Levitt's way with the the data they want to use, John Donohue now says what they wanted to do:

It may be asking too much of the numbers to convince everybody. “The debate over abortion and crime will not be resolved within the parameters of our paper,” says Mr Donohue. He thinks the arrest figures are “muddy” and the state population data “sloppy”. Combining the two generates so much noise, it is hard for the statistical tests to hear anything. Ted Joyce, a professor at Baruch College (part of the City University of New York), who has had his own methodological disagreements with Messrs Donohue and Levitt, also thinks the debate is stretching the data too far. He points out that if you add controls for 50 states and 12 years—as Messrs Foote and Goetz do, and as Messrs Donohue and Levitt meant to do—you are, in effect, holding another 600 things constant. This robs the data of most of their variety, and of much of their ability to explain anything.

There was no warning that these were the wrong tests at all in either of their previous papers or in Levitt's book. Levitt now refers to the “collage of evidence," but the panel data was the only test that really amounted to anything worthwhile. The rest of the data was merely cross sectional or time series.

Incorrect claims about the data in More Guns, Less Crime

I have received an email comparing my work to that of Steve Levitt's regarding claimed coding errors in More Guns, Less Crime. Of course, this discussion is not very accurate.

1) There are no coding errors in the data used in “More Guns, Less Crime.” The book used crime data from 1977 to 1996, and as far as I know, no academics have claimed that there were any coding errors in that data. An interesting and useful Stanford Law Review article by Plassmann and Whitley added an additional four years to the data and the problem arose in this additional data. Overall, less than 200 cells out of 7.5 million cells were accidentally left blank. More important, the results that Plassmann and Whitley noted were the results which they thought were the correct ones were not affected at all by this minor change in the data.

2) I had put the data set on my website as a favor to Plassmann because it was very large and he did not have the ability to put it up for people to download. The data was corrected as soon as the missing cells were discovered and it was available for people to download. A note was added to the site to alert people to the correction. While I had helped them out on their paper, I let Plassmann and Whitley make their own statements about their results. My website reported the regressions in the same way that I had done them in all my previous estimates.

3) In statements here and here, I have discussed what was involved in Levitt's errors. The regressions where they left out the fixed effects were essentially the only ones that were at all useful in testing their hypothesis because they were the only ones that didn't require large degrees of aggregation to get to their "effective abortion rate" that were related to total murder rates in a state. The work that I did with Whitley directly related the age of the murder to the number of abortions when that murderer was born.

Thanks to Bob Thomas for his email.

Levitt's Abortion "Disappeared" When "Programming" Error Fixed

Levitt's Abortion results disappeared when programming Error Fixed

"The Boston Fed's Mr. Foote says he spotted a missing formula in the programming oversight of Mr. Levitt's original research. He argues that the programming oversight made it difficult to pick up other factors that might have influenced crime rates during the 1980s and 1990s . . . . He also argues that Levitt should have counted arrests on a per-capita basis. Instead, he coundted overall arrests. After he adjusted for both factors, Mr. Foote says, the abortion effect disappeared.

Christopher L. Foote and Christopher F. Goetz's paper can be found here. Personally, I think calling this a "programming oversight" is being much too nice (See my post here to see an example of what you would have to miss to not notice whether you have used fixed effects). More importantly everyone who works with panel data knows that you use fixed effects.

My own work concentrated on murder rates, but I also included fixed effects. Donohue and Levitt never provided us with all their data or their regressions and would never answer any questions that we had so I just assumed that they had included fixed effects from the beginning. It would have been nice if they had provided us with this same information years ago.

My paper with John Whitley is available here.
A letter that I had in the WSJ is available here.
James Q. Wilson's review of this debate is available here.

UPDATE:

See also Steve Sailer's past work on this topic.
See also Steve Sailer's compliation of what others are saying here.

UPDATE II (I have been asked about inaccurate claims regarding my own research in More Guns, Less Crime):

See this for a discussion about claimed errors in More Guns, Less Crime

Revisiting John Donohue's cancelling of our gun debate last fall

1) It was very disappointing that John Donohue backed out of our debate on guns last December with just two days to go. I had looked forward to the debate, and obviously had my plane tickets well in advance. It was a date that Donohue had picked (I had said that I could do all but one day over the end of November through December and I would have gone beyond December if necessary), and the first email that I had confirming the event was received in September. I ended up speaking at Chicago on another topic so that a later debate could be attempted to be set up again with Donohue.

2) Unfortunately, the second planned debate was also cancelled. The debate was rescheduled for April 13th this year, and I made sure to reconfirm it because of the previous cancelation. The student at Chicago who set up the debate said that even though he had confirmed the debate with me multiple times and even though we had taken a date that I was told that Donohue wanted, the claim is that the debate somehow hadn't been completely confirmed with Donohue. (You would think that if the previous cancelation was an accident, Donohue would have been careful to double check this time just in case there had been any possible misunderstandings, but no one is claiming that he checked anything.) Attempts were made to get Donohue's coauthor Steve Levitt to substitute for Donohue and debate either guns or abortion claims were unsuccessful. From an email that the organizer sent me after the second debate was cancelled: "You have been the most patient and flexible speaker I could imagine . . . ."

3) In conversations with Joe Cascio (the speaker coordinator for the UC Fed Soc Chapter), I had said several times if we were going to schedule a third try at a debate, we should do it on the abortion and crime claims, especially given the amount of attention that this poorly done empirical claim had gotten. Despite Donohue claiming that the debate would actually take place this time, he was unwilling to debate this hot topic. Steve Levitt was unable to debate (he was debating the topic but only with people who didn't seem to know much about the issue) and Dubner, Levitt's coauthor, was unwilling to debate the abortion claims. The University of Chicago Federalist Society was again willing to have me give another talk that wasn't in a debate format, but they insisted that I remove the statements regarding the canceled debates from my website. (They simply viewed my posts about the April 13th event being critical of the Federalist Society (I disagree), and the earlier post that they wanted removed because they didn't want to be involved in a debate about a debate.) I told them that I would correct anything that they told me was wrong, but I wasn't going to remove the postings. If people set up debates and back out at the last moment, they should be held accountable.


On the substance of Donohue's work on guns, I would direct people to this paper by Plassmann and Whitley. There are two straightforward points that they make. 1) That the graphs at the beginning of the Ayres and Donohue work are very misleading because they don't make it clear that the sample of states is changing. The crime rates clear fall for a decade and a half and the sudden increase isn't real for the remaining states. It just looks like an increase because the couple of rural states that remain did not experience the decline in murder rates that more urban states experienced after right-to-carry laws were passed. 2) The second claim has to do with them fitting an intercept and line to the data and then limiting their reported impact on crime to only five years. Often there is no problem with this, but as Plassmann and Whitley clearly show it doesn't fit in this case. The Ayres and Donohue approach implies that crime initially increase (despite the fact that the year by year data discussed in point (1) doesn't, and it is just an artifact of their estimates over predicting in the early years because they are fitting this straight line with an intercept shift to crime that is falling at an increasing rate. But to compound the problem, they only discuss what the estimates mean for the first five years. Even with this approach if they had picked the sixth year it would have reversed their claims.

I also think that it is pretty clear why they find it difficult to debate either guns or abortion. Steve Landsburg was very nice to suggest in April that Levitt and I have a discussion at Rochester this fall, but I haven't heard anything back. I immediately wrote back saying that I was interested. Landsburg wrote in April that:

"Our department would like to arrange, sometime in the fall, to have you and Levitt visit simultaneously and give back-to-back workshops on guns, abortion or both. We want to hear your criticisms of each others' work and your responses to those criticisms. We *don't* want this to turn into a debate; we want it to be an academic seminar with all the usual rules of logic and evidence. The presenter is presumed to have the floor; there is an expectation of periodic interruptions from the audience; but there's also someone in charge to cut off questioning if it starts to take over the seminar. We would most assuredly *not* advertise this event to the general public, because we don't want an audience of advocates for any point of view. Ideally, we'd have you both here for two days, with you presenting on one day and Levitt presenting on the other. We are open to alternative suggestions. We'd pay you both an honorarium of some size to be determined, though surely far less than you deserve. We'll do what we can to make it attractive, though."

I am posting this because of an email that I received asking questions about Donohue's cancelation last fall.

John Donohue's inaccurate claims

The fact that John Donohue's own research does not show any statistically significant increase in violent crime after right-to-carry laws are passed apparently doesn't stop him from claiming that to the media. A nice discussion of Ayres and Donohue's results can be found here. The estimates range from not finding any affect of the law to those finding large drops in violent crime rates. That doesn't stop Donohue, though. Of course, it would be nice if he decided to show up for scheduled debates (see the posts for 11-30-04 and 4-7-05, an additional attempt to set up a debate is discussed here: 5-7-05).

Thanks to Alphecca for pointing to this article.

Abortion and Crime "debate" at the University of Chicago Scheduled for May 25th

The University of Chicago Federalist Society has tried for a third time to set up a debate between myself and John Donohue. Since the last two debates on the issue of guns at the University of Chicago were canceled at the last moment with Donohue withdrawing from one debate with just 2 days to go, I thought that we might have more luck scheduling a debate on another topic that is getting a lot of attention these days: abortion and crime. Donohue and Steve Levitt were the coauthors on a paper that got a lot of attention on this issue and Levitt as also recently coauthored a book with Steve Dubner that again goes over the issue. All three were asked to pick a time to debate the issue, but even though Donohue is free on the 25th and despite all the attention currently being given to the abortion research, none of them were willing to debate their work on abortion with me. (I think that I know why.) I will still be presenting on the 25th with the hope that Donohue will change his mind and defend his research.

Further discussions about John Donohue, including his backing out of one of the two previously scheduled debates on guns, are here.

Debate with John Donohue is again canceled, this time with only six days to go

It seems like we have been through this before. In December, John Donohue cancelled our debate that was scheduled at the University of Chicago for December 2 on November 30th. After what had happened in the Fall, I double checked to make sure everything was confirmed before I turned down another talk that I had the chance to give, and I was assured that the event was set and that I should get my plane ticket. This time our debate that was scheduled at the University of Chicago on April 13th and canceled with just six days to go.

Further discussion of Ian Ayres & John Donohue are here.

Update: Here is an email that I received from Joe Cascio:


On Apr 7, 2005, at 3:08 PM, wrote:

Dear Dr. Lott,

I know that you are not going to be happy about this, but unfortunately, Professor Donohue cannot make it on the 13th of April. I am currently trying to work out the 21st (a Thursday with him). This time it really is more my fault. If you think that you could make it on the 21st (if that works out--I will let you know when Professor Donohue has his plane reservations in his hands), then we, of course, will pay for your current ticket and/or any changes that you have to make (or for both your current ticket and a new one if it is unalterable). If, on the other hand, you don't think you might be able to make it on that date or any other later in the quarter, I completely understand (obviously, we will still take care of your current reservation).

You have been the most patient and flexible speaker I could imagine and I feel as though you have been treated rather shabbily. I am really sorry about that. I still really hope that this event works out, if it does, I really feel like it might the best of our entire year. However, if it does not, I have nevertheless enjoyed working with you and I still really appreciate everything that you did for us in the fall at the last minute as well. Thank you very much, and I regret that I, once again, must convey my apologies.

Joe

Notes from me: Bold markings added. Before I bought my plane ticket, I had asked Joe in an email well before this one whether everything had been set up and he confirmed that they had. Unlike the first event being cancelled by Donohue with just two days to go, I have my suspicions about what happened here and I suspect that Joe is falling on his sword to provide cover. I was so mad that Donohue would cancel the first event the way he did, I have my suspicions that Joe is trying to keep things undercontrol by taking the blame "this time."

Donohue withdraws from Thursday's University of Chicago debate on Guns

Disappointingly, John Donohue has at the last minute withdrawn from our scheduled debate on Thursday (see note for 11/29 below). I will still give a talk, though I will instead discuss the changing judicial confirmation process. Here was the advertising of the debate in the Chicago Daily Law Bulletin:


Chicago Daily Law Bulletin

November 26, 2004, Friday

SECTION: Pg. 3

LENGTH: 921 words

HEADLINE: A Web of plans at Illinois schools

BYLINE: JERRY CRIMMINS

. . .

University of Chicago Law School

- At 12:15 p.m. Thursday, Dec. 2, at the law school, the Federalist Society will present a debate entitled, "Do More Guns Result in Less Crime?" John Lott, resident scholar at the American Enterprise Institute for Public Policy Research and author of " More Guns, Less Crime: Understanding Crime and Gun Control Laws," will debate Yale Law School Professor John Donohue, author of "Shooting Down the ' More Guns, Less Crime' Hypothesis."

UPDATE: When I gave my talk at Chicago, I asked the students if they knew what had happened, but no one apparently understood why Donohue really backed out of the debate just a couple of days before the event. The debate had been set up months in advance. I had received the same emails that Donohue and Bernard Harcourt, the moderator, received confirming the event more than a month in advance. The emails were cc'd to my address, Donohue's, and Harcourt's at the same time. If there had been some misunderstanding, Donohue should have notified everyone much earlier than two days before the event.

UPDATE 2: Here are some of my very early email correspondance with Joe Cascio, with the University of Chicago Federalist Society, from the just the first half of September. The claim has been made that the event wasn't confirmed until November 29th, but I have it first confirmed in September and then several emails after that checking on the event (including just asking if people had any questions, confirming flights, and setting up the ground rules for the debate).

From: jcascio@uchicago.edu
Date: September 13, 2004 9:29:10 PM EDT
To: John Lott
Subject: Re: Speaking to the Federalist Society w/John Donohue


Dear Dr. Lott,

Great. The topic is "Do more guns (i.e. less restrictive gun laws) mean less crime?" As far as format, anything mutually agreeable to you and Professor Donohue would be fine, but we had in mind a debate with fifteen minutes for you to speak, then fifteen minutes for Professor Donohue, then ten minutes for you, then ten minutes for him, followed by ten minutes of student questions. Please let me know if that's acceptable to you. In any case, thank you very much for accepting our invitation and I'm really looking forward to this event. Please let me know if you will need a hotel room the night of the first.

Joe

Quoting John Lott :



That should be fine. What exactly is the topic? What are the ground
rules?

On Sep 9, 2004, at 6:40 PM, jcascio@uchicago.edu wrote:


Dear Dr. Lott,

I'm sorry I didn't respond sooner to your message. I was out of town for a bit, then I checked with Donohue and Harcourt to see what day that you had said you would be available would work for them. How about December 2? We could have the debate at lunch and if you need to stay the night before we could put you up at the Omni downtown. In any case, please let me know if this can work out and thanks for your patience.

Sincerely,

Joe Cascio



Debate with John Donohue is again canceled, this time with only six days to go.

Further discussion of Ian Ayres & John Donohue