My paper with Mustard as well as my book looked at all the crime data available when those pieces were written and I updated that data with each successive updated edition of my book.
-- Paper with David Mustard: crime data for all the counties and states in the US from 1977 to 1992.
-- First edition of MGLC: crime data for all the counties and states in the US from 1977 to 1992 as well as up to 1994 for a comparison. Literally hundreds of different factors that could impact crime rates were accounted for.
-- Second edition of MGLC: crime data for all the counties, cities, and states in the US from 1977 to 1996.
-- Third edition of MGLC: crime data for all the counties and states in the US from 1977 to 2005.
The only survey discussion that I made in my first two editions of MGLC was for the 1988 and 1996 voter exit poll surveys. Those two exit polls included a question on gun ownership. The third edition of MGLC updates the data to include the 2004 exit poll survey. The reason for using those large exit polls is that they can contain up to 32,000 people surveyed (though in other years it might only be about 3,600) and that allows one to breakdown the data on a state by state basis to see how gun ownership is changing across different states. The GSS survey only has data for 600 to 800 observations at a time every two years. Some other surveys may occasionally have up to 1,200 people, but those samples are just too small to make cross state comparisons. So I wasn’t looking at these exit poll surveys to check general gun ownership rates for the whole US, but to look at the data for specific states.
Dennis Hennigan writes, “the absence of an effect on robbery does much to destroy the theory that more law-abiding citizens carrying concealed guns in public deter crime.”
My response to this type of point is available here in MGLC.
Frank Zimring and Gordon Hawkins as well as Dan Black and Daniel Nagin are intertwined here.
Black and Nagin noticed that there were large variations in state-specific estimates for the effect of “shall-issue” laws on crime. For example, Lott’s findings indicated that right-to-carry laws caused “murders to decline in Florida, but increase in West Virginia. Assaults fall in Maine but increase in Pennsylvania.” In addition, “the magnitudes of the estimates are often implausibly large. The parameter estimates that RTC laws increased murders by 105 percent in West Virginia but reduced aggravated assaults by 67 percent in Maine. . . .
Again, DeFilippis and Hughes ignore that I have extensive discussions on this in both MGLC and a 1998 paper published in the Journal of Legal Studies.
1) Note that even throwing out all counties with populations below 100,000 and Florida, still produced statistically significant drops in some violent crime categories. They thus removed about 89 percent of the data in the study. There are so many combinations of county sizes and states that could have been dropped from the sample -- for example, why not Georgia or Pennsylvania or Virginia or West Virginia or any of the other six states? Why not drop counties with populations under 50,000? Black and Nagin never really explain the combination that they pick.
2) More importantly, even when they drop out counties with fewer than 100,000 people as well as Florida, Black and Nagin still find statistically significant drops in aggravated assaults (significant at the 5% level) and robberies (significant at the 8% level) and no evidence that any type of violent crime increases. Note that they also didn't report over all violent crime, and the reason that they don't report that is because even with their choices the drop in over all violent crime would have been statistically significant.
3) As to the increase in West Virginia, there was only one county in WV (Kanawha County) with more than 100,000 people in it. What they showed is not that crime increased in WV (it fell over all), but that there was an increase in one type of violent crime in one county in WV.
4) DeFilippis and Hughes continually write about "Florida" being removed from the sample, but it is Florida as well as counties with fewer than 100,000 people.
5) If one is interested in my other responses, I suggest that people read both MGLC and the paper published in the Journal of Legal Studies.
Regarding Ted Goertzel's comments, DeFilippis and Hughes plagiarize/copied his comments in their discussion of Dan Black and Nagin. In general their approach is to copy, slightly rewrite other critiques, and then ignore what I have written in response.
DeFilippis and Hughes write:
Within a year, two econometricians, Dan Black and Daniel Nagin validated this concern. By altering Lott’s statistical models with a couple of superficial modeling changes, or by re-running Lott’s own methods on a different grouping of the data, they were able to produce entirely different results.
Goertzel wrote:
Within a year, two determined econometricians, Dan Black and Daniel Nagin (1998) published a study showing that if they changed the statistical model a little bit, or applied it to different segments of the data, Lott and Mustard's findings disappeared. Black and Nagin found that when Florida was removed from the sample there was "no detectable impact of the right-to-carry laws on the rate of murder and rape." They concluded that "inference based on the Lott and Mustard model is inappropriate, and their results cannot be used responsibly to formulate public policy."
This is one time where DeFilippis and Hughes pretend that they are actually linking to what I wrote in response to Goertzel, but instead they misstate what I wrote and link back again to Goertzel. My responses to Goertzel were similar to what I just note above in response to Black and Nagin.
DeFilippis and Hughes claim "Lott’s response to Goetzl was to shrug him off, insisting that he had enough controls to account for the problem." But that is not accurate. I point out that I was also concerned that the sensitivity of specifications. That is why I pointed to papers such as the one by Bartley and Cohen that provided tests of whether the results were indeed sensitive.
As to Ayres and Donohue's 2003 law review paper, DeFilippis and Hughes are just simply wrong about the facts. They write:
"Fortunately, Lott’s data set ended in 1992, permitting researchers to test Lott’s own model with new data. Researchers Ian Ayres, from Yale Law School, and John Donohue, from Stanford Law School, did just this, and examined 14 additional jurisdictions between 1992 and 1996 that adopted concealed carry laws."
The 2nd edition of MGLC came out in 2000 and, as noted above, it had data through 1996. I provided Ayres and Donohue with my data set and they added one year to the study, 1997. That single year did not change the results. While Ayres and Donohue also claimed that the my research had ended with 1992, anyone who checks the 2nd edition of the book or reads chapter 9 in the third edition will see that I had looked at data from 1977 to 1996.
The reply to Ayres and Donohue in the law review was by Florenz Plassmann and John Whitley. I had helped them out and Whitley notes "We thank John Lott for his support,
comments and discussion." There were minor data errors in the additional years that they added from 1997 to 2000, but those errors didn't alter their main results that dealt with count data. They had accidentally left 180 cell blank out of some 7 million cells. Donohue has himself made much more serious data errors in his own work on this issue. For example, he repeats the data for one county in Alaska 73 times, says that Kansas' right to carry law was passed in 1996 and not 2006, and made other errors. I did co-author a corrected version of the Plassmann and Whitley paper that fixed the data errors and is available here. But DeFilippis and Hughes can't even get it straight what paper I co-authored.
In any case, for those who want my response, you can read what I wrote in MGLC (the link only provides part of my discussion).
Again, talk about DeFilippis and Hughes cherry-picking, there are several ways of responding to the quotes by Kleck and Hemenway.
1) Note that Kleck has also said many positive things about my research. For example, see this quote: “John Lott has done the most extensive, thorough, and sophisticated study we have on the effects of loosening gun control laws. Regardless of whether one agrees with his conclusions, his work is mandatory reading for anyone who is open-minded and serious about the gun control issue. Especially fascinating is his account of the often unscrupulous reactions to his research by gun control advocates, academic critics, and the news media.”
2) I have discussed Kleck's quote in MGLC (see attached file).
3) The vast majority of peer reviewed research that looks at national data on crime rates supports my research (see table 2 here and also here).
4) There are a lot of prominent academics and people involved in law enforcement who have said positive things about my research. I can list a few here, but I don't really see the point.
“John Lott documents how far ‘politically correct’ vested interests are willing to go to denigrate anyone who dares disagree with them. Lott has done us all a service by his thorough thoughtful, scholarly approach to a highly controversial issue.”
— Milton Friedman, Nobel prize winning economist
“John Lott is a scholar’s scholar and a writer’s writer — and this book shows why. That gun ownership might bring social benefits as well as costs is a story we do not often see in the press, and Lott here explains why. With a blend of new data, evidence, and examples, he unpacks the bias against such stories in the media.”
— Mark Ramseyer, Harvard University
“For anyone with an open mind on either side of this subject this book will provide a thorough grounding. It is also likely to be the standard reference on the subject for years to come.”
—Stan Liebowitz, University of Texas at Dallas
“John Lott’s work to uncover the truth about the costs and benefits of guns in America is as valuable as it is provocative. Too much of today’s public debate over gun ownership and laws ignores the empirical evidence. Based on carefully proven facts, Professor Lott shatters the orthodox thinking about guns and debunks the most prominent myths about gun use that dominate the policy debate. For those who are convinced that the truth matters in formulating public policy and for anyone interested in the role of guns in our society, More Guns, Less Crime is must reading.” —Edwin Meese III, U.S. Attorney General, 1985–88
“More Guns, Less Crime is one of the most important books of our time. It provides thoroughly researched facts on a life-and-death subject that is too often discussed on the basis of unsubstantiated beliefs and hysterical emotions.”
—Thomas Sowell, Stanford University
“Armed with reams of statistics, John Lott has documented many surprising linkages between guns and crime. More Guns, Less Crime demonstrates
that what is at stake is not just the right to carry arms but rather our performance in controlling a diverse array of criminal behaviors. Perhaps most disturbing is Lott’s documentation of the role of the media and academic commentators in distorting research findings that they regard as politically incorrect.”
—W. Kip Viscusi, Cogan Professor of Law and Director of the Program on Empirical Legal Studies, Harvard Law School
“Until John Lott came along, the standard research paper on firearms and violence consisted of a longitudinal or cross-sectional study on a small and artfully selected data set with few meaningful statistical controls. Lott’s work, embracing all of the data that are relevant to his analysis, has created a new standard, which future scholarship in this area, in order to be credible, will have to live up to.”
—Dan Polsby, Kirkland & Ellis Professor of Law, Northwestern University
“His empirical analysis sets a standard that will be difficult to match. . . . This has got to be the most extensive empirical study of crime deterrence that has been done to date.”
—Public Choice
Up to this point in their list, I have tried to go through each of DeFilippis and Hughes' claims. What should be clear is that I haven't skipped points and I have already answered these claims elsewhere and the same is true for their other assertions. I would suggest that people get a copy of MGLC for issues up to 2010 and look at my later academic papers at the Social Science Research Network or the Crime Prevention Research Center website. Regarding their attack on "The Vanishing Survey," they again completely ignore what I have already written on the issue. Without any attempt to address any of the responses that I have already made to these points, DeFilippis and Hughes' is just a big waste of people's time.
For example, regarding Donohue's latest piece that DeFilippis and Hughes you can see discussions here, here, and here.
I will make one final point. DeFilippis and Hughes incorrectly describe the National Research Council report. Their report examined seemingly ever possible gun law that has been studied by academics, but the panel could not identify one single law that made a statistically significant difference. They made the same response regarding right-to-carry, but unlike all the other laws studied the discussion on right-to-carry laws was the only one that drew a dissent by James Q. Wilson, who pointed out that all of the panel's own regressions found that right-to-carry regressions reduced murder rates. In 15 years prior to that there had only been one other dissent. Academics who don't sign on to a NRC report are not invited back to be on future panels. That creates pressure for people not to dissent, but it also means that virtually all the reports indicate that they can't say anything matters.
