Issues with a Washington Post Poll that claims: "77 percent of Maryland residents who bought a gun in the past 10 years say they went through a background check"

According to the Washington Post article, 81% say that they bought a gun at a traditional gun store, but fewer, 77 percent, say that they had a background check.  Even if only half bought their guns through dealers at gun shows (4.5 percent and that is undoubtedly extremely conservative since my understanding is that the gun shows in Maryland don't even let you have a table unless you are a registered dealer), that leaves a big gap between people buying from places where we would know that they would have to have background checks and those who say that they have had a check.  81% + 4.5% would imply at least 85.5% would be in places where they would have for sure had checks, and presumably even a couple of the 10 percent who bought the guns elsewhere would have had to get checks.

From the Washington Post story:
In the new poll, 77 percent of Maryland residents who bought a gun in the past 10 years say they went through a background check; 21 percent say they did not. (Given the rarity of Marylanders purchasing guns, there is a sizable 11.5 percentage point error margin for the results among this group.)
The vast majority of buyers — 81 percent — say they made their purchase at a traditional gun store rather than a gun show (9 percent) or somewhere else (10 percent). The poll finds little correlation between location of purchase and undergoing a background check. . . .
UPDATE: Glenn Kessler was nice enough to email back and forth with me on this issue.  The WP pollster responded this way:
Your researcher’s intuition is correct, but I think slicing up the data this finely goes a little beyond the capabilities of the survey. Just over 100 gun buyers were interviewed, only 8 of whom bought from a gun show. I think the survey alone is not sufficient to conclude that there’s widespread shirking of background checks at places where they are required.    
The underlying notion is correct—some survey respondents report buying guns from traditional gun stores without background checks, and gun show and private sales don’t account for the 21 percent who report no background check. As we note in the article, “it’s not clear the people  who say they didn’t undergo background checks broke the law. The survey asked about gun buying experiences over a 10-year period, so respondent recall may be an issue. The point precision is also not exact, given the relatively small number of Marylanders who say they have purchased a gun in the past decade.”    
We are working to understand these numbers deeper, and will hopefully understand better any possible issues with recall or exemptions (law enforcement officers and military/vets are exempted from some background checks). 
This is how I responded:
Thanks very much, Glenn, though I don't think that answers my question.  I am not so much trying to get into what individual categories are as seeing whether the two totals should be somewhat roughly the same.  I am really just saying that the number of purchases through FFLs has to be significantly above 81%.  I was just trying to motivate with a simple example how much it has to be above 81%, not that I was trying to breakdown the components.  Either these stores or places didn't have checks and broke the law or people didn't understand that they were having checks done or after up to 10 years they don't really remember whether a check was done.  I have a hard time believing that the first option is true.  The results indicate to me that they are significantly biased towards making the number of sales without background checks look larger than they are. Do I have permission to share this response?  Thank you very much. 
My biggest concern about the survey is how long of a period that people are asked about purchasing guns over.

I know that shorter time period reduces sample size a lot, but my biggest concern is the 10 year time window.  All the surveys that I have been involved with indicates to me that the error rate in answers increases dramatically once you go back more than one year (possibly like what we were discussing below).  If you ask people about more than a year you will start getting them including purchases outside that window and you will also tend to get more unusual events simply because people are more likely to quickly recall very unusual events when they are suddenly asked about them.

One obvious example involves defensive gun uses.  The bigger the window that you ask people about the greater the share of defensive uses will involve people firing their gun. 

In the case of purchases, the buying of a gun is probably a much more memorable event than whether there was a background check.  People will remember the purchases, but not remember whether there was a background check that went along with it.

Labels: ,


Blogger Willorith said...

Apparently the poll relies on the memory of the respondent, and accepts the respondent's statement to establish whether or not the background check took place. A technique not quite firmly rooted in scientific accuracy.

3/01/2013 4:58 PM  

Post a Comment

<< Home