The Defensive Gun Use Lie and the Gun Lobby’s Firehose of Falsehood - Part 6
The Defensive Gun Use Lie and the Gun Lobby’s Firehose of Falsehood - Part 6
An alternative to Kleck’s numbers: The National Crime Victimization Survey
By: Devin Hughes
This is Part 6 of a 12-part series debunking the defensive gun use myth. Part 1 examined recent high-profile incidents of DGUs gone wrong, how the NRA has seized on the defensive gun use narrative to further its guns everywhere agenda, and what constitutes a DGU. Part 2 looked at the academic origins of the DGU myth and its massive flaws. Part 3 delved into why surveys of statistically rare events produce substantial overestimates. Part 4 explored the surprising parallel pro-gun academic Gary Kleck draws between defensive gun use and using cocaine. Part 5 explained how most DGUs reported in surveys are likely aggressive and illegal.
Today we will look at the National Crime Victimization Survey’s DGU numbers as an alternative to private surveys.
Part 6: An alternative to Kleck’s numbers: The National Crime Victimization Survey
Until 2014, The National Crime Victimization Survey (NCVS) was the only real alternative to private surveys that measure DGUs on a national scale. The NCVS is conducted semi-annually by the Bureau of Justice Statistics, and surveys a nationally representative sample of 135,000 households “on the frequency, characteristics, and consequences of criminal victimization in the United States.” During the period of Kleck’s study, there were “only” 50,000 households in the sample.
The survey is primarily focused on crime, not defensive gun use, and has been carefully refined over its decades-long history to be as accurate as possible. Unlike regular law enforcement data, it is able to collect data on incidents that are not reported to the police, providing a fuller picture of crime in the U.S.
From 2007-2011, NCVS extrapolated 235,700 DGUs in the United State in response to violent nonfatal crimes, which translates to slightly more than 47,000 annually. During the same period, NCVS estimated 103,000 victims of property crime used a firearm in self-defense, or approximately 20,000 annually, for a total of 67,000 total DGUs annually.
These figures represent approximately 1% of all violent nonfatal crimes and 0.1% of property crimes (86% of property crime victims were not there during the incident, which indicates 0.7% of victims who were present during the property crime used a firearm defensively). For 2014-2018, NCVS extrapolated roughly 70,000 DGUs annually. Right before the time of Kleck’s survey (1993), NCVS estimated 83,000 DGUs annually.
The primary advantages of the NCVS are its size, frequency, and sophistication.
With 135,000 households surveyed, the NCVS dwarfs one-time surveys like Kleck’s with a sample size of merely 5,000. This decreases the chances that outlier responses will significantly shift results. Unlike one-time surveys, the NCVS has been conducted since 1973, usually semi-annually, which allows researchers to follow-up with participants and correct for telescoping, a key factor in false positives. The semi-annual nature of the NCVS also provides researchers a significant opportunity to analyze and revise methodology to enhance the survey’s accuracy.
The primary disadvantage of the NCVS survey is that the survey does not explicitly ask about DGUs, instead asking participants about attempted or completed crimes against them more generally.
Unlike Kleck’s survey, which directly asks participants whether they used a firearm in self defense, the NCVS asks the open ended questions, “What did you do?” and “Anything else?” to determine whether the participants took any defensive action. Without a direct prompt, participants might fail to specifically disclose that a firearm was used. Participants also would not have the opportunity to disclose a DGU if they used their firearm to stop a crime happening to someone else, leaving these types of incidents uncounted.
Kleck and others whose research fuels the widespread defensive gun use myth also argue that the ordering of NCVS questions is problematic. Respondents must first indicate they were the victim of a crime before they disclose a DGU. Hence, if a participant doesn’t think they were the victim of a crime (i.e. they stopped the perpetrator before a crime was committed), they won’t have an opportunity to report their firearm usage.
This concern, however, is not actually a problem. The NCVS is careful to ask about attempted crimes and threats as well as completed crimes. If a respondent wasn’t at least on the receiving end of a threat, justifying the resulting firearm use as “defensive” is difficult, if not impossible. Therefore, any DGUs that the NCVS might be excluding are much more likely to be criminal assaults rather than self defense.
Further, it provides an excellent bulwark against false positives, as some people who might have falsely claimed a DGU if they had the opportunity are filtered out when they answer “no” to the question of whether they were the victim of an attempted or completed crime.
Comparing the NCVS’s annual estimate of 60,000-80,000 DGUs with Kleck’s annual estimate of 2.5 million leads to three potential conclusions concerning the veracity of Kleck’s claims:
Asking respondents whether they were the victim of an attempted or completed crime causes at least 97% of them to forget their DGUs, a rate of induced forgetfulness unprecedented in survey history.
At least 97% of DGUs reported in Kleck’s surveys are criminal behavior given that they weren’t the victim of an attempted or completed crime.
The difference between NCVS and Kleck is the result of the elimination of false positives.
Given the available evidence, option three is by far the most likely.
The NCVS’ semi-annual schedule and the ordering of its questions are both important mechanisms to prevent false positives resulting from telescoping, social desirability bias, and lying. However, the NCVS still faces the same structural problems that plague Kleck’s and other’s one-time surveys.
Assuming a true DGU incidence of roughly 50,000 (in line with NCVS estimates), the false negative rate would have to be nearly 5,000 times larger than the false positive rate to not produce an overestimate. The order of the questions (and not asking a direct priming question on firearm use) and measures to prevent telescoping are unlikely sufficient to combat such a differential — especially in light of strong social incentives to embellish, fabricate, or mis-remember defensive gun use details.
While NCVS defensive gun use estimate totals are within the realm of plausibility, the foundational problem of false positives in surveys of rare events indicates that the NCVS also produces an overestimate.
Stay tuned for Part 7 of our 12-part series on defensive gun use, which will investigate how false statements about DGUs ended up in the 2013 National Academy of Sciences Report.
Devin Hughes is the President and Founder of GVPedia, a non-profit that provides access to gun violence prevention research, and data.
Image by Gerd Altmann from Pixabay