The Defensive Gun Use Lie and the Gun Lobby’s Firehose of Falsehood - Part 8
The Gun Violence Archive and national empirical data
By: Devin Hughes
This is Part 8 of a 12-part series debunking the defensive gun use myth. Part 1 examined recent high-profile incidents of DGUs gone wrong, how the NRA has seized on the defensive gun use narrative to further its guns everywhere agenda, and what constitutes a DGU. Part 2 looked at the academic origins of the DGU myth and its massive flaws. Part 3 delved into why surveys of statistically rare events produce substantial overestimates. Part 4 explored the surprising parallel pro-gun academic Gary Kleck draws between defensive gun use and using cocaine. Part 5 explained how most DGUs reported in surveys are likely aggressive and illegal. Part 6 looked at the National Crime Victimization Survey’s DGU numbers as an alternative to private surveys. Part 7 investigated the lie that there are more defensive gun uses than offensive uses, and how that lie found its way into the 2013 National Academy of Sciences Report on firearms.
Today, we examine the Gun Violence Archive’s data on defensive gun use and how it debunks the widespread DGU myth.
Part 8: The Gun Violence Archive and national empirical data
To their credit, both sides of the defensive gun use debate in the late 1990s and early 2000s enthusiastically called for more and better national empirical data on DGUs.
Empirical data allows us to directly test the DGU results from Kleck and the NCVS, unlike external validation that relies on comparisons with other crime numbers. The reason NCVS data is considered the gold standard of crime surveys is that its findings on other types of crime generally align with the data collected by the law enforcement agencies themselves (recognizing that there will always be some incidents not reported to law enforcement agencies).
If such data revealed a number of DGUs that is close to private surveys’ prediction of reported DGUs, that would be strong evidence that those surveys are fairly accurate. Even if the empirical data showed a higher number of DGUs than what NCVS shows, without reaching the millions predicted, that could still provide credence to Kleck’s hypothesis.
However, empirical data showing a low number of reported DGUs combined with the external validity tests and false positive problems mentioned previously would be fatal to the claim of widespread defensive gun use. Further, figuring out the percentage of DGUs reported to police or media sources is a challenging but important exercise to assess the validity of survey results.
The first attempt to systematically track hard data on DGUs came in a small 2004 study. It examined DGU incidents in the Phoenix metropolitan area over the course of three and a half months using newspaper reports, supplemented by police and court records.
The results: “Two DGUs involving killing assailants and one involving firing at an assailant were found. The three DGUs stemmed from cases of ‘mutual combat’ or exposed bystanders to gunfire.” As the authors concluded, “These findings cast doubt on rates of DGUs reported in an influential study by Kleck and Gertz, which predict that the police should have known about 98 DGU killings or woundings and 236 DGU firings at adversaries during the time the newspaper was surveyed. The findings reported here were closer to predictions based on the National Crime Victimization Survey, which suggest that the police should have known about eight DGU killings or woundings and 19 DGU firings at adversaries.”
While an important first step, this small study alone is not sufficient to debunk the claim of widespread defensive gun use. It is quite possible that the media reports didn’t capture all of the DGU incidents reported to police. It’s also possible that the three and a half months over which the study was conducted was a very quiet period for DGUs, and had the survey gone on longer it would have revealed substantially more cases. Also, it’s possible (though very unlikely) that the Phoenix area is an extreme outlier when it comes to people unwilling to protect themselves with firearms. National, year-round data was still needed.
Into this void stepped the Gun Violence Archive (GVA) which was founded in 2012 to comprehensively track gun violence incidents.
Utilizing more than 7,500 media and police sources across the country, GVA tracks in near real-time all forms of gun violence, except suicides. Suicide is tallied after the CDC publishes its data, and has a lag of more than a year. GVA classifies defensive gun uses as: “The reported use of force with a firearm to protect and/or defend one's self or family. Only verified incidents are reported.”
Since GVA’s founding, they have found between 1,195 and 2,119 DGUs annually.
These numbers are devastating for the claim of widespread DGUs. They are 1,000 times smaller than what Kleck’s results would predict for reported DGUs, and 10-20 times smaller than NCVS results would indicate.
When combined with the false positive problem inherent in surveys of rare events and Kleck’s results uniformly failing external validity tests, there is no longer a leg to stand on for those still supporting the widespread defensive gun use myth.
It is important to point out that these verified defensive gun use totals are potentially an undercount of the actual prevalence of overall DGUs. As GVA itself notes under its methodology, “There are sometimes questions about Defensive Gun Uses which are not reported to police. GVA can ONLY list incidents which can be verified. Our policies do not take into account stories not reported, ‘I can't believe this happened to me’ scenarios, or extrapolations from surveys. Our position is that if an incident is significant enough that a responsible gun owner fears for their life and determines a need to threaten lethal force, it is significant enough to report to police so law enforcement can stop that perpetrator from harming someone else.”
In an interview, Mark Bryant, President of GVA, fully acknowledged that GVA would miss some defensive gun uses, though he rejected assertions that media sources would deliberately avoid reporting on such cases. Rather than being seen as a socially undesirable scourge that Kleck’s writing would indicate, people with DGUs are often treated as local heroes. If the media hears about a DGU story, they often will report it. Further, unlike what gun advocates such as John Lott and Tomislav Kovandzic falsely indicate, GVA relies on both police and media sources, not just the media.
It is a completely legitimate line of inquiry to speculate how big an undercount GVA data presents; however, it is one that will never be fully satisfied. When survey estimates of defensive gun use rates are applied to GVA’s data, we obtain the following results:
If the Kleck and NCVS surveys are correct about more than half of DGUs reported to law enforcement, there are likely somewhere in the vicinity of 3,000 to 4,000 total DGUs.
If Kleck’s finding that only around a quarter of cases involve shots being fired is correct, and those are the only DGUs that are reported, then that would indicate around 8,000 total DGUs.
If Lott’s survey indicating that only 5% of DGUs involve shots being fired (an extreme outlier result) is accurate, that would indicate around 40,000 total DGUs, which is beginning to approach NCVS defensive gun use territory.
Bryant confirmed that DGU cases involving only brandishing rather than shots fired would be less likely to be picked up by police and media sources, but stressed that such missed cases are very unlikely to approach the territory necessary to support Kleck’s DGU estimates or even those by the NCVS.
Proponents of the widespread DGU theory contend that police and media reports won’t capture the overwhelming majority of DGUs. As Kovandzic has stated, “Nobody who has done their homework on defensive gun use could possibly believe reading news articles accurately captures anything but an infinitesimal share of defensive gun uses…. The only way to measure defensive gun uses is with surveys. While there is no such thing as a perfect measure of anything, the fact that they consistently show large numbers of defensive gun uses can’t be ignored.”
The Heritage Foundation makes a similar claim on its own defensive gun use tracking project (which, ironically, finds fewer DGUs than GVA): “According to the Centers for Disease Control and Prevention, almost every major study on defensive gun use has found that Americans use their firearms defensively between 500,000 and 3 million times each year. There’s good reason to believe that most defensive gun uses are never reported to law enforcement, much less picked up by local or national media outlets.”
These explanations, however, directly contradict the results contained within Kleck’s survey and the NCVS — both of which find that more than 60% of respondents claim that police found out about their DGU.
Unless police reports miss more than 99.9% (which is necessary for Kleck’s estimate to be accurate) or 96% (for the NCVS estimate to be accurate) of DGUs, the widespread DGU theory lacks any empirical support.
Under reasonable assumptions about the ratio of unreported to reported DGUs, GVA data will provide a better benchmark than surveys. Further, Kovandzic and Lott’s claims are an example of shifting goal-posts. Before empirical data on DGUs existed, both sides of the debate recognized the importance of such data. Now that such data is available, it is rejected by pro-gun commentators.
The undercounting speculation runs into the further problem that if the Kleck and NCVS surveys are completely off on the total number of DGUs, they are likely completely off on all their related DGU findings as well. As such, speculation on how much GVA is undercounting DGUs becomes a question of how socially undesirable most DGUs are.
It is important to note that even GVA’s tally is not a total of socially desirable gun uses. Many of the cases involve shootouts where neither side can reasonably claim the mantle of “good guy,” or cases where one drug dealer may break into another drug dealer’s home and is fought off with a gun.
When survey data and empirical data fundamentally contradict each other, the best course is to rely on the hard data, particularly when there is significant evidence that the surveys in question suffer from foundational flaws.
While GVA’s verified DGU count of approximately 2,000 annually is an undercount of overall DGUs, the best available evidence and logic dictates that it is closer to the true number than small private surveys and the NCVS.
Ironically, the more DGUs one believes GVA is missing, the more socially undesirable or illegal gun uses one is required to admit. After all, failure to report incidents to the police suggest one of two scenarios: either the DGU was probably illegal, or the DGU was justified but the defendant was irresponsible by not reporting a dangerous criminal to police.
Therefore, arguing that a massive number of DGUs aren’t reported is synonymous with an admission that defensive gun use, in the aggregate, is likely not beneficial for society.
Stay tuned for Part 9 of our 12-part series on defensive gun use, which will explore attempts by pro-gun commentators to resurrect the widespread defensive gun use myth.
Devin Hughes is the President and Founder of GVPedia, a non-profit that provides access to gun violence prevention research and data.
Woman with umbrella image by Topi Pigula from Pixabay; image of police by F. Muhammad from Pixabay