ICYMI - A Few Brief Gun Violence Data Manipulation Updates
Among all the recent political noise, gun violence disinformation is still spreading
While many of us are trying to enjoy our summer, disinformation never takes a holiday.
Hence a couple of weeks ago, GVPedia President Devin Hughes gave us an update on the gun lobby’s favorite misinformation maestro, John Lott, exposing his most recent firearm fallacies.
In case you missed it, here it is again, below.
Myth-Busting Monday: A Few Brief John Lott Data Manipulation Updates
While global turmoil has largely supplanted gun violence in the headlines over the past few months, that does not mean the purveyors of gun violence disinformation have been silent.
Chief among these disinformation-spreaders is John Lott, President of the Crime Prevention Research Center, whom we have debunked numerous times previously. Here are a few brief updates on some of his recent falsehoods that are receiving traction in the media.
By: Devin Hughes
Myth: Armed Civilians Are Stopping Active Shooters
In a widely circulated, self-published paper released in early May, Lott and his co-author, Dr. Carlisle Moody, argue that armed civilians are more effective at stopping active shooting incidents than police and suffer fewer casualties. However, the paper’s conclusions are the product of substantial data manipulation.
Back in 2022, Lott claimed that the Federal Bureau of Investigation (FBI) was missing dozens of cases in which a “good guy with a gun” had stopped an active shooter event. As we reported at the time, Lott achieved this result by covertly modifying the FBI’s definition of an active shooter event. While the FBI uses “active shooting” to refer to attempted mass public shootings that aren’t part of another crime, Lott extends it to all shootings in public that aren’t part of another crime.
Lott then applies his revised definition to only cases involving a defensive gun use (DGU). The result is that Lott deliberately excludes thousands of shootings in which a DGU did not occur from his analysis.
While one can argue whether Lott’s definition is better than the FBI’s there are only two possible conclusions:
The FBI definition is proper, in which case Lott is falsely adding incidents that aren’t real active shootings.
Lott’s definition is proper, in which case Lott is failing to include thousands of applicable shootings that don’t have a DGU.
The end result in either scenario is data fraud.
Lott and Moody’s paper from May builds on this fraudulently constructed data with a statistical analysis finding that civilians are better at stopping active shootings than police. Yet this conclusion is pre-ordained by the underlying data manipulation, but now with a new statistical analysis on top of the foundation.
Sadly, Lott is functionally immune from the consequences of data fraud as the head of his own organization, as well as hesitation by the courts to get involved in what they perceive as academic disputes. However, it is surprising that Dr. Carlisle Moody would participate in this enterprise given that he is still bound by the ethics code of William & Mary (though this is not the first time Moody has provided false data and analysis to support Lott).
A Convenient Switch in Crime Narrative
Crime is surging and the FBI is covering it up… is what Lott was arguing last October in the middle of the 2024 Election. As we reported at the time, these incendiary — and objectively false — claims were universally adopted by right-wing media, the Trump campaign itself, and boosted by Elon Musk, the richest man in the world, on X and with his political PAC.
Now, eight months later, Lott’s tune has changed: “As Deportations Rise, The U.S. Is On Track For The Lowest Murder Rate On Record.” No data has emerged between October and now to account for such a dramatic shift. The only difference is that Donald Trump is now in the White House, Lott takes the FBI at its word, and the fact that the FBI was reporting record drops in violent crime last year are ignored.
The national crime numbers are now useful for Lott’s political narrative, and with no evidence, he is claiming that the years-long downward trajectory in crime is the result of Trump’s draconian immigration actions over the past few months.
Unsurprising, but still shameful.
Surveys of Statistically Rare Events
For the final entry, Lott is correct about a survey, but misses the broader implications.
He is currently on the warpath against a survey that contends 18 million American adults were present at mass shootings. To demonstrate that the survey’s figure is an overestimate, he produces his own similarly worded survey that finds similar topline results, but then asks the participants if they were injured in the shootings. The results indicate that the number who replied “yes” is around 71 times higher than those who were shot during mass shootings.
This is substantial evidence that the original survey’s results shouldn’t be trusted. However, the problem rests not on the surveyors themselves or anything they did wrong with the questions. Rather, it is the direct result of asking about statistically rare events in a survey.
Because of the nature of false positives and false negatives (for a more detailed breakdown, check out Part 3 of our DGU series), these surveys are almost always going to overestimate the rare event. And the rarer the event, the worse the overestimation gets, and this is true for all such surveys.
In particular, and what Lott fails to grasp, is that this artifact of surveys is at the core of the defensive gun use debate. Lott is a major proponent of the claim that there are millions of DGUs each and every year. Yet, these numbers come from surveys of statistically rare events. Indeed, if we applied Lott’s own analysis of this mass shootings survey to those of DGU surveys, even by the most extreme estimate of gunshots in DGU cases (which is by Lott himself), Lott’s DGU numbers are off by at least 62 times based on observable data, and likely much more. Not far off the error rate Lott alleges in the mass shootings survey.
The moral of the story: don’t trust surveys of statistically rare events, unless they are backed up by substantial empirical data. And don’t trust John Lott.
Devin Hughes is the President and Founder of GVPedia, a non-profit that provides access to gun violence prevention research and data.
Top photo by Jayson Hinrichsen on Unsplash.
Don’t forget that Lott is not just publishing papers loaded with BS, but he is also going round the country “testifying” to state legislatures at the request of local gun groups. I have personally witnessed him spout complete nonsense to state legislative committees and seen the smiles and nods from GOP legislators who eat up all his words like candy!