The Overpromise of Risk Assessments

Can a Questionnaire Predict the Future? Common Sense and Now Harvard Says No.

Remember back 13 years ago, when the Arnold Foundation (now called Arnold Ventures) was in the middle of a world tour touting its predictive criminal justice algorithm. Anne Milgram, former New Jersey Attorney General, did a very famous TED talk entitled, “Why Smart Statistics Are the Key to Fighting Crime.” She referred to what Arnold Ventures was doing as “Moneyballing the criminal justice system.” Of course, referring to the analytical player evaluation approach used by the Oakland Athletics in the movie, Moneyball. This TED talk received enormous publicity and risk assessments and Arnold Ventures became the belles of the bail reform ball.

I remember at the time thinking how absurd this whole concept was. You are telling me that the Arnold Foundation figured out a way to not only predict human behavior, but more incredibly, predict human “criminal” behavior by simply asking them 7-10 simple questions? I didn’t believe it then and I definitely don’t believe it now. And just for the record, while the “moneyball” analytical data technique was new and interesting, the Oakland A’s still never won a championship using it. Why? Because sometimes things happen that go beyond data. Life is not an algorithm. It is a series of unplanned events and circumstances that can’t be predicted or foretold. The Arnold Tool, or Public Safety Assessment, was and is just another shiny new toy for the bail reform proponents to try and use to push their cashless bail agenda through our communities. And just like every other shiny new idea and toy they have tried over the past 50 years, the PSA overpromised and underdelivered.

Last week, an article was published in an obscure Wisconsin newspaper talking about a study done by two Harvard professors. The study was an analysis of the effectiveness of the PSA in Dane County, Wisconsin. The results of the study were mind-blowing, to say the least. After all the promises and lectures on the science of predicting crime, the PSA was found to be “ineffective.” One of the researchers who was interviewed for the article said that,

“…to the extent that the county is spending money to produce the PSA, it may be that that money could be better spent elsewhere. Maybe you can spend it on something that would make things better, or maybe you can just save the taxpayers’ money.”

Another interesting thing about the study is that it was completed in 2022. That is over 4 years ago. And the results are just now coming out? How many jurisdictions have wasted tax dollars and implemented a PSA during that time? How many jurisdictions were duped into believing that they could safely release criminals into their communities? How many more victims have been created because of this catchy marketing promise of “moneyballing” the system?

Trying to fix a problem is one thing but overpromising an impossible and farfetched idea from the beginning and potentially creating more problems and more crimes should be unacceptable.

If risk assessments aren’t already dead, then this should be the last nail in the coffin. Human beings’ futures and safety should not be left to an ineffective and dangerous algorithm.

Hopefully, this Arnold Ventures “moneyball” experiment will open our community’s eyes to the reality of the challenges in our criminal justice system. Bail and bail agents are not the problem. Releasing more people or detaining more people is not the problem. We need solutions that speed up the process by which defendants are moved through the system. We need solutions that get people the help they need to prevent recidivistic behavior early on and prevent future crime. What we don’t need is another shiny new idea from people who don’t want to improve the safety of our communities but would rather just end financially secured bail.

It is time to stop throwing spaghetti at the wall looking for a solution that is right in front of our faces. It is time to bring all of the stakeholders together to use common sense and solve the real problems in our criminal justice system without worrying about who we are offending or upsetting. Public safety is not an algorithm and it is not politically correct. It is an essential and important part of a functioning society and we need it more now than ever.

Additional Sources:
Retrospective Study for the Use of the Arnold Public Safety Assessment (PSA)
Crime Research Group, September 2022