Search Site
Menu
They May Be Intelligent, But Are They Wise?

Speaking of fair shakes, here is a wise word of caution about the emerging, expanding use of computer programs to evaluate people in the justice system, whether at bail hearings, sentencings, or elsewhere.

The author is a former software engineer at Facebook who’s now studying law at Harvard. Her point isn’t that we shouldn’t use or consult these programs, but we should know what we’re getting into and proceed with caution. It’s troubling, for example, if we use programs that no one in the field fully understands—not judges, not lawyers, not probation—because the manufacturer won’t disclose a proprietary algorithm.

She says we turn to computers in part to control for our own biases, “[b]ut shifting the … responsibility to a computer doesn’t necessarily eliminate bias; it delegates and often compounds it.” That’s because these programs mimic the data we use to train them, so even the ones that accurately reflect our world will necessarily reflect our biases. Plus, they work on a feedback loop, so if they’re not constantly retrained, they lean in toward those biases and drift even further from reality and fairness. So they don’t just parrot our own biases; they amplify them. She saw this phenomenon time and again as a software engineer.

She agrees that algorithms can work for good. They’ve reportedly helped New Jersey reduce its pretrial jail population, for example.

But let’s proceed with caution, she says:

“Computers may be intelligent, but they are not wise. Everything they know, we taught them, and we taught them our biases. They are not going to un-learn them without transparency and corrective action by humans.”

Contact us

Please fill out the form below and one of our attorneys will contact you.

Quick Contact Form

Office Location
  • Irvine Office
    1 Park Plaza
    Suite 600
    Irvine, California 92614
    Phone: 714-673-6798
    Fax: 714-673-6812
Do you need an attorney?
I highly recommend Mani Dabiri

Mr. Dabiri is an excellent attorney. He is professional, ethical and sensitive to his clients situation. I highly recommend Mr. Dabiri. He helped me navigate uncharted legal waters successfully.

- a client (5 star review)

Read more

Ratings and Reviews