Computers Are Making Decisions for Us, and Not Always Good Ones.

Great Power

Algorithms are powerful tools. With these mathematical instructions for solving problems or completing tasks, you can receive potential dates from match-making services, movie recommendations from Netflix, and the ads most likely to entice you from online advertisers.

While algorithms can save us time and make decision-making more convenient, our use of them comes at a cost, especially when human lives are at stake. For example, when algorithms are used to decide who is hired or which person is given a loan, human beings are reduced to data points, and human judgement is stripped from what should be case-by-case decisions. Facebook's struggle with fake news demonstrates that algorithms don't always have the discernment a human would.

Worse yet, algorithms can reinforce racist or classist stereotypes and create problems for disadvantaged populations that are already worse off than the general population. Take the Harvard study that found that when people Googled a name linked with black people, the search engine was more likely to show the search ads selling access to criminal records. Justin Reich, executive director at the MIT Teaching Systems Lab, suggested that algorithms will inevitably benefit the people who design them — namely, educated white and Asian men.

“Most people in positions of privilege will find these new tools convenient, safe, and useful,” Reich said in a survey for a Pew Research study. “The harms of new technology will be most experienced by those already disadvantaged in society, where advertising algorithms offer bail bondsman ads that assume readers are criminals, loan applications that penalize people for proxies so correlated with race that they effectively penalize people based on race, and similar issues.”

Algorithms have the potential to widen the gap between those with power and those without it. Those who understand how these algorithms work — and how to most effectively use them — will stand to benefit from them. However, those who have less education and fewer resources will be left further and further behind, said Ryan Hayes, owner of Fit to Tweet.

“Twenty years ago we talked about the ‘digital divide’ being people who had access to a computer at home vs. those that didn’t,” Hayes said for the Pew study. “Ten years from now, though, the life of someone whose capabilities and perception of the world is augmented by sensors and processed with powerful AI and connected to vast amounts of data is going to be vastly different from that of those who don’t have access to those tools or knowledge of how to utilize them. And that divide will be self-perpetuating, where those with fewer capabilities will be more vulnerable in many ways to those with more.”

Great Responsibility

While the potential consequences of algorithms are becoming more widely recognized, the use of this tool remains widespread. Many leaders, including former president Obama, are calling for greater oversight and transparency of algorithms that impact the daily lives of citizens. In order for the risks of algorithms to be minimized, they must be made understandable for both users and expert assessors, suggested Ben Shneiderman, professor of computer science at the University of Maryland.

“When well-designed, algorithms amplify human abilities, but they must be comprehensible, predictable, and controllable,” Shneiderman commented in the Pew survey. “This means they must be designed to be transparent so that users can understand the impacts of their use and they must be subject to continuing evaluation so that critics can assess bias and errors.”

If we can harness the power of algorithms without allowing them to perpetuate injustices or dehumanize users, we could all benefit from these technological wonders, said Jonathan Grudin, principal researcher at Microsoft.

“The algorithms are not in control; people create and adjust them. However, positive effects for one person can be negative for another, and tracing causes and effects can be difficult, so we will have to continually work to understand and adjust the balance,” Grudin said in the Pew survey. “I’m optimistic that a general trend toward positive outcomes will prevail, given the tremendous potential upside to technology use.”