A Guide: How to Create Malevolent Artificial Intelligence

Here's how one could create a malevolent artificial intelligence, and how we may be able to stop it.

5. 24. 16 by Federico Pistono
Image by Getty

The Age of the Machine

Technology is arguably the predominant reason that we live safer, longer, and healthier than ever before, particularly when we include medical technology – sanitation, antibiotics, vaccines – and communication technologies – satellites, the internet, and smartphones. It has immense potential, and it has been the driving force for innovation and development for centuries.

But it has a dark side. Technology, once a strong democratizing force, now drives more inequality. It allows governments and corporations to spy on citizens on a level that would make Orwell's worst nightmares look like child's play. It could lead to a collapse of the economic system as we know it, unless we find, discuss, and test new solutions.

To a certain extent, this is already happening, albeit not in a uniformly distributed fashion. If we consider a longer timeframe – perhaps a few decades – things could get far more worrisome. I think it's worth thinking and preparing sooner, rather than despair once it's too late.

Many distinguished scientists, researchers, and entrepreneurs have expressed such concerns for almost a century. On January 2015 dozens, including Stephen Hawking and Elon Musk, signed an Open Letter, calling for concrete research on how to prevent certain potential pitfalls, noting that, "artificial intelligence has the potential to eradicate disease and poverty, but researchers must not create something which cannot be controlled."

And this is exactly what Roman Yampolskiy and I explored in a paper we recently published, titled Unethical Research: How to Create a Malevolent Artificial Intelligence.

Understanding The Threat

Cybersecurity research involves investigating malicious exploits as well as how to design tools to protect cyber-infrastructure. It is this information exchange between ethical hackers and security experts, which results in a well-balanced cyber-ecosystem. In the blooming domain of AI Safety Engineering, hundreds of papers have been published on different proposals geared at the creation of a safe machine, yet nothing, to our knowledge, has been published on how to design a malevolent machine.

It seemed rather odd to us that virtually all research so far had been focused preventing the accidental and unintended consequences of an AI going rogue – i.e. the paperclip scenario. While this is certainly a possibility, it's also worth considering that someone might deliberately want to create a Malevolent Artificial Intelligence (MAI). If that were the case, who would be most interested in developing it, how would it operate, and what would maximize its chances of survival and ability to strike?

Availability of such information would be of great value particularly to computer scientists, mathematicians, and others who have an interest in AI safety, and who are attempting to avoid the spontaneous emergence or the deliberate creation of a dangerous AI, which can negatively affect human activities and in the worst case cause the complete obliteration of the human species.

This includes the creation of an artificial entity that can outcompete or control humans in any domain, making humankind unnecessary, controllable, or even subject to extinction. Our paper provides some general guidelines for the creation of a malevolent artificial entity, and hints at ways to potentially prevent it, or at the very least to minimize the risk.

We focused on some theoretical yet realistic scenarios, touching on the need for an international oversight board, the risk posed by the existence of non-free software on AI research, and how the legal and economic structure of the United States provides the perfect breeding ground for the creation of a Malevolent Artificial Intelligence.

I am honored to share this paper with Roman, a friend and a distinguished scientist who published over 130 academic papers and has contributed significantly to the field.

I hope our paper will inspire more researchers and policymakers to look into these issues. You can read the full text here: Unethical Research: How to Create a Malevolent Artificial Intelligence

Federico Pistono is an AI researcher, social entrepreneur, computer scientist, and public speaker. He holds a BSc in Computer Science from the University of Verona, and graduated from Singularity University, NASA Ames Research Park. His book "Robots Will Steal Your Job, But That's OK: How to Survive the Economic Collapse and Be Happy" explores the impact of accelerating technological innovation in society. Read more work from Federico over at his blog.

Disclaimer: The views and opinions expressed are solely those of the author. They do not necessarily represent the views of Futurism or its affiliates.

Share This Article

Keep up.
Subscribe to our daily newsletter to keep in touch with the subjects shaping our future.
I understand and agree that registration on or use of this site constitutes agreement to its User Agreement and Privacy Policy
Copyright ©, Camden Media Inc All Rights Reserved. See our User Agreement, Privacy Policy and Data Use Policy. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with prior written permission of Futurism. Articles may contain affiliate links which enable us to share in the revenue of any purchases made.
Fonts by Typekit and Monotype.