MIT is Trying to Unlock the Power of the "Hive Mind"

Capturing the Crowd's Intellect

Harnessing the wisdom of the hive mind — the collective opinion of a group compared to that of a single expert — is a valuable means of gaining knowledge, but simply polling a crowd is not a foolproof way to arrive at a correct answer. That's why scientists from MIT's Sloan Neuroeconomics Lab and Princeton University decided to look for a better way to harvest the boundless potential of the hive mind.

Through their research, which is published in the journal Nature, they were able to develop a technique that they dubbed the "surprisingly popular" algorithm. This algorithm can more accurately pinpoint correct answers from large groups of people through a rather simple technique. People are asked a question, and they must give two answers. The first is what they think the correct answer is, and the second is what they think the popular opinion will be. The overall deviation between the crowd's two responses indicates the correct answer.

For example, the researchers tested their algorithm by asking the group a simple yes-or-no question: "Is Philadelphia the capital of Pennsylvania?" They then asked those people to predict how many others in the group would answer in the affirmative. Those who incorrectly thought that Philadelphia is the capital also thought most others would agree with them and also answer "Yes." Those who knew that Harrisburg is actually the capital also suspected that a majority of people would get the answer wrong and answer "Yes."

 

Thus, the percentage of people who answered "Yes" to the first question and the percentage who answered "Yes" to the second question was not equal. Though mostly everyone believed that others would answer "Yes" to the prediction question, only a small percentage (incorrectly) answered "Yes" to the first question. By going against the prediction, "No" proved to be the "surprisingly popular" answer that was also indicative of the correct answer: "No, Philadelphia is not the capital of Pennsylvania."

Using this algorithm, the researchers surveyed the crowd on subjects ranging from U.S. state capitals to medical diagnoses. They found that their technique resulted in a a 21.3 percent decrease in errors compared to simple majority votes, a 24.2 percent decrease compared to confidence-weighted votes, and a 22.2 percent decrease compared to a second form of confidence-weighted votes, ultimately generating the highest average of correct answers.

The Future of the "Surprisingly Popular" Principle

Unlike typical hive mind studies, this one adds more weight to the specialized knowledge of a particular subgroup of people within the larger group. It does not depend on the absolute majority opinion. "The argument in this paper, in a very rough sense, is that people who expect to be in the minority deserve some extra attention," said Drazen Prelec, co-author of the study and professor at the MIT Sloan School of Management, in a press release. The minority meaning those who know the correct answer but believe others will not.

Unfortunately, the technique is not yet completely foolproof. People could anticipate which answer will be "surprisingly popular" and try to avoid it. Doing do would be difficult though, and the beauty of hive mind predictions is that one outlier won't throw off the whole study. In the future, the scientists hope to utilize their method in a number of different settings, such as political forecasting, making economic predictions, pricing artwork, or grading research proposals. One day soon, the hive mind may be used as the primary way for us to make predictions and prepare for whatever the future holds.