When BuzzFeed announced last month that it would be integrating OpenAI's text generating technology into its iconic quizzes, we were intrigued.
Not all BuzzFeed writers were thrilled with the experiment, but its premise seemed to sidestep the worst ills of publications like CNET and Men's Journal, which have used AI to crank out content that later turned out to contain serious errors and even plagiarism. At least BuzzFeed was announcing the project up front, we figured, and it's not like a silly quiz can spread misinformation.
But when BuzzFeed actually revealed the "Infinity Quizzes" today, we couldn't help but feel that they fell short of the project's potential. We've already seen AI used exploitatively, sure, but it can also be a mind-bending creative tool — and we were honestly intrigued to see what the mad geniuses at BuzzFeed were able to do with it.
The most obvious problem, though, is that the quizzes — which carry both a human byline and that of "Buzzy the Robot — just seem kind of broken. Take this one, which is supposed to cook up fan fiction about a date with your celebrity crush. We tried it a bunch of times, and the most glaring issue is that it seems to be glitching out and putting stuff in parentheses that the AI was failing to fill in. (A BuzzFeed representative didn't respond by press time, but after this story ran they reached out to say that the issue was related to a service outage with OpenAI.)
Here's what it's generating for us:
Another issue, which also seems pretty significant, is that there's hardly an "infinite" selection of scenarios on display here, as suggested by the "Infinity Quizzes" branding. It seems to be chronically looping back to Uber Blacks, a lighter shaped like a little hamburger, dumpsters, a bedroom, and a huge tattoo — which, when you subtract the details that seem to have glitched out, is pretty much the entire story. Not a lot of points for creativity, Mr. Buzzy! (Close readers will also notice that it seems to be repeatedly screwing up the "behind" line, which seems to be fed by a field in the quiz that asks for the reader's first job.)
After this story ran, a BuzzFeed spokesperson said that repeated elements had to do with the input added by the human writer.
"It looks like you tapped right into pieces of the narrative our quiz writers add to shape this new form of storytelling alongside AI," they said.
All in all, though, BuzzFeed's quiz-o-matic feels more like a particularly uninspired game of Mad Libs than anything particularly creative, or even up to the standards of your standard ChatGPT output. Given the fanfare with which the company announced its algorithmic pivot, we expected something a bit, well, better.
Points where points are due, some of BuzzFeed's other quizzes do seem somewhat better than others. One intended to generate a breakup text performed passably when fed information about a Futurism reporter's cat, although it got the grammar kind of scrambled again:
Yet another quiz, sponsored by the plant fertilizer Miracle-Gro, boldly asks the question of what the reader's soulmate would be if it was a plant. Vaguely fun, but it mainly just feels like an excuse to run some sponcon.
Maybe part of the problem is that ChatGPT does best with wildly open-ended prompts. Squished down into the standard format of a standard BuzzFeed quiz, perhaps it can't stretch its creative wings. But BuzzFeed's hands are sort of tied there — in the end, readers could probably be doing a lot more by just logging into ChatGPT, which is currently free.
In BuzzFeed's defense, the spokesperson said the company was still experimenting with the concept, and to expect further adjustments.
"We're still in the experimental phase," they said. "In the future, as we develop more of these, we're looking closely at behaviors and trends to determine the next direction — does the audience want the quizzes to be more open-ended? Or should the quizzes craft a weirdly specific narrative?"
Stronger AI rollouts are possible, though. Compare and contrast the whole thing to the New York Times' GPT-powered Valentine generator, for instance, which also dropped today and spits fire like this:
It's hard to say that BuzzFeed's effort really holds up, right? It lacks the punch, creativity, and acerbic edge that the best AI text generation efforts can bring.
All told, we were hoping for more. Obviously the stakes are low with online quizzes, but it seemed like a nice excuse to show readers a lighthearted but technically impressive side of AI. Who knows, though. Maybe BuzzFeed will improve its AI quizzes as time goes on.
Updated with input from BuzzFeed.
More on AI: Asking Bing's AI Whether It's Sentient Apparently Causes It to Totally Freak Out
Share This Article