Deep Learning Expert Says GPT Startups May Be in for a Very Rude Awakening

Generative AI exploded into the mainstream last year. Led by the Elon Musk cofounded OpenAI — the creator of both DALL-E 2, a text-to-image generator, and ChatGPT, an impressive text-generating system — the industry has absolutely exploded, as these generative tools and others, notably the image-generating systems Stable Diffusion and Midjourney, have dazzled investment firms and the broader public alike.

"Generative AI is well on the way to becoming not just faster and cheaper, but better in some cases than what humans create by hand," reads a blog post by top investment firm Sequoia Capital, published September 2022. "If we allow ourselves to dream multiple decades out, then it's easy to imagine a future where Generative AI is deeply embedded in how we work, create and play."

But despite the hefty amount of investment cash — an estimated $1.37 billion across 78 deals in 2022 alone, according to The New York Times — that VCs are throwing at generative AI companies, not everyone in the field is convinced that these generative machines are really the Earth-shifting force that both creators and investors believe them to be.

"The current climate in AI has so many parallels to 2021 web3 it's making me uncomfortable," François Chollet, an influential deep learning researcher at Google and the creator of the deep learning system Keras, wrote in a blistering Twitter threat. "Narratives based on zero data are accepted as self-evident."

In other words, Chollet is arguing that in eerily similar fashion to the blockchain bubble, hype — as opposed to firm data and proven results — is in the industry driving seat. And considering the current state of affairs over in Web3land, if Chollet's right? A failure for VC-predicted returns to materialize could spell some grim consequences for the broader AI industry.

"Everyone is expecting as a sure thing 'civilization-altering' impact (and 100x returns on investment) in the next 2-3 years," he continued. "Personally I think there's a bull case and bear case. The bull case is way way more conservative than what the median person on my TL considers as completely self-evident."

The bull case, he believes, is that "generative AI becomes a widespread [user experience] paradigm for interacting with most tech products." But Artificial General Intelligence (AGI) — AI that operated at the level of a human or above — remains a "pipe dream." So, startups based on OpenAI tech might not be rendering us humans obsolete quite yet, but they could well find a long-term role within specific niches.

The bear case, meanwhile, would be a scenario in which large language models (LLMs) like GPT-3 would find "limited commercial success in SEO, marketing, and copywriting niches" and ultimately prove to be a "complete bubble." (He does offer that image generation would be far more successful LLMs, but would peak "as an XB/y industry" around 2024.)

That all said, Chollet believes the most likely case is somewhere in between.

But even so, even Chollet's best case prediction is still way out of alignment with VC enthusiasm, where acolytes are writing checks sized to match their optimism for the tech — OpenAI, for example, is in talks to close an investment deal that would bring the company's value to nearly $30 billion.

"It's the new 'mobile' kind of paradigm shift that we've been all waiting for," Niko Bonatsos, an investor at the venture capital firm General Catalyst, told the NYT. "Maybe bigger, too."

To investors' credit, the algorithms are cool. Text-to-image-generators are genuinely impressive, and open up broad new creative frontiers for people without Photoshop chops. GPT systems, at the very least, are lots of fun to play around with.

That said, they also have a lot of problems. ChatGPT, for example, isn't always right about the very confident statements it provides, and experts fear that the tech may make it very simple to easily and efficiently generate misinformation. And though industry CEOs are open about the fact that these programs are still in relative infancy, the very real potential for destruction and blurred creative lines that they present is tough to ignore, even when backdropped against a bright — if still mostly imagined — future.

And to Chollet's point, it takes more than a product being cool and fun, or even very useful for niche things, to really be a "paradigm shift." VCs may well be taking a much bigger risk than they think they are, both fueling and feeding off of a hype cycle of half-baked products, rather than making measured calls about a situationally promising, though still quite limited, burgeoning market.

"The fact that investment is being driven by pure hype, by data-free narratives rather than actual revenue data or first-principles analysis," Chollet's thread concluded. "The circularity of it all — hype drives investment which drives hype which drives investment... narratives backed by nothing somehow end up enshrined as self-evident, common wisdom simply because they get repeated enough times by enough people."

"Everyone starts believing the same canon (especially those who bill themselves as contrarians)," he said.

More on ChatGPT: There's a Problem With That App That Detects GPT-Written Text: It's Not Very Accurate