OpenAI's GPT-4 Just Smoked Basically Every Test and Exam Anyone's Ever Taken

Hot damn!

Freak Out

OpenAI's GPT-4 is officially here — and the numbers speak for themselves.

Hot on the heels of its announcement, OpenAI has released a bunch of stats about its even-more-powerful new large language model — and reader, we're both spooked and skeptical in equal measures.

According to a new white paper, the algorithm got incredibly good scores on a number of exams including the Bar, the LSATs, the SAT's Reading and Math tests, and the GRE.

To put these high scores in perspective, it's important to look at the average scores for all the exams GPT-4 appears to have aced. For instance, the LLM got a 163 out of 180 on the LSAT, which is more than ten points higher than the median score of 152 (per the Princeton Review) and, perhaps even more remarkably, almost twice as good as its predecessor, GPT-3.

Limiting Factors

While these stats — which, to be very clear, were released by OpenAI itself and were undoubtedly tailored to make the LLM look as impressive as possible — are indeed stunning, the firm also admitted that its latest LLM is still suffering from the same drawbacks as its predecessors.

"Despite its capabilities, GPT-4 has similar limitations as earlier GPT models," OpenAI noted on its website. "Most importantly, it still is not fully reliable (it 'hallucinates' facts and makes reasoning errors)."

"Great care should be taken when using language model outputs," the AI firm added,  "particularly in high-stakes contexts, with the exact protocol (such as human review, grounding with additional context, or avoiding high-stakes uses altogether) matching the needs of a specific use-case."

Mixed Reviews

What is clear, if nothing else, is that OpenAI is racing ahead with the release of its LLMs — GPT-3 was released in the summer of 2020; GPT 3.5, the update that gave the world ChatGPT, dropped on the first of December of last year, and now, just three-ish months later, GPT-4.

While we're still waiting to find out about GPT-4's full capabilities, it's pretty obvious at this point that there's a lot of growing momentum — and financial interest — in the AI space.

If you want to give the new model a spin, it's a lot easier than you might think: Microsoft has already confirmed that it's been using GPT-4 all along for its Bing AI search assistant.

More on OpenAI: OpenAI Confused by Why People Are So Impressed With ChatGPT