Retrospective: OpenAI's GPT-4 Unveiled – A New Benchmark for Large Language Models

On March 14, 2023, OpenAI released GPT-4, its most capable multimodal model, setting new standards for AI performance and safety considerations.

A Landmark Introduction: GPT-4 Redefines AI Capabilities

On March 14, 2023, OpenAI, a prominent artificial intelligence research company, announced the release of GPT-4, which it described as its “most capable and aligned model yet” (OpenAI GPT-4 Blog). This launch represented a significant moment in the rapidly evolving field of large language models (LLMs), building upon the widespread attention generated by its predecessor, GPT-3.5, particularly through the public interface of ChatGPT.

The debut of GPT-4 immediately positioned it as a new benchmark for AI performance, demonstrating advancements across reasoning, factuality, and safety. The announcement marked a pivotal step towards more sophisticated and reliable AI systems, drawing considerable interest from the technology industry, researchers, and the public alike during the week following its release.

Key Features and Performance Metrics

GPT-4’s introduction brought several notable advancements. Perhaps the most significant was its multimodal capability, meaning the model was designed to accept not only text but also images as input, generating text outputs in response (OpenAI GPT-4 Blog). While initial public access primarily featured text input, the underlying capability suggested a broader range of potential applications.

OpenAI highlighted substantial improvements in GPT-4’s performance on various professional and academic benchmarks. According to OpenAI, GPT-4 passed the simulated Uniform Bar Examination with a score in the top 90th percentile, a stark contrast to GPT-3.5’s performance in the bottom 10th percentile (OpenAI GPT-4 Blog). The model also achieved an impressive score of 1410 out of 1600 on the SAT, further illustrating its enhanced reasoning abilities. Other tests, including various AP exams and the GRE, reportedly showed GPT-4 performing at or near human-level competence (GPT-4 Technical Report).

The model was made available in two context window variants: an 8K (8,192 tokens) context and a larger 32K (32,768 tokens) context. This expanded context window allowed the model to process significantly more text than previous iterations, enabling more complex and extended conversations or document analysis.

Availability and Early Access

Upon its release, GPT-4 was immediately accessible to subscribers of ChatGPT Plus, OpenAI’s premium offering for ChatGPT, priced at $20 per month. Developers interested in integrating GPT-4 into their own applications were invited to join a waitlist for API access. The swift integration into ChatGPT Plus meant a broad user base could quickly experience the model’s new capabilities.

Notably, on March 14, Microsoft confirmed that its Bing Chat feature, which had launched in early February 2023, had already been running on GPT-4 for several weeks. This revelation provided real-world validation of GPT-4’s capabilities and stability prior to its public announcement (OpenAI GPT-4 Blog).

Emphasis on Safety and Alignment

OpenAI placed a strong emphasis on the safety and alignment efforts undertaken for GPT-4. The company published a “System Card,” detailing its safety testing, mitigations, and potential societal impacts of the model (OpenAI System Card). This document outlined a multi-faceted approach, including expert adversarial testing, red-teaming, and continuous monitoring, indicating a more proactive stance on mitigating harmful outputs compared to earlier models.

According to OpenAI, GPT-4 was 82% less likely to respond to requests for disallowed content and 40% more likely to produce factual responses on their internal evaluations compared to GPT-3.5 (OpenAI GPT-4 Blog). The System Card also acknowledged ongoing challenges and limitations, particularly concerning potential biases, hallucinations, and misuse.

The Technical Report and Omitted Details

The accompanying GPT-4 Technical Report offered a high-level overview of the model’s architecture and capabilities but notably omitted key details about its training data, model size, and computational resources used (GPT-4 Technical Report). OpenAI cited “the competitive landscape and the safety implications of large-scale models” as reasons for this decision, a departure from prior practices that sparked discussion among researchers regarding transparency in AI development.

Industry Reaction and Competitive Landscape

The immediate industry reaction to GPT-4 was overwhelmingly positive, marked by widespread discussions of its enhanced reasoning capabilities and impressive benchmark scores. Many developers and researchers lauded the multimodal potential and improved reliability. The release further solidified OpenAI’s position at the forefront of generative AI development, intensifying the competitive landscape surrounding LLMs.

At the time, other major technology companies and research institutions were also actively developing and deploying their own advanced language models. Google had recently introduced its PaLM 2 model and was integrating AI features across its product suite, while other entities were pursuing various architectural and application-specific innovations. GPT-4’s launch served to accelerate this competitive pace, compelling others to showcase their own advancements in the pursuit of more capable and versatile AI. As of March 21, 2023, the full implications of GPT-4’s release were still unfolding, but it had clearly set a new high bar for AI models.