Anthropic has released Claude 3, claiming new industry benchmarks that see the family of three new large language models approaching “near-human” cognitive capability in some instances. Accessible via Anthropic’s website, the three new models — Claude 3 Haiku, Claude 3 Sonnet and Claude 3 Opus — represent successively increased complexity and parameter count. Sonnet is powering the current Claude.ai chatbot and is free, for now, requiring only an email sign-in. Opus comes with the the $20 monthly subscription for Claude Pro. Both are generally available from the Anthropic website and via API in 159 countries, with Haiku coming soon.

“All three feature a 200,000-token context window,” writes Ars Technica, explaining that “the context window is the number of tokens — fragments of a word — that an AI language model can process at once.”

Anthropic explains in a news announcement that its most intelligent Claude 3 model, Opus, “exhibits near-human levels of comprehension and fluency on complex tasks.”

The announcement features charts demonstrating how Claude 3 “outperforms its peers on most of the common evaluation benchmarks for AI systems, including undergraduate level expert knowledge (MMLU), graduate level expert reasoning (GPQA), basic mathematics (GSM8K), and more.”

“According to Anthropic, Claude 3 Opus beats GPT-4 on 10 AI benchmarks,” Ars Technica says, noting “all Claude 3 models ‘show increased capabilities in analysis and forecasting, nuanced content creation, code generation, and conversing in non-English languages like Spanish, Japanese, and French,’ Anthropic says.”

Ars Technica points out that in previous comparisons, Claude and Claude 2 “fell slightly behind OpenAI’s best models in capability while surpassing them in terms of context window length. With Claude 3, “Anthropic has perhaps finally caught up with OpenAI’s released models in terms of performance.”

Anthropic trained the Claude 3 series on internal data as well as “a proprietary mix of publicly available information on the Internet as of August 2023, as well as non-public data from third parties and data provided by data labeling services and paid contractors,” the company said on a research paper released in conjunction with the new models.

The Verge reports the Claude 3 models “were trained using hardware from Amazon’s AWS and Google Cloud,” noting that Claude 3 will be available on AWS’s model library Bedrock and in Google’s Vertex AI. The two tech giants have invested in Anthropic, Amazon to the tune of $4 billion and Google $2 billion.

Related:
Anthropic’s Claude 3 Knew When Researchers Were Testing It, VentureBeat, 3/4/24
Anthropic’s Claude 3 Causes Stir by Seeming to Realize When It Was Being Tested, Ars Technica, 3/5/24
AI Startup Anthropic Says New Models Cut Hallucination Risks, Bloomberg, 3/4/24