Google's Gemini 1.5 Pro Tops AI Leaderboards, Excels in Multi-Lingual and Complex Tasks
August 6, 2024Google has launched its latest AI model, Gemini 1.5 Pro, for early testing through Google AI Studio and the Gemini API.
This new release builds on the previous Gemini 1.5 model, which was unveiled earlier this year in February.
Gemini 1.5 Pro has quickly achieved the top position on the LMSYS Chatbot Arena leaderboard with an impressive ELO score of 1300.
This score places Gemini 1.5 Pro ahead of competitors like OpenAI's GPT-4o and Anthropic's Claude-3.5 Sonnet.
The model excels in multi-lingual tasks and demonstrates strong performance in mathematics, complex prompts, and coding.
A notable feature of the 1.5 series is its ability to process up to two million tokens, enabling in-depth analysis of large documents and media.
In practical tests, the Jamba model, developed by AI21 Labs, boasts a 256,000-token context window, which is double that of Meta's Llama 3.1.
Shoham claims that Jamba is the only model, besides Gemini, that maintains its performance with a large context window.
AI21 Labs has received positive feedback from customers exploring retrieval-augmented generation (RAG) solutions and expects to demonstrate results soon.
Shoham emphasizes the importance of context length in enhancing the effectiveness of RAG systems.
Despite the advancements in generative AI, Shoham argues that simply increasing the size of large language models will not solve all AI challenges.
He warns that generative AI still faces issues such as hallucinations and the potential for incoherent outputs.
Summary based on 2 sources
Get a daily email with more Tech stories
Sources
ZDNET • Aug 5, 2024
AI scientist: 'We need to think outside the large language model box'OODA Loop • Aug 5, 2024
Google Gemini 1.5 Pro leaps ahead in AI race, challenging GPT-4o