This new cloud service allows developers to build and run AI models with low latency, surpassing the speeds of competitors like OpenAI and Google.

SambaNova Systems has launched SambaNova Cloud, an AI inference platform that claims to offer the fastest inference speeds globally, powered by their innovative SN40L AI chip.

The SambaNova Cloud service utilizes a patented dataflow design and a three-tier memory architecture, enabling it to achieve unprecedented speeds that exceed current industry standards.

The platform runs Meta's Llama 3.1 model at impressive speeds, processing the 70 billion parameter version at 461 tokens per second and the 405 billion version at 132 tokens per second.

Rodrigo Liang, CEO of SambaNova, emphasized that their service offers world-record speeds and full 16-bit precision, making it a compelling choice for developers.

SambaNova Cloud is available in three tiers: Free, Developer, and Enterprise, allowing users to access the API for free while providing options for higher rate limits and scalable solutions.

Developers can create generative AI applications using the Llama 3.1 models through a free API, enabling rapid development of agentic applications.

Liang highlighted the platform's versatility, catering to enterprise needs with both high-speed and high-fidelity model options, which are crucial for various AI workflows.

SambaNova Cloud offers better performance and energy efficiency compared to conventional GPU systems, thanks to its use of application-specific integrated circuits (ASICs).

The platform's ability to handle the 405 billion parameter model at full precision distinguishes it from competitors that often sacrifice precision for performance.

While AI agents hold significant potential for automating processes, there remains skepticism regarding their maturity in effectively solving real-world problems.

Bigtincan, a SaaS company, has expressed enthusiasm for partnering with SambaNova, aiming for significant efficiency improvements through the platform's performance.