The partnerships are noted as the first of their kind and come as concerns about AI risks prompt calls for regulatory measures.

The U.S. AI Safety Institute has entered into groundbreaking agreements with OpenAI and Anthropic to enhance AI safety research, testing, and evaluation.

Established under an executive order from the Biden administration, the institute operates as part of the National Institute of Standards and Technology.

In exchange for collaboration, the AI Safety Institute will offer feedback on security improvements, working closely with the UK's AI Security Institute.

These agreements are part of a broader global effort to assess the risks associated with powerful AI algorithms, amid increasing regulatory scrutiny.

OpenAI's CEO, Sam Altman, has emphasized the need for national-level testing and oversight to maintain U.S. leadership in AI development.

While participation in safety evaluations remains voluntary for AI developers, there are concerns about the ambiguous definition of 'safety'.

Previously, the Biden administration engaged with AI companies through voluntary commitments focused on safety, with 16 companies participating.

Despite the institute's establishment, it lacks legislative power to enforce compliance, relying instead on voluntary commitments from AI developers.

The White House is pursuing voluntary commitments from major tech companies to implement AI safety measures, focusing on cybersecurity and content watermarking.

Generative AI poses safety risks, including inaccuracies and misinformation, highlighting the importance of responsible development.

This initiative aims to balance innovation with safety, particularly as U.S. legislators consider regulations for AI technology.