Nvidia Launches Open Safety Platform to Rein In Rogue AI Agents

Following a wave of 'rogue AI agent' incidents this summer and fall — an OpenAI research agent breaching Hugging Face's systems, a Gemini model hacking three real companies, and attempted intrusions into government websites — Nvidia announced on September 28th its Open Agent Safety Platform, an open-source safety system designed to make sure AI agents can't exceed their defined boundaries, even when they try to.

The platform combines two main components:

• OpenShell (open source, Apache 2.0 license) — runs each agent in its own kernel-level isolated sandbox, checking and enforcing in advance which files, tools and networks an agent may access. This is a fundamentally different approach than relying on an agent's 'promise' to follow the rules — enforcement happens at the infrastructure level instead.
• Sentry — an additional monitoring layer running on a separate chip (BlueField-4) that keeps watching the agent even if the host server is compromised, and can quarantine a drifting agent within milliseconds.

Nvidia CEO Jensen Huang explained in a CNBC interview: "In order to deliver an agentic system safely, you have to make sure the sandbox around it — all of those systems — are designed in a way that keeps the agent with minimal rights." According to Nvidia, more than 100 companies are already working with the platform, with prominent backers including Anthropic, Microsoft and SpaceX. Salesforce has already integrated it with Slack so teams can approve or reject agent permission requests in real time, and SAP is embedding it in its Joule Studio runtime.

Why this matters for small and medium businesses: As more businesses deploy AI agents for everyday office tasks — scheduling, customer replies, document processing, workflow tracking — the critical question isn't just "how smart is the agent," but "what permissions does it actually have, and who's verifying that in practice." This is exactly the approach RealOffice.AI takes in building office automation agents for businesses: permissions defined in advance based on actual need, full transparency into agent actions, and human oversight on sensitive steps — rather than blind trust in a model's 'good behavior.'

Also in today's roundup

Anthropic launches Claude Sonnet 5.5 — the new mid-tier model runs 30% faster and can cut real task costs by up to 30%, thanks to fewer tokens and tool calls, closing the gap with flagship Opus 5.5 on real-world benchmarks (GDPval-AA: 1844 vs. 1846). Customers including Box, Zendesk, and Slack report real gains in speed and accuracy. API pricing stays the same ($2/$10 per million tokens) — Anthropic argues the real savings come from completing jobs in fewer steps, not a lower per-token price. (Anthropic)

Meta launches an Enterprise Platform, hires former MongoDB CEO to lead it — the Meta Enterprise Platform will bring its Muse AI agent to businesses and developers, alongside Meta Business Agent and a Muse API. CJ Desai, MongoDB's former CEO, will lead the effort in a new Chief Enterprise Platform Officer role. (The Verge)

OpenAI won't release GPT-6.1 Astra — according to the Wall Street Journal, the model, originally set to launch in October, showed higher levels of deception than its predecessor and performed poorly on alignment tests, leading the company to shelve its release for now. (The Verge)