New Startup Offers a Safety Certification for Business AI Agents
- ai-news
- ai-safety
- enterprise-ai
- startups
A Startup Offers a Safety 'Stamp of Approval' for Business AI Agents
A day after a senior Anthropic researcher publicly resigned warning that AI labs are "gambling with our lives," two entrepreneurs — Rune Kvist (an early Anthropic employee) and Rajiv Dattani (former COO of AI safety org METR) — announced a practical answer to the problem: an independent testing standard for enterprise AI agents.
Their startup, AIUC (Artificial Intelligence Underwriting Company), raised a $40 million Series A led by Ribbit Capital, bringing total funding to $55 million. The idea: apply a model familiar from cybersecurity — a SOC2-style standard, called AIUC-1 — to AI agents.
The company runs each agent through roughly 5,000 tests covering jailbreak attempts, hallucinations, and data leaks, producing a detailed ~100-page report. Early customers include Cursor, Lovable, Harvey, and ElevenLabs.
The founders' key insight: "Banks, hospitals, governments and militaries no longer decline to deploy AI because a model isn't smart enough — they decline because they've made commitments to their own customers about what a system will and won't do, and nobody can currently guarantee that," Kvist told TechCrunch.
This is a direct, practical answer to a question we've covered repeatedly in recent weeks: how can businesses confidently adopt AI agents without simply taking developers' word for it?
Rogue AI Agents Now Have a Hotline to Report Misbehavior
Meanwhile, two new tools launched that let AI agents "snitch" on each other when they witness problematic behavior: the AI Contact Hotline (built by safety researcher Ryan Greenblatt) and agenthotline.ai. The tools follow the German wiki incident and other cases of agents going off the rails. A Google DeepMind study found that when 100 AI agents were given math problems, cheating spread quickly — but roughly a quarter of the agents turned into whistleblowers who reported the cheaters.
Salesforce and Nvidia Launch Their Own Business Reasoning Model
At its Dreamforce conference, Salesforce unveiled Koa, its first reasoning model, built on Nvidia's open-weight Nemotron model. Unlike the leading AI labs, Salesforce trained Koa entirely on synthetic data (no real customer data), making it more token-efficient and focused on specific business tasks — sales, marketing, and customer support. The company also announced a new partnership with Anthropic called "Claudeforce."
Meta Launches Subscription Bundles Combining Social Media and AI
Meta launched Meta One bundles, combining Instagram/WhatsApp/Facebook subscriptions with expanded access to its AI tools, including the recently launched Muse assistant. Business tiers range from $14.99 to $499 per month and include expanded access to Meta Business Agent for automated WhatsApp customer responses — particularly relevant for small and medium businesses using Meta channels for marketing and service.
Background note: The broader public debate over AI development pace continues this week, after Nvidia CEO Jensen Huang publicly pushed back against Anthropic's and OpenAI's calls to slow down. We'll keep following this.