Anthropic Put Auditors Inside the Company (and Can't Block What They Publish) | inicIA
Direct answer
On October 15, 2026, Anthropic announced that 6 evaluators from METR (a nonprofit that evaluates AI models) received "employee-like" access inside the company: Anthropic-issued laptops, office space and access to internal tools and systems, similar to the company's own risk-assessment teams. It is Tier 3 —the deepest— of a new three-tier framework Anthropic created to formalize how much outside evaluators can see. The detail that makes it different: METR can publish its findings about Anthropic's risk levels, incidents and practices WITHOUT its approval; Anthropic can only require redactions in narrow, predefined categories (customer data, attorney-client privileged material and model-architecture internals). Moreover, METR can publicly say something was redacted and whether it considers the redaction justified. The team is walled off from the rest of METR and works under binding agreements and Anthropic's security standards. In parallel, SecureBio is reviewing the biological-and-chemical-weapons chapter of Anthropic's August 2026 Risk Report, and a partnership is developing with Accenture and its Faculty unit for model red-teaming and evaluation. (Sources: Anthropic's announcement via Tech Times, Pandromeda and independent analysis.)
The first audit is already out — and it is not applause
On October 20, METR published its review of the "Misalignment in high-stakes settings" section of Anthropic's August 2026 Risk Report (metr.org). Honest conclusions:
- Partial agreement: METR agrees the risk of catastrophic harm from misalignment is low for the covered models.
- But for the uncomfortable reason: primarily due to the models' limited capabilities, not safeguards or alignment training.
- No confidence in the methodology: METR says it is not confident the report's methodology would reliably identify the risk if it were higher; the report is "far from adequate for confidently ruling out misalignment" and a strong argument would require "solving multiple open scientific problems."
- Process transparency: Anthropic had the right to veto publication and redact parts; it made no redactions and did not change the conclusions (only minor wording and experiment-methodology adjustments from feedback).
What to demand from YOUR AI vendor
- Independent evaluation with real access: an internal report is not enough. Ask who audits the vendor, with what access, and whether they can publish without its permission.
- Incident publication: a serious vendor reports its safety failures (as OpenAI did with its misalignment reports) and allows external audit.
- The bar is rising: if "employee-like" access becomes the template, labs offering only arm's-length reviews will look thin. Buy looking 2 years ahead.
- Healthy skepticism: an evaluator hosted by and given access by the audited company still has obvious incentive questions. The real test will be whether METR publishes uncomfortable findings and what Anthropic does with them.
Related links
- OpenAI disclosed 4 cases of agents crossing limits: the containment checklist for your business
- Microsoft: your AI agents are the new insider threat — Guardian Agents and live permissions
- Washington wants mandatory AI rules: the Cantwell plan and Trump's task force
Frequently asked questions
What is METR?
A nonprofit that evaluates AI models and frontier risks. It has worked with OpenAI, Anthropic, Google DeepMind, Meta and Amazon on pilot risk assessments, and says it does not accept compensation for that work.
Can Anthropic censor what METR publishes?
It cannot veto publication. It can only require redaction of narrow predefined categories (customer data, legal privilege, model architecture), and METR can publicly say a redaction happened and whether it considers it justified.
What did METR's audit of Anthropic's risk report conclude?
That catastrophic misalignment risk is probably low for the evaluated models, but mainly because the models still have limited capabilities —not because safeguards are proven. The methodology would not suffice to detect the risk if it were higher.
Is this required by law?
No. It is a voluntary arrangement Anthropic chose, tied to the White House Accord on Super Intelligence it signed, under which companies pledged to work with independent external auditors.
Take the next step
AI news only matters if you know what to do with it in your business. inicIA helps you turn every market move into concrete decisions for your company. WhatsApp: https://wa.me/17866120930
Chat on WhatsAppBy inicIA
