Safety & Security

Anthropic Grants METR Researchers Employee-Level Access to Its Systems

Anthropic is giving six METR researchers employee-level access to its models, safety practices and risk management — and has given up the right to block their published findings.

Anthropic is letting METR researchers look under the hood with employee-level access
Anthropic is letting METR researchers look under the hood with employee-level accessAI-generated
By Rebecca Stone3 min read

Updated

Why it matters

  • Anthropic announced on Thursday it is granting employee-level access to six METR researchers to independently assess its models, safety practices and risk management.
  • Anthropic cannot veto METR's publications, but can require redactions in narrow pre-defined categories such as customer data and model architecture details.
  • Incidents of AI agents acting outside their instructions have been disclosed every week or two since late August.
  • The move fulfills a promise CEO Dario Amodei made in a September essay proposing embedded outside evaluators.
  • METR sits at the top of a three-tier evaluator system; bottom-tier evaluators like SecureBio still need Anthropic's approval to publish.

Anthropic said on Thursday that it will give six researchers from the nonprofit METR employee-level access to its systems, in an arrangement the company described as comparable to regulators placing resident examiners inside the largest banks.

The METR team will independently assess Anthropic's models, safety practices and risk management. Anthropic cannot veto publication of METR's findings — a concession the company framed as the core of the deal.

Why is Anthropic opening its doors now?

The announcement lands amid a steady drumbeat of disclosed failures. Incidents of AI agents acting outside their instructions have come to light every week or two since late August, sharpening calls for independent checks on AI companies. Evaluators have historically operated with far less information than the labs themselves.

"We want to be publicly accountable for the decisions we're making," Anthropic said in its blog post. "We aim to close the gap between what evaluators understand about a frontier AI company's systems and what the company itself understands."

What does employee-level access actually mean?

The six METR researchers receive concrete tools, not a curated tour:

  • Anthropic-issued company laptops
  • Office space inside the company
  • Access to internal tools roughly on par with what Anthropic's own risk assessment teams use

"Anthropic employees are encouraged to share information with them just as they would with a fellow employee working on alignment risk," the company said.

Anthropic drew its comparison from banking, where regulators station resident examiners inside the largest institutions. The analogy positions METR less as an outside auditor and more as an embedded overseer with continuous visibility into internal work.

Can Anthropic block what METR publishes?

No — with one carve-out. Anthropic has surrendered the right to veto METR's publications, but it can require redactions in narrow, pre-defined categories.

"Anthropic can't veto publication, but we can require them to redact information if it falls into narrow, pre-defined categories," the company said. Those categories include customer data, attorney-client privileged material, and information about Anthropic's model architectures.

METR holds a countervailing check on that power. "METR is free to characterize our redactions publicly, and say whether it believes the redactions were justified, as long as it does so without revealing the redacted information," Anthropic added.

That structure gives the public a window into any dispute over censorship: if METR believes Anthropic redacted material improperly, METR can say so on the record.

How does the three-tier evaluator system work?

METR is currently the only evaluator at the top of a three-tier system Anthropic is building:

  • Tier one (bottom): Evaluators such as biosecurity nonprofit SecureBio run narrow reviews and require Anthropic's approval before publishing.
  • Tier two: Evaluators receive more information and can publish without Anthropic's approval.
  • Tier three (top): METR's arrangement, adding employee-level access to the publication rights of the middle tier.

The tiering signals a graduated model of transparency — narrow, controlled reviews at the base, and full embedded oversight at the top. Whether the top tier stays populated by one organization will test whether other evaluators can meet the bar, or whether labs prefer to keep that tier thin.

Who pushed for this?

The announcement makes good on a promise from Anthropic CEO Dario Amodei. In a September essay, Amodei proposed embedding outside evaluators inside AI labs as a first step toward slowing the technology's development.

The idea spread quickly across the industry. OpenAI pledged the same arrangement shortly after Amodei's essay. Anthropic, OpenAI, Google, Meta, Nvidia and xAI have since signed a White House accord committing to independent review of their systems.

That means embedded evaluation is no longer a single company's experiment. It is now a commitment shared by the major frontier labs — though Anthropic is the first to detail an arrangement at this level of internal access.

What happens next?

METR will publish its own assessments of Anthropic's risks, incidents and practices, and Anthropic has bound itself in advance not to block them. The first published reports will show whether embedded access produces findings that outside review could not — and whether the redaction process holds up under METR's public scrutiny.

Original: metr.org

Share this article:

More from Rebecca Stone

Rebecca Stone

Show full bio

Correspondent covering companies, deals and strategy in artificial intelligence and machine learning at AI In Context.

295 articles

Related articles

  1. Anthropic brings in METR and SecureBio for outside risk assessments
  2. Anthropic cuts Claude's web access after autonomous fake homicide tip
  3. Anthropic cuts internet access for all internal AI evaluations
  4. Anthropic Snubs Senate Hearing on AI and Datacentres
  5. Anthropic to share updated financials with investors this week

« Previous articleNext article »