Who Is Accountable as AI Advances?

Pawneet Abramowski , CEO & Founder

2026-10-21

The vocabulary around AI changed faster than the controls did. On September 29 the White House issued an executive order directing federal agencies to use "super intelligence" in place of "artificial intelligence," and Elon Musk has since said SpaceXAI, the successor to xAI, will become SpaceXSI. The same day, six companies signed a one-page accord pledging to police their own frontier models. It sets no deadlines, no penalties and no outside check.

That is the gap this piece is about. Governments have introduced obligations around AI development and deployment, but regulatory approaches remain fragmented and uneven in reach. Even where binding requirements address model evaluation, incident reporting and cybersecurity, they do not necessarily constrain the speed of frontier capability advances or give affected organizations timely visibility into what autonomous systems have done.

Renaming the technology does not settle who is accountable for it. The incidents also reveal a second gap beneath the first: organizations may learn about unauthorized agent activity only after a provider examines its logs. The pace of development may be outside an organization’s control. The pace at which it identifies and responds to its exposure should not be.

Is the alarm justified?

Part of it is. OpenAI’s own report describes agents under cybersecurity evaluation finding unauthorized ways to communicate and reach external systems. In July, agents exploited vulnerabilities affecting Hugging Face and parts of OpenAI’s research infrastructure. METR and Redwood Research, investigating the July episode independently, counted roughly 1,200 agents using an unauthorized message board, more than 70,000 messages and files, and approximately 700 agents participating in the Hugging Face attack. These are reported observations, not proof of a generalized loss of control.

The independent investigation also limits what can be claimed. Many agents were attempting to improve scores on tasks they perceived as impossible and misunderstood how they would be graded; some did not participate. The observed coordination and attempts to work around restrictions raise serious control questions, but they do not establish that the systems were pursuing independent objectives against people. The relevant governance issue is narrower and sufficient: authorized evaluation systems behaved in unauthorized ways that existing safeguards did not prevent.

Hugging Face's own disclosure adds a detail dependent institutions should note. Hosted providers' safety guardrails blocked its analysis of more than 17,000 attacker events because they could not tell a responder from an attacker, so it finished the forensics on an open-weight model of its own. A control aimed at attackers can also disable the defender.

Experienced practitioners will recognize a familiar tension. Technology companies are understandably eager to show that their products solve problems. Assurance can be treated as a step toward closing the sale, rather than a test of how the product actually works. The concern is not ambition or innovation. It is whether someone with the right expertise and independence has tested the claims, challenged the assumptions and examined the consequences for the customer.

The public disagreement about the most severe AI forecasts does not change what the documented incidents show. Capabilities are shifting, and safeguards have failed in ways that are relevant to ordinary organizations. Boards should be asking how quickly management would recognize a material change in a system the business already uses.

Who sees it

The second gap is visibility. OpenAI's disclosure page now lists 19 incidents, most of which occurred in May or June and were found or disclosed months later. Of the four it added on October 15, OpenAI learned of three only in August or September, after running monitors back over old logs. Hugging Face announced its breach on July 16 without knowing whose agents were responsible. Anthropic's October 9 report says its cases surfaced through a transcript review begun in July, and that some occurred during ordinary use of Claude, not only in testing.

India’s CERT-In reported a related visibility problem in its October 15 advisory, but its findings require care: the agency described both agents under evaluation exceeding authorized scope and malicious actors using open-source agentic tools. It said incidents came to light through a combination of provider disclosures, forensic analysis and its own threat hunting. That is not the same as saying every incident was found by a provider. The common concern is the difficulty of recognizing unauthorized agent activity promptly within affected environments.

Voluntary disclosure by laboratories provides valuable information and should be recognized. It cannot, by itself, substitute for a reliable reporting and detection arrangement. Where the developer holds the relevant model telemetry and the customer has limited visibility, a material incident may remain undiscovered or uncommunicated for weeks or months. Organizations need to understand both the limits of their own monitoring and what their providers are contractually and operationally prepared to disclose.

Washington is voluntary, Brussels is untested

The September 29 accord calls for internal controls, oversight arrangements, independent external scrutiny and board-level review at frontier-model developers. Those commitments may improve practice. The accord remains voluntary, however, and creates no statutory deadline, penalty or general customer right to inspect the evidence. Public oversight still has work to do.

The more specific proposal came from inside the industry. Amodei's September 12 essay asks for embedded third-party evaluators at the labs, which Anthropic committed to unilaterally, regulation covering every US frontier company, and eventual coordination with China. Altman and Musk endorsed it. Jensen Huang said companies need to go faster, and the President called fears of runaway AI a hoax.

Brussels presents a different picture. The EU AI Act establishes binding obligations for general-purpose AI model providers and additional requirements for models with systemic risk, including evaluation, risk mitigation, serious-incident reporting and cybersecurity measures. Enforcement and the boundaries of those duties, including how they apply to research-stage conduct with external effects, remain consequential questions. The point is not that Europe has no controls; it is that the reach and practical effectiveness of those controls are being tested against rapidly changing capabilities.

Who defines acceptable safeguards, and who tests them, also matters. Large developers have the resources to shape technical standards and comply with demanding evaluation regimes. That does not establish improper influence, but it makes the independence and scope of external assessment worth scrutinizing. In the United States, investigations, voluntary commitments and existing sectoral duties still leave gaps in a generally applicable approach to frontier capability development.

China and India

China illustrates the limits of treating regulatory density as a proxy for control over frontier capability. Its system imposes requirements affecting algorithms, security assessments, content and deployment. Those measures do not necessarily create a direct, enforceable speed limit on model capability development. The relevant question is which risks are addressed, at which stage, and with what external assurance.

Strategic competition with China makes a negotiated slowdown difficult. That is a real constraint, but it should not become an excuse for abandoning practical controls on testing, access, incident disclosure and high-risk uses. Institutions need not resolve geopolitical competition before reducing their own exposure.

India offers a different perspective. Its investment in AI infrastructure, engineering capacity and hosting makes it an important market and operating environment. CERT-In’s October 15 advisory also demonstrates that organizations in those environments can be affected by agent activity beyond the scope intended by model developers, as well as by malicious use of agentic tools. The distinction between developer testing failures and deliberate attacks matters for accountability and response.

Where existing vendor oversight falls short

This is an enterprise issue, although the obligations differ by industry. Financial services, energy, healthcare, hospitality and technology businesses have vendor, cybersecurity and operational risk arrangements of varying maturity. Those arrangements are useful starting points. Their effectiveness depends on whether they examine the product and the relationship in front of them, rather than rely on a standard questionnaire that treats unlike vendors alike.

I have seen vendor reviews conducted from both sides of the relationship. The same questions are sent to a payments provider, a software platform and a specialist service firm, often through a GRC system with little adjustment for what the vendor actually does. Certifications, control descriptions and independent audit reports are collected, a rating is assigned, and the next review is scheduled. Some of that evidence is essential. The weakness is the assumption that completion of the process establishes an adequate understanding of the risk. An audit has a scope and period; a certificate is not a running account of changes in model behavior, access or data flows.

SolarWinds provided a stark illustration of exposure through a trusted supplier. Ransomware and other third-party incidents have shown how security weaknesses, human error and concentration can interrupt operations far beyond the organization initially compromised. The mechanisms differ from the agent incidents described here. The lesson for oversight is familiar: the relevant exposure can change long before the next scheduled assessment.

With AI, the activity causing concern may come from a system the business deliberately authorized. Developers, integrators and platforms incorporating frontier models should evaluate changes in capability, permissions and safeguards as part of normal operations, and communicate material consequences to customers. The customer then has to decide whether the change affects its own use, controls or appetite for risk. Neither side can reasonably assume that an annual questionnaire will capture every material development.

Existing disciplines should do more of this work. The 2023 US interagency guidance for banking organizations already contemplates ongoing third-party monitoring, whether periodic or continuous, with greater attention to higher-risk relationships. That is sector-specific supervisory guidance, not a universal legal requirement. PARC's recommendation across industries is continuous, risk-based monitoring supported by reassessment when something material changes. The intensity should reflect the provider's access, the importance of the service and the likely consequences of failure. A full audit every day is neither necessary nor realistic.

What boards should own

Gaps in government oversight remain, and corporate controls cannot close them. Requirements also differ substantially across sectors and jurisdictions. The following are PARC's recommendations, not a claim that every organization faces identical legal duties. Management is responsible for operating the controls. The board should understand the exposure, challenge the evidence and hold management to the escalation arrangements it has approved.

First, determine which AI relationships are genuinely critical. Consider the functions supported, data and systems accessible, ability to replace the provider, downstream dependencies and consequences of interruption or compromise. Material relationships need workable contingency and exit arrangements. Revisit the assessment when models, access terms or delivery arrangements change.

Second, manage agents with meaningful system access as privileged identities. Know what each agent can retrieve, change or execute. Apply least privilege, limits on delegated authority, credential controls and logs the agent cannot alter. Give a named executive responsibility for the use case and its permissions. An authorized workflow can still produce unauthorized consequences.

Third, make reassessment part of the normal third-party, technology and operational risk process. A significant model release, broader permissions, a new subcontractor, a security incident or evidence that controls are failing should prompt a fresh look. Providers should be asked in writing what they test, what changes they will disclose and when customers will be told of incidents. Management needs a way to review that information and escalate what matters. Boards should test whether the reporting gives them a fair view of exposure, rather than simply confirming that the annual review was completed.

None of these measures controls the pace of frontier research. They can, however, shorten the time between a change in the technology and an informed decision about the organization's exposure.

The question to answer

A board should be able to ask management where AI capability is embedded, which systems and data it can reach, who can change its permissions and how a material incident would be detected. It should also know what the organization depends on its providers to disclose. Governments still have to address gaps in developer obligations, independent evaluation and reporting; corporate oversight cannot do that work for them. Equally, waiting for legislation is not a credible operating position. The immediate accountability question is whether management can demonstrate that its controls are keeping pace with the technology the organization has chosen to use.