A cybersecurity advisory firm deployed 18 leading artificial intelligence systems against a live corporate network to test whether they could penetrate its defences. The index, released on Wednesday, evaluated nine models from each of two countries. The assessment tracked how far each system could advance through an actual breach scenario without human intervention.

Anthropic's Claude Mythos was the sole model to complete the course. The findings were presented by a correspondent on a prominent broadcast network that Wednesday evening.

A dedicated attacker machine was assigned to each model, with instructions delivered sequentially. The advisory firm provided neither a set of instruments nor any accompanying software, and the firm stated that was by design. The company sought to evaluate each model's standalone capabilities, independent of the technical infrastructure typically built around it.

The final tally combined points from two evaluation periods. A separate assessment tested whether an AI system could identify security flaws in pre-compiled binary code without access to the underlying source. A second measure tracked the depth of penetration achieved during a breach of a protected Active Directory environment, with the sequence extending from initial entry through to complete administrative control of the domain.

The consultancy assessed performance against what network logs actually recorded — drawing on traffic data, security records, and intrusion-detection telemetry — rather than relying on figures the models themselves reported. The advisory firm noted that a gap exists between evaluating stored knowledge and measuring other capabilities. Claude Mythos received a score of 80 on that same measure.

Using a compromised staff login, Mythos obtained administrative access on every test run. The system independently developed an alternative pathway to elevated access, rather than adhering to a predetermined set of instructions. In the more rigorous evaluation, lacking any authentication credentials, the system accessed the network from an external position and obtained domain control regardless.

Grok-4.5 placed second with a score of 49, while GPT-5.6 Sol recorded 46 to finish third, as no other model reached that benchmark. Scores of 38 were recorded jointly by Meta's Muse Spark 1.1 and Moonshot's Kimi K3, while Z.ai's GLM-5.2 reached 37 and Claude Opus 4.8 achieved 36. Qwen3-Coder, developed by Alibaba, placed fourth from the bottom with a score of four.

An additional three models achieved complete domain control alongside Mythos. Of the remaining models, four traversed the network horizontally, with nearly all gaining entry without assistance.

Claude Sonnet 5 ranked 15th out of 18 models with a total score of 13. The advisory firm noted that this result appears to contradict the methodology underpinning the rankings themselves. The firm combined that model with an orchestration layer — software that links a model to offensive tools and maintains its operational focus — and a deficit of 67 points was erased through improvements to that infrastructure, drawing it level with Mythos. The firm noted that by providing a structured environment, a model can maintain concentration, adjust to changing conditions, recover when errors arise, and link individual steps into a continuous sequence of actions.

The advisory firm also noted that its own ranking position within the index should not be treated as the principal takeaway from the report. The consultancy's report conveyed its conclusions without qualification, stating that risk now attaches to the overall architecture, not to any individual component within it. The organisation also disclosed gaps in its own data collection, noting that no evaluation had been conducted on Chinese models or open-weight systems paired with tuned harnesses.

The group's data indicate that fully functional systems are already in place. One version of a model cited a lack of authorisation as its reason for not completing an assigned task. A cybersecurity-optimised version of the same model, presented with the same request, agreed and followed through. From that pattern, the advisory firm derived a broader principle: the effectiveness of model guardrails varies depending on how the system is configured and the context in which it operates, and each instance stands independently and cannot be used to draw conclusions about others.

Scores across the board approached the upper limit when tested against an intentionally embedded error. The origin of a model — whether American or Chinese, proprietary or open-source — had no bearing on the results. The report found that all nine frontier models received a score of zero when tested against a previously undetected vulnerability embedded within a large production library. The assessment of the affected component was accurate, yet the conclusion reached was that no risk existed. The report noted that no other frontier models detected the vulnerability in question.

According to the document, Mythos was the only model with sufficient understanding of the platform to exploit it independently. In May, Mythos identified 10,000 vulnerabilities over the course of a single month, though the consulting firm characterised the gap between controlled testing and real-world conditions as providing additional response time for those on the defensive. The firm stated that practical attack capabilities in the field continue to fall short of what controlled testing environments measure.

The company stated it anticipates that most of the 18 models will attain Mythos's performance tier within a six-month period. The report describes AI-assisted attacks as forthcoming in the near term.

The advisory firm released the index together with a tool designed to counter automated attacks. According to the same document, coordinated defence strategies against AI-driven threats reduced attacker effectiveness by more than 95 per cent in the company's internal evaluations. The firm acknowledged that figure has not been independently corroborated and that it supports the position of the party promoting it.

The document calls for binding, industry-specific timelines requiring operators to demonstrate their capacity to manage a security breach. It also calls for a nationwide initiative to evaluate both foreign and open-weight models under real-world conditions, and for controlled access to allow authorised security professionals to work with the tools they are tasked with countering.

According to the document, an incident in July marked the point at which an AI system executed the full attack sequence outside of a controlled environment. One particular model does not appear within the index: the AI system known as Astra, developed by OpenAI, was not evaluated, though the company announced on Tuesday that it had met its internal cybersecurity benchmark. The advisory firm noted that by the time the index is published, the underlying data it tracks has already shifted.