UK AISI: Open-Weight AI Is Shrinking Cyber’s Preparation Window
UK AISI: Open-Weight AI Is Shrinking Cyber’s Preparation Window
Four to seven months is not a comfortable lead. For a large company, it can be shorter than the time needed to inventory a legacy system, approve a replacement, test the change, and put it into production.
That is why the UK AI Security Institute’s new comparison matters. In its first public cyber-specific analysis of open and closed models, AISI concludes that recent leading open models now sit roughly four to seven months behind on its methodology, although the detailed match varies by model and test. In similar internal tests during much of 2025, AISI says the gap was six to ten months.
The easy headline is that open-weight AI is almost at the cyber frontier. That is too loose. The best attempts from July 2026’s leading closed models completed AISI’s full 32-step range; the tested open models did not. The month figure is a comparison between release dates, not a confidence interval, a countdown, or proof that an open model can autonomously compromise a defended enterprise.
The more useful reading is less reassuring: the preparation buffer is compressing while the economics and controls around capability are changing even faster. A slightly weaker model can become operationally important before it reaches parity if it is cheap enough to retry, easy to scaffold, and available as weights that no provider can recall or patch everywhere.
RohitAI’s earlier GLM-5.2 analysis asked when an open coding model becomes good enough for serious agent work. AISI has now supplied an independent cyber signal. The question is no longer only whether open models can catch the frontier. It is whether defenders can use the remaining months better than deployers can use cheap, persistent access.
A calendar headline with three large footnotes
AISI ran two different kinds of evaluation. The first was a suite of narrow tasks: vulnerability research and exploitation, reverse engineering, web exploitation, and cryptography. The second was a long-horizon simulated corporate-network range called The Last Ones.
Those tests answer different questions. Narrow tasks ask whether a model has a specific skill. The range asks whether an agent can preserve state, choose tools, recover from mistakes, and keep making progress through a chained operation.
| Evidence track | Test design | AISI’s comparison | What it supports |
|---|---|---|---|
| Narrow skills | 70 tasks, five attempts each, up to 2.5M tokens per attempt | GLM-5.2 matched Opus 4.6 and GPT-5.3-Codex; DeepSeek matched Opus 4.5 | A four-to-five-month skill comparator on this suite |
| Long-horizon range | 32 steps, four subnets, about 20 hosts, ten runs per model, 100M-token cap | GLM reached about as far as Opus 4.5; DeepSeek fell below Sonnet 4.5 | A larger planning-and-execution gap than narrow tasks show |
| Current frontier check | Best attempts shown separately from ten-run averages | GPT-5.6 Sol and Mythos 5 best attempts reached all 32 steps | The open models did not establish July 2026 frontier parity |
| 2025 baseline | Similar internal tests of open models released January–September 2025 | AISI reports a six-to-ten-month lag | A directional trend, not independently reproducible from the post |
The narrow suite contains 18 technical non-expert tasks, 25 apprentice tasks, 19 practitioner tasks, and eight expert tasks. AISI says using all 96 tasks and a 50-million-token budget per task did not change the selected closed-model comparators. That is a useful robustness check.
But the institute did not publish raw attempt-level data, exact checkpoint hashes, a machine-readable score table, or numerical uncertainty values. Figure 1 visibly includes error bars without defining their statistic, while the range chart uses min–max shading. Its six-to-ten-month historical baseline does not name the models or provide enough detail to reproduce it. And AISI explicitly says it did not optimize its setup for either open model, which may slightly understate their attainable performance.
One gap, four clocks
The month gap is easy to remember because it collapses a messy system into one number. That convenience is also its weakness.
Cyber risk does not move on one clock. It moves on at least four.
The same checkpoint can look months behind on capability while becoming immediately usable, cheaper, better scaffolded, and harder to recall.
The capability clock asks which older closed release shows similar skill on a defined test.
The autonomy clock measures whether that skill survives a long trajectory. Planning, context management, recovery logic, and tool use can move this clock without changing a single weight.
The economics clock measures when many attempts become affordable and available at useful scale. An API price cut can move this clock without changing a single weight.
The recoverability clock measures whether anyone can contain a failure after deployment. A provider can patch a classifier, throttle an account, or withdraw an API. Nobody can reliably update every redistributed copy of public weights.
Time to unmonitored capability is a better risk metric than release-date lag alone. A model can remain behind the frontier while crossing a practical threshold for low-cost, persistent use.
Cheap retries can outrun a capability deficit
AISI’s cost estimates are the most consequential numbers in the report, even though they are not the headline.
These are illustrative price estimates, not measured hardware costs. AISI did not use first-party providers for the open-model runs, and self-hosting changes the accounting. Prices can move quickly: as of July 18, DeepSeek’s official V4-Pro rate card lists $0.003625 per million cache-hit input tokens, $0.435 per million cache-miss input tokens, and $0.87 per million output tokens, with a concurrency limit of 500. Z.ai lists GLM-5.2 at $1.40 input, $0.26 cached input, and $4.40 output per million tokens.
The practical point is not that an attacker will buy one 100-million-token run. It is that low marginal cost changes the value of retries. A model with a lower pass-at-one rate can still become useful when many attempts, parallel exploration, and longer trajectories are affordable. A refusal that disappears after a few repeats is not much of a barrier in that setting; AISI says DeepSeek’s occasional refusals, mostly on reverse-engineering tasks, were bypassed with a small number of repeat attempts.
That is why teams should price threat models with pass-at-k, concurrency, and cost per successful trajectory, not only a benchmark’s first-attempt score.
The orchestration tax is hiding inside the gap
The open models looked closer to older closed releases on isolated tasks than on the long-horizon range. That difference is not noise to wave away. It is a measurement of the agent system around the model.
GLM-5.2 tracked Opus 4.6’s trajectory through step 11 in The Last Ones before stalling. AISI could not tell whether the stall came from missing cyber knowledge, weak planning, context loss, tool errors, or scaffold mismatch. Those causes imply very different futures. Missing knowledge may require new training. Bad state management can be fixed next week in the harness.
AISI’s underlying cyber-range research found that increasing the run budget from 10 million to 100 million tokens improved progress by as much as 59%, with no observed plateau. Its broader Frontier AI Trends Report says an optimized scaffold lifted cyber-development success by nearly ten percentage points and needed about 13% of the non-optimized token budget to reach the same 25% success rate.
Independent practitioner work points in the same direction, with important caveats. Semgrep reported 39% F1 for GLM-5.2 on one IDOR benchmark using a prompt-only harness, versus 37% for Claude Code with Opus 4.6. Its purpose-built pipeline scored 53–61%. That is not a universal model ranking; it is evidence that the analysis system can outweigh a small checkpoint difference.
Semgrep then audited the benchmark’s grounding and found only 14% recall per GLM scan on its hardest repository, 22% union coverage across repeats, and 29.6% pooled recall across five models. All five collectively missed 19 of 27 known issues. One repository cannot establish a universal rate, but it exposes instability hidden by a single aggregate score.
The cyber frontier is a model-plus-scaffold frontier. Evaluations that omit checkpoint, quantization, prompts, tools, permissions, context policy, retries, and token budget are not detailed enough for procurement or safety decisions.
The safety divide is recoverability, not refusal
It is tempting to describe closed models as safe and open weights as unsafe. Reality is less tidy.
Closed systems can be jailbroken. Hosted controls can miss abuse. Legitimate defenders can also be blocked by conservative policy, an outage, or a provider unwilling to process sensitive incident artifacts. Open weights offer real benefits: private deployment, stable access, model inspection, customization, and safety research that requires the parameters.
The decisive difference is what happens after a control fails.
A hosted provider can monitor patterns across requests, revoke credentials, rate-limit a campaign, patch a classifier, change serving behavior, or withdraw a model. Once weights are downloadable and redistributed, the original developer cannot impose those repairs globally. AISI’s open-weight risk research also found that some post-training defenses could be undone with dozens of examples, while pretraining-data filtering was more than ten times as resistant in its tests.
Open does not mean laptop-scale here. GLM-5.2’s FP8 checkpoint is roughly 756 GB, while DeepSeek V4-Pro’s instruct repository is roughly 865 GB. That makes these models institutionally portable before they are personally ubiquitous. But low-cost hosted routes can diffuse practical access faster than hardware ownership.
Monitoring, account enforcement, rapid patches, and withdrawal remain possible. The tradeoff is provider dependence, policy boundaries, retention rules, and less control over the underlying system.
The inference host can police its own route, but it cannot govern copies served elsewhere. Verify the exact checkpoint, provider controls, logging, retention, and concurrency rather than relying on the model name.
Privacy, continuity, and inspectability improve. So does the operator’s obligation to isolate tools, constrain permissions, monitor behavior, preserve evidence, and stop unsafe trajectories.
The useful design choice is therefore not “open or closed?” in isolation. It is: who controls the runtime, what authority does the agent receive, and can a bad deployment be contained?
What the range does not show
The Last Ones is a serious test, but it is deliberately simplified. It begins after initial network access. It has no active defenders, no defensive tooling, no penalty for generating alerts, and no production change-management friction. AISI’s March paper estimated the range at about 14 expert hours; the July post says roughly 20, without explaining the difference.
Most importantly, the evaluation does not show that GLM-5.2 or DeepSeek V4-Pro can select a real target, gain initial access, evade a mature security operation, and persist across a hardened enterprise end to end.
The US CAISI’s broader May evaluation reinforces why scope matters. It placed DeepSeek V4-Pro about eight months behind the US frontier across five domains, and reported an IRT-imputed 32% from a subset of its 285-challenge cyber suite, versus 46% for Opus 4.6 and 71% for GPT-5.5. UK AISI, US CAISI, and independent evaluators can all be directionally credible while producing different “month lag” answers because they test different tasks, budgets, checkpoints, and harnesses.
The defender bottleneck is moving to remediation throughput
Faster vulnerability discovery helps defenders too. The awkward part is what happens after the findings arrive.
An AI system can generate hypotheses, inspect code, retry a task, and produce candidate fixes at machine speed. A company still has to map the affected asset, identify an owner, validate the finding, test the fix, schedule a change, coordinate with suppliers, deploy safely, and verify that nothing else broke.
The operating equation is simple:
finding velocity > validation + ownership + patch + recovery capacity
↓
the exposure backlog grows
The Bank of England’s July Financial Stability Report makes the same economic point: open models may become dangerous and practical before full parity because they can be cheaper and their safeguards easier to remove. It warns that faster patching can itself increase operational errors, especially across interconnected firms and shared technology suppliers. In its 2026 H1 survey, 82% of respondents named cyberattack among the top five risks to the UK financial system, and 26% called it the single biggest risk.
The Five Eyes agencies’ operational advice through the UK NCSC is deliberately unglamorous: reduce attack surface, patch faster, retire unsupported systems, strengthen identity and access, rehearse incidents, and use AI defensively. Those actions matter because a four-month preparation window is worthless if the organization needs nine months to approve a patch.
This is also why open-weight access should not be treated as attacker-only. A sealed local model can preserve incident-response capability when privacy constraints or hosted policies block sensitive analysis. RohitAI’s account of the Hugging Face agentic intrusion describes a local GLM-based defensive path that maintained forensic continuity. The safe pattern is to separate analysis from authority: let the model inspect artifacts inside a controlled environment, while external scanning, exploit execution, remediation, and egress require a different tool boundary and human approval.
A deployment gate for builders and security teams
Do not answer this news by banning open models or by rushing them into an internal agent platform. Turn it into an acceptance test.
For products, one additional rule matters: evaluate the model in the deployment shape you will actually use. A first-party API, a third-party host, and a self-hosted quantization can share a model name while differing in checkpoint, context handling, rate limits, monitoring, and behavior. Those are separate systems.
Kimi K3 is the next falsification test
AISI says it will evaluate Kimi K3 after its weights become public. That makes K3 important, but not evidence for the trend yet.
As of July 18, Moonshot offers the hosted Kimi K3 API and says the model has 2.8 trillion total parameters, activates 16 of 896 experts, supports native multimodality, and has a one-million-token context window. Its launch post promises the full weights by July 27, alongside more technical material. No public checkpoint, weight license, model card, technical report, or AISI cyber result is available yet.
The evaluation protocol will matter. Moonshot’s K3 API guide currently supports only maximum reasoning effort and requires clients to return the complete assistant message in multi-turn and tool-call workflows. Dropping reasoning state or swapping models mid-session can change long-horizon behavior. RohitAI covered that hidden systems issue in the Kimi K3 harness-contract analysis.
If K3 matches or exceeds GLM-5.2 on narrow tasks but still stalls on the range, the orchestration-tax thesis gets stronger. If a faithful harness closes both gaps, it suggests useful cyber capability can diffuse through software improvements before another training run. If the promised weights slip or differ materially from the hosted system, K3 should not be counted as an open-weight data point at all.
The RohitAI read: stop tracking one frontier
The industry will outgrow a single “months behind” number because it hides the variables that decide whether capability matters in practice.
Here is the more useful model:
- There is a capability frontier. Can the checkpoint solve the narrow task?
- There is an autonomy frontier. Can the agent sustain a long trajectory and recover from failure?
- There is an economics frontier. Can many attempts be run cheaply enough to change behavior at scale?
- There is a recoverability frontier. Can access be monitored, constrained, patched, or revoked after something goes wrong?
UK AISI’s result suggests the capability and autonomy gaps may be narrowing, although the unpublished 2025 baseline is not enough to establish a stable trend. Its cost data says the economics clock may already be ahead of the calendar comparison. The nature of downloadable weights pushes the last clock in the opposite direction: deployments become less recoverable as distribution expands.
Three predictions follow.
First, serious cyber evals will become system cards. Within a year, a credible result will need to name the checkpoint, scaffold, tools, permissions, context policy, retry budget, and cost together. “Model X scored Y” will look as incomplete as publishing a database benchmark without its index or hardware.
Second, enterprises will adopt split-authority local agents. Privacy-sensitive incident analysis will increasingly run on sealed open-weight or private routes, while exploit execution, external scanning, production changes, and egress remain in separately approved tools. Local control will be valuable, but only when authority is deliberately fragmented.
Third, the first broad enterprise impact will be patch pressure, not autonomous cyberwar. Cheap models will help more people find, validate, and retry against weaknesses. The immediate bottleneck will be the queue of changes that humans and institutions can safely absorb. Fully autonomous compromise of hardened networks is a higher bar, and AISI’s range does not show it today.
Final take
UK AISI has not shown that GLM-5.2 or DeepSeek V4-Pro equals the July 2026 closed cyber frontier. It has shown something more actionable: the period between frontier capability and cheaper, less recoverable access may be getting short enough to collide with ordinary enterprise timelines.
The number to remember is not only four to seven months. It is the mismatch between that window and the speed of inventory, patching, identity cleanup, supplier coordination, incident rehearsal, and recovery.
Builders should respond with better evaluation and tighter runtime boundaries. Security leaders should respond by increasing remediation throughput and reducing old exposure. Model developers should respond with pre-release testing and safeguards that survive more than a polite refusal.
The benchmark gap may continue to narrow, widen, or jump around as checkpoints and harnesses change. The preparation window is already expensive to waste.
FAQ
What did UK AISI find about open-weight models and cyber capability?
AISI concluded that recent leading open models sit roughly four to seven months behind on its cyber methodology. In detail, GLM-5.2 matched roughly four-month-old closed models on narrow tasks and a less-than-seven-month-old comparator on the range; DeepSeek matched a five-month-old model on narrow tasks but fell below Sonnet 4.5 on the range. AISI says its overall result is narrower than the six-to-ten-month lag seen in similar internal tests during much of 2025.
Does this mean open-weight models match the current closed frontier?
No. The comparison is to older closed releases on specific evaluations. In AISI’s 32-step range, the best attempts from GPT-5.6 Sol and Mythos 5 completed all steps, while the open models stalled earlier. The study also does not demonstrate autonomous compromise of a defended real enterprise.
Why does cost matter if an open model is less capable?
Low prices make repeated and parallel attempts affordable. A weaker pass-at-one result can become operationally useful when pass-at-k is much higher and the cost per attempt is tiny. AISI estimated a 100-million-token DeepSeek run at $1.19 versus about $85 for its older closed comparators, though real provider and self-hosted costs vary.
What should teams test before deploying an open-weight cyber agent?
Test the complete system: checkpoint, quantization, provider, scaffold, tools, permissions, context policy, token budget, retries, grader, cost, and logging. Use isolated environments, read-only defaults, egress controls, explicit approvals for consequential actions, and both narrow-skill and long-horizon evaluations.