Sakana Fugu-Cyber Makes Verification the Cyber-AI Product
Sakana AI has released Fugu-Cyber, a gated API that sends one security task through an undisclosed pool of specialist agents. The launch headline is a pair of strong benchmark scores. The more consequential choice is what Sakana built around them: manual access review, cyber-specific use rules, metered internal orchestration, enterprise integration, specialist validation, and a human decision before a proposed patch moves forward.
That package deserves more attention than the leaderboard claim.
Cyber models are already good enough to produce more findings than many teams can safely absorb. The scarce resource is shifting from raw model intelligence to evidence that a finding is real, in scope, reproducible, and safe to remediate. Fugu-Cyber is Sakana's attempt to sell that shift as one controlled system rather than another frontier chatbot.
The launch also shows why cyber-AI scores need unusually careful reading. Sakana reports 86.9 on CyberGym and 72.1 on CTI-REALM, but neither number is an independently reproduced measure of production defense. CyberGym tests reproduction of known vulnerabilities. CTI-REALM blends telemetry-grounded detection quality with mappings, tool milestones, and LLM-judged work. Sakana has not published the run metadata needed to compare its system cleanly with public submissions.
So the useful question is not whether Fugu-Cyber won a chart. It is whether a guarded, observable verification loop can turn frontier cyber capability into more validated fixes per analyst-hour. That is the product category taking shape here.
Fugu-Cyber extends the general Fugu orchestration system RohitAI covered in June. The public interface still looks familiar: Sakana's integration guide documents an OpenAI-compatible API at https://api.sakana.ai/v1, with Responses and Chat Completions endpoints. The callable model ID is fugu-cyber; the pricing page uses the versioned SKU fugu-cyber-v1.0.
The same guide advertises a one-million-token context window, text and image input, tool calling, server-controlled parallel tool calls, and high or xhigh reasoning settings. Those are documented interface capabilities, not independently stress-tested limits.
Behind that small interface, Sakana says Fugu dynamically routes work across specialist agents. Its launch post positions the system for vulnerability research, security analysis, and threat investigation, while arguing that raw model output is insufficient without local context and verification.
The resulting product boundary looks less like a model and more like a governed security runtime:
A useful deployment model: authorization travels into the run, orchestration produces candidates, independent evidence filters them, and a human owns the commit boundary.
Each boundary catches a different failure:
Sakana only operates part of that stack. A customer still has to supply asset ownership, clean environments, code and telemetry context, regression tests, reviewers, and an auditable approval process. That is why this release is better read as an enterprise implementation strategy than as autonomous cyber defense in a box.
Sakana's chart puts Fugu-Cyber ahead of the model-branded baselines it selected:
These results signal frontier-class capability. They do not yet establish an overall lead.
CyberGym Level 1 contains 1,507 historical vulnerabilities across 188 open-source projects. An agent receives a vulnerability description and unpatched source code, then has to produce a proof of concept that triggers on the vulnerable build but not the patched build. This is concrete and valuable. It is also narrower than discovering unknown flaws, triaging a live estate, writing a safe patch, or running a SOC.
Sakana's 86.9 exceeds the 85.6 GPT-5.5-Cyber and 83.1 Mythos Preview values in its chart. But the public leaderboard data accessed on July 21 showed Crystalline at 89.6 and Microsoft's more-than-100-agent MDASH at 88.45. Fugu-Cyber was absent. Because Fugu-Cyber is itself an orchestration system, agent-system comparisons are more revealing than comparisons only with model-branded entries.
There is a second caveat. CyberGym's submission standard asks teams to disclose scaffold and models, attempts, tokens, cost, time, network policy, environment, and sample trajectories. Sakana has not published those fields. Without them, 86.9 cannot tell a buyer whether the advantage came from worker quality, more retries, a larger token budget, permissive network access, or a better harness.
CTI-REALM evaluates agents that turn public threat reports and telemetry into detection artifacts. Its 50 tasks span Linux, Azure Kubernetes Service, and Azure cloud environments. That makes it closer to defensive operations than a quiz, but the final number still compresses unlike things.
| CTI-REALM score component | Weight | What it represents |
|---|---|---|
| KQL F1 | 50% | Telemetry-grounded precision and recall for the generated query |
| LLM-judged CTI analysis | 12.5% | Quality of the threat-intelligence analysis |
| LLM-judged Sigma quality | 15% | Quality of the generated portable detection rule |
| MITRE and data-source mappings | 17.5% | Technique and telemetry-source identification |
| Successful query execution | 5% | Whether the agent reached a working tool call |
The open scorer shows why 72.1 is not detection accuracy. Half the score is KQL F1, 27.5% comes from LLM judges, and the rest rewards mappings and execution. The benchmark documentation also reports uneven judge calibration, particularly for the CTI-analysis component. A few points of separation should not be treated as statistically settled without repeated, matched runs.
Fugu-Cyber is not instant self-serve access. Sakana's application asks for verified contact and organizational details, relevant background, a detailed use case, acceptable-use confirmation, and expected first-month spend. It also asks about company revenue, current AI spending, and interest in an enterprise contract. Sakana says manual review may take a few days.
Those questions serve two purposes. They create friction around a dual-use capability, and they identify customers with valuable workloads and implementation budgets. That is sensible business design, but buyers should not mistake admission screening for runtime control. A manually approved account can still be given excessive credentials, stale authorization, or an unsafe production target.
The acceptable-use policy forbids unauthorized access, exploitation, social engineering, interception, denial of service, exfiltration, malware and phishing activity, and bypassing security controls. It also says security testing, verification, evaluation, and other experimental security activity require advance written approval. National-security, military, and intelligence uses require approval too.
That creates an unresolved operational question: does approval for the Fugu-Cyber endpoint also cover a customer's recurring internal evaluations, or is separate written approval required? Until Sakana clarifies the relationship, teams should resolve it in writing and store the answer alongside the run policy.
Availability has other limits. Sakana's service terms exclude users in the EEA, United Kingdom, and Switzerland, and provide no service-level commitment for uptime, response time, or accuracy. “Available” therefore means a live endpoint exists—not that every organization or workflow can depend on it today.
Sakana is right that useful cyber defense needs deep local context. A model cannot reliably assess exploitability or propose a safe patch without build instructions, dependency state, configuration, telemetry, and often proprietary source code.
That makes the default data terms unusually important. Sakana's service terms say inputs and outputs may be used for training and analysis, and that contractors or human reviewers may process content. The company offers a non-retroactive training opt-out. Its privacy policy describes collecting prompts, uploads, outputs, session data, and security data, with international processing that can include Japan and the United States.
This does not mean an enterprise deployment is inherently unsafe. It means the public contract should not be assumed to fit proprietary vulnerability research. Before sending source, logs, credentials, exploit details, or customer data, negotiate the requirements that matter: zero or bounded retention, training exclusion, reviewer access, subprocessors, residency, breach notice, deletion, and audit rights.
The tension is structural. The more context that makes the model useful, the more consequential the data route becomes. We saw the same pattern when examining how an agentic intrusion turned a dataset pipeline into the attack surface: model capability does not compensate for weak credential topology or unclear data boundaries.
Fugu-Cyber is pay-as-you-go only. Its standard rates are a 20% premium over Fugu Ultra, but that headline understates the possible difference because Sakana separately meters internal orchestration tokens.
| Token category, per 1M | Up to 272K context | Above 272K context | Operational implication |
|---|---|---|---|
| Input | $6 | $12 | Repository-scale context can cross the price step quickly |
| Cached input | $0.60 | $1.20 | Stable policy and code context should be cacheable where safe |
| Output | $36 | $54 | Verbose agent and validator output is expensive |
| Internal orchestration | Same token rates | Same stepped rates | Visible prompt and answer length do not reveal total run cost |
The pricing documentation exposes orchestration input, cached-input, and output usage fields and includes them in total billed tokens. That is welcome observability. It also means teams should benchmark cost distributions, not a single demo invoice. A difficult request may fan out across specialists, validators, and retries even when the final answer is short.
OpenAI compatibility has another cost wrinkle. Sakana's integration guide says previous_response_id is unsupported, so a client has to resend full conversation history. In a long investigation, repeated history can grow the visible context toward the 272,000-token price step while internal orchestration expands the bill separately. Complex turns may also need longer client timeouts. Production integrations should be asynchronous, cancellable, and explicit about both context growth and total usage.
Use Fugu-Cyber when exploit reproduction, detection engineering, or independent validation can remove expensive analyst work and justify the orchestration premium.
Summarization, classification, basic enrichment, and low-risk document work rarely need the full cyber endpoint or its hidden fan-out.
Keep secrets, regulated telemetry, unreleased vulnerabilities, and contractually restricted source on an approved local or private path until data terms fit.
The strongest line in Sakana's launch is its admission that isolated models produce false positives and need specialist and human verification. Independent context supports that caution: Axios reported that a month-long Palo Alto Networks evaluation across more than 130 products found 75 legitimate vulnerabilities, while still producing roughly 30% false positives and requiring substantial environment-specific harness work. That was not a Fugu-Cyber test, but it shows why raw discovery volume is a poor purchasing metric.
Evaluate the whole loop on representative repositories and telemetry:
authorized target
-> candidate finding
-> clean-room reproduction
-> evidence package
-> independent retest
-> patch and regression tests
-> code-owner approval
-> monitored deployment
The primary KPI should be validated findings or safe patches per analyst-hour. Supporting measures include precision, duplicate rate, reproducibility, regression rate, time to validation, time to remediation, reviewer effort, p95 latency, and total billed tokens. Track how many candidates die at each stage. A model that produces fewer findings but wastes much less human time can be the better system.
Long-running orchestration also makes the session a security boundary. As RohitAI argued in its analysis of long-horizon agent failures, static tool permissions are not enough. Record context hashes, tool activity, retries, spend, validator outputs, patch diffs, and approval events. Add pause and cancellation paths. Keep the merge, disclosure, and deployment steps outside the model's unilateral control.
Three implications extend beyond this launch.
Once a system routes among agents, benchmark performance belongs to the whole execution policy: worker selection, context construction, tools, retries, budgets, validators, and stopping rules. “Which model scored 86.9?” is the wrong abstraction. The defensible question is “Which disclosed system produced 86.9 under which constraints?”
This makes evaluation artifacts commercially important. Buyers will ask for route-level traces, reproducibility packages, and cost-normalized results. Sakana does not need to reveal proprietary routing logic, but it will face pressure to publish enough metadata for a serious comparison.
Today's application form screens the customer once. Mature cyber orchestration will have to evaluate authority continuously: who owns this target, what testing is allowed, which tools may run, where evidence may go, and who can approve a side effect. Those answers should be machine-readable inputs, not a PDF remembered by the security team.
That is a deeper form of access control than API-key possession. It turns policy into execution state.
Fugu can reduce dependence on the quality or availability of one frontier model by routing across a pool. It does not remove dependence on frontier providers. Sakana has not disclosed the cyber worker set, and the service terms acknowledge third-party dependencies. A provider policy change, outage, price move, or regional restriction can still alter the product from underneath the customer.
The likely enterprise response is pressure for approved worker pools, route auditability, data-path disclosure, and fallback guarantees. Hidden orchestration is attractive as an abstraction until compliance or incident response needs to know where a sensitive task went.
No public Fugu-Cyber system card, worker manifest, training description, cyber-specific technical report, evaluation appendix, or downloadable model was located as of July 21. Sakana has not disclosed its benchmark versions, task splits, number of trials, tools, network policy, retries, budgets, latency, variance, category scores, or trajectories. No independent test of Fugu-Cyber was available on launch day.
Those gaps do not erase the launch. They define the diligence queue.
Sakana's general Fugu technical report explains learned worker selection and multi-agent workflow construction, but it predates Fugu-Cyber and does not validate the new scores. Until a cyber-specific artifact appears, buyers should treat 86.9 and 72.1 as promising vendor evidence, not settled capability facts.
No. Sakana presents it as an API endpoint that behaves like one model while dynamically orchestrating specialist agents. The worker models, routing distribution, and cyber-specific training details are not public.
No. It is a normalized composite reward. Half comes from KQL F1, while the remainder combines LLM-judged analysis and Sigma quality, MITRE and data-source mappings, and successful query execution.
Not on the public launch-day snapshot. Sakana's 86.9 vendor claim beats the model-branded comparators in its chart, but the public data listed Crystalline at 89.6 and Microsoft MDASH at 88.45. Fugu-Cyber was not yet present.
Below the 272,000-token context threshold, Sakana lists $6 per million input tokens, $36 per million output tokens, and $0.60 per million cached input tokens. Higher-context rates are $12, $54, and $1.20. Internal orchestration tokens are also billed.
Do not assume so. Sakana manually reviews access, while its acceptable-use policy separately says testing, verification, security evaluation, and experimental security activity require advance written approval. Customers should clarify the scope in writing.
Fugu-Cyber is a meaningful release because Sakana has packaged cyber capability as a controlled orchestration service rather than pretending one smart answer equals enterprise defense. The scores are strong enough to warrant testing. The missing evaluation artifacts are significant enough to prevent a victory lap.
The company has also identified the right bottleneck. As cyber agents improve, finding candidates becomes cheap. Proving them, prioritizing them, fixing them without regressions, and preserving an audit trail remain expensive.
If Fugu-Cyber succeeds, it will not be because 86.9 became a memorable number. It will be because the system helped defenders move from authorized question to reproducible evidence to safe remediation—with less wasted human attention and no ambiguity about who approved the final step.