What’s new: GPT-6.1 Sol matched GPT-6 Astra’s evidence-backed verdict counts in both tiers of our 48-case agentic investigation benchmark. Its measured run cost was $4.83. Astra’s recorded workload is estimated at $31.01–$53.31 at current Standard rates. Both used High effort.

Why it matters: An alarming command can be an intrusion or routine administration. Analysts need a correct classification and the records that establish it. Verdict accuracy checks the classification; evidence-backed accuracy also requires accepted supporting records retrieved and cited by the agent.

We compare seven models at High effort on the same DFIR 0.6 cases. The GPT-5.6 rows are earlier baselines; GPT-6 Sol has no measured row in this collection. ChatGPT and Claude app features are outside this model comparison.

Basic cases: Sol and Astra tie on accepted proof

Basic investigations · 36 cases per model · DFIR 0.6 · One pass
Model Effort Correct verdict Verdict with proof
GPT-6.1 Sol High 72.2%26 / 36 69.4%25 / 36
GPT-6 Astra High 77.8%28 / 36 69.4%25 / 36
GPT-6 Luna High 63.9%23 / 36 5.6%2 / 36
GPT-5.6 Sol High 80.6%29 / 36 19.4%7 / 36
GPT-5.6 Luna High 83.3%30 / 36 5.6%2 / 36
Claude Opus 5.5 High 91.7%33 / 36 33.3%12 / 36
Claude Sonnet 5.5 High 91.7%33 / 36 30.6%11 / 36

Sol 6.1 supplied accepted proof in 25 of 36 basic cases, matching Astra and exceeding Opus 5.5’s 12 and Sonnet 5.5’s 11. But it classified 26 of 36 correctly, below Astra’s 28 and both Claude models’ 33.

Against GPT-5.6 Sol, basic proof counts rose from 7 to 25, while correct verdicts fell from 29 to 26.

Intrusions: Sol, Astra and Opus share the proof score

Multi-host intrusions · 12 cases per model · DFIR 0.6 · One pass
Model Effort Correct verdict Verdict with proof
GPT-6.1 Sol High 83.3%10 / 12 41.7%5 / 12
GPT-6 Astra High 66.7%8 / 12 41.7%5 / 12
GPT-6 Luna High 75.0%9 / 12 8.3%1 / 12
GPT-5.6 Sol High 91.7%11 / 12 0.0%0 / 12
GPT-5.6 Luna High 100.0%12 / 12 8.3%1 / 12
Claude Opus 5.5 High 100.0%12 / 12 41.7%5 / 12
Claude Sonnet 5.5 High 83.3%10 / 12 33.3%4 / 12

Sol 6.1, Astra and Opus 5.5 each met the proof requirement in 5 of 12 intrusions. Their verdict counts differed: Sol got 10 right, Astra eight and Opus all 12. Sonnet 5.5 got 10 right, with accepted proof in four.

Cost:

Full 48-case pass · 36 basic + 12 intrusion cases · Standard rates as of October 14, 2026
Model Effort Actual spend Est. Standard cost Workflows completed
GPT-6.1 Sol High $4.83 $4.83 48 / 48
GPT-6 Astra High $15.51–$26.66 $31.01–$53.31 48 / 48
GPT-6 Luna High $0.25 $0.51 47 / 48
GPT-5.6 Sol High $2.91 $11.64 48 / 48
GPT-5.6 Luna High $0.41 ≈$0.82* 48 / 48
Claude Opus 5.5 High $54.57–$55.35 $54.57–$55.35 48 / 48
Claude Sonnet 5.5 High $21.57 $21.57 48 / 48

The Lunas were much cheaper, but each met the intrusion proof requirement in only one case. GPT-5.6 Luna’s 12/12 correct intrusion verdicts therefore deserve a different reading from 12 defensible, cited conclusions. GPT-6 Luna also had one invalid basic workflow; it remains in the denominator.

* Provisional: GPT-5.6 Luna assumes the historical Flex rates were 50% of Standard. Its full endpoint snapshot is unavailable.

Cost ranges retain unresolved charges in the Astra and Opus releases. Standard estimates hold token use, caching and retries fixed; they do not measure a Standard-tier rerun or predict an uncached investigation’s bill.

Pricing sources and calculation

Prices checked October 14, 2026: OpenAI Standard rates, GPT-5.6 Sol, Opus 5.5 and Sonnet 5.5. Archived rates for input, output, cache reads, cache writes and long-context bands scale uniformly: 2× for Astra and GPT-6 Luna; 4× for GPT-5.6 Sol; 1× for GPT-6.1 Sol and both Claude models.

GPT-5.6 Sol’s archived Flex endpoint also had a promotional discount. Today’s published $4 input / $20 output rates per million tokens imply $11.64 for the recorded workload. The historical promotional Standard equivalent was $5.82; that is not the current-price basis used here.

Recorded provider pins remain openai for Sol 6.1, openai/flex for the other GPT rows, and anthropic for Claude. The September per-request journals are absent from this checkout, so those estimates use archived rate ratios rather than a token-by-token replay. Sol 6.1’s 711 receipts independently reproduce its $4.83 total at Standard rates. Outcomes and latency remain those of the original runs.

Updated October 14, 2026: Added current Standard-rate estimates alongside actual spend. Measured scores and release hashes are unchanged.

What the result supports

The useful distinction: Sol 6.1 paired Astra’s accepted-proof counts with a lower bill in these passes. Claude 5.5 produced more correct basic verdicts. Classification, proof and workflow completion should stay separate when judging an investigation agent.

The limits: Each model ran once per case. The 36 basic cases belong to 12 related families; the 12 intrusions belong to four. Related variants are not independent incidents. These preliminary counts need replication and do not establish statistical superiority or broad real-world performance.

Technical settings and verification

All seven models used High effort through pinned OpenRouter endpoints, with no provider fallback. Each response allowed 16,384 output tokens. Cases, prompts, tools, evidence limits and grading share the same frozen protocol. Sol 6.1 completed all 48 workflows; independent replay passed and all 711 billing receipts settled, with no unresolved reservations.

Data sources and release details

Original GPT and Opus results · Sonnet 5.5 results · GPT-6.1 Sol results · Scoring and methodology · Main benchmark tables

Release hashes

Original cohort: 5e38e2573494a3ce7136d2c680c7dfd55d10d55834c6c8d5883e189b91abcb33

Sonnet 5.5: ba4bc2aab466f11fa90a06f073f57401f7f5c67bdc1cbc575d84bad84d6a3042

GPT-6.1 Sol: 91d579a0c603731ad72df3e99a76b070d856bedcacbf2f4c95b9c9519e127edf