FireFlash: An Open Recipe for Long-Horizon Agentic RL Achieving the Performance-Cost Pareto Frontier

Introduction

We introduce FireFlash, a 300B-scale model tailored for long-horizon agentic tasks. FireFlash excels as a terminal and coding agent, achieving 46.2% on Terminal-Bench 4.0¹, putting it on par with Fable 5 and GPT-6 Sol, while costing over 30× less per task than Fable 5 at maximum effort (Figure 1). These gains also transfer to software engineering benchmarks, including DeepSWE v1.1² and SWE-Bench Pro Verified³ (Table 1).

While frontier labs scale reinforcement learning across thousands of GPUs, we reach the performance-cost Pareto frontier under 5 days using 64 NVIDIA B300 GPUs. Three key factors make this possible: (1) careful curation of agentic RL environments so that every task provides a reliable signal at the right difficulty; (2) an RL recipe built for long, stale, and low-precision rollouts; and (3) an optimized coding harness tailored to the model for both training and evaluation. Together, these components bring frontier efficiency to a hundred-GPU-scale cluster, and we are releasing the full recipe.

Method

We perform full-parameter RL starting from GLM-5.3-Flash, training all 320B language parameters, including the MTP layer, while keeping the vision tower and DSA indexer frozen. We revisit and optimize five key components of the post-training pipeline, sharing our key lessons and findings in the following sections.

Infrastructure: Fully Asynchronous RL

Key lesson 1Idle GPUs cost more than stale rollouts: let sampling and training each run at full speed.

• Dedicated trainer and rollout GPUs. Half of the GPUs train in BF16 using slime⁵, while the other half run SGLang⁶ engines with FP8 weights and an FP8 KV cache. Updated weights are synchronized to the rollout engines over NCCL after every step, so neither training nor sampling stalls on the other (Figure 2).
• Training with stale rollouts. Because sampling proceeds continuously, a turn may be generated by weights several steps behind the trainer. A turn is discarded only if it was generated more than 6 weight versions ago. Policy staleness incurs minimal drift: the gap between trainer and rollout log-probabilities increases only from 0.065 at a staleness of one version to 0.08 at four to six.
• Fast, isolated rollouts. Speculative decoding with the model's native multi-token prediction (MTP) head delivers a 1.38× decoding speedup. Every rollout and grading run executes in its own Firecracker microVM via AgentENV⁴, isolating attempts from one another and reducing the share of rollout groups lost to infrastructure errors from 12% to 0.5%.

Task Selection: Train Where the Model Is Uncertain

Key lesson 2Spend rollouts only where the model is uncertain, and measure that uncertainty under the exact conditions it will train in.

GRPO learns from relative differences across attempts on the same task. A task the model always solves—or never solves—consumes a full group of rollouts while providing zero learning signal. We therefore profile each candidate task with at least 5 rollouts of the base model and retain those with an initial pass rate between 15% and 85%. Profiling runs under the exact training configuration—the same harness, reasoning effort, context window, turn budget, and sandbox—since a task's pass rate depends on each of them. This filtering cuts the share of zero-signal rollout groups from 48% to 10.2%. Figure 3 shows the resulting training mixture.

Grader and Reward: Robust Grader and Prevent Reward Hacking

Key lesson 3A task the model never solves is often a broken task: verify every grader before its reward reaches the model.

When we audited never-solved tasks, nearly half were broken rather than hard: malformed, reliant on brittle tests that reject correct solutions, or containing protected-file traps. Other graders awarded partial credit for outputs that merely matched the expected format, or crashed on real agent outputs. To eliminate these failure modes, every new task must pass nine verification gates before entering the training pool (Figure 4).

Reward design. Each attempt is scored once upon completion with a terminal reward in [0, 1]. A full solve earns 1.0, while an incomplete attempt earns partial credit for the verification checks it passes—or, on software engineering tasks, for the failing tests it resolves. Successful attempts also earn a length bonus of up to 0.2, scaled by how many fewer tokens they use relative to the longest solve in the same group. This shaping reduces tokens per solve to 0.84× with no drop in evaluation accuracy.

Harness: Built for Long Attempts, Identical in Training and Evaluation

Key lesson 4The harness is part of the model: train in exactly the one you evaluate in.

Our harness adopts Claude Code conventions: four tools (Bash, Read, Write, Edit), native tool calls, retained reasoning history, and context compaction. Crucially, the exact same harness drives training rollouts and evaluation trials, and automated unit tests enforce code and prompt parity between the two. Prior to any RL, it lifts GLM-5.3-Flash 4.4 points above its official Claude Code entry on Terminal-Bench 4.0 (Table 3).

Long-horizon attempts must also remain affordable. State-preserving context compaction triggers at 85% of the context window and preserves prefix caching by appending a sectioned summary prompt to the existing conversation history. Commands write stdout to files and return a truncated preview referencing the filename, while long-running commands continue asynchronously in the background rather than terminating at timeout. Together, these features cut per-trial cost by about a third. Agentic prompt guidelines that enforce efficient workflows (in-place edits, single-pass writes, no sleep polling) save a further 16%.

RL Objective: Learn from Stale Samples Without Drifting

Key lesson 5Correct stale, low-precision samples instead of discarding them, and treat rising entropy as the early warning.

Rollouts are sampled in FP8 on weights up to 6 versions old, so they come from a policy that differs slightly from the BF16 policy being trained. Rather than discarding these stale trajectories, our training objective directly corrects for them. The core algorithm is GRPO with 12 attempts per task and a single optimizer update per step; each attempt's advantage combines its reward and its length bonus:

A  = (R' − mean R') / std R'  +  (b − mean b) / 0.5     over the 12 attempts at one task
R' = 1.5                      if the attempt solved the task
R' = 0.5 × R                  otherwise (R = the terminal reward in [0, 1] above)
b  = 0.2 × (1 − tokens / max tokens among solves)   for a solve (≥ 2 solves in the group), else 0

• Truncated importance sampling. Each token is weighted by the ratio of the BF16 trainer probability to the FP8 rollout probability, clipped to [0.5, 2.0] to bound the influence of any single token; only 1–3% of tokens hit the clip boundary.
• Score centering. The rollout engine returns each token's top-32 log-probabilities, and the trainer centers each token's policy-gradient score by subtracting the sampler's expected score over those candidates⁷. This eliminates the residual drift toward the stale FP8 sampler that truncation alone leaves behind, at a cost of 7.6% in step time.
• Success offset. A full solve (R = 1.0) enters the advantage as 1.5, whereas an unresolved attempt enters as half its partial credit⁸: the best near-miss (R = 0.5) counts as 0.25 against a solve's 1.5, so partial credit shapes learning without rivaling an actual solve.
• Loss normalization floor. Each attempt's loss is normalized by its token count, with a minimum denominator of 2,048 tokens. This prevents a degenerate rollout of only a few tokens from receiving an outsized per-token weight and dominating the gradient.

With these corrections in place, we use no KL penalty and no entropy bonus. In a healthy run, policy entropy decreases slowly, from roughly 0.46 to 0.33; rising entropy serves as an early warning sign of instability.

Results

We evaluate FireFlash in the same harness it was trained in, and compare it with the official GLM-5.3-Flash entry it starts from and with published scores of other models (Figure 5). FireFlash improves on its base on all three benchmarks: by 10.4 points on Terminal-Bench 4.0, where it passes Fable 5 and GLM-5.3, its full-size sibling; by 5.4 on DeepSWE v1.1, passing Fable 5.1; and by 5.1 on SWE-Bench Pro Verified, passing Kimi K3 and GPT-5.6 Sol.

FireFlash against published scores

Cost and performance

FireFlash sits on the cost-performance frontier of Terminal-Bench 4.0 (Figure 1): of the more than forty models and effort settings plotted, only four cost less per trial, and none of them scores above 35.8%. At GLM-5.3-Flash's list rate a trial costs $0.72, against $22.02 for Fable 5 at maximum effort and $1.49 for GPT-6.1 Terra, the next point on the frontier at 55.6%. Every model that scores higher than FireFlash costs at least twice as much per trial, and GLM-5.3, its full-size sibling, costs $8.27 per trial at maximum effort for a lower 41.8%.

Final Remarks

FireFlash demonstrates that targeted post-training can turn an open-weight model into a highly capable domain-specialized agent, reaching frontier-level performance at a fraction of the inference cost. We believe that jointly optimizing models, training environments, harnesses, and inference systems offers another path to building capable and efficient agents beyond scaling.

FireFlash is our first step in this direction. We will extend this approach beyond coding and terminal tasks to more complex, long-horizon, recurring work.

Authors

Wenbo Hu, Jesse Cai^†, Po-nien Kung^†, Yuxuan Lu^†, Di Wu^†, Huimin Zeng^†, Nanyun Peng, Dakuo Wang, Kai-Wei Chang

^†Listed in alphabetical order.

References

1. Terminal-Bench 4.0 and its leaderboard. tbench.ai.
2. DeepSWE v1.1 leaderboard. deepswe.datacurve.ai.
3. SWE-Bench Pro Verified. arXiv:2609.08149.
4. AgentENV: Firecracker microVM sandboxes behind an E2B-style API. github.com/kvcache-ai/AgentENV.
5. slime, the RL framework our trainer forks. github.com/THUDM/slime.
6. SGLang, the inference engine behind every rollout. github.com/sgl-project/sglang.
7. Score centering on top of truncated importance sampling (the TIS + SC arm). arXiv:2609.20807.
8. The success offset in the GRPO term. arXiv:2610.22192.

Citation

Please cite this work as:

@misc{fireflash2026,
  title  = {FireFlash: An Open Recipe for Long-Horizon Agentic RL Achieving the Performance-Cost Pareto Frontier},
  author = {Hu, Wenbo and Cai, Jesse and Kung, Po-nien and Lu, Yuxuan and Wu, Di and Zeng, Huimin and Peng, Nanyun and Wang, Dakuo and Chang, Kai-Wei},
  year   = {2026},
  month  = nov,
  note   = {Blog post}
}