infrastructure
Decision Trees Stop Recomputing Their Trunks
Flide flattens branching decision requests into one execution stream and reports answers as much as 9.5 times faster than vLLM and SGLang.
Summary
Flide flattens branching decision requests into one execution stream and reports answers as much as 9.5 times faster than vLLM and SGLang.
Decision models repeatedly evaluate branches that share a long prefix, but generation-oriented engines either recompute that prefix or alter output probabilities. Flide separates the logical request tree from its physical execution. ForkAttention uses GPU-resident indices across standard attention and recurrent Gated DeltaNet layers so each branch is equivalent, in exact arithmetic, to an isolated pass. The engine also resumes expanding histories from earlier checkpoints. On the reported public models and workloads, Flide answered queries up to 9.5 times faster than vLLM and SGLang while staying closest to a float32 reference in total-variation distance. The speedup is workload-specific and does not imply the same gain for ordinary free-form generation.
Why it matters
Flide flattens branching decision requests into one execution stream and reports answers as much as 9.5 times faster than vLLM and SGLang.
Limits and context
- The speedup is workload-specific and does not imply the same gain for ordinary free-form generation.
Key claims
Flide flattens branching decision requests into one execution stream and reports answers as much as 9.5 times faster than vLLM and SGLang.
Qualification: The speedup is workload-specific and does not imply the same gain for ordinary free-form generation.
Evidence: source-2026-11-02-004
Sources
- arXiv preprint 2610.42148, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.