chips infrastructure
The Fleet Stopped Profiling When the Decision Stopped Moving
FleetSieve measures only the LLM-serving configurations likely to change a resource-coupled allocation.
Summary
FleetSieve measures only the LLM-serving configurations likely to change a resource-coupled allocation.
On a fixed H100 grid for a 31-billion-parameter open model, the method matched the oracle aggregate choice with 6.9 percent fewer GPU-seconds than uniform random profiling in the fixed comparison. Its joint capacity and tail-latency model also avoided a configuration whose 46.4-second p99 violated a 30-second service objective, although the paper reports that FleetSieve did not use the fewest GPU-seconds for every workload.
Why it matters
FleetSieve measures only the LLM-serving configurations likely to change a resource-coupled allocation.
Limits and context
- Its joint capacity and tail-latency model also avoided a configuration whose 46.4-second p99 violated a 30-second service objective, although the paper reports that FleetSieve did not use the fewest GPU-seconds for every workload.
Key claims
FleetSieve measures only the LLM-serving configurations likely to change a resource-coupled allocation.
Qualification: Its joint capacity and tail-latency model also avoided a configuration whose 46.4-second p99 violated a 30-second service objective, although the paper reports that FleetSieve did not use the fewest GPU-seconds for every workload.
Evidence: source-2026-08-23-004
Sources
- arXiv preprint 2608.19659arXiv · primary research
Corrections
No corrections have been recorded for this story.