infrastructure
The Input Context Has an Energy Bill Too
Profiling found linear output scaling but both linear and quadratic components for long-input prefill compute time and energy.
Summary
Profiling found linear output scaling but both linear and quadratic components for long-input prefill compute time and energy.
Energy estimates for language models often count generated tokens and neglect the long prompts common in agent workflows. A profiling study varied both prefill and decode lengths and fit generalized linear models to compute time and measured energy. Decode cost scaled linearly with output length, while prefill showed linear and quadratic components as input grew; compute time also tracked energy closely enough to serve as a proxy in the tested setup. The work offers an estimation framework, not a measurement of every proprietary API or deployment stack.
Why it matters
Profiling found linear output scaling but both linear and quadratic components for long-input prefill compute time and energy.
Limits and context
- The work offers an estimation framework, not a measurement of every proprietary API or deployment stack.
Key claims
Profiling found linear output scaling but both linear and quadratic components for long-input prefill compute time and energy.
Qualification: The work offers an estimation framework, not a measurement of every proprietary API or deployment stack.
Evidence: source-2026-10-31-008
Sources
- arXiv preprint 2610.39621, original v1arXiv · primary research
Corrections
No corrections have been recorded for this story.