developer tools
The Specification Had to Agree Across Every Implementation
A semantic-block model reduced task context by about 71 percent, while execution tests exposed a 14.4-point variability floor.
Summary
A semantic-block model reduced task context by about 71 percent, while execution tests exposed a 14.4-point variability floor.
The work defines specification determinacy as agreement among all conforming implementations and encodes specifications as blocks, dependencies, owned rules, decision points and open questions. In an Oracle-to-PostgreSQL migration case, dependency closures reduced mean per-task context by about 71 percent and covered 85.5 percent of the study’s construct taxonomy. Repeated LLM implementations still showed a median 14.4-point arm-delta spread, supporting determinacy as a formal concept but not a sufficient standalone quality metric.
Why it matters
A semantic-block model reduced task context by about 71 percent, while execution tests exposed a 14.4-point variability floor.
Limits and context
- Repeated LLM implementations still showed a median 14.4-point arm-delta spread, supporting determinacy as a formal concept but not a sufficient standalone quality metric.
Key claims
A semantic-block model reduced task context by about 71 percent, while execution tests exposed a 14.4-point variability floor.
Qualification: Repeated LLM implementations still showed a median 14.4-point arm-delta spread, supporting determinacy as a formal concept but not a sufficient standalone quality metric.
Evidence: source-2026-08-21-010
Sources
- arXiv preprint 2608.19475arXiv · primary research
Corrections
No corrections have been recorded for this story.