research
The Sentence Could Not Carry the Whole Meaning
An information-theoretic analysis bounds how well any text representation can recover a speaker's intent when language leaves context unresolved.
Summary
An information-theoretic analysis bounds how well any text representation can recover a speaker's intent when language leaves context unresolved.
The authors model language use as a joint distribution over meanings, contexts and utterances, then derive upper bounds for any decoder working from a representation of the utterance alone, including modern language-model hidden states. The remaining uncertainty separates into an irreducible component and a component that only extralinguistic context can resolve. Experiments on artificial languages, Mandarin zero-pronoun resolution and color reference support the theory; the result is a limit on text-only recovery, not a claim that useful language learning from text is impossible.
Why it matters
An information-theoretic analysis bounds how well any text representation can recover a speaker's intent when language leaves context unresolved.
Limits and context
- The remaining uncertainty separates into an irreducible component and a component that only extralinguistic context can resolve.
- Experiments on artificial languages, Mandarin zero-pronoun resolution and color reference support the theory; the result is a limit on text-only recovery, not a claim that useful language learning from text is impossible.
Key claims
An information-theoretic analysis bounds how well any text representation can recover a speaker's intent when language leaves context unresolved.
Qualification: The remaining uncertainty separates into an irreducible component and a component that only extralinguistic context can resolve.
Evidence: source-2026-08-31-001
Sources
- arXiv preprint 2608.28560arXiv · primary research
Corrections
No corrections have been recorded for this story.