| title | Paragraph Boundaries Are Not White Space:Compression Depth as the Signature of Hierarchical Structure |
|---|---|
| authors | Shuyang Xiang |
| date | 2026-09-20 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2609.23551 |
| upvotes | 7 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Standard positional encodings represent position as a one-dimensional reading-order coordinate, but reading order alone does not determine hierarchical textual structure. We use a hierarchical rotary positional encoding (hRoPE) that represents paragraph, sentence, and token indices as separate channels, hold the token sequence fixed, intervene on the paragraph coordinate p1, and measure cross-paragraph attention with a token-distance-exact estimator. Attention is compressed relative to a token-distance-matched baseline in every corpus, but compression alone is not diagnostic of true structure: an architecturally identical channel with density-matched random labels is compressed too, more shallowly. What distinguishes real structure is the depth of compression, which is greater and corpus-dependent while the control's is not. Comparing eight corpus-only quantities across three constructs (lexical persistence, paragraph length, embedding-based coherence), none fully reproduces the cross-corpus ordering of depth, though embedding-based coherence comes closest. Compression depth, not its location, is the reproducible signature of genuine paragraph structure in our setting. |
cite
@misc{xiang2026paragraph,
title={Paragraph Boundaries Are Not White Space:Compression Depth as the Signature of Hierarchical Structure},
author={Shuyang Xiang},
year={2026},
eprint={2609.23551},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2609.23551}
}