The Knowledge LedgerThe Knowledge LedgerFollow on WhatsApp

A curated stream of high-signal insights across Technology, Healthcare, Lifesciences, AI, Oncology, Business, Entrepreneurship, Leadership, Philosophy and beyond.

Thoughtful, cross-disciplinary content designed to expand how you think, not just what you know.

No noise. Only substance.

Filtered by #inference-systemsClear filter
Sep 4, 2026ยทarxiv.org

Language Models Can Control Thier Own Attention

Language models spend most of their attention on a small fraction of context, yet they still read the entire KV cache to find it. Ask about a detail in a long conversation and every token of the reply may require a full scan. Most sparse methods still pay an O(N) tax just to decide what matters.

Declarative Attention takes the opposite bet. The model already knows which parts are relevant. It declares the scope inside its chain of thought, in three modes. Global: look at everything to navigate. Focus: attend to a specific region. Local: use only recent output to synthesize. The inference engine parses those declarations like tool calls and skips most of the cache read.