Most local serving systems treat a personal computer as a small GPU that must fit the model. FreeToken starts from a different premise. The machine is a unified, elastic inference platform whose CPU, GPU, and interconnect can be orchestrated continuously as workloads and available resources change.

The system is designed for Mixture-of-Experts models and for agentic workloads that repeatedly edit context and call tools. Its core techniques include bandwidth-adaptive execution (deciding in real time whether missing experts should be moved to the GPU or run on the CPU), semantic-aware state caching at natural boundaries, full-layer double buffering during prefill, and elastic reallocation of GPU memory between KV cache and experts.

Most systems still give models knowledge the hard way. They retrieve raw chunks at query time and force the model to re-derive understanding on every question. Nothing accumulates.

A better pattern has quietly emerged in 2026.

Directly asking LLMs for numerical ratings (like purchase intent on a 1–5 scale) usually produces unrealistic, overly positive, or narrowly distributed answers that don’t match real human data.

This paper introduces Semantic Similarity Rating (SSR)— a simple method that first lets the LLM generate free-text reasoning about a product, then maps that text to a realistic Likert distribution using embedding similarity to anchor statements.