Jun 12, 2026·arxiv.org
Goodbye A/B Testing: LLMs Simulate Real Consumer Responses
Directly asking LLMs for numerical ratings (like purchase intent on a 1–5 scale) usually produces unrealistic, overly positive, or narrowly distributed answers that don’t match real human data.
This paper introduces Semantic Similarity Rating (SSR)— a simple method that first lets the LLM generate free-text reasoning about a product, then maps that text to a realistic Likert distribution using embedding similarity to anchor statements.