YORO
open sourceAn LLM cache that knows when its answers went stale, and replays the reasoning instead of re-deriving it.
YORO (You Only Reason Once) is an OpenAI-compatible caching proxy. A semantic cache serves an answer whenever a request looks familiar, which means it keeps serving a dead answer once the world changes. YORO fingerprints what each answer depends on, invalidates when those inputs move, then replays the stored derivation against the new inputs. It drops in front of vLLM, llama.cpp, or any OpenAI-compatible endpoint.
Benchmarked on gpt-oss-120B and Qwen2.5-32B: accuracy under drift held at 0.96 versus 0.16 for a naive cache, at 10 to 21% of the no-cache output tokens.
- Python
- OpenAI-compatible proxy
- vLLM
- llama.cpp
- sentence-transformers