How do you implement caching strategies for LLM applications?

Several layers, from cheapest to smartest. Exact-match cache: hash the full prompt and store the response — trivial and safe for deterministic (low-temperature) calls. Semantic cache: embed the query and return a stored answer when a past query is similar enough (above a tuned threshold) — powerful but risky, since near-duplicates can have different correct answers, so set the threshold conservatively. Prompt/prefix caching: many providers cache a shared system prompt or document prefix so you only pay to process the changing suffix — big savings for RAG and long instructions. Also cache embeddings and retrieval results. Always add TTLs and cache-busting for content that changes, and never cache personalised or permission-sensitive responses across users.