mediumAI System DesignReviewed Jul 17, 2026

How would you design a conversational AI system with memory across sessions?

Separate short-term from long-term memory. Short-term is the current conversation, kept in the context window — summarise or truncate older turns when it fills. Long-term memory persists across sessions: after each conversation, extract durable facts (preferences, key details) and store them, either as structured records or as embedded snippets in a vector store keyed by user. At the start of a session, retrieve the relevant memories and inject them into the prompt — retrieval, not stuffing everything in. Design choices: what's worth remembering (avoid hoarding noise), how to update or contradict stale facts, and privacy — users must be able to view and delete their memory. Guard against prompt-injected 'remember this' instructions poisoning long-term state.

system-designmemory

More AI System Design questions

See all AI System Design questions →