How would you design a RAG system to chat with a company's documents?

Two pipelines. Offline ingestion: load documents, chunk them (respecting structure — headings, tables), embed each chunk, and store vectors plus metadata in a vector database, re-running on updates. Online query: embed the user's question, retrieve the top-k relevant chunks (often hybrid — vector plus keyword — then re-rank), assemble a grounded prompt with citations, and call the LLM. Stream the answer back. Key design choices: chunk size and overlap, how many chunks fit the context budget, access control so users only retrieve permitted documents, and caching of embeddings and frequent queries. Add evaluation on a golden question set and a 'no relevant context found' path so the model says it doesn't know rather than hallucinating.