How would you design a multimodal search system over text, images, and video?

Use a shared embedding space so a query in one modality can retrieve another. A model like CLIP maps images and text into the same vector space; for video, sample keyframes (and optionally transcribe audio to text) and embed those. Offline: extract, embed, and index all content with metadata in a vector database. Online: embed the query — text, image, or both — and run nearest-neighbour search, then re-rank. Design choices: how finely to sample video (cost vs recall), combining modalities in one query, and hybrid search that also uses captions/transcripts as text signals so keyword matches aren't lost. Watch storage and compute — video embeddings are large — and evaluate cross-modal retrieval quality on a labelled set, since shared-space alignment is imperfect.