Most multimodal RAG stacks have a “split-brain” design: embeddings in a vector DB, relationships in a graph DB, and images/video/audio in blob storage. For an agent, that becomes a slow three-hop loop—similarity search, graph traversal, then a network fetch to load the actual frames for a VLM. This talk proposes a Lance-native Multi-Modal Knowledge Graph (MMKG) on lance-context + lance-graph, unifying media bytes, embeddings, and graph topology in one columnar dataset. Nodes become first-class media objects (images, clips, audio segments) stored directly in Lance, enabling zero-copy retrieval during traversal and faster “watch and reason” agents.

We’ll outline a high-level schema: MediaNodes for rich payloads with embeddings, timestamps, and structured metadata; TemporalEdges for explicit ordering (frames/scenes/clips) to support playback and “flashback” queries; and SemanticEdges linking media to entities and concepts. This design enables hybrid retrieval that combines graph constraints with multimodal similarity (e.g., “CEO smiling”), temporal window reasoning (“what happened in the 5 seconds before?”), and other agent workflows. We’ll close with a roadmap of our lance-graph open source project: ingestion utilities and a small set of multimodal graph primitives, and we welcome the graph community to join in the effort!