Shekhar Iyer | Multi-Modal Retrieval Over Knowledge Graphs
KGC 2021 Conference, Workshops and Tutorials • 24m
Recent Advances in representation learning and application of Deep Neural Nets towards structured data and Knowledge Graphs (KG) is enabling opportunities for multi-modal representation of entities and relations. We can now aspire to build access to data encoded in knowledge Graphs through one of many modalities (image, audio, text or video) and also train joint representations to cross over from one modality to another (e.g. text-to-image, audio-to-text). These kinds of capabilities allow us to build applications that can use entity information in entirely new ways, to exploit the sensors available in modern multi-modal devices like glasses, watches, smart earphones etc... These contextually aware smart devices provide a better model for the users to interact with the world and consequently need a more robust support from a multi-model knowledge Graph to help them contextualize the users environment. In this presentation I will talk about a retrieval architecture to support a multi-modal KG. I will also show some examples of prototypes we have built to multi-modal retrieval.
Up Next in KGC 2021 Conference, Workshops and Tutorials
-
Sergio Baranzini | SPOKE: A Biomedica...
SPOKE is a biomedical knowledge graph containing factual data on various biomedical fields including genetics, molecular biology, physiology, metabolomics, pharmacology and clinical medicine. SPOKE can be used to repurpose medications, predict patient outcomes, and accelerate drug development, am...
-
Ryan Wisnesky | How To Optimally Merg...
In this talk we describe a new technique for merging knowledge graphs: translating the knowledge graph schemas into categories and the knowledge graph data into functors, then applying the "co-limit/pushout" construction from a branch of mathematics called category theory to merge these categorie...
-
Roi Krakovski | The Usearch Contextua...
We exploit the recent breakthroughs in Neuroscience to build web search engines based entirely on AI-generated data, thus eliminating the need to collect users’ data. We show how to generate search queries that are almost identical to real users’ queries. We use the generated queries to build a ...
