Karthik Soman, SAP, Palo Alto, Senior Data Scientist

In the current landscape dominated by Large Language Models, embeddings from models like Sentence Transformers have become the de facto standard for document representation, excelling at capturing semantic relationships. While these embeddings are powerful, they often treat documents as independent units, missing the rich structural and relational information that exists between documents and their components. I present an approach that demonstrates how graph neural networks can enhance these semantic embeddings by acting as knowledge multiplexers, preserving semantic richness while simultaneously encoding structural relationships in the same space.

Through a comparative analysis of semantic and graph-based embeddings, I show that graph neural networks achieve this multiplexing effect through a non-linear relationship between geometric and directional similarities in the embedding space. The analysis reveals that documents can maintain high directional similarity (capturing thematic relationships) while preserving geometric distances (reflecting categorical distinctions), demonstrating true multiplexing of different types of document relationships. Quantitative analysis shows graph embeddings achieve better cluster separation and stability than semantic embeddings, suggesting they can serve as a bridge between pure semantic understanding and structural knowledge representation, thereby enabling better nuanced document representation and information retrieval.