IME: Integrating Multi-curvature Shared and Specific Embedding for Temporal Knowledge Graph Completion

Introduction

Knowledge Graphs (KGs) are structured data representations that organize information as entities and their relationships. Triplets consisting of head entity, relation, and tail entity model real-world facts. For example, (Albert Einstein, born_in, Germany) states that Albert Einstein was born in Germany. KGs allow machines to understand and reason about structured knowledge, uncovering patterns and supporting applications like recommendation systems, information retrieval, and semantic search. Their semantic modeling of knowledge enables informed decision-making by machines based on the encapsulated structured data.

The paper introduces Temporal Knowledge Graphs (TKGs) which incorporate temporal information into traditional knowledge graphs, allowing the tracking of knowledge evolution over time. TKGs aim to represent knowledge as quadruplets (subject, relation, object, timestamp). Despite existing large TKGs, their incompleteness hinders knowledge-driven systems, highlighting the importance of Temporal Knowledge Graph Completion (TKGC) to predict missing elements.

The paper notes that different geometric spaces (hyperspherical, hyperbolic, Euclidean) excel at capturing different data structures. However, TKGs exhibit complex structures, and most TKGC methods model them in a single space, failing to effectively capture their intricate geometry. Current methods also overlook the spatial gap between curvature spaces, limiting expressive capacity. Additionally, existing feature fusion approaches have high computational complexity despite pooling strategies' effectiveness in reducing it.

To address these challenges, the paper proposes the Integrating Multi-curvature shared and specific Embedding (IME) model for TKGC tasks. IME simultaneously models TKGs in hyperspherical, hyperbolic, and Euclidean spaces, introducing a quadruplet distributor in each space. IME learns space-shared properties to mitigate the spatial gap by capturing shared information across curvature spaces, and space-specific properties to capture complementary information in each space.

The paper also introduces an Adjustable Multi-curvature Pooling (AMP) approach to learn appropriate pooling weights for superior pooling strategies and effective information retention. AMP aggregates space-shared and -specific representations for downstream predictions.

Key contributions include the novel IME model with space-shared and -specific properties, the AMP module for effective pooling, introducing structure loss for structural similarity across curvature spaces, and achieving competitive performance on TKGC tasks.

Related work

The passage provides an overview of knowledge graph completion (KGC) methods from two perspectives: Euclidean embedding-based methods and Non-Euclidean embedding-based methods.

Euclidean Embedding-based Methods:

• Static knowledge graph completion (SKGC) focuses on predicting missing triplets in static knowledge graphs.

• Methods include translation-based (TransE, RotatE, TransR, SimplE, BoxE), semantic matching-based (DistMult, ComplEx, CapsE, TuckER, McRL, MLI), and convolutional neural network-based (ConvE, R-GCN, TDN).

• Temporal knowledge graph completion (TKGC) aims to predict missing quadruplets (entities, relations, timestamps) in temporal knowledge graphs.

• Methods like TTransE, TA-TransE, TA-DistMult, ChronoR, TuckERTNT, BoxTE, HyTE, TeRo, TComplEx, DE-SimplE, ATiSE, TeLM, EvoExplore, BDME, and QDN are discussed.

Non-Euclidean Embedding-based Methods:

• These methods embed knowledge graphs into non-Euclidean spaces to capture complex geometric structures.
• For SKGC, methods like ATTH, BiQUE, MuRMP, and GIE are discussed.
• For TKGC, methods such as DyERNIE and BiQCap are mentioned.

The passage does not provide summaries for specific sections.

Problem Definition

The paper introduces the concept of a temporal knowledge graph, denoted as 𝒢={ℰ,ℛ,𝒯,𝒬}𝒢ℰℛ𝒯𝒬\mathcal{G}={\mathcal{E},\ \mathcal{R},\ \mathcal{T},\ \mathcal{Q}}caligraphic_G = { caligraphic_E , caligraphic_R , caligraphic_T , caligraphic_Q }. This graph consists of a set of entities ℰℰ\mathcal{E}caligraphic_E, relations ℛℛ\mathcal{R}caligraphic_R, and timestamps 𝒯𝒯\mathcal{T}caligraphic_T. Each element in the graph is a quadruplet (𝐬,𝐫,𝐨,𝐭)∈𝒬𝐬𝐫𝐨𝐭𝒬(\mathbf{s},\ \mathbf{r},\ \mathbf{o},\ \mathbf{t})\in\mathcal{Q}( bold_s , bold_r , bold_o , bold_t ) ∈ caligraphic_Q, where 𝐬,𝐨∈ℰ𝐬𝐨ℰ\mathbf{s},\ \mathbf{o}\in\mathcal{E}bold_s , bold_o ∈ caligraphic_E are the head and tail entities, 𝐫∈ℛ𝐫ℛ\mathbf{r}\in\mathcal{R}bold_r

Methodology

The section describes the proposed model IME (Intrinsic Multi-curvature Embeddings) for temporal knowledge graph reasoning. IME consists of three main stages:

1. Multi-curvature Embeddings: Entities, relations, and timestamps are simultaneously modeled in Euclidean, hyperbolic, and hyperspherical spaces to capture complex geometric structures in temporal knowledge graphs. An information aggregation and distribution process is used to update the representations across spaces.

2. Space-shared and -specific Representations: Encoding functions with shared and specific parameters are employed to capture the commonalities and unique characteristics across different curvature spaces.

3. Adjustable Multi-curvature Pooling (AMP): An adjustable pooling approach is proposed to effectively retain important information when aggregating the space-shared and -specific representations for prediction.

Additionally, the loss function consists of four components: the task loss for link prediction, similarity loss to bridge spatial gaps, difference loss to capture space-specific features, and structure loss to ensure structural similarity across spaces.

The model is designed to comprehensively capture the complex structures and characteristics present in temporal knowledge graphs across multiple curvature spaces.

Experiment

This section describes the datasets used, experimental setup, evaluation metrics, and analysis of results for the proposed temporal knowledge graph completion (TKGC) model.

Three commonly used temporal knowledge graph (TKG) datasets are employed: ICEWS14, ICEWS05-15, and GDELT. The proposed model is compared against various baselines, including static KG completion (SKGC) methods like TransE, DistMult, SimplE, RotatE, and temporal KGC (TKGC) methods such as TA-DistMult, TeRo, ChronoR, ATiSE, TeLM, TuckERTNT, BoxTE, BDME, EvoExplore, DyERNIE, BiQCap, and QDN.

Link prediction metrics used are Mean Reciprocal Rank (MRR) and Hits@N (N=1,3,10). Hyperparameters like loss weights, embedding dimension, and learning rate are tuned using grid search on the validation set.

The key results and analyses are:

1. The proposed model outperforms state-of-the-art baselines across most metrics on all three datasets, showing its effectiveness in modeling complex geometric structures and reducing spatial gaps between different curvature spaces.

2. Ablation studies highlight the importance of the similarity loss in learning common features, the difference loss in capturing space-specific characteristics, and the structure loss in constraining embeddings across spaces.

3. Using an adjustable multi-curvature pooling approach improves performance over max pooling.

4. There is an optimal embedding dimension (500) beyond which performance degrades, likely due to overfitting or increased complexity.

The results demonstrate the proposed model's ability to effectively represent and complete temporal knowledge graphs by leveraging multiple curvature spaces and reducing spatial gaps between them.

Conclusion

The paper proposes a novel temporal knowledge graph completion (TKGC) method called Integrating Multi-curvature shared and specific Embedding (IME). IME models temporal knowledge graphs in multi-curvature spaces to capture complex geometric structures. It learns space-specific properties to comprehensively capture characteristic information and space-shared properties to reduce spatial gaps caused by heterogeneity across different curvature spaces.

IME innovatively introduces an Adjustable Multi-curvature Pooling (AMP) approach to effectively strengthen the retention of important information. Experimental results on several well-established datasets demonstrate that IME achieves competitive performance compared to state-of-the-art TKGC methods.

The work was funded by the National Key R&D Program of China, National Natural Science Foundation of China, and the R&D Program of Beijing Municipal Education Commission.