Ontology Alignment Evaluation Initiative - OAEI-2022 Campaign OAEI

Common Knowledge Graphs

This track evaluates the ability of matching systems to map the schema (classes) of large common knowledge graphs such as DBpedia, YAGO and NELL. Publicly available knowledge graphs are highly complementary, and known for sharing data about real-world entities such as person, organization and place. The goal of this task is to align classes from highly influential and domain-independent knowledge graphs.

Schedule

The schedule is that of OAEI 2022

Evaluation campaign results

The results of the OAEI 2022, for the Common Knowledge Graphs track are available here

Dataset

The Common Knowledge Graphs dataset composed of two knowledge graphs extracted from DBpedia and NELL, which is a semi-automatically constructed knowledge graph. The process of creating the gold standard is discussed in [1]. The second dataset mapps classes from Yago and Wikidata. Details about the two datasets are shared in the following table.

Knowledge Graph # Classes # Instances Avg #instance per class
NELL 134 1,184,377 8,905
DBpedia 138 631,461 4,576
Yago 304 5,149,594 33,691
Wikidata 304 2,160,102 12,576

The dataset is available here and here to download in order to test your matcher locally. Also, you can test your matcher on this task by using MELT platform (see MELT evaluation instructions). This year, we have three test cases:

MELT track repository:

For running the Common Knowledge Graph test suite in SEALS, you will have to specify the following input parameters:
Testsuite

Evaluation Modalities

The class alignments resulting from each matcher will be evaluated based on precision, recall, and f-measure. The gold standard of the NELL and DBpedia test case is only a partial gold standard. Therefore, our evaluation does not over-penalized systems that may discover reasonable matches that are not coded in the gold standard.

Organizers

This track is organized by:

For any questions or suggestions about the track please email: oafallatah1 at sheffield dot ac dot uk

References

[1] Fallatah, O., Zhang, Z., Hopfgartner, F. A gold standard dataset for large knowledge graphs matching, Proceedings of the 15th Ontology Matching workshop (2020). [pdf]