Ontology Alignment Evaluation Initiative - OAEI-2021 Campaign OAEI

Common Knowledge Graphs

This track evaluates the ability of matching systems to map the schema (classes) of large common knowledge graphs such as DBpedia, YAGO and NELL. Publicly available knowledge graphs are highly complementary, and known for sharing data about real-world entities such as person, organization and place. The goal of this task is to align classes from highly influential and domain-independent knowledge graphs (i.e., NELL and DBpedia).

Schedule

The schedule is that of OAEI 2021

Evaluation campaign results

The results of the OAEI 2021, for the Common Knowledge Graphs track are available here

Dataset

The Common Knowledge Graphs dataset composed of two knowledge graphs extracted from DBpedia and NELL, a semi-automatically constructed knowledge graph. The process of creating the gold standard is discussed in [1]. Details about this dataset are shared in the following table.

Knowledge Graph # Classes # Instances Avg #instance per class
NELL 134 1,184,377 8,905
DBpedia 138 631,461 4,576

The dataset is available here to download in order to test your matcher locally. Also, you can test your matcher on this task by using MELT platform (see MELT evaluation instructions).

MELT track repository:

For running the Common Knowledge Graph test suite in SEALS, you will have to specify the following input parameters:
Testsuite

Evaluation Modalities

The class alignments resulting from each matcher will be evaluated based on precision, recall, and f-measure. The gold standard of this task is only a partial gold standard. Therefore, our evaluation does not over-penalized systems that may discover reasonable matches that are not coded in the gold standard.

Organizers

This track is organized by:

For any questions or suggestions about the track please email: oafallatah1 at sheffield dot ac dot uk

References

[1] Fallatah, O., Zhang, Z., Hopfgartner, F. A gold standard dataset for large knowledge graphs matching, Proceedings of the 15th Ontology Matching workshop (2020). [pdf]