Ontology Alignment Evaluation Initiative - OAEI Campaign OAEI OAEI

Knowledge Graph Track

The Knowledge Graph Track contains isolated knowledge graphs with instance and schema data. The goal of the task is to match both the instances and the schema. The knowledge graphs were created in the course of the DBkWik by running the DBpedia extraction framework on Wikis from the Fandom Wiki hosting platform. The evaluation process of the Knowledge Graph Track will be supported by MELT toolkit.

Data set

The data set is available from this Web page (see below). You can either download them for local analysis and processing, or you can directly use the MELT toolkit which contains many OAEI tracks and allows for fast and fine-grained evaluation (test case coordinates are also online).

MELT track repository:
For running the Knowledge Graph test suite in SEALS, you will have to specify the following input parameters:

For the evaluation, we use a gold standard of correspondences both on the schema and the instance level. While the schema level correspondences were created by experts, the instance correspondences were extracted from the "External links" section of the wiki pages. Due to the large amount of instances and classes this gold standard is only a partical gold standard.

The following table describes the knowledge graphs and the sources they were created from:

Source Source URL LanguageHubTopic#Instances#Properties#ClassesDump
Star Wars Wiki http://starwars.wikia.com en Movies Entertainment 145,033 700 269 rdf/xml
The Old Republic Wiki http://swtor.wikia.com en Games Gaming 4,180 368 101 rdf/xml
Star Wars Galaxies Wiki http://swg.wikia.com en Games Gaming 9,634 148 67 rdf/xml
Marvel Database http://marvel.wikia.com en Comics Comics 210,996 139 186 rdf/xml
Marvel Cinematic Universe Wiki http://marvelcinematicuniverse.wikia.comen Movies Entertainment 17,187 147 55 rdf/xml
Memory Alpha http://memory-alpha.wikia.com en TV Entertainment 45,828 325 181 rdf/xml
Star Trek Expanded Universe http://stexpanded.wikia.com en TV Entertainment 13,426 202 283 rdf/xml
Memory Beta http://memory-beta.wikia.com en Books Entertainment 51,323 423 240 rdf/xml

Each dump contains the following output files of the DBpedia Extraction Framework, similar to DBpedia (combined in one single rdf/xml file):
Name Comment Example/Preview
anchor-text (wiki page, dbkwik:wikiPageWikiLinkText>, text) triples
text appears as link text of the wiki page
article-categories (wiki page, dct:subject, category) triples preview
category-labels (category, rdfs:label, label) triples preview
disambiguations (wiki page, dbkwik:wikiPageDisambiguates, wiki page) triples preview
external-links (wiki page, dbkwik:wikiPageExternalLink, url) triples.
all links to external uri
images triples with foaf:depiction, foaf:thumbnail, dc:rights preview
infobox-properties extracted triples from infoboxes
uri of property: http://dbkwik.webdatacommons.org/{wiki}/property/{name}
infobox-property-definitions defines the type of properties to be rdf:Property and
contains corresponding labels
infobox-template-type (wiki page, rdf:type, class) triples, where template name contains "infobox" preview
infobox-template-type-definitions defines the type of properties to be rdf:Property and
contains corresponding labels
labels (wiki page, rdfs:label, label) triples
the label is usually the title of the wiki page
long-abstracts (wiki page, dbkwik:abstract, abstract) triples
the abstract is the text until the toc or first header.
short-abstracts (wiki page, rdfs:comment, comment) triples
short abstract between [200,600] see extractor
skos-categories skos:prefLabel and skos:broader of categories
(category tree)
template-type (wiki page, rdf:type, class) triples preview
template-type-definitions label and type of classes. preview
page-links (wiki page, dbkwik:wikiPageWikiLink, wiki page) triples
represents any link between two pages

This year there are the following test cases: An example for the matching task is depicted below. Schema alignments are shown in blue, while instance alignments are shown in red:


The alignments will be evaluated based on Precision, Recall and F-Measure. We will compare the overall performance, as well as the performance on instance and schema level in isolation.
The matching system does not need to match instances of classes "http://dbkwik.webdatacommons.org/ontology/Image" and skos:Concept. These are only included to help the system to find good matches. For the evaluation it doesn't make a difference because instances of these classes are not evaluated at all.


The schedule is available at the OAEI main page.



[1] Sven Hertling, Heiko Paulheim: The Knowledge Graph Track at OAEI - Gold Standards, Baselines, and the Golden Hammer Bias. Extended Semantic Web Conference (ESWC) 2020. [pdf]

[2] Sven Hertling, Heiko Paulheim: DBkWik: A Consolidated Knowledge Graph from Thousands of Wikis. International Conference on Big Knowledge 2018. [pdf]

[3] Alexandra Hofmann, Samresh Perchani, Jan Portisch, Sven Hertling, and Heiko Paulheim. DBkWik: Towards Knowledge Graph Creation from Thousands of Wikis. International Semantic Web Conference (Posters & Demos) 2017. [pdf]