Ontology Alignment Evaluation Initiative - OAEI-2018 CampaignOAEI OAEI

Knowledge Graph Track

The Knowledge Graph Track contains nine isolated knowledge graphs with instance and schema data. The goal of the task is to match both the instances and the schema. The knowledge graphs were created in the course of the DBkWik by running the DBpedia extraction framework on Wikis from the Fandom Wiki hosting platform. The evaluation process of the Knowledge Graph Track will be supported by SEALS platform as well as Hobbit platform.

Data set

The data set is available from this Web page (see below). You can either download them for local analysis and processing, or you can directly use the SEALS or Hobbit platform. Both options are possible, but ultimately, we require you to upload your matching system to the Hobbit platform.
For running the Knowledge Graph test suite, you will have to specify the following input parameters:

Seals Testsuite

Hobbit Testsuite

For the evaluation, we use a gold standard of correspondences both on the schema and the instance level. While the schema level correspondences were created by experts, the instance correspondences were crowd sourced using Amazon MTurk. Due to the large amount of instances and classes this gold standard is only a partical gold standard.

The following table describes the knowledge graphs and the sources they were created from:

Source Source URL LanguageHubTopic#Instances#Properties#ClassesDump
RuneScape Wiki http://runescape.wikia.com en Games Gaming 200,605 1,998 106 rdf/xml
Old School RuneScape Wiki http://oldschoolrunescape.wikia.com en Games Gaming 38,563 488 53 rdf/xml
DarkScape Wiki http://darkscape.wikia.com en Games Gaming 19,623 686 65 rdf/xml
Marvel Database http://marvel.wikia.com en Comics Comics 56,464 99 2 rdf/xml
Hey Kids Comics Wiki http://heykidscomics.wikia.com en Comics Entertainment 158,234 1,925 181 rdf/xml
DC Database http://dc.wikia.com en Comics Lifestyle 128,495 177 5 rdf/xml
Memory Alpha http://memory-alpha.wikia.com en TV Entertainment 63,240 326 0 rdf/xml
Star Trek Expanded Universe http://stexpanded.wikia.com en TV Entertainment 17,659 201 3 rdf/xml
Memory Beta http://memory-beta.wikia.com en Books Entertainment 63,223 413 11 rdf/xml


Each dump contains the following output files of the DBpedia Extraction Framework, similar to DBpedia (combined in one single rdf/xml file):
Name Comment Example/Preview
article-categories (wiki page, dct:subject, category) triples preview
category-labels (category, rdfs:label, label) triples preview
disambiguations-redirected (wiki page, dbkwik:wikiPageDisambiguates, wiki page) triples preview
external-links (wiki page, dbkwik:wikiPageExternalLink, url) triples.
all links to external uri
preview
images triples with foaf:depiction, foaf:thumbnail, dc:rights preview
infobox-properties-redirectedextracted triples from infoboxes
uri of property: http://dbkwik.webdatacommons.org/{wiki}/property/{name}
preview
infobox-property-definitions defines the type of properties to be rdf:Property and
contains corresponding labels
preview
labels (wiki page, rdfs:label, label) triples
the label is usually the title of the wiki page
preview
long-abstracts (wiki page, dbkwik:abstract, abstract) triples
the abstract is the text until the toc or first header.
preview
short-abstracts (wiki page, rdfs:comment, comment) triples
short abstract between [200,600] see extractor
preview
skos-categories skos:prefLabel and skos:broader of categories
(category tree)
preview
template-type (wiki page, rdf:type, class) triples
type information for some instances
preview
template-type-definitions label and type of classes. preview


Knowledge graphs extracted from Wikis of the same topic are to be matched with each other. This results in the following subtasks: An example for the matching task is depicted below. Schema alignments are shown in blue, while instance alignments are shown in red:

Modalities

Participants will use the Hobbit platform to upload their matcher. The alignments will be evaluated based on Precision, Recall and F-Measure. We will compare the overall performance, as well as the performance on instance and schema level in isolation.

Schedule

The schedule is available at the OAEI main page.

Organizers