Knowledge Graph Track

The Knowledge Graph Track contains nine isolated knowledge graphs with instance and schema data. The goal of the task is to match both the instances and the schema. The knowledge graphs were created in the course of the DBkWik by running the DBpedia extraction framework on Wikis from the Fandom Wiki hosting platform. The evaluation process of the Knowledge Graph Track will be supported by SEALS platform as well as Hobbit platform.

Data set

The data set is available from this Web page (see below). You can either download them for local analysis and processing, or you can directly use the SEALS or Hobbit platform. Both options are possible, but ultimately, we require you to upload your matching system to the Hobbit platform.
For running the Knowledge Graph test suite, you will have to specify the following input parameters:

Seals Testsuite

Repository: http://oaei.webdatacommons.org/tdrs/
Suite-ID: knowledgegraph
Version-ID: v1

Hobbit Testsuite

Benchmark API: bench:knowledgegraph

For the evaluation, we use a gold standard of correspondences both on the schema and the instance level. While the schema level correspondences were created by experts, the instance correspondences were crowd sourced using Amazon MTurk. Due to the large amount of instances and classes this gold standard is only a partical gold standard.

The following table describes the knowledge graphs and the sources they were created from:

Source	Source URL	Language	Hub	Topic	#Instances	#Properties	#Classes	Dump

RuneScape Wiki	http://runescape.wikia.com	en	Games	Gaming	200,605	1,998	106	rdf/xml
Old School RuneScape Wiki	http://oldschoolrunescape.wikia.com	en	Games	Gaming	38,563	488	53	rdf/xml
DarkScape Wiki	http://darkscape.wikia.com	en	Games	Gaming	19,623	686	65	rdf/xml

Marvel Database	http://marvel.wikia.com	en	Comics	Comics	56,464	99	2	rdf/xml
Hey Kids Comics Wiki	http://heykidscomics.wikia.com	en	Comics	Entertainment	158,234	1,925	181	rdf/xml
DC Database	http://dc.wikia.com	en	Comics	Lifestyle	128,495	177	5	rdf/xml

Memory Alpha	http://memory-alpha.wikia.com	en	TV	Entertainment	63,240	326	0	rdf/xml
Star Trek Expanded Universe	http://stexpanded.wikia.com	en	TV	Entertainment	17,659	201	3	rdf/xml
Memory Beta	http://memory-beta.wikia.com	en	Books	Entertainment	63,223	413	11	rdf/xml

Each dump contains the following output files of the DBpedia Extraction Framework, similar to DBpedia (combined in one single rdf/xml file):

Name	Comment	Example/Preview
article-categories	(wiki page, dct:subject, category) triples	preview
category-labels	(category, rdfs:label, label) triples	preview
disambiguations-redirected	(wiki page, dbkwik:wikiPageDisambiguates, wiki page) triples	preview
external-links	(wiki page, dbkwik:wikiPageExternalLink, url) triples. all links to external uri	preview
images	triples with foaf:depiction, foaf:thumbnail, dc:rights	preview
infobox-properties-redirected	extracted triples from infoboxes uri of property: http://dbkwik.webdatacommons.org/{wiki}/property/{name}	preview
infobox-property-definitions	defines the type of properties to be rdf:Property and contains corresponding labels	preview
labels	(wiki page, rdfs:label, label) triples the label is usually the title of the wiki page	preview
long-abstracts	(wiki page, dbkwik:abstract, abstract) triples the abstract is the text until the toc or first header.	preview
short-abstracts	(wiki page, rdfs:comment, comment) triples short abstract between [200,600] see extractor	preview
skos-categories	skos:prefLabel and skos:broader of categories (category tree)	preview
template-type	(wiki page, rdf:type, class) triples type information for some instances	preview
template-type-definitions	label and type of classes.	preview

Knowledge graphs extracted from Wikis of the same topic are to be matched with each other. This results in the following subtasks:

An example for the matching task is depicted below. Schema alignments are shown in blue, while instance alignments are shown in red:

Modalities

Participants will use the Hobbit platform to upload their matcher. The alignments will be evaluated based on Precision, Recall and F-Measure. We will compare the overall performance, as well as the performance on instance and schema level in isolation.

Schedule

The schedule is available at the OAEI main page.

Organizers

Sven Hertling (University of Mannheim, Germany), main contact for the track, sven at informatik dot uni-mannheim dot de
Heiko Paulheim (University of Mannheim, Germany)

References

[1] Sven Hertling, Heiko Paulheim: DBkWik: A Consolidated Knowledge Graph from Thousands of Wikis. International Conference on Big Knowledge 2018. [pdf]

[2] Alexandra Hofmann, Samresh Perchani, Jan Portisch, Sven Hertling, and Heiko Paulheim. DBkWik: Towards Knowledge Graph Creation from Thousands of Wikis. International Semantic Web Conference (Posters & Demos) 2017. [pdf]