Ontology Alignment Evaluation Initiative - OAEI-2017.5 CampaignOAEI HOBBIT

Conference track

Conference track contains 16 ontologies from the same domain (conference organization). These ontologies are suitable for ontology matching task because of their heterogenous character of origin. An evaluation process will be supported by the HOBBIT platform, see section below for list of planned evaluation methods.

News: Ontologies from this track are from the OntoFarm collection. If you use the conference (OntoFarm) ontologies in your work, please cite new paper about OntoFarm [4]: Zamazal, O., Svátek, V. (2017). The Ten-Year OntoFarm and its Fertilization within the Onto-Sphere. Web Semantics: Science, Services and Agents on the World Wide Web, 43, 46-53.

Results

Results will be available according to OAEI schedule.

Data set and HOBBIT Supporting Evaluation

Data set is available from this web-page (see below). You can either download them and try them on your own or you can directly use the HOBBIT platform. Both options are possible, however finally systems willing to participate in the OAEI 2017.5 need to follow the instructions as described on the main page. Systems participating in this track should explicitly mention that implements its API: bench:ConferenceAPI. See the implementation of the LogMap system and its metadata description (system.ttl).

For running the conference track using the HOBBIT platform (see more information) you can use the web interface and selecting:

Dataset description

The collection of ontologies is dealing with conference organization. They have been developed within OntoFarm project. 16 ontologies are included; the last one (linklings.owl) is in the OWL 2 profile. Each of them is shortly described in the following table. Ontologies have been based upon three types of underlying resources:

Name is derived from name of conference or conference organisation tool. For downloading this ontology, just click on its name or you can download all ontologies at once.
DL expressivity was obtained by Pellet reasoner.

NameTypeNumber of ClassesNumber of Datatype Properties Number of Object PropertiesDL expressivityRelated link
EkawInsider74033SHINhttp://ekaw.vse.cz
SofsemInsider601846ALCHIF(D)http://www.sofsem.cz
SigkddWeb491117ALEI(D)http://www.acm.org/sigs/sigkdd/kdd2006
IastedWeb140338ALCIN(D)http://iasted.com/conferences/2005/cancun/ms.htm
MicroWeb32917ALCOIN(D)www.microarch.org
ConfiousTool57552SHIN(D)http://www.confious.com
PcsTool231424ALCIF(D)http://precisionconference.com
OpenConfTool622124ALCOI(D)http://www.zakongroup.com/technology/openconf.shtml
ConfToolTool382313SIN(D)http://www.conftool.net
CrsTool14215ALCIF(D)http://www.conferencereview.com
CmtTool361049ALCIN(D)http://msrcmt.research.microsoft.com/cmt
CocusTool55035ALCIFhttp://cocus.create-net.it/
PaperdyneTool472161ALCHIN(D)http://www.paperdyne.com/
EdasTool1042030ALCOIN(D)http://edas.info/
MyReviewTool391749ALCOIN(D)http://myreview.intellagence.eu/
LinklingsTool371631SROIQ(D)http://www.linklings.com/

Modalities

The HOBBIT platform will deliver results in terms of precision, recall and fmeasure for the open reference alignment. The alignments generated by systems via the HOBBIT platform will be further evaluated based on

The confidence values for all matches in the crisp reference alignment are 1.0, while the uncertain version of the reference alignment has confidence values that reflect the degree of agreement of a group of twenty people on the validity of the match. Crisp reference alignment contains 21 alignments (test cases), which corresponds to the complete alignment space between 7 ontologies (Cmt, ConfTool, Edas, Ekaw, Iasted, Sigkdd, Sofsem) from the data set. This is a subset of all ontologies within this track (16). Total number of possible test cases is hence 120. There are three variants of reference alignment:

Schedule

The schedule is available at the OAEI main page.

Organizers

References

[1] Michelle Cheatham, Pascal Hitzler: Conference v2.0: An Uncertain Version of the OAEI Conference Benchmark. International Semantic Web Conference (2) 2014: 33-48.

[2] Alessandro Solimando, Ernesto Jiménez-Ruiz, Giovanna Guerrini: Detecting and Correcting Conservativity Principle Violations in Ontology-to-Ontology Mappings. International Semantic Web Conference (2) 2014: 1-16.

[3] Alessandro Solimando, Ernesto Jiménez-Ruiz, Giovanna Guerrini: Minimizing conservativity violations in ontology alignments: algorithms and evaluation. Knowl. Inf. Syst. 51(3): 775-819 (2017)

[4] Ondřej Zamazal, Vojtěch Svátek. The Ten-Year OntoFarm and its Fertilization within the Onto-Sphere. Web Semantics: Science, Services and Agents on the World Wide Web, 43, 46-53. 2017.