Ontology Alignment Evaluation Initiative - OAEI-2016 Campaign

Conference track

Conference track contains 16 ontologies from the same domain (conference organization). These ontologies are suitable for ontology matching task because of their heterogenous character of origin. An evaluation process will be supported by SEALS platform, see section below for list of planned evaluation methods.

Results

Results are available.

Data set

Data set is available from this web-page (see below). You can either download them and try them on your own or you can directly use SEALS platform. Both options are possible, however finally you have to wrap your tool against the SEALS client provided via the OAEI main page and run this client against the Conference testsuite. For running the Conference testsuite you will have to specify the following input parameters:

The collection of ontologies is dealing with conference organization. They have been developed within OntoFarm project. 16 ontologies are included; the last one (linklings.owl) is in the OWL 2 profile. Each of them is shortly described in the following table. Ontologies have been based upon three types of underlying resources:

• actual conference (series) and its web pages (indicated in column Type with mark Web),
• actual software tool for conference organisation support (indicated in column Type with mark Tool),
• experience of people with personal participation in organisation of actual conference (indicated in column Type with mark Insider).

These ontologies are in English. Based on this collection multilingual collection MultiFarm have been made and included into the OAEI campaign as MultiFarm track.

Participants will use SEALS platform to deliver their results. Alignments submitted via SEALS platform will be evaluated based on

• crisp reference alignment,
• its uncertain version (presented at ISWC 2014 [1]) and
• logical reasoning evaluation based on violations of conservativity principle (presented at OWLED 2014 and at ISWC 2014 [2,3]).
• further modalities will be added soon

  The confidence values for all matches in the crisp reference alignment are 1.0, while the uncertain version of the reference alignment has confidence values that reflect the degree of agreement of a group of twenty people on the validity of the match. Crisp reference alignment contains 21 alignments (testcases), which corresponds to the complete alignment space between 7 ontologies (Cmt, ConfTool, Edas, Ekaw, Iasted, Sigkdd, Sofsem) from the data set. This is a subset of all ontologies within this track (16). Total number of possible testcases is hence 120. There are two variants of reference alignment:

  • original reference alignment (ra1) which you can download - please let us know how you use this reference-alignment (outside the range of the OAEI campagin) and data set (ondrej.zamazal at vse dot cz).
  • entailed reference alignment (ra2) generated as a transitive closure computed on the original reference alignment (ra1). In order to obtain coherent reference alignment set, conflicting correspondences have been inspected and resolved by evaluators. As a result the degree of correctness and completeness of ra2 is probaly slightly better than for ra1. However, the differences are relatively restricted. ra2 is not available.

  Furthermore, participants can directly submit their 'interesting individual correspondences' ('nuggets') which could represent complex correspondences to organizers of this conference track, for contacts please see below. These correspondences will be evaluated manually.

  Schedule

  The schedule is available at the OAEI main page.

  Organizers

  • Ondřej Zamazal (University of Economics, Prague), main contact for the track, ondrej dot zamazal at vse dot cz
  • Michelle Cheatham (Wright State University, USA)

  References

  [1] Michelle Cheatham, Pascal Hitzler: Conference v2.0: An Uncertain Version of the OAEI Conference Benchmark. International Semantic Web Conference (2) 2014: 33-48.

  [2] Alessandro Solimando, Ernesto Jiménez-Ruiz, Giovanna Guerrini: Detecting and Correcting Conservativity Principle Violations in Ontology-to-Ontology Mappings. International Semantic Web Conference (2) 2014: 1-16.

  [3] Alessandro Solimando, Ernesto Jiménez-Ruiz, Giovanna Guerrini: A Multi-strategy Approach for Detecting and Correcting Conservativity Principle Violations in Ontology Alignments. OWL: Experiences and Directions Workshop 2014 (OWLED 2014). 13-24.

  http://oaei.ontologymatching.org/2016/conference/index.html