Ontology Alignment Evaluation Initiative - OAEI-2011 Campaign

Oriented Matching 2011

An increasing number of matchers are now capable of deriving mapping relations other than equivalence relations, such as subsumption, disjointness or named relations. This is a necessity given that we need to compute alignments between ontologies at different granularity levels or between ontologies that elaborate on non-equivalent elements. The evaluation of such mappings was addressed already in OAEI (2009) Oriented Matching track. Following these goals we wish to get a better insight into the types of non-equivalence mappings that state-of-the-art tools produce and report on existing methods for computing subsumption mappings. The track aims also to report on evaluation methods and measures for subsumption mappings, in conjunction to the computation of equivalence mappings. Targeting these goals, we have built new benchmark datasets that are described below. The track focuses on a gold standard based evaluation for two given datasets that has been derived from a) the OAEI benchmark dataset (all possible pairs of the real ontologies 301 to 304) and b) all pairs of ontologies in the Illinois course catalogs dataset.

Data sets

Concerning the evaluation of equivalence and subsumption relations the track provides two datasets for the participants:

For each pair of ontologies we provide reference alignments for both equivalence and subsumption mappings and we aim to evaluate tools for their ability to identify both equivalences and subsumptions between classes. However, we aim at distinguishing between tools that (a) compute class equivalences and subsumptions, (b) compute class subsumptions without computing any equivalences and (c) compute class subsumptions and non-class equivalences. These can be done either by declaration of the participants or by providing justifications for the alignments: For subsumption mappings that cannot be inferred (e.g. by exploiting equivalence or subsumption relations between classes/properties), we have added an annotation property called 'Alignment_Argument' in the corresponded alignment cells of the refalign.rdf file. The value of this property is an argument upon the reference alignment. Each tool can assign a value to this property in relation to each mapping relation discovered that provides grounding for its decisions.

It is important to use our related namespace (http://ai-lab-webserver.aegean.gr/ai-lab/oaei2011/alignArguments.rdf) in order to annotate your alignments (similar to our refalign.rdf files provided)!!!

The reference alignments (refalign.rdf) have been constructed by manually creating a merged ontology for each pair of source ontologies as follows:

  1. Import the two source ontologies
  2. Manually introduce equivalences between classes of the two imported ontologies
  3. Classify classes in the final ontology
  4. Introduce additional subsumption relations
  5. The resulting ontology is translated to equivalence and subsumption mappings that involve classes with different URIs to the RDF rendered of the Alignment API (RDFRendererVisitor).

The datasets are accompanied with documentation of the specific changes made to the original datasets. The rationale behind these changes can be found a) in the documentation.txt file in each ontology pair folder, b) as an annotation property in the specification of the subsumption relations in the merged ontology, c) in the corresponded alignment cells of the refalign.rdf file.

Datasets can be downloaded directly from here.


We will use precision, recall and f-measures for the efficacy of tools to produce the mappings. Participants will have to provide an alignment containing both equivalent and subsumption relations, optionally (but desired) with corresponding arguments.


May 30th
datasets are out
June 27th
end of commenting period
July 1st
tests are frozen
September 1st
participants send preliminary results for interoperability-checking (old-style tracks), or ensure that the last version of their tools is submitted to the SEALS platform (SEALS tracks, see further instructions for at the Execution phase)
September 23rd
participants send final results (old-style tracks) or submit the ultimate tool version to the SEALS platform (SEALS tracks). All participants must send their papers.
October 22nd
final results ready and OM-2011 workshop.
November 15th
participants send final versions of papers to Cassia Trojahn and Pavel Shvaiko. Tool

The alignArguments namespace extension in order to be able to annotate your alignments (similar to our refalign.rdf files provided) can be found at http://ai-lab-webserver.aegean.gr/ai-lab/oaei2011/alignArguments.rdf. Additional argumentation extensions, published as part of the on-line Alignment API documentation at http://alignapi.gforge.inria.fr/labels.html, may be used.


George Vouros, Konstantinos Kotis and Vassilis Spiliopoulos, University of the Aegean, Greece E-mails: georgev@aegean.gr, kotis@aegean.gr, 2billy@gmail.com

Original page: http://ai-lab-webserver.aegean.gr/oaei2011/oaei2011.html [cached: 06/12/2011]