Ontology Alignment Evaluation Initiative - OAEI-2024 Campaign

Complex track

General description

Complex alignments are more expressive than simple alignments as their correspondences can contain logical constructors or transformation functions of literal values.

• A correspondence stating that: Forall x, o1:AcceptedPaper(x) = Exists y, o2:acceptedBy(x,y) is a complex correspondence with a logical constructor (property restriction);
• A correspondence stating that: Forall x,y, o1:authorOf(x,y) = o2:writtenBy(y,x) is a complex correspondence with a logical constructor (inverse property);
• A correspondence stating that: The values of the o1:fullName property can be obtained by concatenating the values of o2:firstName and o2:lastName is a complex correspondence with a transformation function of literal values (string concatenation).

With this track, we evaluate systems which can generate such correspondences.

The complex track contains 2 datasets from the conference domain: Conference and Populated Conference. Each of the datasets and corresponding evaluation methods are presented below.

The participants of the track should output their (complex) correspondences in the EDOAL format. This format is supported by the Alignment API. The evaluation will be supported by the MELT framwork this year. The participants have to wrap their tool as described at MELT framework. For executing the tasks in each dataset the parameters are listed in boxes below (repository, suite-id, version-id).

The number of ontologies, simple (1:1) and complex (1:n), (m:n) correspondences for each dataset of this track are summarized in the following table.

Ontologies and correspondences

This dataset is based on the OntoFarm dataset [1] used in the Conference track of the OAEI campaigns. It is composed of 16 ontologies on the conference organisation domain and simple reference alignments between 7 of them. Here, we consider 3 out of the 7 ontologies from the reference alignments (cmt, conference and ekaw), resulting in 3 alignment pairs.

The correspondences were manually curated by 3 experts following the query rewriting methodology in [2]. For each pair o1-o2 of ontologies, the following steps were applied:

• Create a simple equivalence consensus alignment: ra1 was slightly modified.
• For each entity e1 of o1 not in a simple equivalence correspondence, find a semantically equivalent construction from o2 entities. If no equivalence can be found, look for the closest entity or construction from o2 subsumed by e1.
• Repeat previous step for each entity of o2 (constructions from o1 entities)

4 experts assessed the curated correspondences to reach a consensus.

Evaluation modalities

The complex correspondences output by the systems will be manually compared to the ones of the consensus alignment.

For this first evaluation, only equivalence correspondences will be evaluated and the confidence of the correspondenes will not be taken into account.

The systems can take the ra1 simple alignments as input. The ra1 alignments can be downloaded here.

In order to allow matchers which rely on instances to participate over the Conference complex track, we propose a populated version of the Conference dataset. 5 ontologies have been populated with more or less common instances resulting in 6 datasets: (6 versions on the repository: v0, v20, v40, v60, v80 and v100).

Evaluation modalities

The alignments will be evaluated based on Competency Questions for Alignment: basic queries that the alignment should be able to cover [6].

The queries are automatically rewritten using 2 systems:

• that from [7] which covers (1:n) correspondences with EDOAL expressions.
• a system which compares the answers (sets of instances or sets of pairs of instances) of the source query and the source member of the correspondences and which outputs the target member if both sets are identical.

The best rewritten query scores are kept.

A precision score will be given by comparing the instances described by the source and target members of the correspondences.

Organizers

• Cassia Trojahn (IRIT, Toulouse, France), cassia [.] trojahn [at] irit [.] fr
• Ondřej Zamazal (University of Economics, Prague), ondrej [.] zamazal [at] vse [.] cz
• Marta Silva (University of Lisbon), mcdsilva [at] fc [.] ul [.] pt
• Guilheme Sousa (IRIT, Toulouse, France), Guilherme [.] Santos [-] Sousa [at] irit.fr

References

[1] Ondřej Zamazal, Vojtěch Svátek. The Ten-Year OntoFarm and its Fertilization within the Onto-Sphere. Web Semantics: Science, Services and Agents on the World Wide Web, 43, 46-53. 2017.

[2] Élodie Thiéblin, Ollivier Haemmerlé, Nathalie Hernandez, Cassia Trojahn. Task-Oriented Complex Ontology Alignment: Two Alignment Evaluation Sets. In : European Semantic Web Conference. Springer, Cham, 655-670, 2018.

[3] Élodie Thiéblin, Fabien Amarger, Nathalie Hernandez, Catherine Roussey, Cassia Trojahn. Cross-querying LOD datasets using complex alignments: an application to agronomic taxa. In: Research Conference on Metadata and Semantics Research. Springer, Cham, 25-37, 2017.

[4] Lu Zhou, Michelle Cheatham, Adila Krisnadhi, Pascal Hitzler. A Complex Alignment Benchamark: GeoLink Dataset. In: International Semantic Web Conference. Springer, Proceedings, Part II, pp. 273-288, 2018.

[5] Marc Ehrig, and Jérôme Euzenat. "Relaxed precision and recall for ontology matching." K-CAP 2005 Workshop on Integrating Ontologies, Banff, Canada, 2005.

[6] Élodie Thiéblin. Do competency questions for alignment help fostering complex correspondences?. In EKAW Doctoral Consortium, 2018.

[7] Élodie Thiéblin, Fabien Amarger, Ollivier Haemmerlé, Nathalie Hernandez, Cassia Trojahn. Rewriting SELECT SPARQL queries from 1:n complex correspondences. In: Ontology Matching, pp. 49-60, 2016.