Ontology Alignment Evaluation Initiative - OAEI-2018 Campaign

Complex track - GeoLink - Evaluation

Evaluation Tasks

There are three subtasks related to this evaluation:

1. Entity Identification: For each entity in the source ontology, list all ofthe entities that are related in some way in the target ontology.
2. Relationship Identification: Given a dictionary containing entitiesfrom the source ontology paired with all related entities, determine theexpression that specifies the nature of the relation.
3. Full Complex Alignment Identification: A combination of the two former step to determine the complex alignment that exist between thesource and target ontology.

All three subtasks are evaluated based on standard precision, recall, and F-measure.

Results

There are around 16 ontology alignment systems who participate in this year's OAEI. Unfortunately, none of the alignment systems are capable of producing results for subtasks 2 and 3 on GeoLink benchmark. The following table summarizes the evaluation results of the matchers that can produce on subtask 1.

Disscussion

There are 7 systems that can produce the alignment on the subtask 1 of GeoLink dataset. Among these alignments, all correspondences between GeoLink Base Ontology (GBO) and GeoLink Modular Ontology (GMO) are 1-to-1 equivalence. The precision of most of the systems are still relatively high, which means that the traditional ontology alignment systems can handle the simple relations in real-world ontologies as well, since the GeoLink benchmark is created based on a real-world case. But, it is not surprising that the low recall reflects that the current ontology alignment systems are not capable of identifying more complex relations, which we hope that it will be improved in the next future years.