Ontology Alignment Evaluation Initiative - OAEI-2015 Campaign

Results for the Instance Matching Track

The Instance Matching Track aims at evaluating the performance of matching tools when the goal is to detect the degree of similarity between pairs of items/instances expressed in the form of OWL Aboxes.

The track is organized in the following five independent tasks, namely:

The tasks val-sem, val-struct, and val-sem-struct have been generated through SPIMBENCH, an IM benchmark for the assessment of IM techniques based on RDF data with associated schema.

To participate to the Instance Matching Track, submit results related to one, more, or even all the expected tasks.

Each task is articulated in two tests with different scales (i.e., number of instances to match):

In both tests, the goal is to discover the matching pairs (i.e., mappings) among the instances in the source dataset and the instances in the target dataset.

Author Disambiguation Task (author-dis task)

Participants

Expected mappings

The set of expected mappings for the mainbox test (initially hidden to the participants) is now available.

Analysis by precision, recall, and F-measure

Sandbox task
Expected mappings Retrieved mappings Precision Recall F-measure
EXONA 854 854 0.941 0.941 0.941
InsMT+ 854 722 0.834 0.705 0.764
Lily 854 854 0.981 0.981 0.981
LogMap 854 779 0.994 0.906 0.948
RiMOM 854 854 0.929 0.929 0.929
Mainbox task
Expected mappings Retrieved mappings Precision Recall F-measure
EXONA 8428 144827 0.0 0.0 NaN
InsMT+ 8428 7372 0.76 0.665 0.709
Lily 8428 8428 0.964 0.964 0.964
LogMap 8428 7030 0.996 0.831 0.906
RiMOM 8428 8428 0.911 0.911 0.911

Author Recognition Task (author-rec task)

Participants

  • EXONA
  • InsMT+
  • Lily
  • LogMap
  • RiMOM

Expected mappings

The set of expected mappings for the mainbox test (initially hidden to the participants) is now available.

Analysis by precision, recall, and F-measure

Sandbox task
Expected mappings Retrieved mappings Precision Recall F-measure
EXONA 854 854 0.518 0.518 0.518
InsMT+ 854 90 0.556 0.059 0.106
Lily 854 854 1.0 1.0 1.0
LogMap 854 854 1.0 1.0 1.0
RiMOM 854 854 1.0 1.0 1.0
Mainbox task
Expected mappings Retrieved mappings Precision Recall F-measure
EXONA 8428 8428 0.409 0.409 0.409
InsMT+ 8428 961 0.246 0.028 0.05
Lily 8428 8424 0.999 0.998 0.999
LogMap 8428 8436 0.999 1.0 0.999
RiMOM 8428 8428 0.999 0.999 0.999

SPIMBENCH Value Semantics (val-sem task)

Participants

  • STRIM
  • LogMap

Expected mappings

The set of expected mappings for the mainbox test (initially hidden to the participants) is now available.

Analysis by precision, recall, and F-measure

Sandbox task
Expected mappings Retrieved mappings Precision Recall F-measure
STRIM 9649 10641 0.90583 0.99896 0.95012
LogMap 9649 8350 0.993770 0.85998 0.92205
Mainbox task
Expected mappings Retrieved mappings Precision Recall F-measure
STRIM 97256 106232 0.91261 0.99684 0.95287
LogMap 97256 83880 0.99600 0.85902 0.92245

SPIMBENCH Value Structure (val-struct task)

Participants

  • STRIM
  • LogMap

Expected mappings

The set of expected mappings for the mainbox test (initially hidden to the participants) is now available.

Analysis by precision, recall, and F-measure

Sandbox task
Expected mappings Retrieved mappings Precision Recall F-measure
STRIM 10601 10657 0.99436 0.99962 0.99698
LogMap 10601 8779 0.99270 0.82209 0.89938
Mainbox task
Expected mappings Retrieved mappings Precision Recall F-measure
STRIM 106137 105352 0.99386 0.99587 0.99486
LogMap 106137 87137 0.99709 0.81860 0.89907

SPIMBENCH Value Structure Semantics (val-struct-sem task)

Participants

  • STRIM
  • LogMap

Expected mappings

The set of expected mappings for the mainbox test (initially hidden to the participants) is now available.

Analysis by precision, recall, and F-measure

Sandbox task
Expected mappings Retrieved mappings Precision Recall F-measure
STRIM 9790 10639 0.91916 0.99887 0.95736
LogMap 9790 7779 0.99395 0.78978 0.88018
Mainbox task
Expected mappings Retrieved mappings Precision Recall F-measure
STRIM 98144 106576 0.91765 0.99649 0.95545
LogMap 98144 77983 0.99574 0.79119 0.88176
Original page: http://islab.di.unimi.it/im_oaei_2015 [cached: 13/05/2016]
http://oaei.ontologymatching.org/2015/im/results.html