OAEI 2006 food
Thesaurus Mapping Task: results

Data

The results of the participants are in. The following teams have entered their final results: South East University (Falcon-AO), University of Pittsburgh (Prior), Tsinghua University (KEG), University of Leipzig (COMA++), and Università degli Studi di Milano (HMatch).

The results of the participants can be found here
The Gold Standard used to evaluate Precision and Recall can be found here

Evaluation

1. Each participant submits his preliminary mappings, in the common format for alignments, before September 4th.
2. Each participant submits his final mappings before September 15th.
3. The mappings will be distributed at random and anonymously over a group of domain experts. (food safety researchers and food product development researchers)
4. A (small) sample of the mappings are distributed at random and anonymously over the other participants.
5. The domain experts and participants are required to assess the mappings appointed to him before October 5th.
6. The results are published before October 9th.
7. Evaluation measurements of the participants' systems calculated based on this list of reference alignments.

A web tool for the assessment of the results has been set up at the Free University Amsterdam. Participants have been contacted personally about their assessment loads.

An example assessment page can be found here or at the following web address: http://prauw.cs.vu.nl/~wrvhage/oaei2006

Results

Precision

The Falcon-AO system of South East University has a significantly better Precision than all other systems at mapping taxonomical and miscellaneous concepts (such as geographical concepts and food stuffs). The RiMOM system of Tsinghua University (KEG) has a significantly better Precision than all other systems at mapping biological and chemical concepts.

Recall

OAEI 2006 food results presentation

Organization

Send any questions, comments, or suggestions to:
Willem Robert van Hage ( )