Ontology Alignment Evaluation Initiative - OAEI-2008 Campaign

Multilingual directory Data Set (MLdirectoy) - Results

Results for English-English alignment tasks

In the 2008 campaign, we have four participants for the mldirectory data set. The systems include DSSim, Lily, MapPSO and RiMOM. Among the four systems, three of them -- DSSim, MapPSO, and RiMOM -- were used in the alignment task for all five domains in the English-English alignment(ontology 1 and ontology 2), and one of them, Lily, was used in the task for two domains, automobile and movie. The number of alignments found by the systems are shown in Table 1. As can be seen in this table, Lily finds more alignments than do the other systems. Conversely, MapPSO retrieves only a few alignments from the data set.

In order to learn the different biases of the systems, we counted the number of overlap alignments retrieved by the systems. The results are shown in Table 2. The letters D, L, M and R in the top row denote system names DSSim, Lily, MapPSO, and RiMOM, respectively. For example, the DR column is the number of alignments retrieved by both DSSim and RiMOM. We can see that both systems retrieve the same 82 alignments in the movie domain. In this table, we see interesting phenomena. Lily and RiMOM have the same bias for finding the alignments. For example, in the auto domain, 33% of the alignments found by Lily were also retrieved by RiMOM, and 46% of the alignments found by RiMOM were also retrieved by Lily. The same phenomenon is also seen in the movie domain. In contrast, MapPSO has a very different tendency. Although the system found 556 alignments in total, only one alignment was found by the other systems.

We also created a component bar chart for clarifying the sharing of retrieved alignments. In the automobile and movie domains, 80% of the alignments are found by only one system, and most of the other 20% are found by both Lily and RiMOM. From this graph, we can see that Lily has the same bias as RiMOM, but the system still found many alignments that the other systems did not find. For the remaining domains, outdoor, photo and software, the alignments found by only one system reached almost 100%.

Figure 1. Shared alignments.

Results for other alignment tasks

Unfortunately, the results of other alignment tasks such as English-Japanese alignments (ontology 1-3, ontology 1-4, ontology 2-3, and ontology 2-4), Japanese-Japanese alignments (ontology 3-4) were only submitted by RiMOM. The number of alignments by RiMOM are shown in Table 3.

Contacts

If you have any questions and comments, please feel free to contact Ryutaro Ichise ( )

Initial location of this page: http://ri-www.nii.ac.jp/OAEI/2008/result.html.