MultiFarm Results for OAEI 2013

In this page one can find the results of the OAEI 2013 campaign for the MultiFarm track. The details on this data set can be found at MultiFarm data set. If you notice any kind of error (wrong numbers, incorrect information on a matching system, etc.) do not hesitate to contact us (mail see below in the last paragraph on this page).

Generated alignments

You can download the complete set of all generated alignments. These alignments have been generated by executing the tools with the help of the SEALS infrastructure. All results presented in the following are based on analysing these alignments.

Experimental setting

For generating these results, we consider all pairs of languages (36 pairs), excluding the test cases with edas and ekaw ontologies (25 pairs). This results in 900 matching tasks. We can distinguish two types of matching tasks: (i) those test cases where two different ontologies have been translated in different languages (cmt--confOf, for instance); and (ii) those test cases where the same ontology has been translated in different languages (cmt--cmt, for instance). For the test cases of type (ii), good results are not directly related to the use of specific techniques for dealing with ontologies in different natural languages, but on the ability to exploit the fact that both ontologies have an identical structure (and that the reference alignment covers all entities described in the ontologies).

This year, 7 participants (out of 21) implement specific cross-lingual modules : CIDER-CL, MapSSS, RiMOM2013, StringsAuto, WeSeE, WikiMatch, and YAM++. We maintain the number of participants implementing specific modules for dealing with multilingualism as 2012 (ASE, AUTOMSv2, GOMMA, MEDLEY, WeSeE, WikiMatch, and YAM++) and count on 4 new participants (some of them, extensions of systems participating in previous campaigns). The other systems (AML, CroMatcher, HerTUDA, HotMatch, IAMA, LogMap, LogMapLite, MaasMatch, ODGOMS, OntoK, ServOMap, Synthesis, XMapGen, XMapSiG) are not specifically designed to match ontologies in different languages, nor do they make use of a component that can be utilised for that purpose.

Overall results

Before discussing the results per pairs of languages, we present the aggregated results for the test cases within each type of matching task -- types (i) and (ii) -- table below. These results are based on a single run. The systems not listed in this table have generated empty alignments for most test cases or have thrown some exceptions. The results have been generated using the Alignment API 4.5. For computing the aggregated results, we do not distinguish empty and erroneous alignments. The reader can refer to the OAEI paper for a more detailed discussion on these results.

MultiFarm aggregated results per matcher, for each type of matching task -- types (i) and (ii). Runtime is measured in minutes (time for completing all 900 matching tasks)
			Different ontologies (i)			Same ontologies (ii)
	System	Runtime	Precision	Fmeasure	Recall	Precision	Fmeasure	Recall
Specific cross-lingual matchers	CIDER-CL	110	.03	.03	.04	.18	.06	.04
	MapSSS	2380	.27	.10	.07	.50	.06	.03
	StringsAuto	24	.30	.14	.09	.51	.07	.04
	RiMOM2013	13	.52	.21	.13	.87	.14	.08
	WeSeE	2474	.22	.15	.12	.56	.16	.09
	WikiMatch	1284	.34	.27	.23	.65	.18	.11
	YAM++	443	.51	.40	.36	.91	.60	.50


Non-specific matchers	AML	7	.14	.04	.03	.35	.03	.01
	HerTUDA	46	.00	.01	1	.02	.03	1
	HotMatch	17	.00	.00	.00	.41	.04	.02
	IAMA	15	.15	.05	.03	.58	.04	.02
	LogMap	18	.18	.05	.03	.35	.03	.01
	LogMapLite	6	.13	.04	.02	.25	.02	.01
	MaasMatch	70	.01	.02	.03	.62	.29	.19
	ODGOMS	44	.26	.10	.06	.47	.05	.03
	OntoK	1602	.01	.01	.05	.16	.16	.15
	Synthesis	67	.30	.05	.03	.25	.04	.03

We observe significant differences between the results obtained for each type of matching task (specially in terms of precision). While most of the systems that implement specific cross-lingual techniques generate the best results for test cases of type (i) -- YAM++ (.40), WikiMatch (.27), RiMOM2013 (.21), WeSeE (.15), StringsAuto (.14), and MapSSS (.10) -- for the test cases of type (ii), systems no specifically designed for cross-lingual matching -- MaasMatch and OntoK -- are in the top (5) F-measure (together with YAM++, WikiMatch and WeSeE). With respect to CIDER-CL, this system in principle is able to deal with a sub set of languages, i.e., EN, ES, NL, and DE.

Overall (for both types i and ii), in terms of F-measure, most systems implementing specific cross-lingual methods outperform non-specific systems : YAM++ (.50), WikiMatch (.22), RiMOM (.17), WeSeE (.15) -- with MaasMatch given its high scores on cases (ii) -- and StringsAuto (.10).

Language specific results

Language specific results (type i)

Table below presents the results per pair of language, involving matching different ontologies (test cases of type i).

algo	AML			CIDER			HerTUDA			HotMatch			IAMA			LogMap			LogMapL			MaasMatch			MapSSS			ODGOMS			OntoK			RiMOM			StringsAuto			Synthesis			WeSeE			WikiMatch			YAM++
	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R
cn-cz	.00	NaN	.00	.00	.00	.00	.00	.01	1.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.01	.01	.04	.53	.12	.07	.00	NaN	.00	NaN	NaN	.00	.00	NaN	.00	.28	.12	.07	.43	.36	.31
cn-de	.00	NaN	.00	.00	.00	.02	.00	.01	1.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	.00	.01	.00	NaN	.00	.00	NaN	.00	.00	.01	.02	.69	.18	.10	.00	NaN	.00	NaN	NaN	.00	.00	NaN	.00	.26	.15	.11	.43	.37	.33
cn-en	.00	NaN	.00	.00	.01	.02	.00	.01	1.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.01	.01	.04	.42	.25	.18	.00	NaN	.00	NaN	NaN	.00	.00	NaN	.00	.26	.16	.11	.50	.43	.38
cn-es	.00	NaN	.00	.00	.00	.02	.00	.01	1.00	.01	.01	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	.00	.00	.00	NaN	.00	.00	NaN	.00	.01	.01	.04	.77	.17	.10	.00	NaN	.00	NaN	NaN	.00	.00	NaN	.00	.37	.24	.18	.39	.19	.13
cn-fr	.00	NaN	.00	.00	.01	.02	.00	.01	1.00	.03	.01	.01	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.01	.01	.01	.00	NaN	.00	.00	NaN	.00	.01	.01	.05	.79	.17	.09	.00	NaN	.00	NaN	NaN	.00	.03	.01	.01	.18	.10	.07	.50	.42	.37
cn-nl	.00	NaN	.00	.00	.00	.00	.00	.01	1.00	.01	.01	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	.00	.00	.00	NaN	.00	.00	NaN	.00	.01	.01	.04	.51	.16	.09	.00	NaN	.00	NaN	NaN	.00	.00	NaN	.00	.33	.19	.13	.37	.31	.28
cn-pt	.00	NaN	.00	.00	.01	.02	.00	.01	1.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.01	.01	.05	.56	.10	.06	.00	NaN	.00	NaN	NaN	.00	.00	NaN	.00	.30	.18	.13	.36	.32	.29
cn-ru	.00	NaN	.00	.00	.00	.01	.00	.01	1.00	.01	.01	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	.00	.00	.01	.01	.03	.00	NaN	.00	.00	NaN	.00	NaN	NaN	.00	.01	.01	.00	.43	.23	.16	.43	.34	.28
cz-de	.32	.10	.06	.00	.00	.02	.00	.01	1.00	.00	NaN	.00	.35	.10	.06	.36	.09	.05	.30	.09	.05	.01	.01	.02	.44	.14	.09	.43	.13	.07	.01	.01	.04	.71	.24	.15	.47	.22	.15	.88	.11	.06	.60	.02	.01	.36	.26	.21	.52	.48	.44
cz-en	.18	.04	.02	.00	.00	.01	.00	.01	1.00	.00	NaN	.00	.36	.12	.07	.25	.05	.03	.18	.04	.02	.02	.02	.04	.58	.25	.16	.57	.24	.15	.01	.01	.05	.69	.25	.15	.57	.29	.19	1.00	.06	.03	.14	.08	.05	.29	.18	.13	.59	.57	.56
cz-es	.37	.11	.07	.00	.00	.00	.00	.01	1.00	.00	NaN	.00	.38	.11	.07	.44	.11	.07	.36	.11	.07	.02	.02	.04	.49	.18	.11	.52	.18	.11	.02	.03	.10	.65	.24	.15	.51	.22	.14	.90	.14	.07	.35	.09	.05	.38	.29	.24	.56	.19	.12
cz-fr	.07	.01	.01	.00	.00	.01	.00	.01	1.00	.03	.01	.01	.07	.01	.01	.09	.01	.01	.07	.01	.01	.01	.01	.02	.45	.15	.09	.29	.07	.04	.01	.02	.06	.71	.17	.10	.45	.17	.10	1.00	.02	.01	.08	.04	.02	.32	.22	.17	.53	.52	.51
cz-nl	.15	.04	.02	.00	.00	.02	.00	.01	1.00	.00	NaN	.00	.21	.05	.03	.19	.04	.02	.15	.04	.02	.01	.01	.02	.35	.12	.07	.34	.12	.07	.02	.03	.09	.62	.32	.22	.44	.18	.11	1.00	.04	.02	.02	.01	.01	.33	.23	.18	.54	.53	.51
cz-pt	.39	.12	.07	.00	.01	.02	.00	.01	1.00	.00	NaN	.00	.42	.13	.08	.47	.13	.07	.40	.13	.08	.01	.01	.02	.54	.18	.11	.58	.20	.12	.01	.02	.07	.70	.24	.14	.56	.23	.14	1.00	.16	.09	.07	.04	.02	.32	.25	.20	.56	.56	.56
cz-ru	.00	NaN	.00	.00	.01	.03	.00	.01	1.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.01	.01	.04	NaN	NaN	.00	.00	NaN	.00	NaN	NaN	.00	.00	NaN	.00	.30	.25	.21	.52	.48	.45
de-en	.52	.20	.12	.18	.12	.09	.00	.01	1.00	.00	NaN	.00	.54	.21	.13	.60	.22	.13	.52	.20	.12	.04	.05	.09	.63	.28	.18	.65	.30	.19	.01	.01	.05	.85	.39	.26	.63	.33	.22	.94	.22	.12	.34	.33	.32	.37	.32	.28	.55	.50	.46
de-es	.26	.07	.04	.15	.15	.14	.00	.01	1.00	.00	NaN	.00	.26	.07	.04	.42	.09	.05	.22	.06	.03	.02	.02	.04	.63	.26	.16	.61	.24	.15	.01	.01	.05	.82	.31	.19	.64	.29	.19	.77	.08	.04	.30	.28	.26	.33	.29	.25	.56	.19	.12
de-fr	.19	.05	.03	.02	.01	.00	.00	.01	1.00	.00	NaN	.00	.15	.04	.02	.20	.04	.02	.15	.04	.02	.02	.02	.04	.31	.08	.04	.27	.06	.04	.01	.01	.04	.83	.29	.18	.55	.23	.14	1.00	.04	.02	.64	.32	.22	.31	.26	.23	.47	.44	.41
de-nl	.19	.05	.03	.17	.16	.15	.00	.01	1.00	.00	NaN	.00	.18	.04	.02	.21	.04	.02	.17	.04	.02	.03	.04	.07	.49	.17	.11	.55	.21	.13	.01	.01	.04	.71	.30	.19	.48	.19	.12	.86	.05	.02	.34	.31	.28	.35	.27	.22	.45	.40	.37
de-pt	.26	.07	.04	.00	NaN	.00	.00	.01	1.00	.00	NaN	.00	.32	.08	.05	.33	.07	.04	.26	.07	.04	.01	.02	.03	.46	.13	.08	.42	.11	.07	.01	.02	.07	.76	.27	.17	.51	.18	.11	1.00	.09	.05	.63	.34	.24	.32	.26	.22	.44	.41	.39
de-ru	.00	NaN	.00	.00	.00	.02	.00	.01	1.00	.01	.01	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.01	.01	.01	.00	NaN	.00	.00	NaN	.00	.01	.01	.04	NaN	NaN	.00	.00	NaN	.00	NaN	NaN	.00	.00	NaN	.00	.39	.31	.26	.52	.47	.43
en-es	.22	.06	.03	.15	.21	.36	.00	.01	1.00	.00	NaN	.00	.21	.05	.03	.50	.15	.09	.18	.04	.02	.03	.03	.06	.61	.33	.23	.65	.30	.20	.01	.01	.05	.77	.42	.28	.61	.37	.27	.64	.05	.03	.61	.46	.37	.40	.38	.37	.58	.23	.15
en-fr	.22	.06	.03	.00	NaN	.00	.00	.01	1.00	.01	.01	.00	.32	.10	.06	.28	.06	.04	.14	.04	.02	.03	.05	.08	.46	.17	.11	.43	.15	.09	.01	.01	.03	.86	.32	.20	.53	.29	.20	.63	.04	.02	.48	.32	.24	.38	.33	.29	.54	.50	.47
en-nl	.25	.07	.04	.18	.18	.18	.00	.01	1.00	.00	NaN	.00	.26	.07	.04	.28	.07	.04	.30	.10	.06	.03	.05	.08	.50	.22	.14	.60	.24	.15	.01	.02	.07	.64	.35	.24	.44	.24	.17	.83	.08	.04	.56	.40	.31	.38	.32	.28	.55	.52	.49
en-pt	.22	.06	.03	.00	NaN	.00	.00	.01	1.00	.00	NaN	.00	.26	.06	.04	.28	.06	.03	.24	.06	.04	.02	.02	.04	.54	.18	.11	.54	.19	.12	.01	.02	.07	.77	.36	.23	.58	.30	.20	1.00	.07	.04	.58	.41	.32	.41	.37	.34	.59	.57	.55
en-ru	.00	NaN	.00	.00	.00	.01	.00	.01	1.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	.00	.01	.00	NaN	.00	.00	NaN	.00	.00	.01	.02	.00	NaN	.00	.00	NaN	.00	NaN	NaN	.00	.00	NaN	.00	.28	.24	.22	.55	.50	.47
es-fr	.13	.03	.02	.00	NaN	.00	.00	.01	1.00	.00	NaN	.00	.24	.06	.03	.29	.06	.03	.07	.01	.01	.02	.03	.04	.50	.17	.10	.45	.14	.08	.01	.02	.06	.51	.36	.28	.62	.29	.19	1.00	.02	.01	.26	.25	.24	.37	.37	.37	.57	.20	.12
es-nl	.00	NaN	.00	.15	.18	.23	.00	.01	1.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.02	.02	.04	.22	.07	.04	.12	.03	.02	.01	.01	.05	.57	.29	.20	.34	.15	.09	.00	NaN	.00	.35	.34	.33	.35	.32	.30	.49	.16	.09
es-pt	.54	.29	.20	.00	NaN	.00	.00	.01	1.00	.00	NaN	.00	.50	.29	.21	.53	.24	.16	.47	.23	.15	.04	.05	.09	.53	.33	.24	.50	.33	.24	.02	.04	.15	.44	.40	.37	.50	.36	.28	.74	.25	.15	.31	.33	.35	.34	.37	.40	.59	.25	.16
es-ru	.00	NaN	.00	.00	.00	.00	.00	.01	1.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.01	.02	.06	NaN	NaN	.00	.00	NaN	.00	NaN	NaN	.00	.00	NaN	.00	.38	.35	.32	.55	.19	.11
fr-nl	.38	.12	.07	.01	.00	.00	.00	.01	1.00	.01	.01	.00	.39	.12	.07	.44	.13	.07	.38	.12	.07	.04	.06	.10	.46	.16	.09	.43	.13	.07	.02	.03	.11	.79	.30	.19	.59	.26	.17	.91	.15	.08	.29	.24	.20	.37	.34	.32	.48	.46	.44
fr-pt	.07	.01	.01	.00	NaN	.00	.00	.01	1.00	.01	.01	.00	.04	.01	.00	.00	NaN	.00	.00	NaN	.00	.03	.04	.07	.35	.10	.06	.39	.11	.07	NaN	NaN	.00	.47	.26	.18	.52	.19	.12	NaN	NaN	.00	.48	.34	.26	.29	.28	.27	.52	.53	.53
fr-ru	.00	NaN	.00	.00	.01	.03	.00	.01	1.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	.00	.00	.00	NaN	.00	.00	NaN	.00	.01	.01	.05	NaN	NaN	.00	.00	NaN	.00	NaN	NaN	.00	.00	NaN	.00	.35	.33	.30	.49	.46	.44
nl-pt	.07	.01	.01	.00	NaN	.00	.00	.01	1.00	.00	NaN	.00	.10	.02	.01	.09	.01	.01	.07	.01	.01	.01	.02	.03	.13	.03	.02	.14	.04	.02	.01	.01	.03	.51	.15	.09	.25	.07	.04	1.00	.02	.01	.32	.31	.29	.31	.27	.24	.52	.51	.51
nl-ru	.00	NaN	.00	.00	.00	.01	.00	.01	1.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.01	.01	.01	.00	NaN	.00	.00	NaN	.00	.01	.01	.05	NaN	NaN	.00	.00	NaN	.00	NaN	NaN	.00	.00	NaN	.00	.39	.33	.28	.47	.44	.41
pt-ru	.00	NaN	.00	.00	.00	.02	.00	.01	1.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	NaN	.00	.00	.00	.00	.00	NaN	.00	.00	NaN	.00	.01	.01	.05	NaN	NaN	.00	.00	NaN	.00	NaN	NaN	.00	.00	NaN	.00	.32	.29	.27	.50	.47	.45

n/a: result alignment not provided or not readable
NaN: division per zero, likely due to empty alignment.

Language specific results (type ii)

Table below presents the results per pair of language, involving matching the same ontologies (test cases of type ii).

algo	AML			CIDER			HerTUDA			HotMatch			IAMA			LogMap			LogMapL			MaasMatch			MapSSS			ODGOMS			OntoK			RiMOM			StringsAuto			Synthesis			WeSeE			WikiMatch			YAM++
	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R	P	F	R
cn-cz	.00	NaN	.00	.01	.01	.01	.02	.03	1.00	.36	.04	.02	.40	.01	.01	.00	NaN	.00	.00	NaN	.00	.56	.26	.17	.00	NaN	.00	.00	NaN	.00	.15	.15	.15	.78	.10	.05	.00	NaN	.00	.12	.01	.01	.33	.01	.01	.61	.09	.05	.88	.65	.52
cn-de	.00	NaN	.00	.02	.02	.02	.02	.03	1.00	.57	.06	.03	.40	.01	.01	.00	NaN	.00	.00	NaN	.00	.48	.22	.14	.00	NaN	.00	.00	NaN	.00	.12	.12	.12	.88	.13	.07	.00	NaN	.00	.12	.01	.01	.00	NaN	.00	.56	.10	.06	.94	.69	.55
cn-en	.00	NaN	.00	.03	.03	.03	.02	.03	1.00	.43	.04	.02	.40	.01	.01	.00	NaN	.00	.00	NaN	.00	.53	.25	.16	.00	NaN	.00	.00	NaN	.00	.15	.15	.15	.77	.41	.28	.00	NaN	.00	.12	.01	.01	.40	.04	.02	.69	.16	.09	.88	.67	.55
cn-es	.00	NaN	.00	.03	.03	.03	.02	.03	1.00	.39	.04	.02	.40	.01	.01	.00	NaN	.00	.00	NaN	.00	.58	.27	.17	.00	NaN	.00	.00	NaN	.00	.16	.16	.16	.88	.15	.08	.00	NaN	.00	.12	.01	.01	.14	.00	.00	.77	.18	.10	.88	.15	.08
cn-fr	.00	NaN	.00	.02	.02	.02	.02	.03	1.00	.46	.05	.02	.40	.01	.01	.00	NaN	.00	.00	NaN	.00	.52	.24	.15	.00	NaN	.00	.00	NaN	.00	.14	.14	.14	.89	.14	.08	.00	NaN	.00	.12	.01	.01	.46	.05	.02	.62	.11	.06	.92	.71	.58
cn-nl	.00	NaN	.00	.03	.03	.03	.02	.03	1.00	.39	.04	.02	.40	.01	.01	.00	NaN	.00	.00	NaN	.00	.48	.22	.15	.14	.00	.00	.00	NaN	.00	.16	.16	.16	.75	.17	.09	.14	.00	.00	.12	.01	.01	.21	.01	.01	.77	.16	.09	.90	.70	.57
cn-pt	.00	NaN	.00	.03	.03	.03	.02	.03	1.00	.43	.04	.02	.40	.01	.01	.00	NaN	.00	.00	NaN	.00	.60	.28	.18	.00	NaN	.00	.00	NaN	.00	.14	.14	.14	.77	.10	.06	.00	NaN	.00	.12	.01	.01	.00	NaN	.00	.68	.13	.07	.90	.70	.57
cn-ru	.29	.01	.01	.03	.03	.03	.02	.03	1.00	.36	.04	.02	.40	.01	.01	.00	NaN	.00	.00	NaN	.00	.57	.26	.17	.00	NaN	.00	.04	.04	.03	.17	.17	.17	.80	.01	.01	.00	NaN	.00	.12	.01	.01	.27	.04	.02	.73	.13	.07	.92	.68	.53
cz-de	.69	.06	.03	.03	.03	.03	.02	.03	1.00	.43	.04	.02	.77	.07	.04	.64	.05	.03	.52	.05	.02	.63	.29	.19	.77	.09	.05	.79	.08	.04	.14	.14	.15	.92	.12	.07	.78	.10	.05	.41	.08	.04	.90	.03	.02	.59	.13	.08	.91	.71	.58
cz-en	.67	.06	.03	.02	.02	.02	.02	.03	1.00	.39	.04	.02	.75	.07	.04	.55	.04	.02	.48	.04	.02	.66	.31	.20	.78	.11	.06	.83	.10	.06	.15	.15	.15	.85	.15	.08	.78	.11	.06	.34	.05	.03	.55	.09	.05	.62	.13	.07	.94	.77	.64
cz-es	.47	.03	.01	.03	.03	.03	.02	.03	1.00	.43	.04	.02	.63	.04	.02	.53	.03	.01	.33	.02	.01	.62	.29	.19	.70	.05	.03	.70	.05	.03	.18	.18	.18	1.00	.11	.06	.70	.05	.03	.34	.05	.03	.60	.04	.02	.59	.15	.08	.84	.11	.06
cz-fr	.33	.01	.01	.02	.02	.02	.02	.03	1.00	.46	.05	.02	.57	.03	.01	.30	.01	.01	.20	.01	.01	.61	.28	.18	.61	.04	.02	.65	.04	.02	.17	.17	.17	.96	.08	.04	.63	.04	.02	.23	.05	.03	.56	.08	.04	.61	.13	.07	.93	.77	.66
cz-nl	.47	.03	.01	.02	.02	.02	.02	.03	1.00	.46	.05	.02	.65	.05	.02	.53	.03	.02	.38	.03	.01	.66	.30	.20	.73	.07	.03	.74	.07	.04	.14	.14	.14	.94	.17	.09	.77	.08	.04	.29	.04	.02	.49	.07	.03	.59	.14	.08	.91	.72	.59
cz-pt	.47	.03	.01	.03	.03	.03	.02	.03	1.00	.36	.04	.02	.65	.04	.02	.53	.03	.01	.37	.02	.01	.66	.30	.20	.77	.07	.04	.71	.05	.03	.17	.17	.17	.93	.09	.05	.77	.07	.04	.30	.05	.03	.47	.07	.04	.57	.12	.07	.92	.73	.61
cz-ru	.00	NaN	.00	.02	.02	.02	.02	.03	1.00	.36	.04	.02	.40	.01	.01	.13	.00	.00	.00	NaN	.00	.46	.21	.14	.14	.00	.00	.00	NaN	.00	.15	.15	.15	1.00	.01	.00	.14	.00	.00	.19	.03	.02	.20	.00	.00	.53	.13	.07	.92	.70	.57
de-en	.75	.08	.04	.92	.15	.08	.02	.03	1.00	.36	.04	.02	.85	.12	.06	.74	.08	.04	.64	.07	.04	.75	.35	.22	.90	.17	.10	.88	.15	.08	.15	.15	.15	.96	.21	.12	.88	.17	.10	.43	.11	.06	.74	.35	.23	.70	.24	.15	.95	.80	.70
de-es	.43	.02	.01	.85	.12	.06	.02	.03	1.00	.39	.04	.02	.63	.04	.02	.50	.03	.01	.25	.01	.01	.61	.28	.18	.68	.05	.03	.74	.06	.03	.17	.17	.17	.97	.10	.05	.74	.07	.04	.30	.04	.02	.67	.27	.17	.64	.20	.12	.85	.11	.06
de-fr	.65	.05	.03	.00	NaN	.00	.02	.03	1.00	.39	.04	.02	.71	.05	.03	.47	.03	.01	.37	.02	.01	.63	.29	.19	.83	.10	.06	.77	.07	.04	.15	.15	.15	.95	.14	.07	.84	.11	.06	.23	.04	.02	.93	.25	.15	.65	.19	.11	.93	.76	.65
de-nl	.65	.06	.03	.73	.23	.14	.02	.03	1.00	.39	.04	.02	.73	.07	.03	.65	.06	.03	.54	.05	.03	.79	.36	.24	.73	.10	.06	.79	.11	.06	.13	.13	.13	.94	.21	.12	.75	.11	.06	.34	.07	.04	.73	.31	.20	.63	.19	.11	.90	.69	.56
de-pt	.47	.03	.01	.00	NaN	.00	.02	.03	1.00	.43	.04	.02	.68	.05	.02	.50	.03	.01	.33	.02	.01	.61	.28	.18	.74	.06	.03	.63	.04	.02	.16	.16	.16	.90	.09	.05	.77	.07	.04	.27	.04	.02	.93	.26	.15	.60	.17	.10	.91	.71	.58
de-ru	.00	NaN	.00	.02	.02	.02	.02	.03	1.00	.39	.04	.02	.40	.01	.01	.00	NaN	.00	.00	NaN	.00	.52	.24	.16	.14	.00	.00	.00	NaN	.00	.15	.15	.15	1.00	.01	.00	.14	.00	.00	.12	.01	.01	.50	.01	.00	.64	.18	.10	.92	.70	.56
en-es	.58	.04	.02	.80	.49	.35	.02	.03	1.00	.39	.04	.02	.67	.04	.02	.64	.05	.03	.33	.02	.01	.69	.32	.21	.87	.16	.09	.81	.09	.05	.14	.15	.15	.95	.20	.11	.88	.17	.09	1.00	.01	.01	.90	.31	.19	.78	.29	.17	.88	.15	.08
en-fr	.73	.08	.04	.00	NaN	.00	.02	.03	1.00	.36	.04	.02	.83	.10	.06	.70	.07	.03	.56	.05	.03	.77	.36	.23	.88	.19	.11	.88	.14	.08	.18	.18	.18	.97	.21	.12	.88	.20	.12	.39	.09	.05	.90	.33	.20	.77	.29	.18	.95	.83	.73
en-nl	.68	.07	.03	.86	.31	.19	.02	.03	1.00	.39	.04	.02	.77	.08	.04	.71	.08	.04	.63	.08	.04	.73	.34	.22	.83	.17	.10	.83	.11	.06	.17	.18	.18	.86	.23	.13	.82	.18	.10	.43	.10	.05	.92	.37	.23	.65	.23	.14	.91	.76	.65
en-pt	.60	.04	.02	.20	.01	.00	.02	.03	1.00	.36	.04	.02	.73	.06	.03	.67	.06	.03	.43	.03	.02	.74	.34	.22	.87	.17	.10	.80	.08	.04	.16	.16	.16	.90	.18	.10	.87	.18	.10	.31	.06	.03	.90	.32	.19	.68	.25	.16	.94	.77	.65
en-ru	.11	.00	.00	.03	.03	.03	.02	.03	1.00	.43	.04	.02	.50	.02	.01	.22	.01	.00	.20	.01	.01	.63	.29	.19	.25	.01	.00	.33	.01	.01	.15	.15	.15	1.00	.02	.01	.25	.01	.00	.12	.01	.01	.47	.06	.03	.64	.20	.12	.94	.74	.61
es-fr	.47	.03	.01	.10	.00	.00	.02	.03	1.00	.50	.05	.03	.68	.05	.02	.56	.03	.02	.25	.01	.01	.75	.35	.23	.80	.08	.04	.82	.09	.05	.18	.18	.18	.75	.23	.13	.84	.11	.06	1.00	.01	.00	.71	.31	.20	.71	.28	.17	.87	.13	.07
es-nl	.33	.01	.01	.76	.25	.15	.02	.03	1.00	.39	.04	.02	.57	.03	.01	.36	.01	.01	.25	.01	.01	.69	.32	.21	.75	.07	.04	.59	.04	.02	.16	.16	.16	.89	.14	.08	.71	.09	.05	.22	.03	.02	.77	.32	.20	.71	.24	.15	.86	.12	.07
es-pt	.89	.21	.12	.20	.01	.00	.02	.03	1.00	.43	.04	.02	.89	.19	.11	.80	.11	.06	.73	.12	.07	.85	.39	.26	.93	.25	.15	.93	.34	.21	.21	.21	.21	.88	.50	.35	.92	.27	.16	.55	.17	.10	.77	.36	.23	.73	.34	.22	.92	.22	.12
es-ru	.11	.00	.00	.03	.03	.03	.02	.03	1.00	.39	.04	.02	.45	.02	.01	.13	.00	.00	.08	.00	.00	.53	.25	.16	.25	.01	.00	.14	.00	.00	.16	.16	.16	1.00	.01	.01	.25	.01	.00	.12	.01	.01	.45	.05	.03	.61	.20	.12	.86	.13	.07
fr-nl	.58	.04	.02	.10	.00	.00	.02	.03	1.00	.36	.04	.02	.75	.06	.03	.62	.05	.02	.50	.04	.02	.62	.29	.19	.79	.08	.04	.73	.06	.03	.20	.20	.20	.86	.13	.07	.83	.10	.05	.37	.07	.04	.75	.29	.18	.69	.24	.15	.93	.79	.69
fr-pt	.38	.02	.01	.40	.01	.01	.02	.03	1.00	.39	.04	.02	.60	.03	.02	.36	.01	.01	.08	.00	.00	.73	.34	.22	.79	.08	.04	.72	.06	.03	.22	.06	.03	.78	.20	.12	.77	.08	.04	.19	.03	.02	.86	.24	.14	.60	.20	.12	.94	.82	.73
fr-ru	.00	NaN	.00	.02	.02	.02	.02	.03	1.00	.43	.04	.02	.40	.01	.01	.13	.00	.00	.00	NaN	.00	.55	.25	.16	.25	.01	.00	.00	NaN	.00	.15	.15	.15	1.00	.01	.01	.25	.01	.00	.19	.03	.02	.43	.05	.02	.63	.17	.10	.94	.77	.65
nl-pt	.50	.02	.01	.10	.00	.00	.02	.03	1.00	.50	.05	.03	.67	.04	.02	.50	.03	.01	.33	.02	.01	.62	.29	.19	.72	.06	.03	.61	.04	.02	.15	.15	.15	.75	.09	.05	.69	.06	.03	.28	.05	.03	.75	.33	.21	.59	.19	.12	.93	.79	.70
nl-ru	.11	.00	.00	.03	.03	.03	.02	.03	1.00	.39	.04	.02	.45	.02	.01	.13	.00	.00	.08	.00	.00	.53	.24	.16	.25	.01	.00	.14	.00	.00	.15	.15	.15	1.00	.01	.01	.25	.01	.00	.15	.01	.01	.43	.05	.03	.71	.19	.11	.93	.76	.64
pt-ru	.11	.00	.00	.02	.02	.02	.02	.03	1.00	.39	.04	.02	.45	.02	.01	.13	.00	.00	.08	.00	.00	.52	.24	.15	.14	.00	.00	.14	.00	.00	.15	.15	.15	1.00	.01	.00	.14	.00	.00	.15	.01	.01	.20	.01	.00	.56	.17	.10	.89	.74	.62

n/a: result alignment not provided or not readable
NaN: division per zero, likely due to empty alignment.

References

[1] Christian Meilicke, Raul Garcia-Castro, Fred Freitas, Willem Robert van Hage, Elena Montiel-Ponsoda, Ryan Ribeiro de Azevedo, Heiner Stuckenschmidt, Ondrej Svab-Zamazal, Vojtech Svatek, Andrei Tamilin, Cassia Trojahn, Shenghui Wang. MultiFarm: A Benchmark for Multilingual Ontology Matching. Accepted for publication at the Journal of Web Semantics.

An authors version of the paper can be found at the MultiFarm homepage, where the data set is described in details.

Contact

This track is organised by Cassia Trojahn dos Santos, with the help of Roger Granada in 2013. If you have any problems working with the ontologies, any questions or suggestions, feel free to write an email to cassia [.] trojahn [at] irit [.] fr.