Ontology Alignment Evaluation Initiative - OAEI-2011 CampaignOAEI

Ontology Alignment Evaluation Initiative

Benchmark final results

This year 16 systems participated in the benchmark track. We have generated three different benchmarks against which matchers have been evaluated: benchmark (biblio), benchmark2 (ekaw) and benchmark3 (finance). All the details on these datasets and their results can be found at OAEI 2011 final paper. In the following, we summarise the evaluation results, both in terms of runtime and compliance in relation to reference alignments.

Runtime

We used a 3GHz Xeon 5472 (4 cores) machine running Linux Fedora 8 with 8GB RAM. This is a preliminary setting for mainly testing the deployability of tools into the SEALS platform. Table 1 presents the runtime (in minutes) based on 3 runs, and the five best systems in terms of F-measure in each data set (top-5). ‘Error’ indicates that the tool could not run in the current setting or their final version has some packaging error; ‘T’ indicates that tool could not process the single 101 test in less than 2 hours; ‘x’ indicates that the tool breaks when parsing some ontologies. Results in bold face are based on only 1 run.

Table 1: Runtime on 2011 data sets.

Runtimes in Benchmark datasets

CODI is able to run under the platform on Windows, but has specific requirements not met yet on the Linux version that has been used. Hence, we are not able to evaluate CODI's runtime. This matcher has been executed in a different setting and hence we could not compare its runtime with other systems. Furthermore, we were not able to run the uploaded final versions of OMR and OACAS.

Compliance

Table 2 presents the average F-measure for 3 runs, for each data set. These three runs are not necessary: even if matchers exhibit non deterministic behavior on a test case basis, their average F-measure on the whole data set remains the same. This year, although most of the systems participating in 2010 have improved their algorithms, none of them could outperform ASMOV, the best system in the 2010 campaign. With respect to the original benchmark data set and the new generated one (original and biblio in Table 5), we could observe a 1-2% constant and negative variation in Fmeasure, for most of the systems (except CODI and MapEVO). Furthermore, most of the systems perform better with the bibliographic ontology than with ekaw (a variation of 5-15%). The exceptions are LDOA, LogMap and MapPSO, followed by MaasMatch and CIDER with relatively stable F-measures. Although we have not enough results for a fair comparison with finance, we could observe that CSA and MaasMatch are the most stable matchers (with less variation than the others), followed by Aroma, CIDER and AgreementMaker, respectively. Finally, the group of best systems in each data set remains relatively the same across the different seed ontologies. Disregarding finance, CSA, CODI and YAM++ are ahead as the best systems for all three data sets, with MapSSS (2 out of 3) and Agreement- Maker, Aroma and Lily (1 out of 3) as followers.

Table 2: Results obtained by participants on each data set (based on 94 tests), including the results from the participants in 2010, and the top-five F-measure (five better systems in each data set).

Runtimes in Benchmark datasets

Precision/recall graphs

Figure 1 shows the precision and recall graphs of this year. These results are only relevant for the results of participants who provide confidence measures different from 1 or 0. This graph has been drawn with only technical adaptation of the technique used in TREC. Moreover, due to lack of time, these graphs have been computed by averaging the graphs of each of the tests (instead to pure precision and recall).

These results and those presented in Table 2 single out the same group of systems, MapSSS, YAM++, CODI, CASA nad AgrMaker, which seem to perform these tests at the highest level of quality. Of these, MapSSS has slightly better results than the two others. So, this confirms the leadership that we observed on raw results.

Precision/recall graphs for benchmarks. The alignments generated by matchers are cut under a threshold necessary for achieving n% recall and the corresponding precision is computed. The numbers in the legend are the Mean Average Precision (MAP): the average precision for each correct retrieved correspondence. Systems for which these graphs are not meaningful (because they did not provide graded confidence values) are drawn in dashed lines

Full results

algo refalign edna AgrMaker aroma CSA CIDER CODI LDOA Lily LogMap MaasMtch MapEVO MapPSO MapSSS Optima YAM
test Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec.
101 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.96 0.96 0.25 0.15 0.77 0.97 1.00 1.00 1.00 1.00 1.00 0.98 0.99 1.00 0.99 0.89 1.00 1.00 1.00 1.00 1.00 1.00
201 1.00 1.00 0.01 0.01 1.00 0.08 1.00 0.04 0.56 0.45 1.00 0.02 0.96 0.52 0.02 0.01 1.00 0.03 1.00 0.00 1.00 0.00 0.04 0.01 0.65 0.63 0.98 0.55 1.00 0.00 1.00 0.00
201-2 1.00 1.00 0.81 0.81 0.98 0.92 0.95 0.82 0.98 0.94 0.91 0.90 1.00 0.84 0.72 0.80 0.92 0.92 1.00 0.77 1.00 0.75 0.98 0.82 0.93 0.85 1.00 0.96 0.60 0.84 1.00 0.96
201-4 1.00 1.00 0.62 0.62 0.98 0.75 0.92 0.68 0.96 0.88 0.91 0.81 0.95 0.74 0.63 0.62 0.83 0.75 0.98 0.58 1.00 0.51 0.89 0.63 0.96 0.84 1.00 0.88 0.57 0.66 0.98 0.92
201-6 1.00 1.00 0.43 0.43 0.95 0.57 0.89 0.53 0.86 0.75 0.86 0.72 0.90 0.60 0.49 0.42 0.87 0.70 0.98 0.42 1.00 0.30 0.80 0.44 0.79 0.66 0.99 0.83 0.60 0.47 0.96 0.83
201-8 1.00 1.00 0.25 0.25 1.00 0.34 0.95 0.35 0.75 0.58 0.80 0.37 0.91 0.56 0.33 0.25 0.95 0.38 1.00 0.25 1.00 0.15 0.66 0.25 0.77 0.75 1.00 0.78 0.65 0.25 1.00 0.35
202 1.00 1.00 0.01 0.01 1.00 0.08 1.00 0.00 0.54 0.45 1.00 0.00 0.86 0.45 0.02 0.01 1.00 0.00 1.00 0.00 1.00 0.00 0.03 0.01 0.65 0.64 1.00 0.57 1.00 0.00 1.00 0.00
202-2 1.00 1.00 0.81 0.81 0.98 0.91 0.93 0.81 0.98 0.94 0.92 0.91 0.99 0.83 0.73 0.80 0.92 0.92 0.99 0.77 1.00 0.74 0.97 0.81 0.97 0.87 1.00 0.95 0.65 0.84 1.00 0.95
202-4 1.00 1.00 0.62 0.62 0.98 0.74 0.91 0.67 0.96 0.89 0.89 0.78 0.98 0.75 0.63 0.62 0.85 0.77 0.98 0.58 1.00 0.50 0.93 0.62 0.90 0.77 0.99 0.86 0.59 0.66 0.96 0.91
202-6 1.00 1.00 0.43 0.43 0.97 0.57 0.93 0.53 0.85 0.75 0.87 0.71 0.90 0.60 0.48 0.42 0.88 0.69 1.00 0.42 1.00 0.30 0.67 0.42 0.83 0.71 1.00 0.77 0.60 0.47 0.96 0.83
202-8 1.00 1.00 0.25 0.25 1.00 0.33 0.95 0.33 0.73 0.62 0.82 0.35 0.88 0.54 0.33 0.25 0.95 0.34 1.00 0.25 1.00 0.15 0.62 0.25 0.79 0.73 1.00 0.71 0.53 0.25 1.00 0.35
221 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.96 0.96 1.00 1.00 0.77 0.97 0.98 0.97 1.00 1.00 1.00 0.84 0.74 0.33 0.99 0.93 1.00 1.00 0.66 1.00 1.00 1.00
222 1.00 1.00 0.92 1.00 1.00 1.00 0.97 0.97 0.99 0.99 0.94 0.94 1.00 1.00 0.75 0.97 0.94 0.94 1.00 0.88 1.00 0.94 0.82 0.42 0.96 0.81 1.00 1.00 0.63 1.00 1.00 1.00
223 1.00 1.00 1.00 1.00 1.00 1.00 0.97 0.96 1.00 1.00 0.98 0.98 1.00 1.00 0.77 0.97 0.96 0.96 1.00 1.00 1.00 0.98 0.02 0.01 0.96 0.94 1.00 1.00 0.53 1.00 1.00 1.00
224 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.96 0.96 1.00 1.00 0.77 0.97 0.96 0.96 1.00 1.00 1.00 0.98 0.99 1.00 0.99 0.89 1.00 1.00 0.63 1.00 1.00 1.00
225 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.96 0.96 1.00 1.00 0.77 0.97 0.96 0.96 1.00 1.00 1.00 0.98 1.00 1.00 1.00 0.87 1.00 1.00 0.62 1.00 1.00 1.00
228 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.72 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.99 1.00 1.00 0.61 1.00 1.00 1.00
232 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.96 0.96 1.00 1.00 0.77 0.97 0.98 0.97 1.00 1.00 1.00 0.84 0.77 0.34 0.99 0.98 1.00 1.00 0.62 1.00 1.00 1.00
233 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.72 1.00 1.00 0.99 1.00 1.00 1.00 0.79 0.21 0.04 0.97 0.86 1.00 1.00 0.61 1.00 1.00 1.00
236 1.00 1.00 0.88 1.00 0.88 1.00 0.88 1.00 0.88 1.00 0.88 1.00 0.88 1.00 0.63 1.00 0.88 1.00 0.88 1.00 0.88 1.00 0.88 1.00 0.88 0.98 0.88 1.00 0.48 1.00 0.88 1.00
237 1.00 1.00 0.92 1.00 1.00 1.00 0.97 0.97 0.99 0.99 0.92 0.92 1.00 1.00 0.75 0.97 0.94 0.94 1.00 0.88 1.00 0.94 0.72 0.40 0.96 0.70 1.00 1.00 0.63 1.00 1.00 1.00
238 1.00 1.00 1.00 1.00 1.00 1.00 0.97 0.96 1.00 1.00 0.96 0.96 1.00 1.00 0.77 0.97 1.00 1.00 1.00 1.00 1.00 0.98 0.00 0.00 0.97 0.95 1.00 1.00 0.62 1.00 1.00 1.00
239 1.00 1.00 0.88 1.00 1.00 1.00 0.95 0.95 0.97 0.97 1.00 1.00 1.00 1.00 0.69 1.00 0.98 0.98 1.00 1.00 1.00 0.97 0.40 0.09 0.95 0.91 1.00 1.00 0.63 1.00 1.00 1.00
240 1.00 1.00 0.88 1.00 0.88 1.00 0.86 0.97 0.88 1.00 0.88 1.00 0.88 1.00 0.63 1.00 0.88 1.00 0.88 1.00 0.88 1.00 0.00 0.00 0.82 0.94 0.88 1.00 0.55 1.00 0.88 1.00
241 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.72 1.00 1.00 0.99 1.00 1.00 1.00 0.79 0.21 0.05 0.99 0.92 1.00 1.00 0.61 1.00 1.00 1.00
246 1.00 1.00 0.88 1.00 1.00 1.00 0.95 0.95 0.97 0.97 1.00 1.00 1.00 1.00 0.69 1.00 0.98 0.98 1.00 1.00 1.00 0.97 0.39 0.11 0.98 0.95 1.00 1.00 0.63 1.00 1.00 1.00
247 1.00 1.00 0.88 1.00 0.88 1.00 0.86 0.97 0.88 1.00 0.88 1.00 0.88 1.00 0.63 1.00 0.88 1.00 0.88 1.00 0.88 1.00 0.00 0.00 0.84 0.92 0.88 1.00 0.50 1.00 0.88 1.00
248 1.00 1.00 0.00 0.00 1.00 0.03 1.00 0.00 0.36 0.09 1.00 0.00 1.00 0.00 0.02 0.01 1.00 0.00 1.00 0.00 1.00 0.00 0.00 0.00 0.08 0.08 0.89 0.15 1.00 0.00 1.00 0.00
248-2 1.00 1.00 0.81 0.81 0.98 0.84 0.92 0.75 1.00 0.84 0.97 0.86 1.00 0.81 0.73 0.80 0.96 0.84 1.00 0.77 1.00 0.65 0.79 0.28 0.89 0.88 1.00 0.86 0.67 0.84 0.99 0.88
248-4 1.00 1.00 0.62 0.62 0.97 0.66 0.97 0.61 0.99 0.69 0.95 0.67 1.00 0.62 0.64 0.62 0.91 0.63 1.00 0.59 1.00 0.48 0.51 0.21 0.66 0.63 1.00 0.70 0.63 0.66 0.98 0.76
248-6 1.00 1.00 0.43 0.43 0.94 0.48 0.92 0.42 0.93 0.53 0.95 0.56 1.00 0.43 0.49 0.42 0.89 0.47 1.00 0.42 1.00 0.29 0.48 0.15 0.50 0.49 0.98 0.58 0.53 0.47 0.94 0.62
248-8 1.00 1.00 0.25 0.25 1.00 0.26 0.96 0.25 0.87 0.37 0.94 0.30 1.00 0.25 0.33 0.25 0.86 0.24 1.00 0.25 1.00 0.15 0.29 0.08 0.32 0.31 1.00 0.42 0.52 0.25 1.00 0.33
249 1.00 1.00 0.01 0.01 1.00 0.08 1.00 0.00 0.52 0.43 1.00 0.00 0.86 0.46 0.02 0.01 1.00 0.00 1.00 0.00 1.00 0.00 0.04 0.01 0.69 0.68 0.96 0.51 1.00 0.00 1.00 0.00
249-2 1.00 1.00 0.81 0.81 0.98 0.91 0.93 0.81 0.98 0.94 0.94 0.92 1.00 0.84 0.73 0.80 0.94 0.93 0.99 0.77 1.00 0.74 0.97 0.81 0.99 0.79 1.00 0.97 0.60 0.84 0.98 0.94
249-4 1.00 1.00 0.62 0.62 0.98 0.74 0.91 0.67 0.95 0.87 0.91 0.80 0.94 0.73 0.65 0.62 0.85 0.77 1.00 0.58 1.00 0.50 0.93 0.64 0.91 0.80 1.00 0.89 0.61 0.66 0.96 0.91
249-6 1.00 1.00 0.43 0.43 0.97 0.57 0.93 0.53 0.85 0.75 0.87 0.72 0.89 0.59 0.51 0.43 0.90 0.72 0.98 0.42 1.00 0.30 0.80 0.42 0.77 0.68 0.99 0.77 0.60 0.47 0.93 0.81
249-8 1.00 1.00 0.25 0.25 1.00 0.33 0.95 0.33 0.72 0.57 0.79 0.35 0.91 0.56 0.33 0.25 0.94 0.32 1.00 0.25 1.00 0.15 0.57 0.25 0.75 0.65 1.00 0.72 0.62 0.25 1.00 0.35
250 1.00 1.00 0.03 0.03 1.00 0.00 1.00 0.00 0.49 0.45 1.00 0.00 0.81 0.36 0.00 0.00 1.00 0.00 1.00 0.00 1.00 0.00 0.00 0.00 0.64 0.64 0.10 0.01 1.00 0.00 1.00 0.00
250-2 1.00 1.00 0.81 0.81 1.00 0.88 0.94 0.82 0.95 0.95 0.92 0.90 1.00 0.82 0.67 0.81 0.94 0.92 1.00 0.81 1.00 0.77 0.98 0.81 0.97 0.86 1.00 0.82 0.56 0.81 1.00 0.95
250-4 1.00 1.00 0.62 0.62 1.00 0.71 0.87 0.66 0.92 0.90 0.86 0.78 0.96 0.71 0.56 0.62 0.89 0.78 1.00 0.62 1.00 0.51 0.94 0.64 0.93 0.90 0.96 0.62 0.53 0.62 1.00 0.70
250-6 1.00 1.00 0.42 0.42 1.00 0.55 0.93 0.52 0.81 0.74 0.77 0.60 0.89 0.56 0.42 0.42 0.87 0.64 1.00 0.42 1.00 0.32 0.79 0.42 0.73 0.71 0.97 0.44 0.57 0.42 1.00 0.47
250-8 1.00 1.00 0.23 0.23 1.00 0.33 0.93 0.34 0.71 0.47 0.67 0.30 0.92 0.45 0.26 0.23 1.00 0.32 1.00 0.23 1.00 0.15 0.63 0.26 0.74 0.70 0.74 0.23 0.47 0.23 1.00 0.29
251 1.00 1.00 0.00 0.00 1.00 0.08 1.00 0.00 0.28 0.20 1.00 0.00 0.77 0.28 0.02 0.01 1.00 0.00 1.00 0.00 1.00 0.00 0.10 0.03 0.15 0.14 0.50 0.04 1.00 0.00 1.00 0.00
251-2 1.00 1.00 0.75 0.81 0.98 0.86 0.94 0.80 0.97 0.91 0.92 0.88 0.96 0.85 0.71 0.80 0.95 0.89 1.00 0.70 1.00 0.72 0.68 0.35 0.87 0.81 1.00 0.88 0.64 0.81 1.00 0.92
251-4 1.00 1.00 0.58 0.64 0.97 0.70 0.94 0.68 0.87 0.76 0.88 0.73 0.89 0.72 0.63 0.64 0.86 0.70 0.98 0.55 1.00 0.51 0.61 0.28 0.77 0.70 0.99 0.73 0.63 0.64 0.98 0.85
251-6 1.00 1.00 0.40 0.43 0.92 0.49 0.94 0.52 0.80 0.62 0.75 0.57 0.89 0.58 0.48 0.42 0.91 0.53 1.00 0.37 1.00 0.30 0.48 0.21 0.53 0.49 0.98 0.57 0.64 0.43 0.93 0.72
251-8 1.00 1.00 0.24 0.26 1.00 0.33 0.91 0.31 0.84 0.44 0.82 0.32 0.83 0.46 0.32 0.26 0.95 0.36 1.00 0.25 1.00 0.18 0.41 0.11 0.34 0.33 0.98 0.45 0.66 0.26 1.00 0.33
252 1.00 1.00 0.00 0.00 1.00 0.07 1.00 0.00 0.17 0.11 1.00 0.00 0.60 0.31 0.01 0.01 1.00 0.00 1.00 0.00 1.00 0.00 0.00 0.00 0.06 0.06 0.87 0.45 1.00 0.00 1.00 0.00
252-2 1.00 1.00 0.81 0.81 1.00 0.81 0.93 0.81 0.92 0.89 0.90 0.89 0.99 0.83 0.69 0.80 0.90 0.89 0.99 0.77 1.00 0.74 0.02 0.01 0.81 0.78 0.98 0.94 0.64 0.84 0.94 0.91
252-4 1.00 1.00 0.62 0.62 0.95 0.69 0.90 0.66 0.85 0.77 0.86 0.76 0.94 0.73 0.61 0.62 0.83 0.71 0.98 0.58 1.00 0.50 0.02 0.01 0.63 0.60 0.97 0.84 0.62 0.66 0.92 0.85
252-6 1.00 1.00 0.43 0.43 0.92 0.51 0.88 0.50 0.77 0.60 0.79 0.65 0.83 0.56 0.48 0.43 0.79 0.57 0.98 0.42 1.00 0.30 0.00 0.00 0.47 0.46 0.93 0.76 0.61 0.47 0.81 0.65
252-8 1.00 1.00 0.25 0.25 0.91 0.29 0.86 0.30 0.69 0.42 0.76 0.32 0.83 0.51 0.30 0.25 0.95 0.33 1.00 0.25 1.00 0.15 0.00 0.00 0.25 0.25 0.90 0.65 0.57 0.25 1.00 0.33
253 1.00 1.00 0.01 0.01 1.00 0.03 1.00 0.00 0.39 0.10 1.00 0.00 1.00 0.00 0.02 0.01 1.00 0.00 1.00 0.00 1.00 0.00 0.07 0.02 0.10 0.10 0.88 0.13 1.00 0.00 1.00 0.00
253-2 1.00 1.00 0.81 0.81 0.98 0.84 0.92 0.75 1.00 0.84 0.97 0.86 1.00 0.81 0.73 0.80 0.94 0.82 1.00 0.77 1.00 0.65 0.66 0.27 0.86 0.84 1.00 0.87 0.62 0.84 0.99 0.88
253-4 1.00 1.00 0.62 0.62 0.97 0.66 0.97 0.61 0.99 0.69 0.95 0.67 1.00 0.62 0.65 0.62 0.88 0.61 1.00 0.59 1.00 0.48 0.54 0.21 0.74 0.74 1.00 0.73 0.55 0.66 0.96 0.76
253-6 1.00 1.00 0.43 0.43 0.94 0.48 0.92 0.42 0.93 0.53 0.95 0.56 1.00 0.43 0.48 0.43 0.86 0.45 0.98 0.42 1.00 0.29 0.59 0.16 0.51 0.51 0.98 0.58 0.60 0.47 0.93 0.61
253-8 1.00 1.00 0.25 0.25 1.00 0.26 0.96 0.25 0.87 0.37 0.94 0.30 1.00 0.25 0.33 0.25 0.86 0.24 0.96 0.25 1.00 0.15 0.36 0.09 0.34 0.33 0.98 0.42 0.54 0.25 1.00 0.33
254 1.00 1.00 0.00 0.00 1.00 0.00 1.00 0.00 0.00 0.00 1.00 0.00 1.00 0.00 0.00 0.00 1.00 0.00 1.00 0.00 1.00 0.00 0.05 0.01 0.10 0.10 1.00 0.00 1.00 0.00 1.00 0.00
254-2 1.00 1.00 0.81 0.81 1.00 0.81 0.91 0.73 1.00 0.81 0.98 0.79 1.00 0.81 0.67 0.81 1.00 0.79 1.00 0.81 1.00 0.64 0.18 0.04 0.76 0.75 1.00 0.81 0.61 0.81 1.00 0.81
254-4 1.00 1.00 0.62 0.62 1.00 0.62 0.95 0.58 1.00 0.62 0.98 0.60 1.00 0.62 0.56 0.62 0.95 0.58 1.00 0.62 1.00 0.48 0.13 0.03 0.63 0.59 1.00 0.62 0.63 0.62 1.00 0.62
254-6 1.00 1.00 0.42 0.42 1.00 0.44 0.90 0.37 1.00 0.42 0.97 0.41 1.00 0.42 0.42 0.42 1.00 0.42 1.00 0.42 1.00 0.30 0.00 0.00 0.52 0.49 1.00 0.42 0.46 0.42 1.00 0.42
254-8 1.00 1.00 0.23 0.23 1.00 0.25 0.94 0.22 1.00 0.23 0.94 0.22 1.00 0.23 0.26 0.23 1.00 0.22 1.00 0.23 1.00 0.15 0.00 0.00 0.22 0.22 1.00 0.23 0.41 0.23 1.00 0.23
257 1.00 1.00 0.00 0.00 1.00 0.00 1.00 0.00 0.51 0.48 1.00 0.00 0.84 0.37 0.00 0.00 1.00 0.00 1.00 0.00 1.00 0.00 0.05 0.01 0.60 0.59 0.00 0.00 1.00 0.00 1.00 0.00
257-2 1.00 1.00 0.81 0.81 1.00 0.88 0.94 0.82 0.97 0.97 0.94 0.93 1.00 0.82 0.67 0.81 0.94 0.92 1.00 0.81 1.00 0.77 0.97 0.78 0.99 0.92 1.00 0.81 0.62 0.81 1.00 0.96
257-4 1.00 1.00 0.62 0.62 1.00 0.71 0.87 0.66 0.90 0.88 0.88 0.78 1.00 0.74 0.55 0.62 0.95 0.82 1.00 0.62 1.00 0.51 0.90 0.60 0.94 0.88 0.96 0.63 0.58 0.62 1.00 0.70
257-6 1.00 1.00 0.42 0.42 1.00 0.55 0.93 0.52 0.81 0.74 0.72 0.58 0.93 0.59 0.42 0.42 0.96 0.68 1.00 0.42 1.00 0.32 0.67 0.41 0.83 0.82 0.97 0.42 0.56 0.42 1.00 0.47
257-8 1.00 1.00 0.23 0.23 1.00 0.33 0.93 0.34 0.73 0.51 0.67 0.30 0.94 0.47 0.27 0.23 1.00 0.32 1.00 0.23 1.00 0.15 0.65 0.23 0.68 0.66 0.75 0.25 0.46 0.23 1.00 0.29
258 1.00 1.00 0.01 0.01 1.00 0.08 1.00 0.00 0.28 0.20 1.00 0.00 0.74 0.27 0.02 0.01 1.00 0.00 1.00 0.00 1.00 0.00 0.06 0.01 0.10 0.10 0.09 0.01 1.00 0.00 1.00 0.00
258-2 1.00 1.00 0.75 0.81 0.98 0.86 0.94 0.80 0.97 0.91 0.90 0.86 0.95 0.84 0.72 0.80 0.92 0.87 1.00 0.70 1.00 0.72 0.75 0.37 0.92 0.82 1.00 0.88 0.64 0.81 1.00 0.92
258-4 1.00 1.00 0.58 0.64 0.97 0.70 0.94 0.68 0.87 0.77 0.89 0.74 0.88 0.72 0.63 0.64 0.90 0.74 1.00 0.55 1.00 0.51 0.75 0.28 0.78 0.64 0.99 0.73 0.63 0.64 1.00 0.87
258-6 1.00 1.00 0.40 0.43 0.96 0.52 0.94 0.52 0.80 0.61 0.82 0.61 0.87 0.57 0.49 0.42 0.91 0.53 1.00 0.37 1.00 0.30 0.49 0.21 0.48 0.47 0.98 0.61 0.64 0.43 0.96 0.74
258-8 1.00 1.00 0.24 0.26 1.00 0.33 0.91 0.31 0.84 0.44 0.79 0.32 0.80 0.44 0.32 0.26 0.95 0.36 1.00 0.25 1.00 0.18 0.35 0.11 0.36 0.35 0.98 0.43 0.66 0.26 1.00 0.33
259 1.00 1.00 0.02 0.02 1.00 0.06 1.00 0.00 0.21 0.14 1.00 0.00 0.68 0.32 0.01 0.01 1.00 0.00 1.00 0.00 1.00 0.00 0.00 0.00 0.05 0.05 0.86 0.42 1.00 0.00 1.00 0.00
259-2 1.00 1.00 0.81 0.81 1.00 0.81 0.93 0.81 0.92 0.89 0.90 0.89 1.00 0.84 0.70 0.80 0.92 0.92 0.99 0.77 1.00 0.74 0.02 0.01 0.83 0.81 0.99 0.94 0.62 0.84 0.94 0.91
259-4 1.00 1.00 0.62 0.62 0.95 0.69 0.90 0.66 0.85 0.77 0.85 0.75 0.93 0.72 0.61 0.62 0.82 0.70 0.98 0.58 1.00 0.50 0.02 0.01 0.61 0.60 0.98 0.85 0.60 0.66 0.93 0.85
259-6 1.00 1.00 0.43 0.43 0.92 0.51 0.88 0.50 0.75 0.58 0.74 0.61 0.83 0.56 0.48 0.43 0.82 0.60 0.98 0.42 1.00 0.30 0.00 0.00 0.47 0.46 0.93 0.73 0.62 0.47 0.87 0.69
259-8 1.00 1.00 0.25 0.25 0.91 0.29 0.86 0.30 0.69 0.42 0.77 0.32 0.83 0.51 0.31 0.25 0.95 0.35 1.00 0.25 1.00 0.15 0.03 0.01 0.26 0.25 0.90 0.59 0.57 0.25 1.00 0.33
260 1.00 1.00 0.01 0.02 1.00 0.00 1.00 0.00 0.10 0.08 1.00 0.00 0.83 0.23 0.00 0.00 1.00 0.00 1.00 0.00 1.00 0.00 0.00 0.00 0.24 0.22 0.00 0.00 1.00 0.00 1.00 0.00
260-2 1.00 1.00 0.71 0.81 1.00 0.86 0.95 0.81 0.95 0.91 0.92 0.89 0.91 0.83 0.63 0.81 0.97 0.92 1.00 0.81 1.00 0.77 0.29 0.11 0.84 0.83 0.98 0.81 0.63 0.81 0.97 0.89
260-4 1.00 1.00 0.56 0.64 1.00 0.70 0.91 0.67 0.81 0.75 0.86 0.77 0.81 0.67 0.54 0.64 0.94 0.78 1.00 0.64 1.00 0.53 0.29 0.09 0.69 0.67 0.95 0.64 0.62 0.64 1.00 0.69
260-6 1.00 1.00 0.37 0.42 0.94 0.48 0.94 0.50 0.75 0.56 0.74 0.53 0.87 0.52 0.39 0.42 0.94 0.53 1.00 0.42 1.00 0.31 0.25 0.05 0.51 0.50 0.84 0.42 0.61 0.42 1.00 0.47
260-8 1.00 1.00 0.22 0.25 1.00 0.33 0.87 0.31 0.73 0.34 0.67 0.28 0.89 0.38 0.27 0.25 1.00 0.27 1.00 0.25 1.00 0.17 0.08 0.02 0.33 0.33 0.80 0.25 0.57 0.25 1.00 0.28
261 1.00 1.00 0.00 0.00 1.00 0.00 1.00 0.00 0.02 0.01 1.00 0.00 1.00 0.00 0.00 0.00 1.00 0.00 1.00 0.00 1.00 0.00 0.00 0.00 0.14 0.14 0.00 0.00 1.00 0.00 1.00 0.00
261-2 1.00 1.00 0.81 0.81 1.00 0.81 0.94 0.82 0.88 0.88 0.89 0.89 1.00 0.81 0.63 0.81 0.91 0.88 1.00 0.81 1.00 0.77 0.05 0.01 0.76 0.73 1.00 0.81 0.54 0.81 0.91 0.88
261-4 1.00 1.00 0.62 0.62 1.00 0.62 0.85 0.64 0.78 0.74 0.76 0.70 1.00 0.62 0.54 0.62 0.90 0.71 1.00 0.62 1.00 0.51 0.00 0.00 0.61 0.62 0.94 0.63 0.46 0.62 1.00 0.68
261-6 1.00 1.00 0.42 0.42 1.00 0.42 0.85 0.48 0.70 0.53 0.62 0.49 1.00 0.42 0.36 0.42 0.95 0.49 1.00 0.42 1.00 0.32 0.00 0.00 0.36 0.36 0.94 0.42 0.41 0.42 1.00 0.47
261-8 1.00 1.00 0.23 0.23 1.00 0.23 0.81 0.30 0.61 0.32 0.71 0.30 1.00 0.23 0.23 0.23 1.00 0.34 1.00 0.23 1.00 0.15 0.00 0.00 0.18 0.18 0.72 0.25 0.47 0.23 1.00 0.27
262 1.00 1.00 0.01 0.01 1.00 0.00 1.00 0.00 0.00 0.00 1.00 0.00 1.00 0.00 0.00 0.00 1.00 0.00 1.00 0.00 1.00 0.00 0.07 0.01 0.11 0.11 1.00 0.00 1.00 0.00 1.00 0.00
262-2 1.00 1.00 0.81 0.81 1.00 0.81 0.91 0.73 1.00 0.81 0.98 0.79 1.00 0.81 0.67 0.81 1.00 0.79 1.00 0.81 1.00 0.64 0.25 0.03 0.86 0.82 1.00 0.81 0.62 0.81 1.00 0.81
262-4 1.00 1.00 0.62 0.62 1.00 0.62 0.95 0.58 1.00 0.62 0.98 0.60 1.00 0.62 0.56 0.62 1.00 0.60 1.00 0.62 1.00 0.48 0.00 0.00 0.69 0.66 1.00 0.62 0.51 0.62 1.00 0.62
262-6 1.00 1.00 0.42 0.42 1.00 0.44 0.90 0.37 1.00 0.42 0.97 0.41 1.00 0.42 0.43 0.42 1.00 0.42 1.00 0.42 1.00 0.30 0.21 0.05 0.55 0.53 1.00 0.42 0.56 0.42 1.00 0.42
262-8 1.00 1.00 0.23 0.23 1.00 0.25 0.94 0.22 1.00 0.23 0.94 0.22 1.00 0.23 0.27 0.23 1.00 0.22 1.00 0.23 1.00 0.15 0.18 0.03 0.26 0.26 1.00 0.23 0.44 0.23 1.00 0.23
265 1.00 1.00 0.00 0.00 1.00 0.00 1.00 0.00 0.08 0.06 1.00 0.00 0.67 0.19 0.00 0.00 1.00 0.00 1.00 0.00 1.00 0.00 0.05 0.02 0.19 0.19 0.00 0.00 1.00 0.00 1.00 0.00
265-2 1.00 1.00 0.71 0.81 1.00 0.86 0.95 0.81 0.95 0.91 0.92 0.89 0.88 0.80 0.64 0.81 0.98 0.92 1.00 0.81 1.00 0.77 0.35 0.09 0.85 0.83 0.98 0.81 0.63 0.81 0.97 0.89
265-4 1.00 1.00 0.56 0.64 1.00 0.70 0.91 0.67 0.81 0.75 0.86 0.77 0.81 0.67 0.53 0.64 0.96 0.78 1.00 0.64 1.00 0.53 0.47 0.13 0.75 0.72 0.95 0.64 0.62 0.64 1.00 0.69
265-6 1.00 1.00 0.37 0.42 0.94 0.48 0.94 0.50 0.80 0.56 0.65 0.47 0.87 0.52 0.40 0.42 0.97 0.53 1.00 0.42 1.00 0.31 0.23 0.05 0.48 0.47 0.84 0.42 0.61 0.42 1.00 0.47
265-8 1.00 1.00 0.22 0.25 1.00 0.33 0.87 0.31 0.84 0.33 0.67 0.28 0.89 0.38 0.26 0.25 1.00 0.25 1.00 0.25 1.00 0.17 0.07 0.02 0.40 0.39 0.80 0.25 0.57 0.25 1.00 0.28
266 1.00 1.00 0.00 0.00 1.00 0.00 1.00 0.00 0.02 0.01 1.00 0.00 1.00 0.00 0.00 0.00 1.00 0.00 1.00 0.00 1.00 0.00 0.00 0.00 0.09 0.08 0.00 0.00 1.00 0.00 1.00 0.00
266-2 1.00 1.00 0.81 0.81 1.00 0.81 0.94 0.82 0.88 0.88 0.84 0.84 1.00 0.81 0.64 0.81 0.90 0.86 1.00 0.81 1.00 0.77 0.00 0.00 0.77 0.75 1.00 0.81 0.63 0.81 0.91 0.88
266-4 1.00 1.00 0.62 0.62 1.00 0.62 0.85 0.64 0.78 0.74 0.79 0.74 1.00 0.62 0.53 0.62 0.90 0.71 1.00 0.62 1.00 0.51 0.00 0.00 0.52 0.52 0.94 0.63 0.60 0.62 1.00 0.68
266-6 1.00 1.00 0.42 0.42 1.00 0.42 0.85 0.48 0.68 0.52 0.77 0.60 1.00 0.42 0.36 0.42 0.95 0.52 1.00 0.42 1.00 0.32 0.00 0.00 0.36 0.36 0.94 0.42 0.57 0.42 1.00 0.47
266-8 1.00 1.00 0.23 0.23 1.00 0.23 0.81 0.30 0.61 0.32 0.64 0.29 1.00 0.23 0.22 0.23 1.00 0.36 1.00 0.23 1.00 0.15 0.00 0.00 0.28 0.29 0.90 0.25 0.39 0.23 1.00 0.27
H-mean1.00 1.00 0.50 0.52 0.98 0.56 0.93 0.53 0.82 0.65 0.89 0.58 0.93 0.60 0.51 0.51 0.93 0.57 0.99 0.50 0.99 0.44 0.54 0.22 0.64 0.61 0.96 0.64 0.60 0.53 0.97 0.60

$Id: benchmarks.html,v 1.3 2011/09/28 10:25:04 trojahnd Exp $