Currently Available Suites
The SEALS project is
dedicated to the evaluation of semantic web technologies. To that
extent, it is creating a platform for easing this evaluation,
organising evaluation campaigns, and building the community of tool
providers and tool users around this evaluation activity.
SEALS is also focussing on setting up
a repository that hosts useful testcases processable by
those tools that have implemented the required interfaces. A set
of associated testcases is called a testsuite, which can have several
versions. The following listing refers to those testsuites that have
already been tested and/or are intended to be used in one of the
following evaluation campaigns. We refers in most cases to the newest
stable version currently available.
How to use these testsuites is explained in detail here in the section "Tutorial on Tool Wrapping". In particular, it is explained how the IDs of the listed suites can be used as input to a client which allows to run locally evaluation experiments with these datasets.
Listing of Testsuites
- Benchmark1 - v2.0
- Repository: http://seals-test.sti2.at/tdrs-web/
- Suite-ID: a4e0a9c6-2128-4728-8abb-cd4151ed6479
- Version-ID: dcd9be96-a5a1-4943-a16b-dd467ca545f2
- Description: Classical OAEI Benchmark based on the bibliography ontology
- Link: http://oaei.ontologymatching.org/2011/benchmarks/
- Benchmark2 - 2011-v2
- Repository: http://seals-test.sti2.at/tdrs-web/
- Suite-ID: b50cbebb-5be3-4fc6-a02d-3a801791770c
- Version-ID: ebee3a34-a155-4d50-879f-33372822db4d
- Description: OAEI Benchmark based on the EKAW ontology from the conference dataset
- Link: http://oaei.ontologymatching.org/2011/benchmarks2/
- Conference - 2010
- Repository: http://seals.sti2.at/tdrs-web/(take care: this respository url differs from the others)
- Suite-ID: Conference+Testsuite
- Version-ID: 2010 (or use 2010complete, which includes thoses testcases without reference alignments)
- Description: Classical OAEI Benchmark based on the bibliography ontology
- Link: http://oaei.ontologymatching.org/2011/conference/
- Anatomy - Version 2011
- Repository: http://seals-test.sti2.at/tdrs-web/
- Suite-ID: ca1cddfe-c728-4207-b33d-ca245642f4c9 (there was a mistake in this ID in previous versions of this webpage)
- Version-ID: 712477b7-e1f9-4633-8d52-17d2b25af008
- Description: Dataset used in the previous years for the OAEI Anataomy track
- Link: http://oaei.ontologymatching.org/2011/anatomy/
- Multifarm - de-fr-multifarm (collection of approx 40 testsuites, this is an example for a specific one)
- Repository: http://seals-test.sti2.at/tdrs-web/
- Suite-ID: de-fr-multifarm
- Version-ID: de-fr-multifarm-v1.0
- Description: Comprehensive collection ontologies and reference mappinggs based on the translated Conference dataset, for a full listing of all combinations search in the SEALS respository and type "multifarm" in the name field.
- Link: http://oaei.ontologymatching.org/2011.5/multifarm/
- Large Biomedical Ontologies - v1.1.2011.5
- Repository: http://seals-test.sti2.at/tdrs-web/
- Suite-ID: cf0378d9-da30-4b58-b937-192028ed4961
- Version-ID: a9fe4a9b-54c0-4ecf-bdfe-f405cf72193d
- Description: Includes the biomedical ontologies FMA, NCI and SNOMED, plus reference mappings between them based on the UMLS Metathesaurus.
- Link: http://oaei.ontologymatching.org/2011.5/largeBioMed/
Do not get confused by the different naming styles used for IDs. In some cases IDs are randomly generated strings like "1dc20a...", sometimes they are words or abbreviations of words.
Acknowledgement
The testsuites listed above have been developed by different groups of researchers. We thank all those people who
wrapped their testsuites (or allowed us to wrap their testsuites) under a common format in the SEALS platform to further improve the quality of ontology matching evaluation.
