Ontology Alignment Evaluation Initiative - OAEI-2017 Campaign

Disease and Phenotype Track

OAEI 2017::Disease and Phenotype Track (phenotype)

NEWS:

• 2017 results available now!

General description

The 2017 edition is composed by four tasks requiring the pairwise alignment of:

1. Human Phenotype Ontology (HP) to Mammalian Phenotype Ontology (MP);
2. Human Disease Ontology (DOID) to the Orphanet Rare Disease Ontology (ORDO);
3. Human Phenotype Ontology (HP) to Medical Subject Headings (MESH); and
4. Human Phenotype Ontology (HP) to Online Mendelian Inheritance in Man (OMIM).

Currently, mappings between these ontologies are mostly curated by bioinformatics and disease experts who would benefit from automation of their workflows supported by implementation of ontology matching algorithms.

Table 1 summarizes the ontology versions and sources of the ontologies used in the OAEI 2017. Note that the version and source of HP, MP, DOID and ORDO are different from the ones used in 2016.

We have extracted "baseline" reference alignments based on the available BioPortal mappings (July 8, 2017). Most of the BioPortal mappings are generated automatically by the LOOM system; which should only be considered as a baseline since it is incomplete or may contain erroneous mappings.

The ontologies and baseline mappings for the OAEI 2017 campaign can be downloaded as zip files: Phenotype2017_Ontologies.zip [25Mb] and Phenotype2017_BaselineMappingsBioPortal.zip [0.6Mb]. The OAEI 2017 campaign datasets can also be accessed via the SEALS platform (see dataset identifiers below).

Evaluation process

• Precision and Recall with respect to a voted reference alignment that will be automatically built by merging/voting the outputs of the participating systems. We will use vote=2 and vote=3.
• Recall with respect to manually generated mappings.
• Manual assessment of a subset of the generated mappings, specially the ones that are not suggested by other systems.

Additionally, systems are also welcome to provide all pair of entities with some similarity. These pairs do not necessary need to represent equivalence or a narrower/broader correspondences between the entities, but entities that have some degree of similarity. System developers can submit these correspondences directly to ernesto [.] jimenez [.] ruiz [at] gmail [.] com (in OAEI-RDF Alignment format) or create a configuration flag in the SEALS package, e.g. flag "all_corresponences=true" outputs all pairs of entities with some similarity, while "all_corresponences=false" outputs only the correspondences to evaluate Precision and Recall against the reference alignment.

SEALS support

The evaluation of the Disease and Phenotype Track will be run with support of the SEALS infrastructure. This requires that you wrap your matching system in a way that allows us to execute it using the SEALS client (see OAEI 2017 evaluation details).

HP-MP matching problem

This task consists of matching the HP (31,034 classes) and MP (30,273 entities) ontologies. The BioPortal-based alignment contains 696 mappings.

DOID-ORDO matching problem

This task consists of matching the DOID (38,240 classes) and ORDO (13,504) ontologies. The BioPortal-based alignment contains 1,237 mappings.

HP-MESH matching problem

This task consists of matching the HP (31,034 classes) and MESH (305,349) ontologies. The BioPortal-based alignment contains 2,466 mappings.

HP-MESH matching problem

This task consists of matching the HP (31,034 classes) and OMIM (93,048) ontologies. The BioPortal-based alignment contains 3,768 mappings.