Ontology Alignment Evaluation Initiative - MELT evaluation for OAEI 2021 Campaign

MELT evaluation for OAEI 2021

MELT is a powerful maven framework for developing, tuning, evaluating, and packaging ontology matching systems. Please consider citing MELT in your academic work.

Instructions for participants

• Create a Web-based package: this is the preferred option. MELT will automatically wrap a server around the matcher and put it all in a docker container. See documentation and an example matcher. For developers not using Java, MELT can also be used to facilitate the wrapping (see example python matcher here). Note that you still need Java and Maven installed to create the package (via command maven package).


• LogMap's Web-based wrapping (Java): logmap-melt (extending MactherURL or MatcherYAAAOwlApi)


• Create a SEALS-based package: to provide backwards compatibility and also participation in the Interactive track, SEALS-packages (especially from previous years) using MELT will also be accepted. See documentation here.


• Create a Web service: alternatively one can also provide the URL of a system running as a remote service. One should follow similar instructions as for a Web-based package. This option should be motivated by hardware and/or software requirements.

See other packaging examples here.

Matching process

Alignment systems in the OAEI are expected to accept as input two ontologies (or their URL reference) and should provide as output an alignment file in RDF Alignment format (or its URL reference). Options to produce the alignment in the expected OAEI format:

• Use the Alignment class from the MELT framework (see example packages).
• Rely on the Alignment API.
• Create your own script/class to generate the mappings as done in LogMap.

Evaluation instructions for participants and organisers

Participants are encouraged to evaluate their wrapped system before submission.

Evaluation instructions are available here. MELT can evaluate built-in matchers, matchers wrapped as a Web service (MatcherDockerFile), SEALS-based matchers (MatcherSeals), and matchers accessible via a running remote service (see details here). This is done using Java. See also the test class included in LogMap's Web-based wrapping.

A command line client is also available to facilitate the system evaluation for non-java developers.

MELT includes several built-in evaluation tracks: built-in datasets. One can also evaluate systems with local tracks or with just a pair of ontologies (see details here).

New organisers can perform the evaluation via a Local Track or contacting the platform chairs to create a new built-in track.