Results for Crosswalks Data Schema Matching

This track aims at evaluating the ability of systems to deal with the schema metadata matching task, in particular, with a subset of a collection of crosswalks from fifteen research data schemas to Schema.org [1].

This year a subset of the 16 metadata schemas aligned to schema.org has been considered. This subset involves: Data Catalogue Vocabulary (DCAT-v3), Data Catalogue Vocabulary - Application Profile (DCAT-AP), DataCity, Dublin Core (DC), ISO19115-1 schemas (ISO) and RIFCS.

Experimental setting and participation

We have conducted an open evaluation. The systems have been executed on a Ubuntu Linux machine configured with 32GB of RAM running under a Intel Core CPU 2.00GHz x8 processors. All measurements are based on a single run. 4 systems have registered to participate in the track: AMD, Matcha, LogMap, LogMapLite and Matcha.

Evaluation results

Table below shows the results for the systems that have generated correspondences. AMD was not able to generate any one. LogMapLite and Matcha are the only systems ableto generated (few) correct correspondences. These correspondences involve properties where labels are equal, for instance: https://schema.org/distribution and http://www.w3.org/ns/dcat#distribution. With respect to the pairs, both systems generated a higher number of correspondences for DCAT-v3 and RIFCS. LogMapLite is the system that is able to deal with a higher number of matching pairs. Systems output empty results for DC.

Matcha
	correct	output	expected
dcat3	3	17	42
datacity	0	4	34

LogMap
	correct	output	expected
dcat3	0	12	42
datacity	0	3	34
rifcs	0	11	24
dcat-ap	0	2	34

LogMapLite
	correct	output	expected
dcat3	3	41	42
datacity	0	4	34
rifcs	0	9	24
dcat-ap	0	4	34
iso	0	2	42

Conclusions

This task mostly deal with properties of metadata schemas. This year, we have used the schemas for which an RDF serialization is available. A first future improvement is to provide an OWL serialisation and/or provide a task dedicate do those specific types of format.

Contacts

This track is organized by

Chantelle Verhey, WDS-ITO
Mingfang Wu, Australian Research Data Commons
Cassia Trojahn, IRIT, France

References

[1] Wu, M., Hagan, P., Cecconi, B., Richard, S. M., Verhey, C., & RDA Research Metadata Schemas WG. (2022). A Collection of Crosswalks from Fifteen Research Data Schemas to Schema.org. Research Data Alliance. https://doi.org/10.15497/RDA00069