Add SKOS lookup function in Fix

[TobiasNx]

In the Destatis-Fächerklassifikation Vocab there are now english prefLabels and in order to add them with metamorph/fix we need to use different mapping files for each language in order to get the prefLabels we want like https://gitlab.com/oersi/oersi-etl/-/blob/master/data/maps/subject-labels.tsv
For an english version we would need an additional list, that would need to be cared about.

But since we have a ScoHub Vocabs/Skos-‘ttl‘-files it would be nice to use them as lookup so that we do not need to create and update additional lists.

For the lookup should ttl file should be the target: e.g.: https://github.com/dini-ag-kim/hochschulfaechersystematik/blob/master/hochschulfaechersystematik.ttl
(Other skos serialization could follow)

Nice would be something like the following with mock code:

@base <https://w3id.org/kim/hochschulfaechersystematik/> .
@prefix dct: <http://purl.org/dc/terms/> .
@prefix skos: <http://www.w3.org/2004/02/skos/core#> .
@prefix schema: <http://schema.org/> .
@prefix vann: <http://purl.org/vocab/vann/> .

...

<n4> a skos:Concept ;
  skos:prefLabel "Mathematik, Naturwissenschaften"@de, "Mathematics, Natural Sciences"@en ;
  skos:narrower   <n36>, <n37>, <n39>, <n40>, <n41>, <n42>, <n43>, <n44> ;
  skos:notation "4" ;
  skos:topConceptOf <scheme> .

...

Idea for Fix function:

skos_lookup("element-path" ,file="[path/url]", 
[match="attribute that should be matching", matchLanguage="language of the replaced value"], 
target="attribute to be replace with", targetLanguage="language of replacing value")

file= could be a URL or a local file,
match= is default id
match= and matchLanguage= are optional
target= and targetLanguage= are always needed

Use case 1:

Find matching subject and return object of targeted predicate.

in: https://w3id.org/kim/hochschulfaechersystematik/n4

skos_lookup("path", file="https://raw.githubusercontent.com/dini-ag-kim/hochschulfaechersystematik/master/hochschulfaechersystematik.ttl", target="prefLabel", targetLanguage="de")

out: Mathematik, Naturwissenschaften

Use case 2:

Find matching object value in selected predicate and return its subject.

in: Mathematics, Natural Sciences

skos_lookup("path", file="https://raw.githubusercontent.com/dini-ag-kim/hochschulfaechersystematik/master/hochschulfaechersystematik.ttl",match="prefLabel", matchLanguage="en", target="id")

out: https://w3id.org/kim/hochschulfaechersystematik/n4

Use case 3:

Find matching object value in selected predicate and return object of targeted and connected predicate.
This could be also interesting if we have SKOS files with hiddenLabels or altLabels.

in: Mathematics, Natural Sciences

skos_lookup("path", file="https://raw.githubusercontent.com/dini-ag-kim/hochschulfaechersystematik/master/hochschulfaechersystematik.ttl", match="prefLabel", matchLanguage="en", target="prefLabel", targetLanguage="de")

out: Mathematik, Naturwissenschaften

Code review: @fsteeg
Functional review: @TobiasNx @acka47

[acka47]

In today's meeting we decided to:

1. work on a concrete use case (from RPB, @acka47 will provide it)
2. focus on an implementation for Metafix
3. support different RDF serializations
4. take into account that we would have to address the considerations below at a later point without breaking the implementation

Further considerations:

• What about concept schemes that are not available as single files, where you have to traverse the graph from ConceptScheme over hasTopConcept and narrower to get all the data for lookup function? (E.g. SkoHub Vocabs doesn't require you to have the ConceptScheme in one file, you could also have one file per Concept.)
• For use cases I know from @sroertgen one would need to specify different fields (e.g. prefLabel, altLabel, hiddenLabel) to be taken into account for matching.

[acka47]

As required, here is my use case.

In RPB data we only have notations for RPB subject, e.g. #30 _sn584060_[/]#30a_sn584070_, see here.

I can create the correct concept URI with Fix, resulting in:

{
   "subject":[
      {
         "id":"http://purl.org/lobid/rpb#n584060",
         "label":"Platzhalter Schlagwortlabel",
         "type":[
            "Concept"
         ],
         "source":{
            "id":"http://purl.org/lobid/rpb",
            "label":"Systematik der Rheinland-Pfälzischen Bibliographie"
         }
      },
      {
         "id":"http://purl.org/lobid/rpb#n584070",
         "label":"Platzhalter Schlagwortlabel",
         "type":[
            "Concept"
         ],
         "source":{
            "id":"http://purl.org/lobid/rpb",
            "label":"Systematik der Rheinland-Pfälzischen Bibliographie"
         }
      }
   ]
}

As you can see, for the label I added a generic "Platzhalter Schlagwortlabel" for now as I can not (yet) lookup labels in a SKOS file. I'd be happy to in the future do something like this in the fix:

add_field("label", lookup:"prefLabel@de", in:"http://purl.org/lobid/rpb", basedOn:"id", match:"${s}")

Where I basically specify what content should be added to the new "label" field by indicating:

• which string to add, here the skos:prefLabel of the matched resource with language code "de"
• the ConceptScheme to do the lookup in, here http://purl.org/lobid/rpb
• the source field from my data to do the lookup with, here id
• the field(s) to look for a match, here it is the subject URI in the RDF: ${s}

[fsteeg]

add_field("label", lookup:"prefLabel@de", in:"http://purl.org/lobid/rpb", basedOn:"id", match:"${s}")

In general, I think we should implement this like the existing lookup, so something like:

lookup("subject.label", "rpb.ttl", someOption: ..., anotherOption2: ...)

[blackwinter]

In general, I think we should implement this like the existing lookup, so something like:

lookup("subject.label", "rpb.ttl", someOption: ..., anotherOption2: ...)

Are you sure that lookup() should be overloaded? Catmandu has lookup_in_store(), so I'd suggest either modeling it as a store and implementing that function or just naming it accordingly (lookup_in_rdf()). Otherwise, rdf_lookup() might be an acceptable name.

[fsteeg]

My view would be that essentially, we want to support one additional file format, TTL, in addition to CSV and TSV.

Since we'd probably implement this based on an RDF model anyway, we might as well support other SKOS RDF serializations (though I'm not even sure I like that idea, I'd prefer to stick to actual use cases, and we use TTL files). But a generic RDF lookup would be quite a different thing. For that, something like lookup_in_store (and then using something like a triple store) might make more sense.

[blackwinter]

My view would be that essentially, we want to support one additional file format, TTL, in addition to CSV and TSV.

In principle, yes, but lookup() is specifically meant for dictionaries. And an RDF file, whatever its serialization, is conceptually quite different from a simple delimited file with key-value pairs.

But I'm unsure myself. I just think we might regret it if we overwhelmed lookup() with too many features.

[Come to think of it, maybe we shouldn't even have added local maps to it. lookup_in_map() or lookup_in_store(..., Memory) might have been more appropriate.]

[dr0i]

I agree to @blackwinter - while it's in principle possible to make a Map out of RDF files, it may get complicated. And since there are Further considerations it may be better to go with an RDF store from the beginning.
As I am not a fan of external databases (brings complexity) and our scenarios make use of only little data I would start with an in-memory RDF store/model.