Rethinking datasets and graphs?

[iherman]

I decided to start with some controversy:-)

In short, I have always been confused by the ways Datasets are treated in JSON-LD, and I propose to re-open that can of worms. I have jotted down my idea in a separate wiki page (it would have been too long for an issue).

TL;DR: My proposal is to start from scratch, ie, deprecating @graph and replacing the functionalities with something cleaner. See the wiki page...

[iherman]

I realize we have an issue with backward compatibility. I would therefore propose that we declare@graph as deprecated, but not removed. Ie, the old features remain valid, and we use a new keyword instead (@dataset). This means that, alas!, 1.1 implementations should implement @graph, too, though we should not include, imho, the graph container feature (we will have to see what to replace it with if necessary).

[gkellogg]

I updated the wiki with my thoughts. I think we should continue to use @graph, and we can adapt for some of the issues you mention.

The graph container feature is required for Verifiable Credentials, and I suspect that @msporny and @dlongley would object to it's being removed.

Creating a top-level dataset using a map structure could be accomplished by leveraging the existing container semantics as reproduced below:

{
  "@context": {
    ...
    "dataset": {"@id": "@graph", "@container": ["@graph", "@id"]}
  },
  "dataset" : {
    "URL1" : {
        Some RDF statements here
    },
    "URL2" : [
        {
            We could also define a bush just like above
        },
        {

        }
    ],
    "@none" : [{
        Default graph statements here
    }]
  }
}

This just says that the use of the "dataset" term treats it like @graph, except to use the "@container": ["@graph", "@id"] mechanism to define a graph map. This avoids needing to introduce values in the default graph that reference the named graph identifiers.

[msporny]

TL;DR: My proposal is to start from scratch, ie, deprecating @graph and replacing the functionalities with something cleaner.

What is the problem or documented author issue we are attempting to solve here? (You will see that I will repeat this question for every new feature/deprecation proposed for JSON-LD 1.1). :)

@graph is something that was designed to be used in @contexts... now, some may be using it in JSON-LD markup, which is okay... but I hesitate to say that it's a best practice. We had originally designed @graph to hold data that is digitally signed and needed to exist in a separate graph from the signature. So, @graph was primarily designed so we can digitally sign information (and the most natural way to do that is to use datasets). @graph wasn't something that most developers/authors would be exposed to.

I get the conceptual purity argument, but I haven't seen folks complaining about @graph. I fully admit that we may not have been exposed to those authors/developers... but again, I'd like to see them writing about this issue rather than deprecating a JSON-LD feature before seeing that data.

also,

I decided to start with some controversy:-)

😆 -- nice to see that your sense of humor hasn't changed, @iherman.

[msporny]

in a format [JSON-LD] that is, at the end of the day, the serialization of RDF

I know I'm sounding like a broken record at this point, but JSON-LD is not primarily a serialization of RDF. It's a graph-based syntax that just so happens to losslessly convert to and from RDF. I think people think I'm kidding when I say this, I'm only half-kidding... JSON-LD started off by attempting to create a new graph model syntax that Web developers would use... RDF compatibility was not as important to our organization as it was to the existing RDF community and it continues to not be a primary goal.

[iherman]

@gkellogg not 100%...

I tested in the dev. version of playground:

{
  "@context": {
    "@version": 1.1,
    "dataset": {"@id": "@graph", "@container": ["@graph", "@id"]}
  },
  "dataset" : {
    "http://www.ex.org/1" : {
        "@id" : "http://www.ivan-herman.net",
        "http://a.b.c/1" : "Ivan Herman"
    },
    "http://www.ex.org/2" : [
        {
         "@id" : "http://p.q.r",
         "http://a.b.c/2" : "Somebody else"
        },
        {
         "@id" : "http://x.w.z",
         "http://a.b.c/3" : "And somebody else again"
        }
    ],
    "@none" : [{
        "@id" : "http://www.w3.org",
        "http://a.b.c/4" : "Nobody"
    }]
  }
}

and what I got was:

<http://p.q.r> <http://a.b.c/2> "Somebody else" <http://www.ex.org/2> .
<http://www.ivan-herman.net> <http://a.b.c/1> "Ivan Herman" <http://www.ex.org/1> .
<http://x.w.z> <http://a.b.c/3> "And somebody else again" <http://www.ex.org/2> .
<http://www.w3.org> <http://a.b.c/4> "Nobody" _:b0 .

Note the last line: I did not get statements in the default graph, but in yet another graph with a blank node as an id...

But yes, it is pretty close.

[iherman]

@msporny

What is the problem or documented author issue we are attempting to solve here?

If I am the only one who is constantly confused on how to use the @graph term then I will of course shut up, and assign it to my own deficiencies. I would have no problem accepting that. But I do believe that the usage of @graph is confusing. As I tried to show that it confuses terms, imposes restrictions (like the usage of blank nodes in graph containers which generate RDF that would be unusable in SPARQL). Its current usage of representing bushes in JSON-LD is confusing, and it is not obvious (only via a conceptually complex trick) to represent elementary datasets. If the reader cannot gain a clear mental model of what is happening, then the only way of encoding data would be to make copy-paste from the examples without really understanding them, which is a problem (in my view).

About the role of JSON-LD: history is what it is, but that is now bygone. JSON-LD has been "marketed", and I daresay extremely successfully so, as an RDF serialization format, too. This is what it has become today and used by various communities. We have to take this connection seriously and try to improve the purity of the relationship. Ie, if our syntax leads to a confusion of RDF Graphs and RDF Datasets I do see that as a problem.

[gkellogg]

No, it’s not implemented in the spec just now, but would be a logical thing to do. Similar to adding ‘@containeron@type` which is also considered elsewhere.

[iherman]

@gkellogg

The graph container feature is required for Verifiable Credentials, and I suspect that @msporny and @dlongley would object to it's being removed.

As I said, I did not thought through how to include graph container feature, I was not saying that the feature itself should be removed. Just its current syntax.

[iherman]

@gkellogg

what about the separate proposal of represent bushes via a simple cross reference to contexts?

[ericprud]

I understand Manu's (provocative) point about about JSON-LD being a graph language first and RDF-compatible second. I believe that the @container: @graph construct doesn't behave as one would expect in a graph language, i.e. that a property points at an non-rooted graph. For instance:

{ "@context": {
    "@version": 1.1,
    "p1": {
      "@id": "http://vocab.ex/p1",
      "@container": "@graph"
    },
    "p2": { "@id": "http://vocab.ex/p2" },
    "p3": { "@id": "http://vocab.ex/p3" },
    "p4": { "@id": "http://vocab.ex/p4" }
  },
  "p1": {
    "p2": {
      "p3": "v3",
      "p4": "v4"
    }
  } }

emits the dataset:

_:b0 ex:p1 _:b1 .
GRAPH _:b1 {
  _:b2 ex:p2 _:b3  .
  _:b3 ex:p3 "v3"  .
  _:b3 ex:p4 "v4"  .
}

Navigating in JSON-land, <p1> is strongly-connected to the object with a <p2> property (_:b2, in RDF land). If I'm navigating this as a graph (RDF graph, property graph, Spark's variant of Cypher, etc), <p1> connects to a bag of triples. The application has to be working with a known schema and valid data to discover _:b2.

A solution that will be irksome to some and blindingly obvious to others is to give the subjects of the nested properties (i.e. <p2>) the same identity as that of the graph. In such a schema,

  "p1": {
    "p2": {
      "p3": "v3"
    },
    "p5": "v5"

would look like:

_:b0 ex:p1 _:b1 .
GRAPH _:b1 {
  _:b1 ex:p2 _:b2  .
  _:b2 ex:p3 "v3"  .
  _:b1 ex:p5 "v5"  .
}

This would eliminate a lot of fuzzy heuristics from query/update/validation.

[BigBlueHat]

@ericprud looks like some wee typos in your last examples (i.e. what happened to p5 and v5? and where did p4 come from?). Could you fix those? Thanks!

[BigBlueHat]

We have to take this connection seriously and try to improve the purity of the relationship.

While I do agree with @iherman about the importance of JSON-LD as a serialization of RDF, I'll also 👍 @msporny's statements that (even regardless of history), JSON-LD's appeal reaches farther than just "RDF-land."

It may be a tricky balance to when addressing situations like this, but it's clear that "confusion" is subjective.

I've no clear technical suggestions at this point, other than that we at least have more to clarify and exemplify (i.e. improve our examples) and would prefer we start there...and see what's still missing.

[gkellogg]

@iherman while adding referencable contexts is feasible, I think it’s a big step, and I don’t think it’s necessary.

@ericprud your thought about preserving the graph name as the implicit subject of triples in the referenced graph has merit, and does solve the nasty rooting problem.

@msporny we settled on using the RDF model as the basis for JSON-LD not just to appease the RDF 1.1 WG, but because it didn’t make sense to introduce yet another model. I think it’s important that the JSON-LD surface syntax remain usable by developers that don’t care, but we need to make sure the underpinnings have a good basis in theory. Perhaps we can use JSON-LD to push forward on some emerging areas of interest, such as property graph alignment via RDF*.