Reference objects don't combine well with “nullable”

[Rinzwind]

Perhaps more of a question than an issue report, but I am not sure how to combine “nullable” with a reference object; I have the impression they don't combine well?

In Swagger-UI issue 3325 I posted an example with the following schema. The intent is to express that both “dataFork” and “resourceFork” contain a Fork, except that “resourceFork” can also be null, but “dataFork” cannot be null. As was pointed out though, the schema is not in accordance with the OpenAPI v3.0.0 specification, due to the use of JSON Schema's {"type":"null"} which has not been adopted in OpenAPI.

...
"File": {
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "dataFork": { "$ref": "#/components/schemas/Fork" },
    "resourceFork": {
      "anyOf": [
        { "type": "null" },
        {"$ref": "#/components/schemas/Fork" } ] },
    ...

I considered I can instead express the example as follows, which as far as I can tell is at least in accordance with the OpenAPI specification? But the use of “anyOf” with only one schema inside seems clumsy, is there a better way to handle this? Essentially, a better way to combine “nullable” with the reference object?

... 
"File": {
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "dataFork": { "$ref": "#/components/schemas/Fork" },
    "resourceFork": {
      "nullable": true,
      "anyOf": [
        { "$ref": "#/components/schemas/Fork" } ] },
    ...

[handrews]

@Rinzwind I use allOf with one entry regularly (for one entry, the *Of's behave the same and allOf is the simplest). And I do it exactly as you show, with the $ref in the allOf and everything else outside of it.

This is how boolean flags like nullable, readOnly, writeOnly, etc. are intended to be used AFAIK (certainly in JSON Schema for readOnly and as of draft-07 writeOnly).

See also #1389 for nullable vs "type": "null" discussion.

[tedepstein]

About the example given in the original post: if we're interpreting nullable as described in #1389, then I don't think an object with resourceFork: null would be valid against the File schema:

    File:
      type: object
      properties:
        name:
          type: string
        dataFork:
          $ref: '#/components/schemas/Fork'
        resourceFork:
          nullable: true
          anyOf:
          - $ref: '#/components/schemas/Fork'
          
    Fork:
      type: object
      description: Information aspect of a file...

I added a 'Fork' stub schema to illustrate. Fork itself is not nullable, it requires an object value.

(And BTW, even if Fork didn't specify a type, it's not clear to me that null is allowed. Does anyone know whether schemas without a type and without an explicit nullable: true would permit null values?)

In the File schema, resourceFork has nullable: true. But in OpenAPI Schema Object, as in JSON Schema, constraints are cumulative and cannot be relaxed or overridden. The allOf expression means that all assertions in the referenced subschema apply here. Those inherited assertions, whether implicit or explicit, prohibit a null value. Adding a new, more permissive type assertion on top of that doesn't change anything.

So I think the first attempted schema in the original post is closer to the mark. But we have to do something kind of awkward to allow either null or Fork:

    File:
      type: object
      properties:
        name:
          type: string
        dataFork:
          $ref: '#/components/schemas/Fork'
        resourceFork:
          oneOf:
          - $ref: '#/components/schemas/Fork'
          - $ref: '#/components/schemas/NullValue'
          
    Fork:
      type: object
      description: Information aspect of a file...
      
    NullValue:
      # Allow null values.
      nullable: true
      # Disallow any non-null value.
      not:
        anyOf:
        - type: string
        - type: number
        - type: boolean
        - type: object
        - type: array
          # Array schema must specify items, so we'll use an 
          # empty object to mean arrays of any type are included.
          items: {}

It might be cleaner to start with a nullable Fork object, and restrict it to non-null as needed:

    File:
      type: object
      properties:
        name:
          type: string
        dataFork:
          allOf:
          - nullable: false
          - $ref: '#/components/schemas/Fork'
        resourceFork:
          $ref: '#/components/schemas/Fork'
          
    Fork:
      type: object
      nullable: true
      description: Information aspect of a file...

[Rinzwind]

But in OpenAPI Schema Object, as in JSON Schema, constraints are cumulative and cannot be relaxed or overridden.

I'm not sure that's right. The JSON schema validation specification says that “validation keywords typically operate independent of each other” (section 4.3) (*), but then makes some exceptions.

The OpenAPI Specification 3.0.2 isn't exactly clear on how it defines “nullable”, but I think it would have to be an exception too. Otherwise “nullable” would seem to always conflict with other keywords, in particular “type”. For example, consider the schema {"nullable":true,"type":"object"}. The instance null would not match this schema if it has to match each keyword independently, as it matches the “nullable” keyword, but not the “type” keyword (null is not an object). In order for this schema to have the intended semantics, I think “nullable” would have to be defined as overriding other keywords: if “nullable” has the value true and the instance is null, it overrides all other keywords. So null would match the schema because null matches “nullable”, and “type” does not apply. Similarly, null would also match the schema {"nullable":true,"anyOf":[{"type":"object","nullable":false}]} because the “anyOf” does not apply.

(*) Section 4.3 in the version of the JSON schema validation specification linked to from the OpenAPI specification 3.0.2. In newer versions, the sentence moved to section 3.1.1.

But we have to do something kind of awkward to allow either null or Fork: […]

I hadn't considered that, if your schema “NullValue” is correct and allowed by OpenAPI, that seems like a way of sneaking the easier to understand {"type":"null"} back in!

[handrews]

Hi @Rinzwind , I'm one of the current primary editors of the JSON Schema spec (and yes, it's my fault that draft-08 is taking forever to appear 😭 )

What "validation keywords typically operate independently" is that in the majority of cases, each keyword's validation outcome can be determined without examining any other keyword.

As of draft-06 and later, that is true of any assertion keyword, such as type, maxLength, or required. These keywords produce their own pass/fail validation results.

Some applicator keywords (which apply subschemas and combine their assertion results, rather than producing their own results) do interact. The most notorious being that determining to which properties additionalProperties applies depends on properties and patternProperties. The fact that that inter-keyword dependency causes so many misunderstandings is good evidence of why we try to minimize this behavior.

Sadly, in draft-04 (which is effectively what OAS uses- there's not really a draft-05, it's a long story), the exclusiveMaximum and exclusiveMinimum assertion keywords were not independent keywords, which also causes problems. That has since been changed.

OAS's nullable is another assertion keyword that is not independent. And that causes many problems when you write complex schemas with *Of and similar keywords. Which is why we removed assertions with that behavior, and have done other things to mitigate the problems seen in the applicator keywords.

That's probably more than you wanted to know, but the upshot is that nullable (if indeed it's default is false and applies when type is absent) violates two fundamental principles of JSON Schema keyword design:

• The empty schema MUST successfully validate any and all possible instances
• Assertion keywords SHOULD function independently; failure to follow this guideline produces behavior that is very confusing in complex schemas

[Rinzwind]

OAS's nullable is another assertion keyword that is not independent. […]
nullable […] violates two fundamental principles of JSON Schema keyword design

OK, but regardless of the violations of nullable with respect to those principles, the intent of the OpenAPI 3.0.2 specification was at least that nullableis not independent, and kind of “overrides” other keywords such that null is validated by all of the following schemas?

• { "nullable": true, "type": "object" }
• { "nullable": true, "anyOf": [ { "type": "object" } ] }
• { "nullable": true, "anyOf": [ { "type": "object", "nullable": false } ] }
• { "nullable": true, "anyOf": [ { "$ref": "#/components/schemas/Fork" } ] }

[tedepstein]

@Rinzwind,

There are discussions in #1389 and #1900 essentially trying to figure out how to interpret nullable.

OK, but regardless of the violations of nullable with respect to those principles, the intent of the OpenAPI 3.0.2 specification was at least that nullableis not independent, and kind of “overrides” other keywords such that null is validated by all of the following schemas?

That's how I assumed it worked. But then I dug deeper and now I'm much less certain of how the founding fathers intended nullable to work, or if they ever discussed use cases cases where nullable might conflict with enum, how it might behave in allOf inheritance hierarchies, etc.

We could say that nullable always takes precedence over other constraints, such that those other constraints only apply to non-null values. If that's what it means, we should document that, and we should expect that it's going to break some implementations that assumed otherwise.

I think we're still waiting to hear from the TSC about this one.