Validating schema with external references

[dlarrick]

Greetings,
I am attempting to use openapi-schema-validator to validate a bare JSON object against a specific OpenAPI Model. This JSON is received by my application by GET from a remote API, similar to the case in python-openapi/openapi-core#154 .

However, the wrinkle is that the model I am trying to validate against has external refs. OpenAPI-Core itself (given a spec_url) is able to resolve these external refs, but I am struggling to find a method to validate bare models that doesn't involve making a fake request and path, as described in 154.

Is there a way to provide OpenAPI-Schema-Valdiator with a resolver that's capable of resolving external refs (filenames)?

Alternatively, is there a way to retrieve a fully-flattened model definition (with $refs expanded) from an existing OpenAPI spec? openapi_spec.get_schema('Modelname') returns the original model with refs in place.

Thanks in advance for any guidance.

[LourensVeen]

I am not exactly sure what you're trying to do, but I believe I have a similar problem and after a lot of digging through source code and documentation I've found a solution. Maybe this will help you or someone else who finds this issue.

I have an OpenAPI specification with an endpoint and a components/schemas section with various type definitions with cross-references between them. Here's a simplified example:

from jsonschema.validators import RefResolver
from openapi_schema_validator import OAS30Validator


schemas = {
        'A': {
            'type': 'array',
            'items': {
                '$ref': '#/components/schemas/B'
                }
            },
        'B': {
            'type': 'object',
            'required': ['i'],
            'properties': {
                'i': {
                    'type': 'integer'
                    }
                }
            }
        }

I can validate against the schema for B just fine, as it has no external references:

# Direct schema validation, no references
b = {'i': 42}
b_validator = OAS30Validator(schemas['B'])
b_validator.validate(b)

b2 = {'x': 42}
# b_validator.validate(b2)        # correctly rejects input

However, validating against A doesn't work:

# References give an error with a plain validator
a = [b]
a_validator = OAS30Validator(schemas['A'])
a_validator.validate(a)         # fails

because it cannot resolve the reference:

Traceback (most recent call last):
  File "/tmp/resolver_test/env/lib/python3.6/site-packages/jsonschema/validators.py", line 811, in resolve_fragment
    document = document[part]
KeyError: 'components'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "resolver_test.py", line 37, in <module>
    a_validator.validate(a)         # fails
  File "/tmp/resolver_test/env/lib/python3.6/site-packages/jsonschema/validators.py", line 352, in validate
    for error in self.iter_errors(*args, **kwargs):
  File "/tmp/resolver_test/env/lib/python3.6/site-packages/jsonschema/validators.py", line 328, in iter_errors
    for error in errors:
  File "/tmp/resolver_test/env/lib/python3.6/site-packages/openapi_schema_validator/_validators.py", line 29, in items
    for error in validator.descend(item, items, path=index):
  File "/tmp/resolver_test/env/lib/python3.6/site-packages/jsonschema/validators.py", line 344, in descend
    for error in self.iter_errors(instance, schema):
  File "/tmp/resolver_test/env/lib/python3.6/site-packages/jsonschema/validators.py", line 328, in iter_errors
    for error in errors:
  File "/tmp/resolver_test/env/lib/python3.6/site-packages/jsonschema/_validators.py", line 259, in ref
    scope, resolved = validator.resolver.resolve(ref)
  File "/tmp/resolver_test/env/lib/python3.6/site-packages/jsonschema/validators.py", line 766, in resolve
    return url, self._remote_cache(url)
  File "/tmp/resolver_test/env/lib/python3.6/site-packages/jsonschema/validators.py", line 781, in resolve_from_url
    return self.resolve_fragment(document, fragment)
  File "/tmp/resolver_test/env/lib/python3.6/site-packages/jsonschema/validators.py", line 814, in resolve_fragment
    "Unresolvable JSON pointer: %r" % fragment
jsonschema.exceptions.RefResolutionError: Unresolvable JSON pointer: 'components/schemas/B'

The OAS30Validator is a jsonschema validator, and jsonschema allows you to insert a custom resolver. In this particular case, jsonschema.validators.RefResolver does what we want out of the box:

# Using a custom resolver to resolve local references
# Note that the reference above has /components/schemas/B, so we need to
# provide those to make it resolve correctly.
rooted_schemas = {
        'components': {
            'schemas': schemas}}

ref_resolver = RefResolver.from_schema(rooted_schemas)
a_validator2 = OAS30Validator(schemas['A'], resolver=ref_resolver)
a_validator2.validate(a)

That solves my problem.

I was in pretty deep by now, so I thought I'd try to solve what I think may be your problem, in case it helps you or anyone else. You mention that your external reference consist of filenames. You can use those, but you have to specify a URL, and the file scheme only supports absolute URLs:

# A schema that references a schema in a file works out of the box, but
# it needs to have the file:// and an absolute path, or
# urllib.requests.urlopen() cannot find it.
schema2 = {
        'type': 'array',
        'items': {
            '$ref': 'file:///tmp/resolver_test/c.json#/C'
            }
        }

c = {'j': 43}
c_array_validator = OAS30Validator(schema2)
c_array_validator.validate([c])

For completeness, here is /tmp/resolver_test/c.json:

{
    "C": {
        "type": "object",
        "required": ["j"],
        "properties": {
            "j": {
                "type": "integer"
            }
        }
    }
}

If you do have plain file names, and cannot change the schema file, then it may be possible to do something with the store or handlers arguments to RefResolver. I think that if you pass a store mapping 'c.json' (in this example) to the contents of c.json then that may work, but you'd have to know and load the files in advance.

Alternatively, maybe passing a handler for the empty scheme '' could work as a more generic solution, then that handler could load files from whichever directory they're in. If you're still having this problem, then maybe you could post actual schema you're trying to match against? (Or a simplified example, if you can't post the real thing.)

[dlarrick]

Thanks. For other reasons, we wound up flattening our schema for runtime into one big file with Speccy. So we were able to find & use the models of interest that way. But your tips here are are quite informative regardless, so thanks again.

[harshsikhwal]

Thanks for this. Can you add this in README or somewhere convenient? This was hugely helpful!

[LourensVeen]

Well, I'm not a maintainer so I can't put it in myself, but you're all welcome to use the above as a whole or in part under the project's 3-clause BSD license.