Deep validation... New release?

[mwmitchell]

Just curious about when there might be a new release? The "deep" validation feature would be super handy to have. Thanks!

[apomelov]

Facing the same issue. When I'm using a simple schema, its validation generates a very nice report. But when I'm using $ref, the validator does not validate the referenced schema. And if it contains errors, document validation will fail with schema fatal error without any report.

[fge]

Uhm, not sure by what you mean with deep validation here?

Do you mean:

• deep syntax validation,
• validation of instance children even if the container fails,
• something else?

[apomelov]

I need to validate schema syntax including all schemas, referenced from it.
E.g. I have two schemas:
item.schema.json:

{
  "properties": { "id": { "type": "string" } }
}

items.schema.json:

{
  "type": "array",
  "items": { "$ref": "classpath:/item.schema.json" }
}

When I'm validating the items.schema.json, the validation succeeded even if item.schema.json contains errors.
When I need to validate json document with items.schema.json, I start with schema validation (it is OK), then document validation (it fails with a fatal error and I can not find out why).
The only way for me is to traverse the schema myself, find all external references and validate that schemas also. To do this I need to deal with schema semantics, but I want it to be encapsulated in the validator library.

[fge]

@mwmitchell can you precise what you mean by deep validation?

@apomelov can you please provide the error? Also, what version are you using?

[apomelov]

Just switched from 2.1.7 to 2.1.8.
Here is the initializing code that I use to resolve classpath references:

URIDownloader downloader = ResourceURIDownloader.getInstance();
LoadingConfiguration loadingConfiguration = LoadingConfiguration.newBuilder().addScheme("classpath", downloader).freeze();
JsonSchemaFactory factory = JsonSchemaFactory.newBuilder().setLoadingConfiguration(loadingConfiguration).freeze();
SyntaxValidator syntaxValidator = factory.getSyntaxValidator();

The error in the referenced schema must not break the wellformeness, just schema semantic. E.g.:

{
  "properties": { "id": { "type": "striiiing" } }
}

Syntax validation of the outer schema will be successful.
The document validation with the outer schema will fail with:

com.github.fge.jsonschema.core.exceptions.InvalidSchemaException: fatal: core.invalidSchema
    level: "fatal"

[fge]

OK, can you try this code:

ProcessingReport report;
try {
    report = syntaxValidator.validateSchema(yourSchema);
} catch (ProcessingException e) {
    System.out.println(e.getProcessingMessage());
}

and paste the output?

[apomelov]

Here is my current validation test:
exampleJson and schemaJson -- input strings for the test.

@Test
public void test() throws Exception {
    JsonNode exampleNode = null;
    try {
        exampleNode = isNotEmpty(exampleJson) ? reader.readTree(exampleJson) : null;
    } catch (JsonParseException e) {
        fail("Failed to parse example:\n " + e.getMessage());
    }

    JsonNode schemaNode = null;
    try {
        schemaNode = isNotEmpty(schemaJson) ? reader.readTree(schemaJson) : null;
    } catch (JsonParseException e) {
        fail("Failed to parse schema:\n " + e.getMessage());
    }

    if (schemaNode != null) {
        ProcessingReport validateSchema = validateSchema(schemaNode);
        System.out.println("report: " + validateSchema);
        assertThat("Schema validation error:\n" + validateSchema, validateSchema.isSuccess());
        if (exampleNode != null) {
            ProcessingReport validateData = validateJson(exampleNode, schemaNode);
            if (!validateData.isSuccess()) fail(formatReport(validateData, mapper));
        }
    }
}

I've just added sysout for schema validation report. It always prints:

report: com.github.fge.jsonschema.core.report.ListProcessingReport: success

But the test fails on validateData phase.

[fge]

Hmm OK, so, here is what happens and, more importantly, why it happens:

• your original schema is valid;
• the referenced schema is not.

Why: syntax validation does NOT process JSON References: it merely checks that if a $ref is present in the schema, its value is a valid JSON Reference (a URI, basically). This is enough, by the spec, to ensure that a schema is valid ;)

The error here occurs during data validation because during data validation, the references which need to be resolved are resolved -- and the result of the resolution is checked for syntax.

The solution here would be to add an option to the validation process so that references be processed, and the schemas between these references be checked.

Note however that this has quite a lot of implications:

• at two distincts instants in time t1 and t2, a referenced schema may change;
• at two distincts instants in time t1 and t2, a referenced schema may, or may not, be available;
• a valid JSON Reference may not point anywhere!

Still, I understand the need. Right now the API does not provide this.

This is therefore a feature request, and I will flag it as such.

In the meanwhile, as a workaround, you can check the syntax of all schemas individually. Schemas under your control, that is. Maybe add a dedicated test?

[apomelov]

I understand the reasons of the test failure and the implications. Let me completely describe my use case.
I'm designing some rest-api. And one artifact is a machine-readable specification. I'm using the RAML format for it. It also contains examples and/or schemas for request and response bodies. My test for this artifact checks the RAML spec for wellformeness, checks syntax of all given examples and schemas. If some rest resource contains both of example and schema, test also validates the given example against that schema. Some schemas a la "baseTypes" do not appear in spec explicitly. In fact all my schemas are local and static, so "deep schema syntax validation" prior to example validation against that schema is suitable for me.

[fge]

Closing this one, not doable for 2.2.

But it WILL be in 3.0.