External reference to SecuritySchemes not loading documents

[dldl-cmd]

When reading an OpenApi document not all documents get loaded into the workspace. When a reference to a security scheme in another file exists this file does not get loaded into the workspace and there the reference cannot be resolved.

OpenApi File & Program example To Reproduce

File: ExampleOpenApi.yaml

openapi: '3.0.2'

info:
    title: Example API
    description: Example for OpenApiWalker reference problem
    version: 1.0.0

servers:
    - url: 'https://localhost'

paths:
    /example:
        get:
            summary: "Test"
            responses:
              '200':
                description: Example response
                content:
                    application/json:
                        schema:
                            $ref: CommonSchemas.yaml#/components/schemas/ExampleSchema

security:
    - TestSecurityScheme: []

components:
    securitySchemes:
        TestSecurityScheme:
            $ref: CommonSecuritySchemas.yaml#/components/securitySchemes/TestSecurityScheme

File: CommonSchemas.yaml

openapi: '3.0.2'

info:
    title: Common definitions
    description: Common definitions for OpenApiWalker reference problem
    version: 1.0.0

paths:

components:
    schemas:
        ExampleSchema:
            type: object
            properties:
              PropA:
                type: string
              PropB:
                type: string

File: CommonSecuritySchemas.yaml

openapi: '3.0.2'

info:
    title: Common definitions
    description: Common definitions for OpenApiWalker reference problem
    version: 1.0.0

paths:

components:
    securitySchemes:
        TestSecurityScheme:
            type: oauth2
            flows:
                clientCredentials:
                    tokenUrl: https://localhost/connect/token
                    scopes:
                        Read: Allows to read data
                        Write: Allows to write data
                        Admin: Allows to change access to data

Example program:

using Microsoft.OpenApi.Models;
using Microsoft.OpenApi.Readers;

await using var openApiContent = new FileStream("ExampleOpenApi.yaml", FileMode.Open, FileAccess.Read);

var readerSettings = new OpenApiReaderSettings
{
    LoadExternalRefs = true,
    BaseUrl = new Uri(AppContext.BaseDirectory, UriKind.Absolute)
};

var reader = new OpenApiStreamReader(readerSettings);
var readResult = await reader.ReadAsync(openApiContent);

var workspace = readResult.OpenApiDocument.Workspace;

foreach (var document in workspace.Documents)
{
    var pathCount = (document.Paths ?? new OpenApiPaths()).Count;
    var schemaCount = (document.Components.Schemas ?? new Dictionary<string, OpenApiSchema>()).Count;
    var securitySchemaCount = (document.Components.SecuritySchemes ?? new Dictionary<string, OpenApiSecurityScheme>()).Count;

    Console.WriteLine($"Paths: {pathCount} Schemas: {schemaCount} SecuritySchemas: {securitySchemaCount}");
}

Result:

Paths: 1 Schemas: 0 SecuritySchemas: 1
Paths: 0 Schemas: 1 SecuritySchemas: 0

Expected behavior

The expectation is, that all three document are found. From the example above it can be seen, that the root document with the path and the reference to the security scheme is found. Furthermore, the document with the schema is loaded but the document with securitySchemes is not loaded.

This then results in problems to try to resolve the reference in the root document.

[dldl-cmd]

The reason, why the document is not loaded is, that the OpenApiRemoteReferenceCollector does not get the reference added to the document with the securitySchemes. This occurs because the OpenApiWalker does not visit references directly in a component. A fix for this would be to visit also unresolved references.

So the following could be changed from:

private bool ProcessAsReference(IOpenApiReferenceable referenceable, bool isComponent = false)
{
    var isReference = referenceable.Reference != null && !isComponent;
    if (isReference)
    {
        Walk(referenceable);
    }
    return isReference;
}

to

private bool ProcessAsReference(IOpenApiReferenceable referenceable, bool isComponent = false)
{
    var isReference = referenceable.Reference != null && 
                      (!isComponent || referenceable.UnresolvedReference);
    if (isReference)
    {
        Walk(referenceable);
    }
    return isReference;
}

I will prepare a PR.

[darrelmiller]

I am concerned about this fix. We have had a number of issues in the past with having $ref at the root of the component. My recollection was that we still didn't support it properly. I can't tell at this point whether this change actually fixed the issue completely, or just exchanged one bug with other another. More investigation on our part is required before we can ship this.