Pre-generate the OpenAPI schema

[candlerb]

NetBox version

v2.11.3

Feature type

Change to existing functionality

Proposed functionality

A feature like "collectstatic" which builds the openapi schema once in ./upgrade.sh, then serves it as a static file.

Currently it's served dynamically and is very slow, taking consistently 7-8 seconds on my system:

# time curl -sS localhost:8001/api/docs/?format=openapi | wc
      0   78785  799797

real	0m7.805s
user	0m0.016s
sys	0m0.005s

Alternatively, cache it in redis (it will take nearly a megabyte of RAM, but worth it)

Use case

ansible netbox inventory

The ansible netbox inventory fetches the whole openapi schema each time it runs. When combined with the actual data queries it does for devices, vms etc, this gives around a 10-second lead time to any playbook which is hitting the netbox API.

Whilst this inventory module has a feature to cache the results of the inventory (if you configure a suitable cache plugin), I want to get the "real-time" up-to-the-minute data from Netbox. The plugin can't be configured to cache the openapi schema only, whilst still making data queries to Netbox.

pynetbox

Any code which uses pynetbox, and calls nb.openapi(), will also benefit.

import pynetbox
nb = pynetbox.api(API_URL, token=API_TOKEN)
nb.openapi()

Database changes

None

External dependencies

None

[mraerino]

This really got worse on 2.11 - my webserver even times out while generating the spec.

Is there a console command you can run to get the openapi json?

[jeremystretch]

This really got worse on 2.11

I haven't noticed any difference from v2.10, and I can't recall any changes we've made that would have a significant impact on it.

[sdktr]

While serving a static version of the spec seems harmless since it's version bound anyway, I'm not sure that it makes sense to request the complete schema each time the plugin starts? No matter how fast netbox spits out the file, is that a roundtrip we can avoid?

[candlerb]

There are two ways to bite the cherry.

1. At the server end: pre-generate the spec and serve it statically
2. At the client end: read the spec once and cache it

Whilst both are possible, (1) gives the maximum benefit:

• It only has to be implemented once
• It benefits all clients, even dumb ones
• Clients don't have to worry about running with a stale copy of the spec when Netbox changes
• TBH, transferring 1MB over the network is cheap these days

Ideally, Netbox would pregenerate the spec and give an ETag - then smart clients can check and don't have to refetch unless it has changed.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Please see our contributing guide.