Enable census collection

[jeremystretch]

NetBox version

v3.4.8

Feature type

New functionality

Proposed functionality

On startup, log the following metrics to census.netbox.dev using an HTTP POST request. (This collector is operated and maintained by the NetBox maintainers team.)

• NetBox version
• Python version
• Random unique ID

This anonymized data will be collected and aggregated for analysis purposes (described below). This is very similar in nature to how many other open source applications track their user base. Only one log message will be sent per NetBox worker instance.

Additionally, introduce a CENSUS_REPORTING_ENABLED configuration parameter, which defaults to True. Setting this to False will disable all census reporting.

Use case

Ever since its initial open source release, we've struggled to ascertain just how many NetBox deployments exist in the world. (The closest analog we have is GitHub stars, which is of very limited utility.) Having NetBox automatically report this anonymized data will allow us to:

• Form a rough idea about how many NetBox deployments exist
• Measure the average intervals between version upgrades
• Determine how long an old release remains in use (eventually)
• Gauge resistance to the adoption of newer Python releases

Database changes

None

External dependencies

None; it should be feasible to implement a lightweight logging function natively.

[abhi1693]

Would this not cause issues in containerized deployments where people are running multiple replicas? Or, what if an instance came online due to restarts or downtime.

There has to be a very deterministic way to generate the UUID to avoid duplicate data to be posted to the endpoint.

[jeremystretch]

There has to be a very deterministic way to generate the UUID to avoid duplicate data to be posted to the endpoint.

The implication of using a UUID was an error on my part when drafting the implementation. I had been using UUIDs for testing, but in practice we can use the unique DEPLOYMENT_ID already generated by NetBox to support Sentry integration.

Ultimately though, it's not a problem if this changes over time, as we're primarily concerned with counting the number of total NetBox deployments at each version at any point in time, and identifying macro trends over long timelines.

[decoupca]

In the spirit of transparency I think it would be appropriate to add CENSUS_REPORTING_ENABLED to configuration_example.py as well.

[davekempe]

Hey @jeremystretch what's the plan with the use of IP address data used in the request to census.netbox? I support this feature, just wondering what the plan is for transparency. I suppose you can get a breakdown of cloud vs on prem.

[davekempe]

Oh another thought I had, lots of our on prem deployments need a proxy to get out to the internet. So netbox would need to know how to use that proxy if you want this to work.

[decoupca]

Oh another thought I had, lots of our on prem deployments need a proxy to get out to the internet. So netbox would need to know how to use that proxy if you want this to work.

the call uses HTTP_PROXIES from configuration.py

[jeremystretch]

Hey @jeremystretch what's the plan with the use of IP address data used in the request to census.netbox?

No plan; we're just counting unique deployments (identified by deployment ID) per NetBox release. I plan to share some stats after a few months.