Filter custom field of type 'selection' for value is null or not-null

[candlerb]

NetBox version

v3.4.2

Feature type

Change to existing functionality

Proposed functionality

I would like a way to be able to filter on custom fields of type "selection", to see if they are set (with any value) or unset.

There is an empty filter documented although only for string fields. There are no examples of how to use it, and no test cases for it in netbox/utilities/tests/test_filters.py

Given a custom selection field snmp_module, the following experimentally don't work:

# These return all devices
/api/dcim/devices/?cf_snmp_module__empty
/api/dcim/devices/?cf_snmp_module__empty=

# These always return zero devices
/api/dcim/devices/?cf_snmp_module__empty=true
/api/dcim/devices/?cf_snmp_module__empty=false
/api/dcim/devices/?cf_snmp_module__empty=1
/api/dcim/devices/?cf_snmp_module__empty=0

It does work with regular string fields though (e.g. name__empty=0 and name__empty=1 are fine).

There are other filters which do work on this custom field. The following returns all devices where the custom field is set and the value contains an underscore:

/api/dcim/devices/?cf_snmp_module__ic=_

The following returns all devices where the custom field is non-null, and also does not contain 'z':

/api/dcim/devices/?cf_snmp_module__nic=z

The following returns all devices where the custom field is non-null and does not start with xxxxx (and therefore works as an ugly workaround for what I want):

/api/dcim/devices/?cf_snmp_module__nisw=xxxxx

But the following doesn't work, as it returns all devices (whether or not the field is null):

/api/dcim/devices/?cf_snmp_module__n=xxxxx

What I'm requesting in this feature request is either:

• the existing empty condition to be extended to mean "null/not null" for selection custom fields; or
• a new condition, e.g. __null=1 or __null=0

---

Aside: it's unclear to me whether "empty" was intended to work this way (i.e. this is a bug, not a feature request). I found this code in netbox/netbox/filtersets.py:

        elif isinstance(existing_filter, (
            django_filters.ModelChoiceFilter,
            django_filters.ModelMultipleChoiceFilter,
            TagFilter
        )) or existing_filter.extra.get('choices'):
            # These filter types support only negation
            return FILTER_NEGATION_LOOKUP_MAP

This suggests to me that choices should only support negation (n) - but I've already shown that "ic" and "nic" do work, so maybe these "selection" fields are being considered as strings anyway.

Use case

To be able to filter devices depending on the presence/absence of a value in a selection custom field.

Specifically: I use the custom field snmp_module to choose an appropriate SNMP mobile to scrape by Prometheus. I need to query Netbox to get a list of targets to scrape, and really that should just be exactly those devices where snmp_module has been set.

Currently I have to use a tag to achieve this: I can filter on tag=prom_snmp. But this requires setting both the custom field and a tag on the device.

Database changes

None

External dependencies

None

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

[candlerb]

I should add, to find devices with custom field unset I also tried

/api/dcim/devices/?cf_snmp_module=

but that returns all devices. In any case, what I need is the opposite: "all devices where cf_snmp_module is set to any (non-empty) value"

[jeremystretch]

#11539 may be related to this, though I don't believe it's blocking.

[jwbensley]

This request is to add the __empty= filter to custom fields which are lists/selections. This filter is currently only available on string/text fields which are native fields on objects in NetBox. I would like to see this filter on custom fields which are strings/text too. We have the need to filter custom fields which are strings, for all which are empty / not empty.

[candlerb]

By chance I came across this in netbox/netbox/settings.py:

FILTERS_NULL_CHOICE_LABEL = 'None'
FILTERS_NULL_CHOICE_VALUE = 'null'

It looks like there's a magic string value here, and experimentally, the following filters do appear to work:

/api/dcim/devices/?cf_snmp_module=null
/api/dcim/devices/?cf_snmp_module__n=null

The latter one is exactly what I was looking for (custom field value is not null).

I would therefore be happy to close this issue, except that as far as I can tell, this functionality is completely undocumented and therefore I'm loathe to depend on it. Could it be documented here? And could it be made clear what the intended difference is between foo__empty=1 and foo=null for filtering?

I observe that there are just two tests using settings.FILTERS_NULL_CHOICE_VALUE in netbox/utilities/tests/test_filters.py

[candlerb]

Unfortunately, now I need this custom field to be multiselect instead of select, and the filter no longer works.

cf_snmp_module=null raises a ValueError exception: (EDIT: now raised separately as #13606)

cf_snmp_module__n=null does nothing: all devices are shown, regardless of whether this custom field is set or not.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.