Overhaul the custom fields model #4878
Description
Proposed Changes
The current approach to storing custom fields employs a CustomField model for each field created by a user, and one instance of a CustomFieldValue per object per field to hold assigned values. This issue proposes replacing the current implementation with a more robust, better-performing solution yet to be identified.
One approach is to employ a new custom_fields JSONField on each model which support custom fields. This field would hold a dictionary mapping field names to their native values. This solution would retain the current CustomField model (possibly with some minor modifications) but ditch CustomFieldValue. There are several benefits to this approach:
- Custom field data is stored locally with every instance.
- JSON data can be filtered against directly when forming a QuerySet (e.g.
Site.objects.filter(custom_fields__foo='abc')). - Custom field values can be set directly on an instance as dictionary keys on its
custom_fieldsattribute. - Data serialization is handled transparently.
Django does not currently support writing JSON data directly via a QuerySet, however issue #29112 has been opened to implement this functionality and appears to be gaining traction. It is also something we could potentially implement locally within NetBox prior to its official implementation.
It's also worth noting that with this approach we lose the ability to easily find all values for a custom field across all models. (This can currently be achieved with e.g. CustomFieldValue.objects.filter(field=some_field).) However, I cannot think of a scenario where we've done this or where it would be especially useful.
Further investigation into this proposal is needed.
Justification
The current implementation has several drawbacks:
- All custom field values are stored in a single database table.
- The low-level process for assigning a new custom field value is cumbersome and unintuitive. (A new CustomFieldValue instance must be created and associated with the instance.)
- Custom field values are serialized to string forms for storage, which can be error-prone and complicates ordering and validation.