Include "private" fields when serializing objects for change logging

[jeremystretch]

NetBox version

v4.0.3

Feature type

Change to existing functionality

Proposed functionality

When logging the creation, change, or deletion of an object, NetBox serializes the Python instance using the serialize_object() utility function. This wraps Django's internal serialization function, with some caveats. First, we ignore MPTT internal fields:

netbox/netbox/utilities/serialization.py

Lines 33 to 36 in 103c08c

	# Exclude any MPTTModel fields
	if issubclass(obj.__class__, MPTTModel):
	for field in ['level', 'lft', 'rght', 'tree_id']:
	data.pop(field)

Second, we omit any "private" fields; those which begin with an underscore. (This is done purely as a means of noise reduction, and not in the interest of privacy or security.)

netbox/netbox/utilities/serialization.py

Lines 48 to 51 in 103c08c

	# Skip excluded and private (prefixes with an underscore) attributes
	for key in list(data.keys()):
	if key in exclude or (isinstance(key, str) and key.startswith('_')):
	data.pop(key)

This issue proposes that we stop omitting these fields from the serialized data. Where it makes sense to omit these fields from display (e.g. when rendering a diff in the change log), the fields can be omitted from the output.

Use case

The exclusion of MPTT internal fields poses an issue for the staged changes API, which has been captured in #13422.

The omission of certain private fields can pose similar problems. For instance, omitting the _name field when serializing a site fails to capture the naturalized form of the site's name. When attempting to restore a site from serialized data using deserialize_object(), the resulting instance will be saved to the database with an empty value for _name, leading to issues with query ordering.

Database changes

None

External dependencies

None