List views of self-nesting (recursive) object types are displayed incorrectly

[jii8by]

Deployment Type

Self-hosted

NetBox Version

v4.2.5

Python Version

3.11

Steps to Reproduce

1. Create three regions without parent: "North America", "South America", "Europe"
2. Create three child regions under North America: "United States", "Canada", "Mexico"
3. Create three grandchildren within United States: "Wisconsin", "Oklahoma", "Tennessee"
4. Navigate to /dcim/regions/ to view all regions

Expected Behavior

The three child regions that were created within North America are displayed directly below NA. Additionally, "Oklahoma", "Wisconsin" and "Tennessee" are indented under the US.

Observed Behavior

Child regions "Canada" and "Mexico" are below "Europe". Grandchild "Oklahoma" is now in "Mexico".

Note that this might not be the only list view that appears to be broken. Replicating the above steps as contact groups behaves in a similar way:

[jnovinger]

This is also happening in the Parent dropdown in create/edit forms and list view filter. The same behavior is happening in Tenant Groups.

[jeremystretch]

Bumped this to medium severity as it seems pretty disruptive. Confirmed that this was introduced in v4.2.5; ordering is working as expected in v4.2.4.

[jeremystretch]

This appears to be a regression introduced by 4ec1e6f under #18729.

[bctiemann]

I was worried about these kinds of less-obvious consequences of that change.

Instead of applying the order_by centrally like in the above commit, maybe we need to do that more surgically (i.e. only on querysets that use annotate) even though that's potentially a lot of changes; or alternatively, maybe it's only these hierarchical view lists that need to be exempt and should not call super().get_queryset().

[bctiemann]

Likely this is going to be happening on any NestedGroupModel. If we can exempt these classes from the central ordering enforcement that might be the way to go.