Closes #5986: Introduce the cloud model

docs/core-functionality/circuits.md

@@ -1,6 +1,7 @@
 # Circuits
 
 {!docs/models/circuits/provider.md!}
+{!docs/models/circuits/cloud.md!}
 
 ---
 

docs/models/circuits/circuittermination.md

@@ -2,9 +2,9 @@
 
 The association of a circuit with a particular site and/or device is modeled separately as a circuit termination. A circuit may have up to two terminations, labeled A and Z. A single-termination circuit can be used when you don't know (or care) about the far end of a circuit (for example, an Internet access circuit which connects to a transit provider). A dual-termination circuit is useful for tracking circuits which connect two sites.
 
-Each circuit termination is tied to a site, and may optionally be connected via a cable to a specific device interface or port within that site. Each termination must be assigned a port speed, and can optionally be assigned an upstream speed if it differs from the downstream speed (a common scenario with e.g. DOCSIS cable modems). Fields are also available to track cross-connect and patch panel details.
+Each circuit termination is attached to either a site or a cloud. Site terminations may optionally be connected via a cable to a specific device interface or port within that site. Each termination must be assigned a port speed, and can optionally be assigned an upstream speed if it differs from the downstream speed (a common scenario with e.g. DOCSIS cable modems). Fields are also available to track cross-connect and patch panel details.
 
-In adherence with NetBox's philosophy of closely modeling the real world, a circuit may terminate only to a physical interface. For example, circuits may not terminate to LAG interfaces, which are virtual in nature. In such cases, a separate physical circuit is associated with each LAG member interface and each needs to be modeled discretely.
+In adherence with NetBox's philosophy of closely modeling the real world, a circuit may be connected only to a physical interface. For example, circuits may not terminate to LAG interfaces, which are virtual in nature. In such cases, a separate physical circuit is associated with each LAG member interface and each needs to be modeled discretely.
 
 !!! note
-    A circuit in NetBox represents a physical link, and cannot have more than two endpoints. When modeling a multi-point topology, each leg of the topology must be defined as a discrete circuit, with one end terminating within the provider's infrastructure.
+    A circuit in NetBox represents a physical link, and cannot have more than two endpoints. When modeling a multi-point topology, each leg of the topology must be defined as a discrete circuit, with one end terminating within the provider's infrastructure. The cloud model is ideal for representing these networks.

docs/models/circuits/cloud.md

@@ -0,0 +1,5 @@
+# Clouds
+
+A cloud represents an abstract portion of network topology, just like in a topology diagram. For example, a cloud may be used to represent a provider's MPLS network.
+
+Each cloud must be assigned to a provider. A circuit may terminate to either a cloud or to a site.

docs/release-notes/version-2.11.md

@@ -70,6 +70,10 @@ This release introduces the new Site Group model, which can be used to organize
 
 The ObjectChange model (which is used to record the creation, modification, and deletion of NetBox objects) now explicitly records the pre-change and post-change state of each object, rather than only the post-change state. This was done to present a more clear depiction of each change being made, and to prevent the erroneous association of a previous unlogged change with its successor.
 
+#### Improved Change Logging ([#5986](https://github.com/netbox-community/netbox/issues/5986))
+
+A new cloud model has been introduced for representing the boundary of a network that exists outside the scope of NetBox. This is analogous to using a cloud icon on a topology drawing to represent an abstracted network. Each cloud must be assigned to a provider, and circuits can terminate to either clouds or sites.
+
 ### Enhancements
 
 * [#5370](https://github.com/netbox-community/netbox/issues/5370) - Extend custom field support to organizational models
@@ -108,6 +112,10 @@ The ObjectChange model (which is used to record the creation, modification, and
   * Added `_occupied` read-only boolean field as common attribute for determining whether an object is occupied
 * Renamed RackGroup to Location
   * The `/dcim/rack-groups/` endpoint is now `/dcim/locations/`
+* circuits.CircuitTermination
+  * Added the `cloud` field
+* circuits.Cloud
+  * Added the `/api/circuits/clouds/` endpoint
 * dcim.Device
   * Added the `location` field
 * dcim.Interface

netbox/circuits/api/nested_serializers.py

@@ -1,16 +1,29 @@
 from rest_framework import serializers
 
-from circuits.models import Circuit, CircuitTermination, CircuitType, Provider
+from circuits.models import *
 from netbox.api import WritableNestedSerializer
 
 __all__ = [
     'NestedCircuitSerializer',
     'NestedCircuitTerminationSerializer',
     'NestedCircuitTypeSerializer',
+    'NestedCloudSerializer',
     'NestedProviderSerializer',
 ]
 
 
+#
+# Clouds
+#
+
+class NestedCloudSerializer(WritableNestedSerializer):
+    url = serializers.HyperlinkedIdentityField(view_name='circuits-api:cloud-detail')
+
+    class Meta:
+        model = Provider
+        fields = ['id', 'url', 'display', 'name']
+
+
 #
 # Providers
 #

netbox/circuits/api/serializers.py

@@ -1,7 +1,7 @@
 from rest_framework import serializers
 
 from circuits.choices import CircuitStatusChoices
-from circuits.models import Provider, Circuit, CircuitTermination, CircuitType
+from circuits.models import *
 from dcim.api.nested_serializers import NestedCableSerializer, NestedSiteSerializer
 from dcim.api.serializers import CableTerminationSerializer, ConnectedEndpointSerializer
 from netbox.api import ChoiceField
@@ -28,6 +28,22 @@ class Meta:
         ]
 
 
+#
+# Clouds
+#
+
+class CloudSerializer(PrimaryModelSerializer):
+    url = serializers.HyperlinkedIdentityField(view_name='circuits-api:cloud-detail')
+    provider = NestedProviderSerializer()
+
+    class Meta:
+        model = Cloud
+        fields = [
+            'id', 'url', 'display', 'provider', 'name', 'description', 'comments', 'tags', 'custom_fields', 'created',
+            'last_updated',
+        ]
+
+
 #
 # Circuits
 #
@@ -47,12 +63,13 @@ class Meta:
 class CircuitCircuitTerminationSerializer(WritableNestedSerializer, ConnectedEndpointSerializer):
     url = serializers.HyperlinkedIdentityField(view_name='circuits-api:circuittermination-detail')
     site = NestedSiteSerializer()
+    cloud = NestedCloudSerializer()
 
     class Meta:
         model = CircuitTermination
         fields = [
-            'id', 'url', 'display', 'site', 'port_speed', 'upstream_speed', 'xconnect_id', 'connected_endpoint',
-            'connected_endpoint_type', 'connected_endpoint_reachable',
+            'id', 'url', 'display', 'site', 'cloud', 'port_speed', 'upstream_speed', 'xconnect_id',
+            'connected_endpoint', 'connected_endpoint_type', 'connected_endpoint_reachable',
         ]
 
 
@@ -77,13 +94,14 @@ class Meta:
 class CircuitTerminationSerializer(BaseModelSerializer, CableTerminationSerializer, ConnectedEndpointSerializer):
     url = serializers.HyperlinkedIdentityField(view_name='circuits-api:circuittermination-detail')
     circuit = NestedCircuitSerializer()
-    site = NestedSiteSerializer()
+    site = NestedSiteSerializer(required=False)
+    cloud = NestedCloudSerializer(required=False)
     cable = NestedCableSerializer(read_only=True)
 
     class Meta:
         model = CircuitTermination
         fields = [
-            'id', 'url', 'display', 'circuit', 'term_side', 'site', 'port_speed', 'upstream_speed', 'xconnect_id',
-            'pp_info', 'description', 'mark_connected', 'cable', 'cable_peer', 'cable_peer_type', 'connected_endpoint',
-            'connected_endpoint_type', 'connected_endpoint_reachable', '_occupied',
+            'id', 'url', 'display', 'circuit', 'term_side', 'site', 'cloud', 'port_speed', 'upstream_speed',
+            'xconnect_id', 'pp_info', 'description', 'mark_connected', 'cable', 'cable_peer', 'cable_peer_type',
+            'connected_endpoint', 'connected_endpoint_type', 'connected_endpoint_reachable', '_occupied',
         ]

netbox/circuits/api/urls.py

@@ -13,5 +13,8 @@
 router.register('circuits', views.CircuitViewSet)
 router.register('circuit-terminations', views.CircuitTerminationViewSet)
 
+# Clouds
+router.register('clouds', views.CloudViewSet)
+
 app_name = 'circuits-api'
 urlpatterns = router.urls

netbox/circuits/api/views.py

@@ -2,7 +2,7 @@
 from rest_framework.routers import APIRootView
 
 from circuits import filters
-from circuits.models import Provider, CircuitTermination, CircuitType, Circuit
+from circuits.models import *
 from dcim.api.views import PathEndpointMixin
 from extras.api.views import CustomFieldModelViewSet
 from netbox.api.views import ModelViewSet
@@ -48,8 +48,7 @@ class CircuitTypeViewSet(CustomFieldModelViewSet):
 
 class CircuitViewSet(CustomFieldModelViewSet):
     queryset = Circuit.objects.prefetch_related(
-        Prefetch('terminations', queryset=CircuitTermination.objects.prefetch_related('site')),
-        'type', 'tenant', 'provider',
+        'type', 'tenant', 'provider', 'termination_a', 'termination_z'
     ).prefetch_related('tags')
     serializer_class = serializers.CircuitSerializer
     filterset_class = filters.CircuitFilterSet
@@ -66,3 +65,13 @@ class CircuitTerminationViewSet(PathEndpointMixin, ModelViewSet):
     serializer_class = serializers.CircuitTerminationSerializer
     filterset_class = filters.CircuitTerminationFilterSet
     brief_prefetch_fields = ['circuit']
+
+
+#
+# Clouds
+#
+
+class CloudViewSet(CustomFieldModelViewSet):
+    queryset = Cloud.objects.prefetch_related('tags')
+    serializer_class = serializers.CloudSerializer
+    filterset_class = filters.CloudFilterSet

netbox/circuits/filters.py

@@ -9,12 +9,13 @@
     BaseFilterSet, NameSlugSearchFilterSet, TagFilter, TreeNodeMultipleChoiceFilter
 )
 from .choices import *
-from .models import Circuit, CircuitTermination, CircuitType, Provider
+from .models import *
 
 __all__ = (
     'CircuitFilterSet',
     'CircuitTerminationFilterSet',
     'CircuitTypeFilterSet',
+    'CloudFilterSet',
     'ProviderFilterSet',
 )
 
@@ -79,6 +80,36 @@ def search(self, queryset, name, value):
         )
 
 
+class CloudFilterSet(BaseFilterSet, CustomFieldModelFilterSet, CreatedUpdatedFilterSet):
+    q = django_filters.CharFilter(
+        method='search',
+        label='Search',
+    )
+    provider_id = django_filters.ModelMultipleChoiceFilter(
+        queryset=Provider.objects.all(),
+        label='Provider (ID)',
+    )
+    provider = django_filters.ModelMultipleChoiceFilter(
+        field_name='provider__slug',
+        queryset=Provider.objects.all(),
+        to_field_name='slug',
+        label='Provider (slug)',
+    )
+    tag = TagFilter()
+
+    class Meta:
+        model = Cloud
+        fields = ['id', 'name']
+
+    def search(self, queryset, name, value):
+        if not value.strip():
+            return queryset
+        return queryset.filter(
+            Q(description__icontains=value) |
+            Q(comments__icontains=value)
+        ).distinct()
+
+
 class CircuitTypeFilterSet(BaseFilterSet, NameSlugSearchFilterSet):
 
     class Meta:
@@ -101,6 +132,11 @@ class CircuitFilterSet(BaseFilterSet, CustomFieldModelFilterSet, TenancyFilterSe
         to_field_name='slug',
         label='Provider (slug)',
     )
+    cloud_id = django_filters.ModelMultipleChoiceFilter(
+        field_name='terminations__cloud',
+        queryset=Cloud.objects.all(),
+        label='Cloud (ID)',
+    )
     type_id = django_filters.ModelMultipleChoiceFilter(
         queryset=CircuitType.objects.all(),
         label='Circuit type (ID)',
@@ -190,6 +226,10 @@ class CircuitTerminationFilterSet(BaseFilterSet, CableTerminationFilterSet, Path
         to_field_name='slug',
         label='Site (slug)',
     )
+    cloud_id = django_filters.ModelMultipleChoiceFilter(
+        queryset=Cloud.objects.all(),
+        label='Cloud (ID)',
+    )
 
     class Meta:
         model = CircuitTermination