V1.3 (#393)

* Adds version analytics event (#306)

* Techdebt: adds state for testrun page (#392)

* Fix tests (#397)

* Fix tests

* Update node version

* 331379891: (feat) disable connection settings when testrun is in progress (#371)

* 331379891: (feat) disable connection settings when testrun is in progress

* 331379891: (fix) include more testrun results as progress

* 331379891: (fix) fix spelling

* 333349715: (fix) GAR 1.3 The disabled system settings panel contains a focusable element (#388)

Co-authored-by: Volha Mardvilka <[email protected]>

---------

Co-authored-by: Volha Mardvilka <[email protected]>

* Disable device item if device in progress (#370)

* Adds ga to track testrun initiation (#415)

Adds ga ti track testrun initiation

* Fix function return value (#421)

* Fix redirect + adds google analytics (#420)

* Adds certificates drawer component with list of certificates (#414)

Adds certificates drawer component with list of certificates. Upload and delete is out of scope

* List modules (#425)

* Adds delete certificate (#424)

Adds delete certificate

* Adds class for links to track report type on download; adds event when download report is clicked on progress page (#434)

* Adds upload certificate (#431)

* Adds upload certificate

* move delete functionality to certificates store

* Catch error on report/certificate deletion (#438)

* Fix button size; fix text-overflow (#440)

* Adds focus on next or close button when certificate is deleted (#439)

* Adds focus on next or close button when certificate is deleted

* Enable test modules for initiate test run dialog (#400)

Enable test modules for initiate test run dialog

* Add steps to resolve to PDF report (#411)

* Add progress for steps to resolve

* Formatting

* Remove print statements

* Allow for no steps to resolve

* Add multipage

* Fix multipage

* Fix config

* update report unit tests

---------

Signed-off-by: J Boddey <[email protected]>
Co-authored-by: jhughesbiot <[email protected]>

* Add new mac addr field for report deleting (#432)

* Bump version to v1.2.2 (#433)

* 337012359: (feat) add snackBar with wait or stop testrun (#444)

* 337012359: (feat) add snackBar with wait or stop testrun

* 337012359: (fix) update to fix failed tests

* 337012359: (fix) add fix for deletCertificate test

---------

Co-authored-by: Volha Mardvilka <[email protected]>

* Feature/port stats (#430)

* Add port speed and duplex tests
Add unit tests for port stats testing
Add place holder for ip port link test

* Add ethtool to system dependencies
Restructure conn stats methods and tests
Resolve port stat information for link test

* Fix runtime issue

* Implement link test
Fix unit tests
Misc port stats updates

* fix pylint issues

* update readme

* pylint fixes

---------

Signed-off-by: J Boddey <[email protected]>
Co-authored-by: J Boddey <[email protected]>

* 339315842: (feat) add risk assessment tab (#450)

* 339315842: (feat) add risk assessment tab

* 339315842: (feat) add risk assessment tab

---------

Co-authored-by: Volha Mardvilka <[email protected]>

* Add steps to resolve to PDF report (#411)

* Add progress for steps to resolve

* Formatting

* Remove print statements

* Allow for no steps to resolve

* Add multipage

* Fix multipage

* Fix config

* update report unit tests

---------

Signed-off-by: J Boddey <[email protected]>
Co-authored-by: jhughesbiot <[email protected]>

* Add new mac addr field for report deleting (#432)

* Remove rebase error

---------

Signed-off-by: J Boddey <[email protected]>
Co-authored-by: Olga Mardvilko <[email protected]>
Co-authored-by: Volha Mardvilka <[email protected]>
Co-authored-by: J Boddey <[email protected]>
Co-authored-by: jhughesbiot <[email protected]>
Co-authored-by: jhughesbiot <[email protected]>

Author: 4 people

.github/workflows/testing.yml

@@ -76,7 +76,7 @@ jobs:
     - name: Install Node
       uses: actions/setup-node@b39b52d1213e96004bfcb1c61a8a6fa8ab84f3e8 # v4.0.1
       with:
-        node-version: 18.13.0
+        node-version: 18.18.0
     - name: Install Chromium Browser
       run: sudo apt install chromium-browser
     - name: Install dependencies
@@ -99,7 +99,7 @@ jobs:
     - name: Install Node
       uses: actions/setup-node@b39b52d1213e96004bfcb1c61a8a6fa8ab84f3e8 # v4.0.1
       with:
-        node-version: 18.13.0
+        node-version: 18.18.0
     - name: Install dependencies
       run: npm install && npm ci
       working-directory: ./modules/ui

cmd/prepare

@@ -20,6 +20,6 @@
 echo Installing system dependencies
 
 # Install system dependencies
-sudo apt-get update && sudo apt-get install openvswitch-common openvswitch-switch python3 libpangocairo-1.0-0
+sudo apt-get update && sudo apt-get install openvswitch-common openvswitch-switch python3 libpangocairo-1.0-0 ethtool
 
 echo Finished installing system dependencies

framework/python/src/api/api.py

@@ -95,6 +95,9 @@ def __init__(self, test_run):
     self._router.add_api_route("/device/edit",
                                self.edit_device,
                                methods=["POST"])
+    
+    self._router.add_api_route("/system/modules",
+                               self.get_test_modules)
 
     # Allow all origins to access the API
     origins = ["*"]
@@ -551,6 +554,20 @@ async def get_results(self, response: Response,
       response.status_code = 404
       return self._generate_msg(False, "Test results could not be found")
 
+  async def get_test_modules(self):
+
+    LOGGER.debug("Received request to list test modules")
+
+    test_modules = []
+
+    for test_module in self._get_test_run().get_test_orc().get_test_modules():
+
+      # Only add module if it is an actual, enabled test module
+      if (test_module.enabled and test_module.enable_container):
+        test_modules.append(test_module.display_name)
+
+    return test_modules
+
   def _validate_device_json(self, json_obj):
 
     # Check all required properties are present
@@ -575,3 +592,6 @@ def _validate_device_json(self, json_obj):
         return False
 
     return True
+
+  def _get_test_run(self):
+    return self._test_run

framework/python/src/common/testreport.py

@@ -58,7 +58,10 @@ def __init__(self,
     self._report_url = ''
     self._cur_page = 0
     # Placeholder until available in json report
-    self._version = 'v1.2.2'
+    self._version = 'v1.3-alpha'
+
+  def get_mac_addr(self):
+    return self._mac_addr
 
   def get_mac_addr(self):
     return self._mac_addr
@@ -94,6 +97,12 @@ def get_report_url(self):
   def set_mac_addr(self, mac_addr):
     self._mac_addr = mac_addr
 
+  def set_mac_addr(self, mac_addr):
+    self._mac_addr = mac_addr
+
+  def set_mac_addr(self, mac_addr):
+    self._mac_addr = mac_addr
+
   def to_json(self):
     report_json = {}
 

framework/python/src/core/testrun.py

@@ -100,14 +100,18 @@ def __init__(self,
     if validate:
       self._session.add_runtime_param('validate')
 
-    self.load_all_devices()
-
     self._net_orc = net_orc.NetworkOrchestrator(
       session=self._session)
     self._test_orc = test_orc.TestOrchestrator(
       self._session,
       self._net_orc)
 
+    # Load device repository
+    self.load_all_devices()
+
+    # Load test modules
+    self._test_orc.start()
+
     if self._no_ui:
 
       # Check Testrun is able to start
@@ -324,8 +328,6 @@ def start(self):
     if self._net_only:
       LOGGER.info('Network only option configured, no tests will be run')
     else:
-      self._test_orc.start()
-
       self.get_net_orc().get_listener().register_callback(
           self._device_stable,
           [NetworkEvent.DEVICE_STABLE]
@@ -382,6 +384,9 @@ def get_config_file(self):
   def get_net_orc(self):
     return self._net_orc
 
+  def get_test_orc(self):
+    return self._test_orc
+
   def _start_network(self):
     # Start the network orchestrator
     if not self.get_net_orc().start():

framework/python/src/net_orc/ip_control.py

@@ -42,7 +42,7 @@ def add_namespace(self, namespace):
       return success
 
   def check_interface_status(self, interface_name):
-    output = util.run_command(cmd=f'ip link show {interface_name}',output=True)
+    output = util.run_command(cmd=f'ip link show {interface_name}', output=True)
     if 'state DOWN ' in output[0]:
       return False
     else:
@@ -81,6 +81,22 @@ def get_links(self):
         netns_links.append(interface_name.strip())
     return netns_links
 
+  def get_iface_connection_stats(self, iface):
+    """Extract information about the physical connection"""
+    response = util.run_command(f'ethtool {iface}')
+    if len(response[1]) == 0:
+      return response[0]
+    else:
+      return None
+
+  def get_iface_port_stats(self, iface):
+    """Extract information about packets connection"""
+    response = util.run_command(f'ethtool -S {iface}')
+    if len(response[1]) == 0:
+      return response[0]
+    else:
+      return None
+
   def get_namespaces(self):
     result = util.run_command('ip netns list')
     #Strip ID's from the namespace results

framework/python/src/net_orc/network_orchestrator.py

@@ -173,6 +173,7 @@ def _device_discovered(self, mac_addr):
         # Ignore discovered device
         return
 
+    self._get_port_stats(pre_monitor=True)
     self._monitor_in_progress = True
 
     LOGGER.debug(
@@ -203,6 +204,8 @@ def _device_discovered(self, mac_addr):
     with open(runtime_device_conf, 'w', encoding='utf-8') as f:
       json.dump(self._session.get_target_device().to_config_json(), f, indent=2)
 
+    self._get_conn_stats()
+
     if device.ip_addr is None:
       LOGGER.info(
           f'Timed out whilst waiting for {mac_addr} to obtain an IP address')
@@ -216,6 +219,31 @@ def _device_discovered(self, mac_addr):
 
     self._start_device_monitor(device)
 
+  def _get_conn_stats(self):
+    """ Extract information about the physical connection
+    and store it to a file for the conn test module to access"""
+    dev_int = self._session.get_device_interface()
+    conn_stats = self._ip_ctrl.get_iface_connection_stats(dev_int)
+    if conn_stats is not None:
+      eth_out_file = os.path.join(NET_DIR, 'ethtool_conn_stats.txt')
+      with open(eth_out_file, 'w', encoding='utf-8') as f:
+        f.write(conn_stats)
+    else:
+      LOGGER.error('Failed to generate connection stats')
+
+  def _get_port_stats(self, pre_monitor=True):
+    """ Extract information about the port statistics
+    and store it to a file for the conn test module to access"""
+    dev_int = self._session.get_device_interface()
+    port_stats = self._ip_ctrl.get_iface_port_stats(dev_int)
+    if port_stats is not None:
+      suffix = 'pre_monitor' if pre_monitor else 'post_monitor'
+      eth_out_file = os.path.join(NET_DIR, f'ethtool_port_stats_{suffix}.txt')
+      with open(eth_out_file, 'w', encoding='utf-8') as f:
+        f.write(port_stats)
+    else:
+      LOGGER.error('Failed to generate port stats')
+
   def monitor_in_progress(self):
     return self._monitor_in_progress
 
@@ -261,6 +289,7 @@ def _start_device_monitor(self, device):
     wrpcap(os.path.join(device_runtime_dir, 'monitor.pcap'),
            self._monitor_packets)
     self._monitor_in_progress = False
+    self._get_port_stats(pre_monitor=False)
     self.get_listener().call_callback(NetworkEvent.DEVICE_STABLE,
                                       device.mac_addr)
 
@@ -498,23 +527,22 @@ def _start_network_service(self, net_module):
     try:
       client = docker.from_env()
       net_module.container = client.containers.run(
-        net_module.image_name,
-        auto_remove=True,
-        cap_add=['NET_ADMIN'],
-        name=net_module.container_name,
-        hostname=net_module.container_name,
-        # Undetermined version of docker seems to have broken
-        # DNS configuration (/etc/resolv.conf)  Re-add when/if
-        # this network is utilized and DNS issue is resolved
-        #network=PRIVATE_DOCKER_NET,
-        privileged=True,
-        detach=True,
-        mounts=net_module.mounts,
-        environment={
-          'TZ': self.get_session().get_timezone(),
-          'HOST_USER': util.get_host_user()
-        }
-      )
+          net_module.image_name,
+          auto_remove=True,
+          cap_add=['NET_ADMIN'],
+          name=net_module.container_name,
+          hostname=net_module.container_name,
+          # Undetermined version of docker seems to have broken
+          # DNS configuration (/etc/resolv.conf)  Re-add when/if
+          # this network is utilized and DNS issue is resolved
+          #network=PRIVATE_DOCKER_NET,
+          privileged=True,
+          detach=True,
+          mounts=net_module.mounts,
+          environment={
+              'TZ': self.get_session().get_timezone(),
+              'HOST_USER': util.get_host_user()
+          })
     except docker.errors.ContainerError as error:
       LOGGER.error('Container run error')
       LOGGER.error(error)

modules/test/conn/README.md

@@ -14,6 +14,9 @@ Within the ```python/src``` directory, the below tests are executed. A few dhcp
 
 | ID | Description | Expected Behavior | Required Result |
 |------------------------------|----------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------|
+| connection.port_link         | The network switch port connected to the device has an active link without errors | When the etherent cable is connected to the port, the device triggers the port to its enabled \"Link UP\" (LEDs illuminate on device and switch ports if present) state, and the switch shows no errors with the LEDs and when interrogated with a \"show interface\" command on most network switches. | Required        |
+| connection.port_speed        | The network switch port connected to the device has auto-negotiated a speed that is 10 Mbps or higher | When the ethernet cable is connected to the port, the device autonegotiates a speed that can be checked with the \"show interface\" command on most network switches.  The output of this command must also show that the \"configured speed\" is set to \"auto\". | Required        |
+| connection.port_duplex       | The network switch port connected to the device has auto-negotiated full-duplex. | When the ethernet cable is connected to the port, the device autonegotiates a full-duplex connection. | Required        |
 | connection.dhcp_address      | The device under test has received an IP address from the DHCP server and responds to an ICMP echo (ping) request | The device is not set up with a static IP address. The device accepts an IP address from a DHCP server (RFC 2131) and responds successfully to an ICMP echo (ping) request. | Required        |
 | connection.mac_address       | Check and note device physical address. | N/A | Required |
 | connection.mac_oui | The device under test has a MAC address prefix that is registered against a known manufacturer. | The MAC address prefix is registered in the IEEE Organizationally Unique Identifier database. | Required |

modules/test/conn/conf/module_config.json

@@ -13,6 +13,24 @@
       "timeout": 1800
     },
     "tests": [
+      {
+        "name": "connection.port_link",
+        "test_description": "The network switch port connected to the device has an active link without errors",
+        "expected_behavior": "When the etherent cable is connected to the port, the device triggers the port to its enabled \"Link UP\" (LEDs illuminate on device and switch ports if present) state, and the switch shows no errors with the LEDs and when interrogated with a \"show interface\" command on most network switches.",
+        "required_result": "Required"
+      },
+      {
+        "name": "connection.port_speed",
+        "test_description": "The network switch port connected to the device has auto-negotiated a speed that is 10 Mbps or higher",
+        "expected_behavior": "When the ethernet cable is connected to the port, the device autonegotiates a speed that can be checked with the \"show interface\" command on most network switches.  The output of this command must also show that the \"configured speed\" is set to \"auto\".",
+        "required_result": "Required"
+      },
+      {
+        "name": "connection.port_duplex",
+        "test_description": "The network switch port connected to the device has auto-negotiated full-duplex",
+        "expected_behavior": "When the ethernet cable is connected to the port, the device autonegotiates a full-duplex connection.",
+        "required_result": "Required"
+      },
       {
         "name": "connection.switch.arp_inspection",
         "test_description": "The device implements ARP correctly as per RFC826",

modules/test/conn/python/src/connection_module.py

@@ -20,15 +20,16 @@
 from dhcp1.client import Client as DHCPClient1
 from dhcp2.client import Client as DHCPClient2
 from dhcp_util import DHCPUtil
+from port_stats_util import PortStatsUtil
 
 LOG_NAME = 'test_connection'
-LOGGER = None
 OUI_FILE = '/usr/local/etc/oui.txt'
 STARTUP_CAPTURE_FILE = '/runtime/device/startup.pcap'
 MONITOR_CAPTURE_FILE = '/runtime/device/monitor.pcap'
 DHCP_CAPTURE_FILE = '/runtime/network/dhcp-1.pcap'
 SLAAC_PREFIX = 'fd10:77be:4186'
 TR_CONTAINER_MAC_PREFIX = '9a:02:57:1e:8f:'
+LOGGER = None
 
 # Should be at least twice as much as the max lease time
 # set in the DHCP server
@@ -38,10 +39,15 @@
 class ConnectionModule(TestModule):
   """Connection Test module"""
 
-  def __init__(self, module):
-    super().__init__(module_name=module, log_name=LOG_NAME)
+  def __init__(self, module, log_dir=None, conf_file=None, results_dir=None):
+    super().__init__(module_name=module,
+                     log_name=LOG_NAME,
+                     log_dir=log_dir,
+                     conf_file=conf_file,
+                     results_dir=results_dir)
     global LOGGER
     LOGGER = self._get_logger()
+    self._port_stats = PortStatsUtil(logger=LOGGER)
     self.dhcp1_client = DHCPClient1()
     self.dhcp2_client = DHCPClient2()
     self._dhcp_util = DHCPUtil(self.dhcp1_client, self.dhcp2_client, LOGGER)
@@ -74,6 +80,18 @@ def __init__(self, module):
     # response = self.dhcp1_client.set_dhcp_range('10.10.10.20','10.10.10.30')
     # print("Set Range: " + str(response))
 
+  def _connection_port_link(self):
+    LOGGER.info('Running connection.port_link')
+    return self._port_stats.connection_port_link_test()
+
+  def _connection_port_speed(self):
+    LOGGER.info('Running connection.port_speed')
+    return self._port_stats.connection_port_speed_test()
+
+  def _connection_port_duplex(self):
+    LOGGER.info('Running connection.port_duplex')
+    return self._port_stats.connection_port_duplex_test()
+
   def _connection_switch_arp_inspection(self):
     LOGGER.info('Running connection.switch.arp_inspection')