@@ -33,12 +33,6 @@ Once completed, one can progress to the
3333&emsp ; [ 6.1 Prerequisite] ( #61-prerequisite ) <br >
3434&emsp ; [ 6.2 Steps for each test case] ( #62-steps-for-each-test-case ) <br >
3535
36- [ 7 Run AWS IoT Device Tester] ( #7-run-aws-iot-device-tester ) <br >
37- &emsp ; [ 7.1 Prerequisite] ( #71-prerequisite ) <br >
38- &emsp ; [ 7.2 Download AWS IoT Device Tester] ( #72-download-aws-iot-device-tester ) <br >
39- &emsp ; [ 7.3 Configure AWS IoT Device Tester] ( #73-configure-aws-iot-device-tester ) <br >
40- &emsp ; [ 7.4 Running the FreeRTOS qualification 2.0 suite] ( #74-running-the-freertos-qualification-20-suite ) <br >
41-
4236## 1 Pre-requisites
4337
4438### 1.1 Hardware Requirements
@@ -610,15 +604,11 @@ I (3444) ota_over_mqtt_demo: Subscribed to topic $aws/things/thing_esp32c3_nonOt
610604
611605### 6.1 Prerequisite
612606
613- - Follow the
614- [ OTA update with AWS IoT Guide] ( #5-perform-firmware-over-the-air-updates-with-aws-iot )
615- to create an OTA update and verify the digital signature, checksum and version
616- number of the new image. If firmware update is verified, you can run the tests on your device.
617607- Run ` idf.py menuconfig ` .
618608- Under ` Featured FreeRTOS IoT Integration ` , choose ` Run qualification test ` .
619609- Under ` Component config -> Unity unit testing library ` , choose ` Include Unity test fixture ` .
620610
621- _ Note: The log of module ` esp_ota_ops ` , ` AWS_OTA ` and ` esp-tls-mbedtls ` will be
611+ _ Note: The log of module ` esp_ota_ops ` and ` esp-tls-mbedtls ` will be
622612disabled when running the qualification test. You can change the log level by
623613` esp_log_level_set ` in [ main.c] ( ./main/main.c ) ._
624614
@@ -635,6 +625,7 @@ disabled when running the qualification test. You can change the log level by
635625 - Set ` Thing Name for Device Advisor Test/OTA end-to-end Test ` to AWS IoT Thing under test.
636626 - Build and run.
637627 - See Device Advisor test result in the console.
628+
6386292 . MQTT Test
639630 - Under ` Featured FreeRTOS IoT Integration -> Qualification Test Configurations -> Qualification Execution Test Configurations ` ,
640631 choose ` MQTT Test ` .
@@ -678,23 +669,7 @@ disabled when running the qualification test. You can change the log level by
678669 I (612755) qual_main: End qualification test.
679670 ```
680671
681- 4. OTA PAL Test
682- - Under `Featured FreeRTOS IoT Integration -> Qualification Test Configurations -> Qualification Execution Test Configurations`,
683- choose `OTA PAL Test`.
684- - Build and run.
685- - See test result on target output.
686- - Example output
687-
688- ```c
689- I (905) qual_main: Run qualification test.
690- ...
691- -----------------------
692- 15 Tests 0 Failures 0 Ignored
693- OK
694- I (113755) qual_main: End qualification test.
695- ```
696-
697- 5. Core PKCS11 Test
672+ 4. Core PKCS11 Test
698673 - Under `Featured FreeRTOS IoT Integration -> Qualification Test Configurations -> Qualification Execution Test Configurations`,
699674 choose `CorePKCS#11 Test`.
700675 - Build and run.
@@ -709,124 +684,3 @@ disabled when running the qualification test. You can change the log level by
709684 OK
710685 I (7518) qual_main: End qualification test.
711686 ```
712-
713- ## 7 Run AWS IoT Device Tester
714-
715- This repository can be tested using
716- [AWS IoT Device Tester for FreeRTOS (IDT)](https://aws.amazon.com/freertos/device-tester/).
717- IDT is a downloadable tool that can be used to exercise a device integration with
718- FreeRTOS to validate functionality and compatibility with Amazon IoT cloud.
719- Passing the test suite provided by IDT is also required to qualify a device for
720- the [Amazon Partner Device Catalogue](https://devices.amazonaws.com/).
721-
722- IDT runs a suite of tests that include testing the device's transport interface
723- layer implementation, PKCS11 functionality, and OTA capabilities. In IDT test
724- cases, the IDT binary will make a copy of the source code, update the header files
725- in the project, then compile the project and flash the resulting image to your
726- board. Finally, IDT will read serial output from the board and communicate with
727- the AWS IoT cloud to ensure that test cases are passing.
728-
729- ### 7.1 Prerequisite
730-
731- - Follow the
732- [OTA update with AWS IoT Guide](#5-perform-firmware-over-the-air-updates-with-aws-iot)
733- to create an OTA update and verify the digital signature, checksum and version
734- number of the new image. If firmware update is verified, you can run the tests
735- on your device.
736- - Run `idf.py menuconfig`.
737- - Under `Featured FreeRTOS IoT Integration`, choose `Run qualification test`.
738- - Under `Component config -> Unity unit testing library`, choose `Include Unity test fixture`.
739- - Under `Featured FreeRTOS IoT Integration -> Qualification Test Configurations -> Qualification Execution Test Configurations`,
740- **DISABLE** all the tests.
741- - Run `idf.py fullclean` to clear local CMAKE cache.
742-
743- _Note: The log of module `esp_ota_ops`, `AWS_OTA` and `esp-tls-mbedtls` will be
744- disabled when running the qualification test. You can change the log level by
745- `esp_log_level_set` in [main.c](./main/main.c)._
746-
747- ### 7.2 Download AWS IoT Device Tester
748-
749- The latest version of IDT can be downloaded from the
750- [here](https://docs.aws.amazon.com/freertos/latest/userguide/dev-test-versions-afr.html).
751- This repository has been qualified by IDT v4.6.0 and test suite version 2.3.0 for
752- [FreeRTOS 202210-LTS](https://github.com/FreeRTOS/FreeRTOS-LTS/tree/202210.01-LTS).
753-
754- ### 7.3 Configure AWS IoT Device Tester
755-
756- Follow [the instructions to setup your AWS account](https://docs.aws.amazon.com/freertos/latest/userguide/lts-idt-dev-tester-prereqs.html#lts-config-aws-account).
757-
758- Extract IDT for FreeRTOS to a location on the file system
759-
760- - The `devicetester-extract-location/bin` directory holds the IDT binary, which
761- is the entry point used to run IDT
762- - The `devicetester-extract-location/results` directory holds logs that are
763- generated every time you run IDT.
764- - The `devicetester-extract-location/configs` directory holds configuration files
765- that are required to setup IDT
766-
767- Before running IDT, the files in `devicetester-extract-location/configs` need
768- to be updated. We have pre-defined configures available in the
769- [idt_config](https://github.com/FreeRTOS/iot-reference-esp32c3/tree/main/idt_config).
770- Copy these templates to `devicetester-extract-location/configs`, and the rest of
771- this section will walk through the remaining values that need to be filled in.
772-
773- You need to configure your AWS credentials for IDT.
774-
775- - In `config.json`, update the `profile` and `awsRegion` fields
776-
777- You need to specify the device details for IDT.
778-
779- - In `device.json`, update `serialPort` to the serial port of your board as from
780- [PORT](./GettingStartedGuide.md#23-provision-the-esp32-c3-with-the-private-key-device-certificate-and-ca-certificate-in-development-mode).
781- Update `publicKeyAsciiHexFilePath` to the absolute path to `dummyPublicKeyAsciiHex.txt`.
782- Update `publicDeviceCertificateArn` to the ARN of the certificate uploaded when
783- [Setup AWS IoT Core](./GettingStartedGuide.md#21-setup-aws-iot-core).
784-
785- You need to configure IDT the build, flash and test settings.
786-
787- - In `build.bat` / `build.sh`, update ESP_IDF_PATH, and ESP_IDF_FRAMEWORK_PATH
788- - In `flash.bat` / `flash.sh`, update ESP_IDF_PATH, ESP_IDF_FRAMEWORK_PATH, and NUM_COMPORT
789- - In `userdata.json`, update `sourcePath` to the absolute path to the root of this repository.
790- - In `userdata.json`, update `signerCertificate` with the ARN of the
791- [Setup pre-requisites for OTA cloud resources](./GettingStartedGuide.md#51-setup-pre-requisites-for-ota-cloud-resources)
792- - Run all the steps to create a
793- [second code signing certificate](./GettingStartedGuide.md#51-setup-pre-requisites-for-ota-cloud-resources)
794- but do NOT provision the key onto your board.
795- - Copy the ARN for this certificate in `userdata.json` for the field
796- `untrustedSignerCertificate`.
797-
798- ### 7.4 Running the FreeRTOS qualification 2.0 suite
799-
800- With configuration complete, IDT can be run for an individual test group, a
801- test case, or the entire qualification suite.
802-
803- List all the available test groups, run:
804-
805- ```sh
806- .\devicetester_win_x86-64.exe list-groups
807- ```
808-
809- Run one or more specified test group, run e.g.:
810-
811- ``` sh
812- .\d evicetester_win_x86-64.exe run-suite --group-id FullCloudIoT --group-id OTACore
813- ```
814-
815- Run one or more specified tests, run e.g.:
816-
817- ``` sh
818- .\d evicetester_win_x86-64.exe run-suite --group-id OTADataplaneMQTT --test-id OTAE2EGreaterVersion
819- ```
820-
821- To run the entire qualification suite, run:
822-
823- ``` sh
824- .\d evicetester_win_x86-64.exe run-suite --skip-group-id FullPKCS11_PreProvisioned_RSA --skip-group-id FullPKCS11_Import_RSA --skip-group-id FullPKCS11_Core --skip-group-id FullTransportInterfacePlainText
825- ```
826-
827- For more information, ` .\devicetester_win_x86-64.exe help ` will show all available commands.
828-
829- When IDT is run, it generates the ` results/uuid ` directory that contains the
830- logs and other information associated with your test run. See
831- [ Understanding results and logs] ( https://docs.aws.amazon.com/freertos/latest/userguide/lts-results-logs.html )
832- for more details.
0 commit comments