Implement `results tree-report` command #213

JenySadadia · 2025-08-18T12:04:45Z

Get tree report.

JenySadadia · 2025-08-19T10:02:31Z

Sample command output without --json option:

$ kci-dev results tree-report --giturl  https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git --branch master --min-age 312 --max-age 340 --group-size 2
Tree: https://d.kernelci.org/tree/next/master/b7d4e259682caccb51a25283655f2c8f02e32d23?o=maestro
Commit: b7d4e259682caccb51a25283655f2c8f02e32d23
Origin: maestro
Checkout start time: 2025-08-06T04:12:56.776000Z

Summary: (pass/fail/inconclusive)
builds:	58/1/1
boots:	89/0/57
tests:	15794/5777/5613

Possible regressions:
Platform: dell-latitude-5400-8665U-sarien
- Config: x86_64_defconfig+lab-setup+x86-board+kselftest
  - Test path: kselftest.cpufreq.hibernate
  · https://d.kernelci.org/test/maestro:6892e1a479d5d217e5451ae7 status:FAIL
  · https://d.kernelci.org/test/maestro:68918ae179d5d217e54314e0 status:PASS

  - Test path: kselftest.cpufreq.hibernate.cpufreq_main_sh
  · https://d.kernelci.org/test/maestro:6892e3b679d5d217e54533e7 status:FAIL
  · https://d.kernelci.org/test/maestro:68918eda79d5d217e5433d1d status:PASS

  - Test path: kselftest.cpufreq.suspend
  · https://d.kernelci.org/test/maestro:6892e1ac79d5d217e5451b2b status:FAIL
  · https://d.kernelci.org/test/maestro:68918ae879d5d217e5431524 status:PASS

  - Test path: kselftest.cpufreq.suspend.cpufreq_main_sh
  · https://d.kernelci.org/test/maestro:6892e49e79d5d217e5454ec9 status:FAIL
  · https://d.kernelci.org/test/maestro:68918ed579d5d217e5433ce9 status:PASS

Fixed regressions:
Platform: beaglebone-black
- Config: multi_v7_defconfig+kselftest
  - Test path: kselftest.seccomp.seccomp_seccomp_benchmark
  · https://d.kernelci.org/test/maestro:6892f25779d5d217e5459341 status:PASS
  · https://d.kernelci.org/test/maestro:6891bf8879d5d217e5438bff status:FAIL

Platform: mt8195-cherry-tomato-r2
- Config: defconfig+lab-setup+arm64-chromebook+CONFIG_MODULE_COMPRESS=n+CONFIG_MODULE_COMPRESS_NONE=y
  - Test path: fluster.debian.v4l2.gstreamer_vp8.validate-fluster-results
  · https://d.kernelci.org/test/maestro:6892df6f79d5d217e54506f5 status:PASS
  · https://d.kernelci.org/test/maestro:68918e8a79d5d217e5433add status:FAIL

Platform: dell-latitude-5400-4305U-sarien
- Config: x86_64_defconfig+lab-setup+x86-board+kselftest
  - Test path: kselftest.cpufreq.hibernate
  · https://d.kernelci.org/test/maestro:6892e1a479d5d217e5451ae4 status:PASS
  · https://d.kernelci.org/test/maestro:68918ae079d5d217e54314dd status:FAIL

  - Test path: kselftest.cpufreq.hibernate.cpufreq_main_sh
  · https://d.kernelci.org/test/maestro:6892f14679d5d217e5459180 status:PASS
  · https://d.kernelci.org/test/maestro:6891aae979d5d217e5438288 status:FAIL

Unstable tests:
Platform: bcm2711-rpi-4-b
- Config: defconfig+lab-setup+kselftest
  - Test path: boot
  · https://d.kernelci.org/test/maestro:6892e0c579d5d217e5451677 status:INCONCLUSIVE (None)
  · https://d.kernelci.org/test/maestro:6892e0ba79d5d217e545162c status:PASS

Platform: beaglebone-black
- Config: multi_v7_defconfig
  - Test path: boot
  · https://d.kernelci.org/test/maestro:6892e3d879d5d217e54535fb status:PASS
  · https://d.kernelci.org/test/maestro:6892e3d679d5d217e54535b6 status:INCONCLUSIVE (MISS)

- Config: multi_v7_defconfig+kselftest
  - Test path: kselftest.landlock
  · https://d.kernelci.org/test/maestro:6892deb279d5d217e544ffc9 status:INCONCLUSIVE (None)
  · https://d.kernelci.org/test/maestro:6891893f79d5d217e5430eb5 status:INCONCLUSIVE (ERROR)

Platform: cd8180-orion-o6
- Config: defconfig+lab-setup+kselftest
  - Test path: kselftest.signal
  · https://d.kernelci.org/test/maestro:6892e0e779d5d217e5451774 status:INCONCLUSIVE (MISS)
  · https://d.kernelci.org/test/maestro:6891905979d5d217e5434afe status:INCONCLUSIVE (ERROR)

Platform: imx8mp-evk
- Config: defconfig+lab-setup+kselftest
  - Test path: kselftest.alsa
  · https://d.kernelci.org/test/maestro:6892e0c879d5d217e5451687 status:INCONCLUSIVE (None)
  · https://d.kernelci.org/test/maestro:6891903279d5d217e54349b6 status:INCONCLUSIVE (MISS)

  - Test path: kselftest.arm64
  · https://d.kernelci.org/test/maestro:6892e0cc79d5d217e54516af status:PASS
  · https://d.kernelci.org/test/maestro:6891903779d5d217e54349fe status:INCONCLUSIVE (MISS)

  - Test path: kselftest.device_error_logs
  · https://d.kernelci.org/test/maestro:6892e0d579d5d217e54516f2 status:FAIL
  · https://d.kernelci.org/test/maestro:6891904279d5d217e5434a50 status:INCONCLUSIVE (MISS)

  - Test path: kselftest.dt
  · https://d.kernelci.org/test/maestro:6892e0db79d5d217e5451722 status:FAIL
  · https://d.kernelci.org/test/maestro:6891904979d5d217e5434a83 status:INCONCLUSIVE (MISS)

- Config: defconfig+arm64-chromebook+kselftest
  - Test path: kselftest.kvm
  · https://d.kernelci.org/test/maestro:6892e70c79d5d217e54566b4 status:FAIL
  · https://d.kernelci.org/test/maestro:68918fed79d5d217e5434654 status:INCONCLUSIVE (MISS)

  - Test path: kselftest.pkvm
  · https://d.kernelci.org/test/maestro:6892e70e79d5d217e54566c3 status:FAIL
  · https://d.kernelci.org/test/maestro:68918fef79d5d217e5434663 status:INCONCLUSIVE (MISS)

JenySadadia · 2025-08-19T10:04:46Z

Sample output with --json and multiple --path options:

$ kci-dev results tree-report --giturl  https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git --branch master --min-age 312 --max-age 340 --group-size 2 --path kselftest.cpufreq.suspend --path boot --json
{
 "dashboard_url": "https://d.kernelci.org/tree/next/master/b7d4e259682caccb51a25283655f2c8f02e32d23?o=maestro",
 "git_url": "https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git",
 "git_branch": "master",
 "commit_hash": "b7d4e259682caccb51a25283655f2c8f02e32d23",
 "origin": "maestro",
 "checkout_start_time": "2025-08-06T04:12:56.776000Z",
 "build_status_summary": {
  "PASS": 58,
  "ERROR": 0,
  "FAIL": 1,
  "SKIP": 0,
  "MISS": 1,
  "DONE": 0,
  "NULL": 0
 },
 "boot_status_summary": {
  "pass_count": 89,
  "error_count": 4,
  "fail_count": 0,
  "skip_count": 0,
  "miss_count": 50,
  "done_count": 0,
  "null_count": 3
 },
 "test_status_summary": {
  "pass_count": 15794,
  "error_count": 37,
  "fail_count": 5777,
  "skip_count": 5449,
  "miss_count": 54,
  "done_count": 0,
  "null_count": 73
 },
 "possible_regressions": {},
 "fixed_regressions": {},
 "unstable_tests": {
  "bcm2711-rpi-4-b": {
   "defconfig+lab-setup+kselftest": {
    "boot": [
     {
      "id": "maestro:6892e0c579d5d217e5451677",
      "start_time": "2025-08-06T04:57:41.785000Z",
      "status": null
     },
     {
      "id": "maestro:6892e0ba79d5d217e545162c",
      "start_time": "2025-08-06T04:57:30.875000Z",
      "status": "PASS"
     }
    ]
   }
  },
  "beaglebone-black": {
   "multi_v7_defconfig": {
    "boot": [
     {
      "id": "maestro:6892e3d879d5d217e54535fb",
      "start_time": "2025-08-06T05:10:48.208000Z",
      "status": "PASS"
     },
     {
      "id": "maestro:6892e3d679d5d217e54535b6",
      "start_time": "2025-08-06T05:10:46.224000Z",
      "status": "MISS"
     }
    ]
   }
  },
  "meson-gxl-s905x-libretech-cc": {
   "defconfig+lab-setup+kselftest": {
    "boot": [
     {
      "id": "maestro:6892e0c079d5d217e5451656",
      "start_time": "2025-08-06T04:57:36.701000Z",
      "status": "MISS"
     },
     {
      "id": "maestro:6892e0be79d5d217e5451645",
      "start_time": "2025-08-06T04:57:34.628000Z",
      "status": "PASS"
     }
    ]
   }
  },
  "rk3399-roc-pc": {
   "defconfig+lab-setup+kselftest": {
    "boot": [
     {
      "id": "maestro:6892e0c279d5d217e545165c",
      "start_time": "2025-08-06T04:57:38.302000Z",
      "status": "MISS"
     },
     {
      "id": "maestro:6892e0bf79d5d217e545164d",
      "start_time": "2025-08-06T04:57:35.454000Z",
      "status": "PASS"
     }
    ]
   }
  },
  "rk3588-rock-5b": {
   "defconfig+lab-setup+kselftest": {
    "boot": [
     {
      "id": "maestro:6892e0c279d5d217e5451660",
      "start_time": "2025-08-06T04:57:38.665000Z",
      "status": "MISS"
     },
     {
      "id": "maestro:6892e0b579d5d217e54515fb",
      "start_time": "2025-08-06T04:57:25.469000Z",
      "status": "PASS"
     }
    ]
   }
  },
  "sun50i-a64-pine64-plus": {
   "defconfig+lab-setup+kselftest": {
    "boot": [
     {
      "id": "maestro:6892e0bf79d5d217e5451650",
      "start_time": "2025-08-06T04:57:35.878000Z",
      "status": "PASS"
     },
     {
      "id": "maestro:6892e0ba79d5d217e5451629",
      "start_time": "2025-08-06T04:57:30.511000Z",
      "status": "MISS"
     }
    ]
   }
  },
  "sun50i-h6-pine-h64": {
   "defconfig+lab-setup+kselftest": {
    "boot": [
     {
      "id": "maestro:6892e0c479d5d217e545166e",
      "start_time": "2025-08-06T04:57:40.037000Z",
      "status": "MISS"
     },
     {
      "id": "maestro:6892e0b679d5d217e54515fe",
      "start_time": "2025-08-06T04:57:26.464000Z",
      "status": "PASS"
     }
    ]
   }
  }
 }
}

padovan · 2025-08-19T11:37:30Z

Thanks for this, Jeny!

I'd try to imitate the email report for the output:

 last run: https://d.kernelci.org/test/maestro:6890eeae236da8c8a0d7faea
 history:  > ❌  > ⚠️  > ✅  > ✅  > ❌

what are min-age and max-age. Why would we need that as parameter?
--group-size rename to --history-size?

Add `dashboard_fetch_tree_report` to get tree report using dashboard API. Signed-off-by: Jeny Sadadia <[email protected]>

Fetch tree report using the dashboard API endpoint and display it. The report will consist of builds, boots, tests summary along with regression, unstable tests, and build issues information. Signed-off-by: Jeny Sadadia <[email protected]>

JenySadadia · 2025-09-09T10:22:17Z

Updated the command to include architecture/compiler information based on kernelci/dashboard#1453.
Also included build issues in the output as per the latest changes to the dashboard API.

$ kci-dev results tree-report --giturl  https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git --branch master --min-age 0 --max-age 124 --history-size 5
Tree: https://d.kernelci.org/tree/mainline/master/76eeb9b8de9880ca38696b2fb56ac45ac0a25c6c?o=maestro
Commit: 76eeb9b8de9880ca38696b2fb56ac45ac0a25c6c
Origin: maestro
Checkout start time: 2025-09-07T22:54:27.618000Z

Summary: (pass/fail/inconclusive)
builds:	53/4/0
boots:	187/3/5
tests:	12133/529/2844

Unstable tests:
Platform: dell-latitude-5400-8665U-sarien
> Config: defconfig+kcidebug+x86-board
 - Architecture/compiler: i386/gcc-12
   - Test path: boot
   last run: https://d.kernelci.org/test/maestro:68be1bbe3273d911562c64a4
   history:❌ > ❌ > ❌ > ⚠️ > ❌

Platform: meson-g12b-a311d-libretech-cc
> Config: defconfig+lab-setup+kselftest
 - Architecture/compiler: arm64/gcc-12
   - Test path: kselftest.arm64
   last run: https://d.kernelci.org/test/maestro:68be1fe13273d911562c83c7
   history:⚠️ > ✅ > ✅

Platform: mt8186-corsola-steelix-sku131072
> Config: defconfig+lab-setup+arm64-chromebook+CONFIG_MODULE_COMPRESS=n+CONFIG_MODULE_COMPRESS_NONE=y
 - Architecture/compiler: arm64/gcc-12
   - Test path: kselftest.cpufreq.suspend
   last run: https://d.kernelci.org/test/maestro:68be1a7e3273d911562c5b9d
   history:⚠️ > ✅ > ✅ > ✅ > ✅

Platform: mt8195-cherry-tomato-r3
> Config: defconfig+lab-setup+arm64-chromebook+CONFIG_MODULE_COMPRESS=n+CONFIG_MODULE_COMPRESS_NONE=y
 - Architecture/compiler: arm64/gcc-12
   - Test path: boot
   last run: https://d.kernelci.org/test/maestro:68be1a663273d911562c5ae2
   history:⚠️ > ❌ > ✅

   - Test path: boot.nfs
   last run: https://d.kernelci.org/test/maestro:68be1a6c3273d911562c5b19
   history:⚠️ > ❌ > ❌ > ❌ > ❌

Platform: sun50i-h5-libretech-all-h3-cc
> Config: defconfig+arm64-chromebook+kselftest
 - Architecture/compiler: arm64/gcc-12
   - Test path: kselftest.uevent
   last run: https://d.kernelci.org/test/maestro:68be1e3f3273d911562c70a2
   history:✅ > ❌ > ✅ > ✅ > ✅

   - Test path: kselftest.uevent.uevent_uevent_filtering
   last run: https://d.kernelci.org/test/maestro:68be330a3273d911562cd016
   history:✅ > ❌ > ✅ > ✅ > ✅

   - Test path: kselftest.uevent.uevent_uevent_filtering_global_uevent_filtering
   last run: https://d.kernelci.org/test/maestro:68be330a3273d911562cd017
   history:✅ > ❌ > ✅ > ✅ > ✅

Platform: mt8395-genio-1200-evk
> Config: defconfig+preempt_rt
 - Architecture/compiler: arm64/gcc-12
   - Test path: rt-tests.rtla-osnoise
   last run: https://d.kernelci.org/test/maestro:68be1e173273d911562c6f6a
   history:⚠️ > ✅ > ✅ > ✅ > ✅

Build issues:
- https://d.kernelci.org/issue/maestro:5ce5dc1862bf3e513dd0196750a67e49f751f73a
  Comment:  in /tmp/kci/linux/tools/testing/selftests/cachestat/test_cachestat (../lib.mk:222) [logspec:kbuild,kbuild.compiler]
  Build: https://d.kernelci.org/build/maestro:68be0d5f3273d911562c0f0a
  version: 1
  Report URL: None
  First seen: 2025-08-05T07:57:31.154863Z
  culprit_code: True

- https://d.kernelci.org/issue/maestro:5ce5dc1862bf3e513dd0196750a67e49f751f73a
  Comment:  in /tmp/kci/linux/tools/testing/selftests/cachestat/test_cachestat (../lib.mk:222) [logspec:kbuild,kbuild.compiler]
  Build: https://d.kernelci.org/build/maestro:68be0d643273d911562c0f0f
  version: 1
  Report URL: None
  First seen: 2025-08-05T07:57:31.154863Z
  culprit_code: True

- https://d.kernelci.org/issue/maestro:5ce5dc1862bf3e513dd0196750a67e49f751f73a
  Comment:  in /tmp/kci/linux/tools/testing/selftests/cachestat/test_cachestat (../lib.mk:222) [logspec:kbuild,kbuild.compiler]
  Build: https://d.kernelci.org/build/maestro:68be0d8c3273d911562c0f51
  version: 1
  Report URL: None
  First seen: 2025-08-05T07:57:31.154863Z
  culprit_code: True

- https://d.kernelci.org/issue/maestro:b521ccaba0bc36c8ba1608c5bba5120a880aa7e4
  Comment:  in /tmp/kci/linux/tools/testing/selftests/acct/acct_syscall (../lib.mk:222) [logspec:kbuild,kbuild.compiler]
  Build: https://d.kernelci.org/build/maestro:68be0dec3273d911562c113a
  version: 1
  Report URL: None
  First seen: 2024-12-01T03:39:21.550276Z
  culprit_code: True

aliceinwire · 2025-09-10T07:13:20Z

looks good for me

aliceinwire · 2025-09-10T07:16:07Z

what is history: > > > representing?
is this a check mark for passed test? not passed test will have a x ?
this should have some documentation on the kci-dev docs

I just discovered that I cannot see your emoji on my Gentoo pc

but yes having documentation would be nice anyway

aliceinwire · 2025-09-10T08:31:00Z

@padovan I would prefer if we use the Unicode version of each emoji symbol,
as not everyone can see it without the font emoji installed

JenySadadia · 2025-09-10T10:57:17Z

what is history: > > > representing? is this a check mark for passed test? not passed test will have a x ? this should have some documentation on the kci-dev docs

That is different test status over different commits for the same test.
See Status history on https://dashboard.kernelci.org/test/ti%3A56e9ad535f9a40eea2d270cb for reference.

aliceinwire · 2025-09-10T15:18:37Z

what is history: > > > representing? is this a check mark for passed test? not passed test will have a x ? this should have some documentation on the kci-dev docs

That is different test status over different commits for the same test.
See Status history on https://dashboard.kernelci.org/test/ti%3A56e9ad535f9a40eea2d270cb for reference.

I just discovered that I cannot see your emoji on my Gentoo pc

As I said in the same comment, I could not see emoji from my Gentoo laptopt as not every os have emoji installed by default

for example I see this code as this:

def status_mapping(status):
    if status == "PASS":
        return ""
    if status == "FAIL":
        return ""
    if status in ["ERROR", "SKIP", "MISS", "DONE", "NULL", None]:
        return ""

I think we should made emoji optional in the future

JenySadadia force-pushed the tree-report-cmd branch 2 times, most recently from 30de9d0 to f24f752 Compare August 19, 2025 09:58

JenySadadia marked this pull request as ready for review August 19, 2025 09:59

JenySadadia force-pushed the tree-report-cmd branch from f24f752 to 9bfd141 Compare August 19, 2025 10:11

JenySadadia requested review from padovan and aliceinwire August 19, 2025 10:11

JenySadadia force-pushed the tree-report-cmd branch 2 times, most recently from 312c653 to 24aa97a Compare September 9, 2025 10:02

Jeny Sadadia added 2 commits September 9, 2025 15:49

Implement function to request /tree-report endpoint

e47c8c3

Add `dashboard_fetch_tree_report` to get tree report using dashboard API. Signed-off-by: Jeny Sadadia <[email protected]>

Implement results tree-report command

8bfa486

Fetch tree report using the dashboard API endpoint and display it. The report will consist of builds, boots, tests summary along with regression, unstable tests, and build issues information. Signed-off-by: Jeny Sadadia <[email protected]>

JenySadadia force-pushed the tree-report-cmd branch from 24aa97a to 8bfa486 Compare September 9, 2025 10:19

aliceinwire approved these changes Sep 11, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Implement `results tree-report` command #213

Implement `results tree-report` command #213

JenySadadia commented Aug 18, 2025

Uh oh!

JenySadadia commented Aug 19, 2025

Uh oh!

JenySadadia commented Aug 19, 2025

Uh oh!

padovan commented Aug 19, 2025

Uh oh!

JenySadadia commented Sep 9, 2025 •

edited

Loading

Uh oh!

aliceinwire commented Sep 10, 2025

Uh oh!

aliceinwire commented Sep 10, 2025 •

edited

Loading

Uh oh!

aliceinwire commented Sep 10, 2025

Uh oh!

JenySadadia commented Sep 10, 2025

Uh oh!

aliceinwire commented Sep 10, 2025 •

edited

Loading

Uh oh!

Uh oh!

Implement results tree-report command #213

Are you sure you want to change the base?

Implement results tree-report command #213

Conversation

JenySadadia commented Aug 18, 2025

Uh oh!

JenySadadia commented Aug 19, 2025

Uh oh!

JenySadadia commented Aug 19, 2025

Uh oh!

padovan commented Aug 19, 2025

Uh oh!

JenySadadia commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aliceinwire commented Sep 10, 2025

Uh oh!

aliceinwire commented Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aliceinwire commented Sep 10, 2025

Uh oh!

JenySadadia commented Sep 10, 2025

Uh oh!

aliceinwire commented Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Implement `results tree-report` command #213

Implement `results tree-report` command #213

JenySadadia commented Sep 9, 2025 •

edited

Loading

aliceinwire commented Sep 10, 2025 •

edited

Loading

aliceinwire commented Sep 10, 2025 •

edited

Loading