Updated link_nvidia_host_libraries.sh for better edge case handling #922

Darkless012 · 2025-02-21T10:45:05Z

Included uncommitted remarks in #754 (review)

Script Overview:

Function definition LINES 70-585:

get_host_ldconfig - Locates the host system's ldconfig, avoiding CVMFS paths.
get_nvlib_list - Downloads or provides default list of required NVIDIA libraries.
check_global_read - Verifies if current umask allows global read access.
check_nvidia_smi_info - Checks for nvidia-smi command and extracts GPU information.
show_ld_preload - Suggests configurations for LD_PRELOAD environment for CUDA libraries
find_cuda_libraries_on_host - Check host's ldconfig, gathers library paths, and filters them on matching.
symlink_mode - Actually symlinks the Matched libraries to correct folders.

Script logic: LINES 585-650

Check EESSI is installed
Check nvidia-smi present
Parse CLI args
Run nvidia-smi to obtain CUDA info
Locate CUDA libraries on HOST (ldconfig)
- Either shows LD_PRELOAD
- Or actually symlinks CUDA libraries to correct places, where EESSI expects them.

…nd better maintainability.

eessi-bot · 2025-02-21T10:45:10Z

Instance eessi-bot-mc-aws is configured to build for:

architectures: x86_64/generic, x86_64/intel/haswell, x86_64/intel/sapphirerapids, x86_64/intel/skylake_avx512, x86_64/amd/zen2, x86_64/amd/zen3, aarch64/generic, aarch64/neoverse_n1, aarch64/neoverse_v1
repositories: eessi.io-2023.06-compat, eessi.io-2023.06-software

riscv-eessi-io-bot · 2025-02-21T10:45:11Z

Instance eessi-bot-riscv is configured to build for:

architectures: riscv64/generic
repositories: riscv.eessi.io-20240402

eessi-bot · 2025-02-21T10:45:11Z

Instance eessi-bot-mc-azure is configured to build for:

architectures: x86_64/amd/zen4
repositories: eessi.io-2023.06-software, eessi.io-2023.06-compat

gpu-bot-ugent · 2025-02-21T10:45:12Z

Instance eessi-bot-vsc-ugent is configured to build for:

architectures: x86_64/amd/zen3
repositories: eessi-hpc.org-2023.06-compat, eessi.io-2023.06-software, eessi-hpc.org-2023.06-software, eessi.io-2023.06-compat

eessi-bot-surf · 2025-02-21T10:45:12Z

Instance eessi-bot-surf is configured to build for:

architectures: x86_64/amd/zen4, x86_64/amd/zen2
repositories: eessi.io-2023.06-compat, eessi.io-2023.06-software, eessi-hpc.org-2023.06-software, eessi-hpc.org-2023.06-compat

eessi-bot-toprichard · 2025-02-21T10:45:12Z

Instance rt-Grace-jr is configured to build for:

architectures: aarch64/nvidia/grace
repositories: eessi.io-2023.06-software

scripts/gpu_support/nvidia/link_nvidia_host_libraries.sh

… commands, fixed paths

.github/workflows/tests_link_nvidia_host_libraries.yml

…SI initialization step

…lt EESSI env vars

…jections directory path from EESSI_HOST_INJECTIONS

…st_injections

…s for symlinks, added fail fast on normal run.

…exit code handling

… ldconfig and linking it

…ions

…k-nvidia-libs.sh to include all fake libraries

… tweaks

ocaisa

Couple of minor tweaks

.github/workflows/tests_link_nvidia_host_libraries.yml

scripts/gpu_support/nvidia/link_nvidia_host_libraries.sh

Co-authored-by: ocaisa <[email protected]>

.github/workflows/tests_link_nvidia_host_libraries.yml

tests/nvidia-libs/mock-nvidia-smi.sh

tests/nvidia-libs/mock-nvidia-libs.sh

Add newlines at end of files

.github/workflows/tests_link_nvidia_host_libraries.yml

ocaisa

LGTM, thanks for effort @Darkless012

ocaisa · 2025-03-18T11:35:49Z

bot: build repo:eessi.io-2023.06-software instance:eessi-bot-mc-aws arch:x86_64/amd/zen2

eessi-bot · 2025-03-18T11:35:52Z

Updates by the bot instance eessi-bot-mc-aws (click for details)

received bot command build repo:eessi.io-2023.06-software instance:eessi-bot-mc-aws arch:x86_64/amd/zen2 from ocaisa
- expanded format: build repository:eessi.io-2023.06-software instance:eessi-bot-mc-aws architecture:x86_64/amd/zen2
handling command build repository:eessi.io-2023.06-software instance:eessi-bot-mc-aws architecture:x86_64/amd/zen2 resulted in:
- submitted job 50947, for details & status see Updated link_nvidia_host_libraries.sh for better edge case handling #922 (comment)

eessi-bot · 2025-03-18T11:35:53Z

Updates by the bot instance eessi-bot-mc-azure (click for details)

received bot command build repo:eessi.io-2023.06-software instance:eessi-bot-mc-aws arch:x86_64/amd/zen2 from ocaisa
- expanded format: build repository:eessi.io-2023.06-software instance:eessi-bot-mc-aws architecture:x86_64/amd/zen2
handling command build repository:eessi.io-2023.06-software instance:eessi-bot-mc-aws architecture:x86_64/amd/zen2 resulted in:
- no jobs were submitted

eessi-bot-trz42 · 2025-03-18T11:35:53Z

Updates by the bot instance trz42-GH200-jr (click for details)

account ocaisa has NO permission to send commands to the bot

eessi-bot-toprichard · 2025-03-18T11:35:53Z

Updates by the bot instance rt-Grace-jr (click for details)

account ocaisa has NO permission to send commands to the bot

eessi-bot · 2025-03-18T11:35:57Z

New job on instance eessi-bot-mc-aws for CPU micro-architecture x86_64-amd-zen2 for repository eessi.io-2023.06-software in job dir /project/def-users/SHARED/jobs/2025.03/pr_922/50947

date	job status	comment
Mar 18 11:35:56 UTC 2025	submitted	job id `50947` awaits release by job manager
Mar 18 11:36:17 UTC 2025	released	job awaits launch by Slurm scheduler
Mar 18 11:42:22 UTC 2025	running	job `50947` is running
Mar 18 11:49:31 UTC 2025	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-50947.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-x86_64-amd-zen2-1742298171.tar.gz` size: 0 MiB (7596 bytes) entries: 1 modules under 2023.06/software/linux/x86_64/amd/zen2/modules/all no module files in tarball software under 2023.06/software/linux/x86_64/amd/zen2/software no software packages in tarball other under 2023.06/software/linux/x86_64/amd/zen2 `2023.06/scripts/gpu_support/nvidia/link_nvidia_host_libraries.sh`
Mar 18 11:49:31 UTC 2025	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ OK ] ( 1/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/29Aug2024-foss-2023b-kokkos %scale=1_node /aeb2d9df @BotBuildTests:x86_64_amd_zen2+default P: perf: 335.48 timesteps/s (r:0, l:None, u:None) [ OK ] ( 2/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/2Aug2023_update2-foss-2023a-kokkos %scale=1_node /04ff9ece @BotBuildTests:x86_64_amd_zen2+default P: perf: 445.288 timesteps/s (r:0, l:None, u:None) [ OK ] ( 3/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /775175bf @BotBuildTests:x86_64_amd_zen2+default P: latency: 1.8 us (r:0, l:None, u:None) [ OK ] ( 4/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /52707c40 @BotBuildTests:x86_64_amd_zen2+default P: latency: 2.77 us (r:0, l:None, u:None) [ OK ] ( 5/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /b1aacda9 @BotBuildTests:x86_64_amd_zen2+default P: latency: 3.8 us (r:0, l:None, u:None) [ OK ] ( 6/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /c6bad193 @BotBuildTests:x86_64_amd_zen2+default P: latency: 4.23 us (r:0, l:None, u:None) [ OK ] ( 7/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /15cad6c4 @BotBuildTests:x86_64_amd_zen2+default P: latency: 0.59 us (r:0, l:None, u:None) [ OK ] ( 8/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /6672deda @BotBuildTests:x86_64_amd_zen2+default P: latency: 2.63 us (r:0, l:None, u:None) [ OK ] ( 9/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /2a9a47b1 @BotBuildTests:x86_64_amd_zen2+default P: bandwidth: 7258.26 MB/s (r:0, l:None, u:None) [ OK ] (10/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /1b24ab8e @BotBuildTests:x86_64_amd_zen2+default P: bandwidth: 7206.68 MB/s (r:0, l:None, u:None) [ PASSED ] Ran 10/10 test case(s) from 10 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-50947.out` ✅ no message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`
Mar 18 11:52:24 UTC 2025	uploaded	transfer of `eessi-2023.06-software-linux-x86_64-amd-zen2-1742298171.tar.gz` to S3 bucket succeeded

eessi-bot-trz42 · 2025-03-18T11:52:05Z

Label bot:deploy has been set by user ocaisa, but this person does not have permission to trigger deployments

eessi-bot-toprichard · 2025-03-18T11:52:06Z

Label bot:deploy has been set by user ocaisa, but this person does not have permission to trigger deployments

ocaisa · 2025-03-18T12:05:53Z

Staging PR has been merged

eessi-bot · 2025-03-18T12:06:01Z

PR merged! Moved ['/project/def-users/SHARED/jobs/2025.03/pr_922/50947'] to /project/def-users/SHARED/trash_bin/EESSI/software-layer/2025.03.18

eessi-bot · 2025-03-18T12:06:02Z

PR merged! Moved [] to /project/def-users/SHARED/trash_bin/EESSI/software-layer/2025.03.18

Updated link_nvidia_host_libraries.sh for better edge case handling a…

5025a66

…nd better maintainability.

ocaisa reviewed Feb 24, 2025

View reviewed changes

scripts/gpu_support/nvidia/link_nvidia_host_libraries.sh Show resolved Hide resolved

Zeldhoron reviewed Feb 26, 2025

View reviewed changes

scripts/gpu_support/nvidia/link_nvidia_host_libraries.sh Outdated Show resolved Hide resolved

Darkless012 added 3 commits February 26, 2025 15:31

Fixed copy/paste error, updated logging, fixed most shellcheck issues.

abe7150

Added Github Action Tests for link_nvidia_host_libraries.sh

26ee21c

updated GHAction test for link_nvidia_host_libraries.sh - unnecessary…

70813b4

… commands, fixed paths

ocaisa requested changes Mar 10, 2025

View reviewed changes

.github/workflows/tests_link_nvidia_host_libraries.yml Show resolved Hide resolved

.github/workflows/tests_link_nvidia_host_libraries.yml Outdated Show resolved Hide resolved

Darkless012 added 16 commits March 10, 2025 10:54

updated GHAction test for link_nvidia_host_libraries.sh - changed EES…

18add4d

…SI initialization step

updated GHAction test for link_nvidia_host_libraries.sh - using defau…

a67c88e

…lt EESSI env vars

updated GHAction test for link_nvidia_host_libraries.sh - get host_in…

a0178df

…jections directory path from EESSI_HOST_INJECTIONS

updated GHAction test for link_nvidia_host_libraries.sh - updating ho…

ed2a0da

…st_injections

updated GHAction test for link_nvidia_host_libraries.sh - updating ho…

3d0ce44

…st_injections

updated GHAction test for link_nvidia_host_libraries.sh - added check…

cb70c60

…s for symlinks, added fail fast on normal run.

updated GHAction test for link_nvidia_host_libraries.sh - fix proper …

a43d6a1

…exit code handling

updated GHAction test for link_nvidia_host_libraries.sh - fix proper …

69134a9

…exit code handling

updated GHAction test for link_nvidia_host_libraries.sh - fix proper …

d9bf093

…exit code handling

updated GHAction test for link_nvidia_host_libraries.sh - fix proper …

0140f92

…exit code handling

updated GHAction test for link_nvidia_host_libraries.sh - fix mocking…

3e539d1

… ldconfig and linking it

updated GHAction test for link_nvidia_host_libraries.sh - fix permiss…

51b5f8b

…ions

updated GHAction test for link_nvidia_host_libraries.sh - updated moc…

8ea284c

…k-nvidia-libs.sh to include all fake libraries

updated GHAction test for link_nvidia_host_libraries.sh - final minor…

ebcfaa5

… tweaks

updated GHAction test for link_nvidia_host_libraries.sh - final minor…

981aa74

… tweaks

Update workflow to trigger only on specific file change.

871ee6a

Darkless012 requested a review from ocaisa March 14, 2025 07:13

ocaisa requested changes Mar 17, 2025

View reviewed changes

.github/workflows/tests_link_nvidia_host_libraries.yml Show resolved Hide resolved

.github/workflows/tests_link_nvidia_host_libraries.yml Outdated Show resolved Hide resolved

scripts/gpu_support/nvidia/link_nvidia_host_libraries.sh Outdated Show resolved Hide resolved

Darkless012 and others added 3 commits March 18, 2025 10:34

Update .github/workflows/tests_link_nvidia_host_libraries.yml

03fac99

Co-authored-by: ocaisa <[email protected]>

Update .github/workflows/tests_link_nvidia_host_libraries.yml

b2de71d

Co-authored-by: ocaisa <[email protected]>

Update scripts/gpu_support/nvidia/link_nvidia_host_libraries.sh

7a938b5

Co-authored-by: ocaisa <[email protected]>

Darkless012 requested a review from ocaisa March 18, 2025 11:12

ocaisa requested changes Mar 18, 2025

View reviewed changes

.github/workflows/tests_link_nvidia_host_libraries.yml Outdated Show resolved Hide resolved

tests/nvidia-libs/mock-nvidia-smi.sh Outdated Show resolved Hide resolved

tests/nvidia-libs/mock-nvidia-libs.sh Outdated Show resolved Hide resolved

Apply suggestions from code review

621a949

Add newlines at end of files

ocaisa reviewed Mar 18, 2025

View reviewed changes

.github/workflows/tests_link_nvidia_host_libraries.yml Outdated Show resolved Hide resolved

Update .github/workflows/tests_link_nvidia_host_libraries.yml

27d940c

ocaisa approved these changes Mar 18, 2025

View reviewed changes

ocaisa added the bot:deploy Ask bot to deploy missing software installations to EESSI label Mar 18, 2025

ocaisa merged commit d4b716a into EESSI:2023.06-software.eessi.io Mar 18, 2025
60 checks passed

Updated link_nvidia_host_libraries.sh for better edge case handling #922

Updated link_nvidia_host_libraries.sh for better edge case handling #922

Uh oh!

Conversation

Darkless012 commented Feb 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eessi-bot bot commented Feb 21, 2025

Uh oh!

riscv-eessi-io-bot bot commented Feb 21, 2025

Uh oh!

eessi-bot bot commented Feb 21, 2025

Uh oh!

gpu-bot-ugent bot commented Feb 21, 2025

Uh oh!

eessi-bot-surf bot commented Feb 21, 2025

Uh oh!

eessi-bot-toprichard bot commented Feb 21, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ocaisa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ocaisa left a comment

Choose a reason for hiding this comment

Uh oh!

ocaisa commented Mar 18, 2025

Uh oh!

eessi-bot bot commented Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eessi-bot bot commented Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eessi-bot-trz42 bot commented Mar 18, 2025

Uh oh!

eessi-bot-toprichard bot commented Mar 18, 2025

Uh oh!

eessi-bot bot commented Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eessi-bot-trz42 bot commented Mar 18, 2025

Uh oh!

eessi-bot-toprichard bot commented Mar 18, 2025

Uh oh!

ocaisa commented Mar 18, 2025

Uh oh!

Uh oh!

eessi-bot bot commented Mar 18, 2025

Uh oh!

eessi-bot bot commented Mar 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Darkless012 commented Feb 21, 2025 •

edited

Loading

eessi-bot bot commented Mar 18, 2025 •

edited

Loading

eessi-bot bot commented Mar 18, 2025 •

edited

Loading

eessi-bot bot commented Mar 18, 2025 •

edited

Loading