[SWDEV-563823][Compiler-rt][ASan] Simplify API Logic 'asan_hsa_amd_ipc_memory_create'. #475

ampandey-AMD · 2025-11-03T16:31:21Z

Use reinterpret_cast for pointer arithmetic.
Add sanitizer interception logic for api 'hsa_amd_pointer_info'.
Allow only valid values of ptr and len in non-ASan mode.
- ptr == Actual agentBaseAddress && len ==
  original_len_used_in_alloc
Allow only valid values of ptr and len in ASan mode.
- Here pinfo is retrieved from external hsa_amd_pointer_info(not
  internal device allocator function AmdgpuMemFuncs::GetPointerInfo)
  - ptr == pinfo.agentBaseAddress && len == pinfo.sizeInBytes
- ptr == original_ptr_returned_by_ASAN && len ==
  original_len_used_in_alloc

z1-cciauto · 2025-11-03T16:32:06Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2637

z1-cciauto · 2025-11-03T16:34:34Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2638

z1-cciauto · 2025-11-03T16:42:30Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2639

b-sumner · 2025-11-03T17:45:36Z

Does the test application in the ticket print the same "base" "len" and "end" as the uninstrumented version (taking into account address randomization)?

ampandey-AMD · 2025-11-04T09:46:48Z

Does the test application in the ticket print the same "base" "len" and "end" as the uninstrumented version (taking into account address randomization)?

Yes. If we include the current fix with/without intercepting hsa_amd_pointer_info.

Non-Instrumented O/P

hsa_amd_pointer_info: test_buff 0x7c947a815000 base 0x7c947a814000 len 16384 end 0x7c947a818000
hsa_amd_ipc_memory_create: handle: [7a814000][7c94][395301][0][4][0][0][5325e42b]

Instrumented O/P

hsa_amd_pointer_info: test_buff 0x7ef7c9509000 base 0x7ef7c9508000 len 16384 end 0x7ef7c950c000
hsa_amd_ipc_memory_create: handle: [c9508000][7ef7][394f4c][0][4][0][0][3a761d78]

b-sumner · 2025-11-04T15:23:06Z

But the output in the ticket, when -fsanitize=address is not used is:

hsa_amd_pointer_info: test_buff 0x7f3df2400000 base 0x7f3df2400000 len 1024 end 0x7f3df2400400

Here the buffer and the base are the same, and len is 1024. Shouldn't our intercepted version of hsa_amd_pointer_info return the same buffer and base, and a len of 1024 instead of 16384?

ampandey-AMD · 2025-11-04T18:02:28Z

But the output in the ticket, when -fsanitize=address is not used is:

hsa_amd_pointer_info: test_buff 0x7f3df2400000 base 0x7f3df2400000 len 1024 end 0x7f3df2400400

Here the buffer and the base are the same, and len is 1024. Shouldn't our intercepted version of hsa_amd_pointer_info return the same buffer and base, and a len of 1024 instead of 16384?

Hi @b-sumner , Ok in my previous o/p, the problem was because of LD_LIBRARY_PATH set to asan instrumented libraries and I compiled non-instrumented o/p without flags -fsanitize=address -shared-libsan. So my executable was actually linked to asan instrumented libraries of rocr and hipamd.

If I avoid adding the /opt/rocm/llvm/lib/asan or /opt/rocm/lib/asan from my LD_LIBRARY_PATH, then I am getting this non-instrumented O/P

hsa_amd_pointer_info: test_buff 0x7b6a28800000 base 0x7b6a28800000 len 1024 end 0x7b6a28800400
hsa_amd_ipc_memory_create: handle: [28800000][7b6a][1feb9][0][4][0][80000000][3d9ec7b6]

I think instrumentation here is not kind of problem, we enable the asan interception of those hsa_amd* api calls in the asanified libraries only as our code is bounded by preprocessor macro SANITIZER_AMDGPU. We actually don't try to instrument the hsa_api* calls itself.

ampandey-AMD · 2025-11-04T18:27:16Z

Hi @b-sumner,
So I tested locally by removing the asan interception code of hsa_amd_pointer_info.

Here are the results without asan_hsa_amd_pointer_info implemented.

Instrumented (ASan enabled via -fsanitize=address -shared-libsan with LD_LIBRARY_PATH set to path/to/lib/asan)

hsa_amd_pointer_info: test_buff 0x75863cea5000 base 0x75863cea4000 len 16384 end 0x75863cea8000
hsa_amd_ipc_memory_create: handle: [3cea4000][7586][22d9b][0][4][0][0][51fdf52f]

Non-Instrumented (ASan disabled with LD_LIBRARY_PATH not set to /path/to/lib/asan)

hsa_amd_pointer_info: test_buff 0x74bd90c00000 base 0x74bd90c00000 len 1024 end 0x74bd90c00400
hsa_amd_ipc_memory_create: handle: [90c00000][74bd][22cd4][0][4][0][80000000][3a14da89]

b-sumner · 2025-11-04T18:44:58Z

Thanks! My thinking is that in order to provide acceptable information to our interception of hsa_amd_ipc_memory_create, our interception of hsa_amd_pointer_info should return the same base, len, and end as observed when the application is not built with instrumentation (or running instrumented libraries). But I would like to hear @bing-ma's take on this.

z1-cciauto · 2025-11-05T13:12:43Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2664

z1-cciauto · 2025-11-05T13:41:15Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2665

z1-cciauto · 2025-11-05T13:45:12Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2666

z1-cciauto · 2025-11-06T05:41:38Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2678

z1-cciauto · 2025-11-06T06:43:46Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2679

z1-cciauto · 2025-11-06T06:46:09Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2680

compiler-rt/lib/asan/asan_allocator.cpp

z1-cciauto · 2025-11-07T04:30:24Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2709

z1-cciauto · 2025-11-07T04:32:44Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2710

compiler-rt/lib/asan/asan_allocator.cpp

z1-cciauto · 2025-11-08T18:03:18Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2737

compiler-rt/lib/asan/asan_allocator.cpp

z1-cciauto · 2025-11-14T05:38:32Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2804

z1-cciauto · 2025-11-14T05:40:58Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2805

compiler-rt/lib/asan/asan_allocator.cpp

z1-cciauto · 2025-11-20T09:32:17Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2894

'asan_hsa_amd_ipc_memory_create'. - Use reinterpret_cast<uptr> for pointer arithmetic. - Add sanitizer interception logic for api 'hsa_amd_pointer_info'. - Allow only valid values of ptr and len in non-ASan mode. - ptr == Actual agentBaseAddress && len == original_len_used_in_alloc - Allow only valid values of ptr and len in ASan mode. - Here pinfo is retrieved from external hsa_amd_pointer_info(not internal device allocator function AmdgpuMemFuncs::GetPointerInfo) - ptr == pinfo.agentBaseAddress && len == pinfo.sizeInBytes - ptr == original_ptr_returned_by_ASAN && len == original_len_used_in_alloc

Improve logic of asan_hsa_amd_ipc_memory_create based on observations.

suggestions. This commit again adds the interception of 'hsa_amd_pointer_info' call.

- Changes to 'asan_hsa_amd_pointer_info' - Initialize status with default value 'HSA_STATUS_ERROR_NOT_INITIALIZE'.

This reverts commit 00898ca.

- Changes to 'asan_hsa_amd_pointer_info' - Initialize status with default value 'HSA_STATUS_ERROR_NOT_INITIALIZE'. - Assure by adding ternary condition to check if ptr_ is not nullptr when calling GetASanChunkByAddr.

Fix Typo.

z1-cciauto · 2025-11-20T09:56:36Z

PSDB Link: https://compiler-ci.amd.com/job/compiler-psdb-amd-staging/2895

b-sumner · 2025-11-20T15:31:58Z

compiler-rt/lib/asan/asan_allocator.cpp

+                     : nullptr;
+  hsa_status_t status = HSA_STATUS_ERROR_NOT_INITIALIZED;
+  if (ptr_ && m) {
+    status = REAL(hsa_amd_pointer_info)(ptr, info, alloc, num_agents_accessible,


Should we use ptr_ here instead of ptr? It might be a bit clearer to do so.

ampandey-AMD force-pushed the fix-swdev-563823-ipc-handles branch from 971d852 to 0c173eb Compare November 3, 2025 16:32

ampandey-AMD requested review from b-sumner and bing-ma November 3, 2025 16:36

ampandey-AMD force-pushed the fix-swdev-563823-ipc-handles branch from 0c173eb to c1765c0 Compare November 3, 2025 16:40

ampandey-AMD force-pushed the fix-swdev-563823-ipc-handles branch from c1765c0 to 0bb451f Compare November 5, 2025 13:11

ampandey-AMD force-pushed the fix-swdev-563823-ipc-handles branch from 0bb451f to 9bc5e55 Compare November 5, 2025 13:39

ampandey-AMD changed the title ~~[SWDEV-563823][Compiler-rt][ASan] Intercept API 'hsa_amd_pointer_info'.~~ [SWDEV-563823][Compiler-rt][ASan] Simplify API Logic 'asan_hsa_amd_ipc_memory_create'. Nov 5, 2025

ampandey-AMD force-pushed the fix-swdev-563823-ipc-handles branch from 9bc5e55 to 339e178 Compare November 5, 2025 13:44

ampandey-AMD force-pushed the fix-swdev-563823-ipc-handles branch from 69a471e to ee235e3 Compare November 6, 2025 06:42

ampandey-AMD force-pushed the fix-swdev-563823-ipc-handles branch from ee235e3 to 8681926 Compare November 6, 2025 06:44

b-sumner reviewed Nov 6, 2025

View reviewed changes

compiler-rt/lib/asan/asan_allocator.cpp Outdated Show resolved Hide resolved

ampandey-AMD force-pushed the fix-swdev-563823-ipc-handles branch from afae2e3 to 619e575 Compare November 7, 2025 04:31

b-sumner reviewed Nov 7, 2025

View reviewed changes

compiler-rt/lib/asan/asan_allocator.cpp Outdated Show resolved Hide resolved

b-sumner reviewed Nov 12, 2025

View reviewed changes

compiler-rt/lib/asan/asan_allocator.cpp Outdated Show resolved Hide resolved

ampandey-AMD force-pushed the fix-swdev-563823-ipc-handles branch from 53c9dda to bed99f8 Compare November 14, 2025 05:39

b-sumner reviewed Nov 14, 2025

View reviewed changes

compiler-rt/lib/asan/asan_allocator.cpp Outdated Show resolved Hide resolved

ampandey-AMD force-pushed the fix-swdev-563823-ipc-handles branch from bed99f8 to c9f7d0d Compare November 20, 2025 09:29

ampandey-AMD added 8 commits November 20, 2025 15:07

Address comments of @bing-ma.

14df989

Improve logic of asan_hsa_amd_ipc_memory_create based on observations.

Improve ASan implementations of hsa api calls based on @bing-ma

1a25efe

suggestions. This commit again adds the interception of 'hsa_amd_pointer_info' call.

Address @b-sumner comments.

c67a73b

Improve code for edge cases testing.

1f9b9c8

- Changes to 'asan_hsa_amd_pointer_info' - Initialize status with default value 'HSA_STATUS_ERROR_NOT_INITIALIZE'.

Revert "Improve code for edge cases testing."

20bc0db

This reverts commit 00898ca.

Improve code for edge cases testing.

601f453

- Changes to 'asan_hsa_amd_pointer_info' - Initialize status with default value 'HSA_STATUS_ERROR_NOT_INITIALIZE'. - Assure by adding ternary condition to check if ptr_ is not nullptr when calling GetASanChunkByAddr.

Address comments of @b-sumner.

eed8503

Fix Typo.

ampandey-AMD force-pushed the fix-swdev-563823-ipc-handles branch from c9f7d0d to eed8503 Compare November 20, 2025 09:55

b-sumner reviewed Nov 20, 2025

View reviewed changes

[SWDEV-563823][Compiler-rt][ASan] Simplify API Logic 'asan_hsa_amd_ipc_memory_create'. #475

Are you sure you want to change the base?

[SWDEV-563823][Compiler-rt][ASan] Simplify API Logic 'asan_hsa_amd_ipc_memory_create'. #475

Uh oh!

Conversation

ampandey-AMD commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

z1-cciauto commented Nov 3, 2025

Uh oh!

z1-cciauto commented Nov 3, 2025

Uh oh!

z1-cciauto commented Nov 3, 2025

Uh oh!

b-sumner commented Nov 3, 2025

Uh oh!

ampandey-AMD commented Nov 4, 2025

Non-Instrumented O/P

Instrumented O/P

Uh oh!

b-sumner commented Nov 4, 2025

Uh oh!

ampandey-AMD commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ampandey-AMD commented Nov 4, 2025

Instrumented (ASan enabled via -fsanitize=address -shared-libsan with LD_LIBRARY_PATH set to path/to/lib/asan)

Non-Instrumented (ASan disabled with LD_LIBRARY_PATH not set to /path/to/lib/asan)

Uh oh!

b-sumner commented Nov 4, 2025

Uh oh!

z1-cciauto commented Nov 5, 2025

Uh oh!

z1-cciauto commented Nov 5, 2025

Uh oh!

z1-cciauto commented Nov 5, 2025

Uh oh!

z1-cciauto commented Nov 6, 2025

Uh oh!

z1-cciauto commented Nov 6, 2025

Uh oh!

z1-cciauto commented Nov 6, 2025

Uh oh!

Uh oh!

z1-cciauto commented Nov 7, 2025

Uh oh!

z1-cciauto commented Nov 7, 2025

Uh oh!

Uh oh!

z1-cciauto commented Nov 8, 2025

Uh oh!

Uh oh!

z1-cciauto commented Nov 14, 2025

Uh oh!

z1-cciauto commented Nov 14, 2025

Uh oh!

Uh oh!

z1-cciauto commented Nov 20, 2025

Uh oh!

z1-cciauto commented Nov 20, 2025

Uh oh!

b-sumner Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ampandey-AMD commented Nov 3, 2025 •

edited

Loading

ampandey-AMD commented Nov 4, 2025 •

edited

Loading