User defined memory allocator for different purposes #3316

dongsheng28849455 · 2024-04-15T02:05:52Z

Some issues that related with memory fragmentation, which may cause the linear memory cannot be allocated. In WAMR, the memory managed by the system is often trivial, but linear memory usually directly allocates a large block and often remains unchanged for a long time. Their sensitivity and contribution to fragmentation are different, which is suitable for different allocation strategies. if we can control the linear memory's allocation, do not make it from system heap, the overhead of heap management might be avoided.

which could know the memory's usage is for linear memory or not

use realloc_func to enlarge memory

move the CONFIG_WAMR_MEM_ALLOC_WITH_USAGE in nuttx

clang format the code

…g28849455/wasm-micro-runtime into feature/user_classified_mem_mng

wenyongh · 2024-04-15T10:16:45Z

@dongsheng28849455 the code adds control for the behavior of both wasm_runtime_malloc and wasm_memory_enlarge, but they are not so relevant, using a flag to control them may make the source code a little strange. I guess in your application the os_mmap uses the same allocator as the allocator of wasm_runtime_malloc, but it seems you just want to control the allocation of linear memory, so is it better to redirect os_mmap to use other allocator? Or add a flag for os_mmap?

dongsheng28849455 · 2024-04-16T01:16:33Z

hi, @wenyongh,
I used to want to modify it in os_mmap, but from the semantics of os_mmap, it seems more suitable for scenarios that require mapping virtual memory and setting properties such as rwx for it. For the allocation that only applies to linear memory, the only available attributes currently are map_prot and map_flags, which are difficult to correspond to the purpose of memory unless a new attribute type is added.

In our current application scenario, we hope to further customize memory allocators, distinguishing between linear memory and regular runtime managed memory. Compared to os_mmp with more abstraction, a customized memory allocator seems more suitable for this user specific usage scenario.

wenyongh · 2024-04-16T03:39:53Z

build-scripts/config_common.cmake

@@ -554,3 +554,6 @@ else ()
  # Disable aot intrinsics for interp, fast-jit and llvm-jit
  add_definitions (-DWASM_ENABLE_AOT_INTRINSICS=0)
 endif ()
+if (WAMR_BUILD_MEM_ALLOC_WITH_USAGE EQUAL 1)


There is already a cmake variable WAMR_BUILD_ALLOC_WITH_USER_DATA, here how about using WAMR_BUILD_ALLOC_WITH_USAGE and putting these lines together with WAMR_BUILD_ALLOC_WITH_USER_DATA?

And it seems only non shared memory is supported since the full_size_mmaped isn't handled in wasm_enlarge_memory_internal, how about adding a check here, e.g. report error if WAMR_BUILD_SHARED_MEMORY is defined.

There is already a cmake variable WAMR_BUILD_ALLOC_WITH_USER_DATA, here how about using WAMR_BUILD_ALLOC_WITH_USAGE and putting these lines together with WAMR_BUILD_ALLOC_WITH_USER_DATA?

I'd like reuse WAMR_BUILD_ALLOC_WITH_USAGE, but that seems will change the API signiture of:
static void *(*malloc_func)(void *user_data, unsigned int size) = NULL;
that will impact its user.
BTW, the user defined malloc_func, realloc_func and free_func seems only have a hidden convention with its user, can we make it more abstract and explicitly declared in wamr_export.h? then we can include all of these things in general API.

Adding WAMR_BUILD_ALLOC_WITH_USAGE is good (together with WAMR_BUILD_ALLOC_WITH_USER_DATA) to me, but since developer can just use wasm_export.h and libvmlib.a or libiwasm.so for his project, normally we should not add macro control in wasm_export.h, it is not easy to declare function prototypes for malloc_func/free_func, how about we just add comments about the prototypes in:

wasm-micro-runtime/core/iwasm/include/wasm_export.h

Lines 123 to 125 in 42199f1

void *malloc_func;

void *realloc_func;

void *free_func;

And if that, we had better update build_wamr.doc about the two cmake flags, and control the code like:

static void *(*malloc_func)( #if WASM_MEM_ALLOC_WITH_USAGE mem_alloc_usage_t usage, #endif #if WASM_MEM_ALLOC_WITH_USER_DATA void *user_data, #endif unsigned int size) = NULL;

what about this in wamr_export.h:

typedef enum { Alloc_For_Runtime, Alloc_For_LinearMemory } mem_alloc_usage_t; typedef struct { mem_alloc_usage_t usage; void *user_data; } user_data_t; typedef void* (*malloc_func_t)(const user_data_t *data, uint64_t size); typedef void* (*realloc_func_t)(const user_data_t *data, void *ptr, uint64_t size); typedef void* (*free_func_t)(const user_data_t *data, void *ptr);

and in wasm_memory.c

static void *allocator_user_data = NULL; static malloc_func_t malloc_func = NULL; static realloc_func_t realloc_func = NULL; static free_func_t free_func = NULL; ... static bool wasm_memory_init_with_allocator(malloc_func_t malloc_func_ptr, realloc_func_t realloc_func_ptr, free_func_t free_func_ptr) { if (malloc_func_ptr && free_func_ptr && malloc_func_ptr != free_func_ptr) { memory_mode = MEMORY_MODE_ALLOCATOR; malloc_func = malloc_func_ptr; realloc_func = realloc_func_ptr; free_func = free_func_ptr; return true; } LOG_ERROR("Init memory with allocator (%p, %p, %p) failed.\n", malloc_func_ptr, realloc_func_ptr, free_func_ptr); return false; }

then we can merge WASM_MEM_ALLOC_WITH_USAGE&WASM_MEM_ALLOC_WITH_USER_DATA together.

My preference is we had better not change the signature of malloc_func/realloc_func functions for general scenarios, they have been used by many developers, changing them will cause many inconvenient.

wenyongh · 2024-04-16T04:06:52Z

core/iwasm/common/wasm_memory.c

+        ret = false;
+        goto return_func;
+    }
+    memory->heap_data = memory_data_new + (heap_data_old - memory_data_old);


The heap data (base address) may be changed, should call mem_allocator_migrate for it like below code since the internal pointers (absolute addresses) inside the heap should be updated, or they will point to invalid buffer. If not, allocation by wasm app or module_malloc by host may fail.

wenyongh · 2024-04-16T04:11:14Z

product-mini/platforms/nuttx/CMakeLists.txt

+if (CONFIG_INTERPRETERS_WAMR_MEM_ALLOC_WITH_USAGE)
+  set(WAMR_BUILD_MEM_ALLOC_WITH_USAGE 1)


How about CONFIG_INTERPRETERS_WAMR_ALLOC_WITH_USAGE and WAMR_BUILD_ALLOC_WITH_USAGE?

wenyongh · 2024-04-16T04:14:13Z

hi, @wenyongh, I used to want to modify it in os_mmap, but from the semantics of os_mmap, it seems more suitable for scenarios that require mapping virtual memory and setting properties such as rwx for it. For the allocation that only applies to linear memory, the only available attributes currently are map_prot and map_flags, which are difficult to correspond to the purpose of memory unless a new attribute type is added.

In our current application scenario, we hope to further customize memory allocators, distinguishing between linear memory and regular runtime managed memory. Compared to os_mmp with more abstraction, a customized memory allocator seems more suitable for this user specific usage scenario.

OK, a concern is that the aot code and some aot data sections are also allocated by os_mmap, will they cause memory fragmentation?

dongsheng28849455 · 2024-04-16T05:25:46Z

OK, a concern is that the aot code and some aot data sections are also allocated by os_mmap, will they cause memory fragmentation?

The feature's purpose is not to eliminate the memory fragmentation, it's focus on processing linear memory separately from system normal heap. so as reduce the probability of being unable to allocate in the pool when fragmentation exists.

wenyongh · 2024-04-17T08:10:06Z

OK, a concern is that the aot code and some aot data sections are also allocated by os_mmap, will they cause memory fragmentation?

The feature's purpose is not to eliminate the memory fragmentation, it's focus on processing linear memory separately from system normal heap. so as reduce the probability of being unable to allocate in the pool when fragmentation exists.

OK, thanks.

1, WAMR_BUILD_MEM_ALLOC_WITH_USAGE -> WAMR_BUILD_ALLOC_WITH_USAGE 2, redefine the malloc, free, realloc signature. 3, let customer handle full_size_mmaped also in its realloc. 4, update the doc

wenyongh · 2024-04-18T03:53:34Z

core/iwasm/common/wasm_memory.c

+    if (!(memory_data_new =
+              realloc_func(Alloc_For_LinearMemory, full_size_mmaped,
+#if WASM_MEM_ALLOC_WITH_USER_DATA != 0
+                           NULL,


Had better use allocator_user_data?

when we handling the linear memory, this paremter is not used here, NULL just for passing build

OK, thanks.

wenyongh · 2024-04-18T03:53:54Z

core/iwasm/common/wasm_memory.c

+    (void)map_size;
+    free_func(Alloc_For_LinearMemory,
+#if WASM_MEM_ALLOC_WITH_USER_DATA != 0
+              NULL,


Same as above.

wenyongh

LGTM

wenyongh · 2024-04-18T04:14:04Z

core/iwasm/common/wasm_memory.c

+    if (!(memory_data_new =
+              realloc_func(Alloc_For_LinearMemory, full_size_mmaped,
+#if WASM_MEM_ALLOC_WITH_USER_DATA != 0
+                           NULL,


OK, thanks.

dongsheng28849455

LGTM

no1wudi · 2024-04-18T13:03:49Z

@dongsheng28849455 Could add essential document to docs instead leave them in commit message? It's not easy for end user to fetch these informations.

dongsheng28849455 · 2024-04-19T01:06:41Z

@dongsheng28849455 Could add essential document to docs instead leave them in commit message? It's not easy for end user to fetch these informations.

good point, I have added some info in the doc:
user-defined-linear-memory-allocator

…e#3316) Some issues are related with memory fragmentation, which may cause the linear memory cannot be allocated. In WAMR, the memory managed by the system is often trivial, but linear memory usually directly allocates a large block and often remains unchanged for a long time. Their sensitivity and contribution to fragmentation are different, which is suitable for different allocation strategies. If we can control the linear memory's allocation, do not make it from system heap, the overhead of heap management might be avoided. Add `mem_alloc_usage_t usage` as the first argument for user defined malloc/realloc/free functions when `WAMR_BUILD_ALLOC_WITH_USAGE` cmake variable is set as 1, and make passing `Alloc_For_LinearMemory` to the argument when allocating the linear memory.

…e#3316) Some issues are related with memory fragmentation, which may cause the linear memory cannot be allocated. In WAMR, the memory managed by the system is often trivial, but linear memory usually directly allocates a large block and often remains unchanged for a long time. Their sensitivity and contribution to fragmentation are different, which is suitable for different allocation strategies. If we can control the linear memory's allocation, do not make it from system heap, the overhead of heap management might be avoided. Add `mem_alloc_usage_t usage` as the first argument for user defined malloc/realloc/free functions when `WAMR_BUILD_ALLOC_WITH_USAGE` cmake variable is set as 1, and make passing `Alloc_For_LinearMemory` to the argument when allocating the linear memory. Signed-off-by: victoryang00 <[email protected]>

…e#3316) Some issues are related with memory fragmentation, which may cause the linear memory cannot be allocated. In WAMR, the memory managed by the system is often trivial, but linear memory usually directly allocates a large block and often remains unchanged for a long time. Their sensitivity and contribution to fragmentation are different, which is suitable for different allocation strategies. If we can control the linear memory's allocation, do not make it from system heap, the overhead of heap management might be avoided. Add `mem_alloc_usage_t usage` as the first argument for user defined malloc/realloc/free functions when `WAMR_BUILD_ALLOC_WITH_USAGE` cmake variable is set as 1, and make passing `Alloc_For_LinearMemory` to the argument when allocating the linear memory.

…e#3316) Some issues are related with memory fragmentation, which may cause the linear memory cannot be allocated. In WAMR, the memory managed by the system is often trivial, but linear memory usually directly allocates a large block and often remains unchanged for a long time. Their sensitivity and contribution to fragmentation are different, which is suitable for different allocation strategies. If we can control the linear memory's allocation, do not make it from system heap, the overhead of heap management might be avoided. Add `mem_alloc_usage_t usage` as the first argument for user defined malloc/realloc/free functions when `WAMR_BUILD_ALLOC_WITH_USAGE` cmake variable is set as 1, and make passing `Alloc_For_LinearMemory` to the argument when allocating the linear memory. Signed-off-by: victoryang00 <[email protected]>

dongsheng28849455 and others added 8 commits April 8, 2024 14:49

use WASM_MEM_ALLOC_WITH_USAGE option define custom allocator

e274d83

which could know the memory's usage is for linear memory or not

use realloc_func to enlarge memory

bcb5cde

use realloc_func to enlarge memory

move the CONFIG_WAMR_MEM_ALLOC_WITH_USAGE in nuttx

9d01595

move the CONFIG_WAMR_MEM_ALLOC_WITH_USAGE in nuttx

clang format the code

468aa43

clang format the code

add WAMR_MEM_ALLOC_WITH_USAGE on nuttx platform

20debc2

Merge branch 'feature/user_classified_mem_mng' of github.com:dongshen…

024340a

…g28849455/wasm-micro-runtime into feature/user_classified_mem_mng

revert incorrect string definition

8881082

rename WAMR_MEM_ALLOC_WITH_USAGE to WAMR_BUILD_MEM_ALLOC_WITH_USAGE

21ca534

wenyongh reviewed Apr 16, 2024

View reviewed changes

add WASM_MEM_ALLOC_WITH_USAGE in wamr.mk

1ec2f5b

update following the reivew comments

25c78af

1, WAMR_BUILD_MEM_ALLOC_WITH_USAGE -> WAMR_BUILD_ALLOC_WITH_USAGE 2, redefine the malloc, free, realloc signature. 3, let customer handle full_size_mmaped also in its realloc. 4, update the doc

wenyongh reviewed Apr 18, 2024

View reviewed changes

wenyongh approved these changes Apr 18, 2024

View reviewed changes

Merge branch 'main' into feature/user_classified_mem_mng

acce5b7

dongsheng28849455 commented Apr 18, 2024

View reviewed changes

wenyongh merged commit ba59e56 into bytecodealliance:main Apr 18, 2024

This was referenced Jan 21, 2025

Missing UserData in LinearMemory for custom allocator #4024

Closed

Update memory allocation functions to use allocator user data #4043

Merged

		if (CONFIG_INTERPRETERS_WAMR_MEM_ALLOC_WITH_USAGE)
		set(WAMR_BUILD_MEM_ALLOC_WITH_USAGE 1)

User defined memory allocator for different purposes #3316

User defined memory allocator for different purposes #3316

Uh oh!

Conversation

dongsheng28849455 commented Apr 15, 2024

Uh oh!

wenyongh commented Apr 15, 2024

Uh oh!

dongsheng28849455 commented Apr 16, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wenyongh commented Apr 16, 2024

Uh oh!

dongsheng28849455 commented Apr 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wenyongh commented Apr 17, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wenyongh left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dongsheng28849455 left a comment

Choose a reason for hiding this comment

Uh oh!

no1wudi commented Apr 18, 2024

Uh oh!

dongsheng28849455 commented Apr 19, 2024

Uh oh!

Uh oh!

dongsheng28849455 commented Apr 16, 2024 •

edited

Loading