[vm] Usage of dart_native_api.h requires linking dlls to either dart.exe or dart_precompiled_runtime.exe

[dcharkes]

On Windows, using dart_native_api.h requires statically linking dlls to dart.exe or dart_precompiled_runtime.exe.

  if (is_win) {
    libs = [ "dart.lib" ]
    abs_root_out_dir = rebase_path(root_out_dir)
    ldflags = [ "/LIBPATH:$abs_root_out_dir" ]
  }

sdk/sdk/BUILD.gn

Line 38 in 76b18c4

# ......dart.lib (import library for VM native extensions on Windows)

All occurrences in our code base:

sdk/runtime/bin/BUILD.gn

Lines 1061 to 1183 in 76b18c4

	shared_library("test_extension") {
	deps = [
	":dart",
	]
	sources = [
	"test_extension.c",
	"test_extension_dllmain_win.cc",
	]
	include_dirs = [ ".." ]
	defines = [
	# The only effect of DART_SHARED_LIB is to export the Dart API.
	"DART_SHARED_LIB",
	]
	if (is_linux || is_android) {
	cflags = [ "-fPIC" ]
	}
	if (is_win) {
	libs = [ "dart.lib" ]
	abs_root_out_dir = rebase_path(root_out_dir)
	ldflags = [ "/LIBPATH:$abs_root_out_dir" ]
	}
	}
	shared_library("entrypoints_verification_test_extension") {
	deps = [
	":dart",
	]
	sources = [
	"entrypoints_verification_test_extension.cc",
	"entrypoints_verification_test_extension_dllmain_win.cc",
	]
	include_dirs = [ ".." ]
	defines = [
	# The only effect of DART_SHARED_LIB is to export the Dart API.
	"DART_SHARED_LIB",
	]
	if (is_linux || is_android) {
	cflags = [ "-fPIC" ]
	}
	if (is_win) {
	libs = [ "dart.lib" ]
	abs_root_out_dir = rebase_path(root_out_dir)
	ldflags = [ "/LIBPATH:$abs_root_out_dir" ]
	}
	}
	shared_library("ffi_test_dynamic_library") {
	deps = [
	":dart",
	]
	sources = [
	"ffi_test/ffi_test_dynamic_library.cc",
	]
	include_dirs = [ ".." ]
	defines = [
	# The only effect of DART_SHARED_LIB is to export the Dart API.
	"DART_SHARED_LIB",
	]
	if (is_linux || is_android) {
	cflags = [ "-fPIC" ]
	}
	if (is_win) {
	libs = [ "dart.lib" ]
	abs_root_out_dir = rebase_path(root_out_dir)
	ldflags = [ "/LIBPATH:$abs_root_out_dir" ]
	}
	}
	shared_library("ffi_test_functions") {
	deps = [
	":dart",
	]
	# The two files here do not depend on each other.
	# flutter/flutter integration tests will only use `ffi_test_functions.cc` -
	# any test functionality using `dart_api.h` has to go into
	# `ffi_test_functions_vmspecific.cc`.
	sources = [
	"ffi_test/ffi_test_functions.cc",
	"ffi_test/ffi_test_functions_vmspecific.cc",
	]
	if (is_win && current_cpu == "x64") {
	sources += [ "ffi_test/clobber_x64_win.S" ]
	} else if (!is_win) {
	sources += [ "ffi_test/clobber_$current_cpu.S" ]
	}
	include_dirs = [ ".." ]
	defines = [
	# The only effect of DART_SHARED_LIB is to export the Dart API.
	"DART_SHARED_LIB",
	]
	if (is_linux || is_android) {
	cflags = [ "-fPIC" ]
	}
	if (is_win) {
	libs = [ "dart.lib" ]
	abs_root_out_dir = rebase_path(root_out_dir)
	ldflags = [ "/LIBPATH:$abs_root_out_dir" ]
	}
	}
	shared_library("sample_extension") {
	deps = [
	":dart",
	]
	sources = [
	"../../samples/sample_extension/sample_extension.cc",
	"../../samples/sample_extension/sample_extension_dllmain_win.cc",
	]
	include_dirs = [ ".." ]
	defines = [
	# The only effect of DART_SHARED_LIB is to export the Dart API.
	"DART_SHARED_LIB",
	]
	if (is_linux || is_android) {
	cflags = [ "-fPIC" ]
	}
	if (is_win) {
	libs = [ "dart.lib" ]
	abs_root_out_dir = rebase_path(root_out_dir)
	ldflags = [ "/LIBPATH:$abs_root_out_dir" ]
	}
	}

A workaround is making different DLLs for using in JIT in AOT, but this requires exposing whether Dart is running in JIT or AOT in dart:ffi to load the right DLL.

A better solution would be to have the Dart runtime as a separate DLL which would be shared between JIT and AOT.

Issue discovered via #40564.

Error without statically linking an executable:

LINK(DLL) ffi_test_functions.dll ffi_test_functions.dll.lib
FAILED: ffi_test_functions.dll ffi_test_functions.dll.lib
C:/src/depot_tools/bootstrap-3_8_0_chromium_8_bin/python/bin/python.exe ../../build/toolchain/win/tool_wrapper.py link-wrapper environment.x64 False link.exe /nologo /IMPLIB:./ffi_test_functions.dll.lib /DLL /OUT:./ffi_test_functions.dll /PDB:./ffi_test_functions.dll.pdb @./ffi_test_functions.dll.rsp
ffi_test_functions.ffi_test_functions_vmspecific.obj : error LNK2019: unresolved external symbol Dart_PostCObject referenced in function "void __cdecl dart::NotifyDart(__int64,class std::function<void __cdecl(void)> const *)" (?NotifyDart@dart@@YAX_JPEBV?$function@$$A6AXXZ@std@@@Z)
ffi_test_functions.ffi_test_functions_vmspecific.obj : error LNK2019: unresolved external symbol Dart_ExecuteInternalCommand referenced in function CollectOnNthAllocation
./ffi_test_functions.dll : fatal error LNK1120: 2 unresolved externals

cc @mraleph @mkustermann @askeksa-google

[dcharkes]

@askeksa-google idea:

1. Make a dart_api.lib which contains the symbols, but non exported.
2. These symbols are implemented with GetModuleHandle(nullptr) and look up the real symbol in the executable.
3. Both when using dart.exe and dart_precompiled_runtime.exe, dlls can statically link against dart_api.lib.

[mkustermann]

Approved crashes of ffi/ffi_native_test/none on windows. Quite positive it's due to this linking issue (like other tests as well that use dart_native_api.h.

/cc @cskau-g

[benfgit]

What are the downsides of this mechanism?

• Use Pointer.fromFunction() to get pointer of a Dart function.
• Register a callback on the native code using the function pointer from the previous step.
• Listen for a spesific event and invoke the callback with the event data.
• On the Dart side, add the event data into a Dart Stream.

This solution does not use Dart_PostCObject from dart_native_api.h and I would like to know if this has any disadvantages over Dart_PostCObject.

[mraleph]

@easeccy Please don't post off-topic questions on random unrelated issues and definitely don't post the same question multiple times. There are other channels for such inquiries (e.g. mailing lists, discord, etc). To answer you question: Dart_PostCObject allows (asynchronous) communication between different threads, while callbacks can only be used on a thread which has a Dart isolate associated with it (in other words they are only suitable for synchronous communication).