What do we consider to be a Single Executable Application?

[RaisinTen]

Does it have to be the entire application code, including native components, all bundled together into one single binary file or is it fine if we produce a main executable that contains all the JS code and place things like native addons as separate files next to it?

This was getting discussed in a thread in a minutes doc addition pr in https://github.com/nodejs/single-executable/pull/31/files#r963027072 where it might get lost very easily, so I think it's worth discussing separately in here.

[ljharb]

I think that it needs to be a single lone file, with nothing alongside.

[jviotti]

Just to bring back some of the bits from the original discussion: while I agree a single file would be fine, I don't think its going to be realistic for every Node.js application, specifically the ones shipping native add-ons on platforms like Windows. In the latter case, we have pretty hard constrains from the Windows operating system that would make single-file applications not work in many cases.

If that's a constrain we can't move, then I don't think we will be able to embrace the single line file approach. Most applications would still be a single file, but there will be cases where that will not be possible. I think we rather accommodate our definition a bit than leave out a set of realistic applications out of the picture.

[guymguym]

Hi. I agree with both @ljharb and @jviotti's points.

I think it should be a single file that can include the addons statically in the binary...

EDIT: I moved the rest of this comment to the other thread - #34 (reply in thread)

[jviotti]

Here is my proposal:

• Most SEA applications will indeed consist of a single file and nothing else
• There are cases where we will need, for practical reasons, to ship certain resource files alongside the binary (like native add-ons). For this reason, an application with a main executable binary + binary resources alongside it still qualifies as an SEA
• An application consisting of a binary executable + textual non-executable resources (like JavaScript files) does NOT qualify as an SEA (as otherwise any Node.js app would be an SEA)

[RaisinTen]

AFAICT, in this approach, the SEA will not truly statically link to the addon because without recompilation, the executable wouldn't have any unresolved symbols for the code that's present inside the static library, so I'm not sure how it would be possible for the executable to call the functions that are inside the static library.

[guymguym]

I believe this is done with weak symbols that the main executable needs to define and check for.
If another static library will define that symbol, it will be used.
https://stackoverflow.com/questions/23079997/override-weak-symbols-in-static-library

[guymguym]

this is a similar fashion to how the main script is optionally loaded by the executable if present.

[guymguym]

but when I think of it - is there really any symbol that needs to be exposed from an addon to the executable? when we create an addon we only initialize it statically by calling the NODE_MODULE macro -

void Initialize(Local<Object> exports);
NODE_MODULE(NODE_GYP_MODULE_NAME, Initialize)

wouldn't that code be run on init the same when added into an executable statically?

[RaisinTen]

I did some digging (macOS only) to understand how Node.js loads addons and this is what I found:

I pasted the contents of the hello.cc file in the example in https://nodejs.org/api/addons.html#hello-world and just ran the preprocessor and this is the relevant bit:

$ clang++ -E hello.cc -I ~/Desktop/git/node/src -I ~/Desktop/git/node/deps/v8/include > preprocessed.cc

preprocessed.cc

...

typedef void (*addon_register_func)(
    v8::Local<v8::Object> exports,
    v8::Local<v8::Value> module,
    void* priv);

...

struct node_module {
  int nm_version;
  unsigned int nm_flags;
  void* nm_dso_handle;
  const char* nm_filename;
  node::addon_register_func nm_register_func;
  node::addon_context_register_func nm_context_register_func;
  const char* nm_modname;
  void* nm_priv;
  struct node_module* nm_link;
};

...

extern "C" __attribute__((visibility("default"))) void node_module_register(void* mod);

...

namespace demo {

...

void Method(const FunctionCallbackInfo<Value>& args) {
  ...
}

void Initialize(Local<Object> exports) {
  node::NODE_SET_METHOD(exports, "hello", Method);
}

extern "C" { static node::node_module _module = { 115, 0, __null, "hello.cc", (node::addon_register_func) (Initialize), __null, "NODE_GYP_MODULE_NAME", __null, __null }; static void _register_NODE_GYP_MODULE_NAME(void) __attribute__((constructor)); static void _register_NODE_GYP_MODULE_NAME(void) { node_module_register(&_module); } }

}

So the way Node.js loads addons is:

1. When the CJS loaders tries to load a .node file, it calls process.dlopen() on that - https://github.com/nodejs/node/blob/de5787c87c557e02523b3e6104445c6427a44f25/lib/internal/modules/cjs/loader.js#L1345-L1354
2. That process.dlopen implementation (DLOpen) in https://github.com/nodejs/node/blob/de5787c87c557e02523b3e6104445c6427a44f25/src/node_binding.cc#L453, calls the actual dlopen() function to load the shared library.
3. When the library gets loaded, macOS' dyld calls the _register_NODE_GYP_MODULE_NAME function because it is marked with __attribute__((constructor)).
4. _register_NODE_GYP_MODULE_NAME calls node_module_register, which is an extern symbol on the Node.js binary - https://github.com/nodejs/node/blob/de5787c87c557e02523b3e6104445c6427a44f25/src/node_binding.cc#L254-L269, and it registers the module.
5. And then finally in the DLOpen code, Node.js calls the nm_register_func function on the node_module struct of the add-on module, i.e., the Initialize function - https://github.com/nodejs/node/blob/de5787c87c557e02523b3e6104445c6427a44f25/src/node_binding.cc#L534.

So as we can see here, the actual entry point of the add-on is the static void _register_NODE_GYP_MODULE_NAME(void) __attribute__((constructor)) function that gets called by macOS' dyld when Node.js calls dlopen to load the addon.

@guymguym could you explain which symbols here should be turned into weak symbols for your approach to work? Also, what's the order of execution you have in mind?

[arcanis]

I think it'd be nice if they could encapsulate full applications, but also that it's likely an unreasonable primary constraint.

• Native arbitrary code cannot be portably executed
• Extracting archives at execution time has significant drawbacks and shouldn't be Node.js' responsibility
• Wasm can be stored and executed from arbitrary buffers, offering an alternative to some native use cases

[jviotti]

What do you mean by full applications?

Native arbitrary code cannot be portably executed

Are you talking about a single package that works across OSes?

[arcanis]

By full app I meant JS code + native libraries.

Are you talking about a single package that works across OSes?

No, just that I'm not aware of portable ways to inject shared libraries in a process from memory. Might be wrong tho, in which case it'd be perfect, but if we have doubts it's probably shouldn't be considered a blocker requirement.

[jviotti]

No, just that I'm not aware of portable ways to inject shared libraries in a process from memory. Might be wrong tho, in which case it'd be perfect, but if we have doubts it's probably shouldn't be considered a blocker requirement.

Yeah, that's my understanding as well.

[rzl]

i want produce a main executable that contains all the JS code and place things like native addons as separate files next to it

1 Not special dependencies, such as. node, can be packaged into a single file for injection and running using tools such as webpack and crollup

2 Most of the applications I have used consist of multiple files, The sea that I have used the most may be the program installation package, and after they are installed, they will also generate many files

3 I think that after sea processing, zip all the files and then copy them to another machine without a node environment installed. After decompression, they can run directly, which is suitable for most scenarios

4 java -jar hello.jar node -nar hello.nar