Avoid compiling native dependencies for common build targets

[jaecktec]

Checklist

Before making a feature request, I have:

• Searched the issues to check that this feature hasn't been requested before
• Checked the documentation to see if it is possible to do what I want already

Feature description

When installing Pact the binaries should be downloaded through an optional platform specific dependency like esbuild

Use case

Due to recent supply chain attacks a lot of developers have npm scripts globally disabled.
Second we wouldn't need to care about proxy settings in the post-install.

[mefellows]

The latest version of Pact JS (10.x.x) doesn't download anything at runtime, however it needs scripts enabled to compile native code. We previously had optional build dependencies (~4-5 years ago), this might be an option long term but is not something I'll be picking up any time soon.

If it were to be done, it would actually be in the pact-js-core project (https://github.com/pact-foundation/pact-js-core/).

[jaecktec]

ah ok.
I was also checking if we could use rust + wasm to not have a binary at all.
But I think that's a big topic.

is the compilation of the native code required?

[mefellows]

is the compilation of the native code required?

Yes, we have raw cpp files that link to the shared library. I believe we could look to precompile them and bundle or ship as optional dependencies. If that's something you'd be interested in contributing, we could definitely discuss further.

I was also checking if we could use rust + wasm to not have a binary at all.

That would be great, however unless I'm mistaken, wasm can't write files to disk or spin up mock servers - so I think it's a non starter.

[jaecktec]

Wasm could export the matcher and plugin part. This would have the benefit that (in theory) all languages could use it and just the http part would need to be implemented in the respective language.

I can look into the pre compile topic. Issue here is that the pact library is surprisingly difficult to wrap the head around, so it might take some time for me 😅

[mefellows]

Wasm could export the matcher and plugin part. This would have the benefit that (in theory) all languages could use it and just the http part would need to be implemented in the respective language.

Unfortunately there is a bit more to it than that. For example, there is also a sophisticated plugin framework that installs, discovers and communicates remotely to plugins.

I can look into the pre compile topic. Issue here is that the pact library is surprisingly difficult to wrap the head around, so it might take some time for me 😅

The key bit is in the pact-js-core. We would need to compile a version of the library for each platform we intend to deploy to (windows 64, linux 32/64/arm, mac 64/aarch64 etc.).

[riesel]

I guess, you could use https://github.com/mapbox/node-pre-gyp to distribute the binaries.

[jaecktec]

but that still means, that we have to have a post install hook

[mefellows]

I guess, you could use https://github.com/mapbox/node-pre-gyp to distribute the binaries.

That's right. I'd prefer to explore the optional dependencies approach if it could be done, but that won't work on platforms like alpine because the os/architecture combination is not enough to determine the runtime needs (for clarity, we don't officially support Alpine because dynamic linking to glibc is challenging.

[d-koppenhagen]

Hey, probably it would be good to deliver some pre-build Docker-Images containing the correct built binaries. What do you think about this? In consequence, the Images versions should always be aligned with the version of pact-js, so consumers can align their image and package.json dependency.

[mefellows]

Is the docker images for building the pact JS versions, or for users to run their Pact JS builds on?

[jaecktec]

I can't figure out how to get node-gyp to cross-compile the binaries 😖. If it's not possible I could create a github action and run it for every major architecture 🤔

[mefellows]

Yeah, I think that is probably what is going to be required. I'm almost certain you won't be able to cross-compile all future targets such Alpine, for example. In fact, I'm not sure how you'd even be able to detect what alpine package to install at runtime as the package constraints aren't that granular - so that almost certainly would have to be a post install script.

Alpine aside, it gets tricky though, because you need to only publish the top level package if all of the optional dependencies publish successfully.

I think tools like https://github.com/semantic-release/semantic-release can help with that kind of process, but it would require substantial changes to our release process (I'm not against it, but it's going to be an involved process to setup a test set of packages first to demonstrate the workflow on a test npm package).

[jaecktec]

is there a difference between alpine and 'standard' linux? esbuild seems to also not make a difference

[mefellows]

Yes, alpine uses musl not glibc

…

On Tue, 27 Sept 2022, 18:37 Constantin, ***@***.***> wrote: is there a difference between alpine and 'standard' linux? esbuild seems to also not make a difference — Reply to this email directly, view it on GitHub <#899 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAANFDHOBMB3BNH24AFQXFLWAKW23ANCNFSM54YLCLEA> . You are receiving this because you commented.Message ID: ***@***.***>