Building nightly node 12 on alpine is broken since 2018-12-06 (alpine linux)

[lsmoura]

I've been building node12 nightly to create alpine linux docker images since December 1st using the source packages available at the nodejs website.

Ever since release nightly20181207cbf0e5a1f4, the build fails with the same error:

  g++ -o /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.host/v8_libbase/deps/v8/src/base/platform/platform-posix-time.o ../deps/v8/src/base/platform/platform-posix-time.cc '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-DV8_GYP_BUILD' '-DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=0' '-DOPENSSL_THREADS' '-DV8_TARGET_ARCH_X64' '-DV8_EMBEDDER_STRING="-node.5"' '-DENABLE_DISASSEMBLER' '-DV8_PROMISE_INTERNAL_FIELD_COUNT=1' '-DV8_INTL_SUPPORT' '-DV8_CONCURRENT_MARKING' '-DDISABLE_UNTRUSTED_CODE_MITIGATIONS' '-DV8_EMBEDDED_BUILTINS' '-DV8_EMBEDDED_BYTECODE_HANDLERS' -I../deps/v8  -pthread -Wall -Wextra -Wno-unused-parameter -m64 -fno-strict-aliasing -m64 -O3 -fno-omit-frame-pointer -fdata-sections -ffunction-sections -O3 -fno-rtti -fno-exceptions -std=gnu++1y -MMD -MF /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/.deps//build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.host/v8_libbase/deps/v8/src/base/platform/platform-posix-time.o.d.raw   -c
  LD_LIBRARY_PATH=/build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/lib.host:/build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../deps/v8/gypfiles; mkdir -p /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen; python ../tools/gen-postmortem-metadata.py "/build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen/debug-support.cc" ../src/objects.h ../src/objects-inl.h ../src/objects/allocation-site-inl.h ../src/objects/allocation-site.h ../src/objects/code-inl.h ../src/objects/code.h ../src/objects/data-handler.h ../src/objects/data-handler-inl.h ../src/objects/fixed-array-inl.h ../src/objects/fixed-array.h ../src/objects/js-array-inl.h ../src/objects/js-array.h ../src/objects/js-array-buffer-inl.h ../src/objects/js-array-buffer.h ../src/objects/js-objects-inl.h ../src/objects/js-objects.h ../src/objects/js-regexp-inl.h ../src/objects/js-regexp.h ../src/objects/js-regexp-string-iterator-inl.h ../src/objects/js-regexp-string-iterator.h ../src/objects/map.h ../src/objects/map-inl.h ../src/objects/name.h ../src/objects/name-inl.h ../src/objects/scope-info.h ../src/objects/script.h ../src/objects/script-inl.h ../src/objects/shared-function-info.h ../src/objects/shared-function-info-inl.h ../src/objects/string.h ../src/objects/string-inl.h
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/tools/icu/icu_implementation.stamp
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/tools/icu/icu_uconfig.stamp
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/tools/icu/icu_uconfig_target.stamp
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.host/tools/icu/icu_implementation.stamp
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.host/tools/icu/icu_uconfig.stamp
make[1]: *** No rule to make target '../deps/v8/test/torque/test-torque.tq', needed by '2f2d73a3ab02a00709f3dbbaf3638568d55301e2.intermediate'.  Stop.

I'm building the code using the base alpine linux docker image for version 3.8.1, with no custom packages (only official ones required to build the image).

Here's how I'm building from the source:

	./configure --prefix=/usr --fully-static --without-npm
	make -j$(getconf _NPROCESSORS_ONLN)

Every nightly on december before that is build properly.

You can check lsmoura/node:12nightly if you want to see other nightly builds working.

Thanks for all the work from the nodejs developers!

[richardlau]

I think deps/v8/test/torque/test-torque.tq came with the V8 7.1 update.

deps/v8/test is excluded from the source tarball:

node/Makefile

Line 984 in a6f69eb

$(RM) -r $(TARNAME)/deps/v8/test

cc @nodejs/build-files @nodejs/v8-update

[lsmoura]

Should I be taking extra steps to prepare the source to build, then? maybe removing the requirement to build deps/v8/test/torque/test-torque.tq from the makefile?

Or maybe the nightly source file is not being built properly?

[richardlau]

Since it's in test, does the gyp file need to reference it? Maybe try removing

node/deps/v8/gypfiles/v8.gyp

Line 27 in a6f69eb

"../test/torque/test-torque.tq",

[richardlau]

@lsmoura It's a gap in our process -- The nightly builds are built from a git checkout, not the source tarball. I'm not aware that we have any builds that build from the source tarball -- cc @nodejs/build ?

[lsmoura]

well, 50% of my motivation to make these builds are to help the nodejs team with something. Even if it is "I've found these errors and I don't know if you're aware of them". Mission accomplished, somehow (=

I'll automate this process soon enough and make logfiles available...

Thanks for the tips. I'll give it a try.

[targos]

I'm preparing a PR to fix this

[lsmoura]

This fixed temporarily on my build script, after unpacking the file:

  cp "deps/v8/gypfiles/v8.gyp" "deps/v8/gypfiles/v8.gyp.tmp"
  grep -v test-torque deps/v8/gypfiles/v8.gyp.tmp > deps/v8/gypfiles/v8.gyp

[lsmoura]

By the end of the build process, I got this one other missing piece:

  g++ -o /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/v8_initializers/gen/torque-generated/builtins-test-from-dsl-gen.o /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen/torque-generated/builtins-test-from-dsl-gen.cc '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-DV8_GYP_BUILD' '-DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=0' '-DOPENSSL_THREADS' '-DV8_TARGET_ARCH_X64' '-DV8_EMBEDDER_STRING="-node.5"' '-DENABLE_DISASSEMBLER' '-DV8_PROMISE_INTERNAL_FIELD_COUNT=1' '-DV8_INTL_SUPPORT' '-DV8_CONCURRENT_MARKING' '-DDISABLE_UNTRUSTED_CODE_MITIGATIONS' '-DV8_EMBEDDED_BUILTINS' '-DV8_EMBEDDED_BYTECODE_HANDLERS' '-DUCONFIG_NO_SERVICE=1' '-DU_ENABLE_DYLOAD=0' '-DU_STATIC_IMPLEMENTATION=1' '-DU_HAVE_STD_STRING=1' '-DUCONFIG_NO_BREAK_ITERATION=0' -I../deps/v8 -I../deps/v8/include -I/build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen -I../deps/icu-small/source/i18n -I../deps/icu-small/source/common  -pthread -Wall -Wextra -Wno-unused-parameter -m64 -fno-strict-aliasing -m64 -O3 -fno-omit-frame-pointer -fdata-sections -ffunction-sections -O3 -fno-rtti -fno-exceptions -std=gnu++1y -MMD -MF /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/.deps//build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/v8_initializers/gen/torque-generated/builtins-test-from-dsl-gen.o.d.raw -Os -fomit-frame-pointer -Os -fomit-frame-pointer -c
g++: error: /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen/torque-generated/builtins-test-from-dsl-gen.cc: No such file or directory
g++: fatal error: no input files
compilation terminated.
make[1]: *** [deps/v8/gypfiles/v8_initializers.target.mk:196: /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/v8_initializers/gen/torque-generated/builtins-test-from-dsl-gen.o] Error 1

[targos]

@lsmoura see https://github.com/nodejs/node/pull/25097/files#diff-011f2f230002e291fd17e99c2012fcab

[rvagg]

So, what might a test worker for src tarballs look like? We can probably only afford to do it on a single platform, and that'll likely be one of the Ubuntu 16.04 or 18.04 Docker containers we have in the mix for these kinds of miscellaneous things.

make tar
tar zxvf <tarball> -C ${NODE_TEST_DIR}
cd ${NODE_TEST_DIR}/<expected path>
./configure <with normal test options>
make <with normal test options>
.... ?

Would we copy in the tests and run a full test suite on the compiled build? Is that necessary or just confirming that it compiles is enough?

[richardlau]

So, what might a test worker for src tarballs look like? We can probably only afford to do it on a single platform, and that'll likely be one of the Ubuntu 16.04 or 18.04 Docker containers we have in the mix for these kinds of miscellaneous things.

make tar
tar zxvf <tarball> -C ${NODE_TEST_DIR}
cd ${NODE_TEST_DIR}/<expected path>
./configure <with normal test options>
make <with normal test options>
.... ?

Would we copy in the tests and run a full test suite on the compiled build? Is that necessary or just confirming that it compiles is enough?

We shouldn't need to copy in any tests (Node.js tests are included in the source tarball). The things that don't get into the source tarball are tests for deps (which we don't run on the regular CI anyway) and things from tools which I think prevents linting.