zig build test -Din_memory=true --summary all doesn't seem to run tests

[johnbcodes]

I'm using zig-sqlite as a reference for my own projects and while I was poking around for answers related to running a full test suite with modules I noticed that running zig build test -Din_memory=true --summary all does not actually print a test summary. So I added a failing test to sqlite.zig

test "this should fail" {
    try testing.expectEqual(1, 2);
}

..and it doesn't fail. I also tried using the bundled option with zig build test -Duse_bundled=true -Din_memory=true --summary all and made sure that build caches were clean. I'm using the master version of Zig so it matches CI which also doesn't print a test summary in the latest runs.

I'm fairly new to Zig so having trouble understanding what's going on.

[vrischmann]

This is weird.

I just tried it with zig version 0.12.0-dev.406+bc626e8b8 and I get this:

$ zig build test -Din_memory=true --summary all
zig test aarch64-native-musl-Debug-multi Debug aarch64-native-musl: error: the following command failed with 1 compilation errors:
/home/vincent/local/bin/zig test /home/vincent/dev/perso/projects/zig-sqlite/sqlite.zig /home/vincent/dev/perso/projects/zig-sqlite/zig-cache/o/56397868824654ab236f2707732408fc/libsqlite.a -lc -fno-emit-bin --cache-dir /home/vincent/dev/perso/projects/zig-sqlite/zig-cache --global-cache-dir /home/vincent/.cache/zig --name aarch64-native-musl-Debug-multi -fno-single-threaded -target aarch64-native-musl -mcpu generic --mod build_options::/home/vincent/dev/perso/projects/zig-sqlite/zig-cache/c/8e5476150c5e55b3a7df3affcb2e73da/options.zig --deps build_options -I /home/vincent/dev/perso/projects/zig-sqlite/c --listen=- 
zig test riscv64-native-musl-Debug-multi Debug riscv64-native-musl: error: the following command failed with 1 compilation errors:
/home/vincent/local/bin/zig test /home/vincent/dev/perso/projects/zig-sqlite/sqlite.zig /home/vincent/dev/perso/projects/zig-sqlite/zig-cache/o/af9ad0f42df8be8d9c5ae3f60b7a64c5/libsqlite.a -lc -fno-emit-bin --cache-dir /home/vincent/dev/perso/projects/zig-sqlite/zig-cache --global-cache-dir /home/vincent/.cache/zig --name riscv64-native-musl-Debug-multi -fno-single-threaded -target riscv64-native-musl -mcpu baseline_rv64 --mod build_options::/home/vincent/dev/perso/projects/zig-sqlite/zig-cache/c/8e5476150c5e55b3a7df3affcb2e73da/options.zig --deps build_options -I /home/vincent/dev/perso/projects/zig-sqlite/c --listen=- 
zig test aarch64-macos-Debug-multi Debug aarch64-macos: error: the following command failed with 1 compilation errors:
/home/vincent/local/bin/zig test /home/vincent/dev/perso/projects/zig-sqlite/sqlite.zig /home/vincent/dev/perso/projects/zig-sqlite/zig-cache/o/92c5ef934e8681981e2c8915ac5bcc35/libsqlite.a -lc -fno-emit-bin --cache-dir /home/vincent/dev/perso/projects/zig-sqlite/zig-cache --global-cache-dir /home/vincent/.cache/zig --name aarch64-macos-Debug-multi -fno-single-threaded -target aarch64-macos -mcpu generic --mod build_options::/home/vincent/dev/perso/projects/zig-sqlite/zig-cache/c/8e5476150c5e55b3a7df3affcb2e73da/options.zig --deps build_options -I /home/vincent/dev/perso/projects/zig-sqlite/c --listen=- 
Build Summary: 26/30 steps succeeded; 3 failed
test transitive failure
├─ zig test native-Debug-multi Debug native success 3s MaxRSS:168M
│  └─ options cached
├─ zig test x86_64-native-musl-Debug-multi Debug x86_64-native-musl success 3s MaxRSS:164M
│  ├─ zig build-lib sqlite Debug x86_64-native-musl success 7s MaxRSS:532M
│  └─ options success
├─ zig test x86-native-musl-Debug-multi Debug x86-native-musl success 3s MaxRSS:166M
│  ├─ zig build-lib sqlite Debug x86-native-musl success 5s MaxRSS:537M
│  └─ options success
├─ zig test aarch64-native-musl-Debug-multi Debug aarch64-native-musl 1 errors
│  ├─ zig build-lib sqlite Debug aarch64-native-musl success 6s MaxRSS:648M
│  └─ options success
├─ zig test riscv64-native-musl-Debug-multi Debug riscv64-native-musl 1 errors
│  ├─ zig build-lib sqlite Debug riscv64-native-musl success 7s MaxRSS:571M
│  └─ options success
├─ zig test mips-native-musl-Debug-multi Debug mips-native-musl success 3s MaxRSS:164M
│  ├─ zig build-lib sqlite Debug mips-native-musl success 7s MaxRSS:509M
│  └─ options success
├─ zig test x86_64-windows-Debug-multi Debug x86_64-windows success 3s MaxRSS:166M
│  ├─ zig build-lib sqlite Debug x86_64-windows success 7s MaxRSS:451M
│  └─ options success
├─ zig test x86-windows-Debug-multi Debug x86-windows success 3s MaxRSS:166M
│  ├─ zig build-lib sqlite Debug x86-windows success 7s MaxRSS:447M
│  └─ options success
├─ zig test x86_64-macos-Debug-multi Debug x86_64-macos success 3s MaxRSS:162M
│  ├─ zig build-lib sqlite Debug x86_64-macos success 7s MaxRSS:535M
│  └─ options success
└─ zig test aarch64-macos-Debug-multi Debug aarch64-macos 1 errors
   ├─ zig build-lib sqlite Debug aarch64-macos success 7s MaxRSS:635M
   └─ options success
/home/vincent/local/lib/zig/std/zig/c_translation.zig:80:53: error: pointer type '?*const fn(?*anyopaque) callconv(.C) void' requires aligned address
            return @as(DestType, @ptrFromInt(castInt(usize, target)));
                                             ~~~~~~~^~~~~~~~~~~~~~~
/home/vincent/local/lib/zig/std/zig/c_translation.zig:16:33: note: called from here
                return castToPtr(DestType, SourceType, target);
                       ~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vincent/dev/perso/projects/zig-sqlite/zig-cache/o/cb5a0b35814d97ef1e9533a464e9d8ba/cimport.zig:1372:67: note: called from here
pub const SQLITE_TRANSIENT = @import("std").zig.c_translation.cast(sqlite3_destructor_type, -@as(c_int, 1));
                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
referenced by:
    bindField__anon_4900: sqlite.zig:1629:110
    bindField__anon_4898: sqlite.zig:1612:43
    remaining reference traces hidden; use '-freference-trace' to see all reference traces
/home/vincent/local/lib/zig/std/zig/c_translation.zig:80:53: error: pointer type '?*const fn(?*anyopaque) callconv(.C) void' requires aligned address
            return @as(DestType, @ptrFromInt(castInt(usize, target)));
                                             ~~~~~~~^~~~~~~~~~~~~~~
/home/vincent/local/lib/zig/std/zig/c_translation.zig:16:33: note: called from here
                return castToPtr(DestType, SourceType, target);
                       ~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vincent/dev/perso/projects/zig-sqlite/zig-cache/o/dacba413597973bdff59eaca4871a613/cimport.zig:1356:67: note: called from here
pub const SQLITE_TRANSIENT = @import("std").zig.c_translation.cast(sqlite3_destructor_type, -@as(c_int, 1));
                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
referenced by:
    bindField__anon_4842: sqlite.zig:1629:110
    bindField__anon_4840: sqlite.zig:1612:43
    remaining reference traces hidden; use '-freference-trace' to see all reference traces
/home/vincent/local/lib/zig/std/zig/c_translation.zig:80:53: error: pointer type '?*const fn(?*anyopaque) callconv(.C) void' requires aligned address
            return @as(DestType, @ptrFromInt(castInt(usize, target)));
                                             ~~~~~~~^~~~~~~~~~~~~~~
/home/vincent/local/lib/zig/std/zig/c_translation.zig:16:33: note: called from here
                return castToPtr(DestType, SourceType, target);
                       ~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vincent/dev/perso/projects/zig-sqlite/zig-cache/o/84348cafe04bffc473cbf8f765c55def/cimport.zig:1372:67: note: called from here
pub const SQLITE_TRANSIENT = @import("std").zig.c_translation.cast(sqlite3_destructor_type, -@as(c_int, 1));
                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
referenced by:
    bindField__anon_6245: sqlite.zig:1629:110
    bindField__anon_6243: sqlite.zig:1612:43
    remaining reference traces hidden; use '-freference-trace' to see all reference traces

(the failures are expected).

Which OS are you running this on ?

[johnbcodes]

Mac OS 13.x x86_64. I was a bit wrong in that I don't quite have Zig master but the daily upload to ziglang.org. Here's my console output:

     ~/dev/thirdparty/zig/zig-sqlite  on    master !1 ─────────────────────────────────────────────────────────────────────────────────────────────────── at 19:51:30 
❯ rm -rf zig-cache/

     ~/dev/thirdparty/zig/zig-sqlite  on    master !1 ─────────────────────────────────────────────────────────────────────────────────────────────────── at 19:51:34 
❯ zig version
0.12.0-dev.403+f40f81cbf

     ~/dev/thirdparty/zig/zig-sqlite  on    master !1 ─────────────────────────────────────────────────────────────────────────────────────────────────── at 19:51:38 
❯ zig build test -Din_memory=true --summary all
Build Summary: 4/4 steps succeeded
test success
└─ zig test x86_64-native-Debug-multi Debug x86_64-native success 520ms MaxRSS:83M
   ├─ zig build-lib sqlite Debug x86_64-native success 6s MaxRSS:492M
   └─ options success

Again, the modified file is just the test I added to sqlite.zig.

The other interesting thing I'l mention is that it doesn't seem to produce a zig-out directory either. Definitely seems platform related since latest GH action runs do the same thing on the macos portions.

My local toy projects run tests okay, so my only thought is the test_targets code in build.zig. I don't see anything obvious there so maybe a zig build/test bug?

[vrischmann]

Tests don't produce anything in zig-out IIRC, so that's fine.

Looks like there's something weird happening but I'm not sure what, I'll have a look.

[vrischmann]

I tested on my Linux PC and I can't make the test fail either with zig build test --summary all so it may not be platform related. It looks like the tests are simply not running for some reason.

[johnbcodes]

Fixed with #134