Skip to content

Conversation

@castholm
Copy link
Contributor

@castholm castholm commented Feb 18, 2024

Closes #18987
Follow-up to #18778 (which added lazy dependencies) and the discussion in #18808 that followed

This adds b.lazyImport, which (in the context of a build script) is to @import what b.lazyDependency is to b.dependency and can be used to import a lazy dependency's build.zig struct.

It enables the use case where a package wants to lazily depend on an @import-style dependency and not fetch it until it is first needed, for example when building for a specific target.

// build.zig of lazy dependency 'foo'
const std = @import("std");

pub fn build(_: *std.Build) void {}

pub const bytes = "Hello!";
// build.zig of consumer
const std = @import("std");

pub fn build(b: *std.Build) void {
    const wf = b.addWriteFiles();

    // the non-lazy way, which won't work for our lazy dependency
    //_ = wf.add("foo.txt", @import("foo").bytes);

    // if 'foo' has not yet been fetched, 'lazyImport' returns 'null' and
    // triggers a recompile of the build script
    if (b.lazyImport(@This(), "foo")) |foo| {
        // 'foo' here is the build.zig struct of the dependency
        // we can access its public decls
        _ = wf.add("foo.txt", foo.bytes);
    }

    b.installDirectory(.{
        .source_dir = wf.getDirectory(),
        .install_dir = .prefix,
        .install_subdir = "",
    });
}

Implementation details

Given the name of a dependency, we want to locate that package.

Just like with dependency and lazyDependency, in order to find the package we need to first obtain a list over which dependencies are available to the dependee, then find the package with the matching name in that list. For dependency/lazyDependency, this list is b.available_deps.

However, because lazyImport returns a comptime-only return type, we can't use runtime arguments like b.available_deps.

All the lists of dependencies of all participating packages can be found in @import("root").dependencies, but given only the name of the package have no way to know at comptime for which package we are calling lazyImport.

The only "clean" solution to this problem that I can come up with is to take a build.zig struct as the second argument:

pub inline fn lazyImport(
    b: *Build,
    /// The build.zig struct of the package importing the dependency.
    /// When calling this function from the `build` function of a build.zig file's, you normally
    /// pass `@This()`.
    comptime asking_build_zig: type,
    comptime dep_name: []const u8,
) ?type

This enables us to compare asking_build_zig against the build.zig struct of every package in @import("root").dependencies to figure out for which package we are calling lazyImport, then search through that package's dependencies as usual.

For good measure we can also throw in a runtime safety check that asserts that the b instance and the build.zig struct both represent the same package.

In practice this means that you almost always invoke the function with @This() as the second argument, such as in b.lazyImport(@This(), "foo").

Testing

This API can't currently easily be tested by the compiler's test suite.

If any maintainers would like to check out this branch and test it locally, I have prepared a little test project that uses lazyImport up to two levels of indirection, which I used myself to test the changes:

test.tar.gz

The project is set up as follows:

  • root lazily depends on alfa
  • root lazily depends on bravo
  • bravo lazily depends on charlie

I used python -m http.server to serve the packages over http://localhost:8000/; you might need to edit them if you want to serve them in a different way.

Running zig build will trigger one initial compile of the build script + two recompiles, then install two plaintext files.

else => continue,
};
return dependencyInner(b, name, build_root, build_zig, &.{}, args);
return dependencyInner(b, name, build_root, build_zig, "anonymous", &.{}, args);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can ignore "anonymous" here, it's just a dummy value that makes it so that package hash comparisons always fail.
Removing anonymousDependency is an accepted proposal and I have a different PR up that removes it (#18590).

@andrewrk andrewrk added release notes This PR should be mentioned in the release notes. zig build system std.Build, the build runner, `zig build` subcommand, package management labels Apr 7, 2024
@andrewrk andrewrk merged commit e204a6e into ziglang:master Apr 7, 2024
@castholm castholm deleted the lazy-build-zig branch April 8, 2024 14:37
@andrewrk andrewrk added this to the 0.12.0 milestone Apr 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release notes This PR should be mentioned in the release notes. zig build system std.Build, the build runner, `zig build` subcommand, package management

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Can't call inline function that returns comptime-known comptime-only type at runtime

2 participants