Incorrect memoization of comptime function calls

[ianprime0509]

Zig Version

0.12.0-dev.1324+5257643d3

Steps to Reproduce and Observed Behavior

Found while debugging a failing test in #17634.

Given the following program:

const std = @import("std");

fn GenericIntApplier(
    comptime Context: type,
    comptime applyFn: fn (context: Context, arg: u32) void,
) type {
    return struct {
        context: Context,

        const Self = @This();

        fn apply(self: Self, arg: u32) void {
            self.any().apply(arg);
        }

        inline fn any(self: *const Self) IntApplier {
            return .{
                .context = @ptrCast(&self.context),
                .applyFn = typeErasedApplyFn,
            };
        }

        fn typeErasedApplyFn(context: *const anyopaque, arg: u32) void {
            const ptr: *const Context = @alignCast(@ptrCast(context));
            applyFn(ptr.*, arg);
        }
    };
}

const IntApplier = struct {
    context: *const anyopaque,
    applyFn: *const fn (context: *const anyopaque, arg: u32) void,

    fn apply(ia: IntApplier, arg: u32) void {
        ia.applyFn(ia.context, arg);
    }
};

const Accumulator = struct {
    value: u32,

    const Applier = GenericIntApplier(*u32, add);

    fn applier(a: *Accumulator) Applier {
        return .{ .context = &a.value };
    }

    fn add(context: *u32, arg: u32) void {
        context.* += arg;
    }
};

fn s() u32 {
    var a: Accumulator = .{ .value = 0 };
    const applier = a.applier();
    applier.apply(1);
    applier.apply(1);
    return a.value;
}

test {
    try comptime std.testing.expect(s() == 2);
}

The test fails. Inserting a @compileLog before returning a.value reveals that the actual value is 1 rather than the expected value of 2.

Expected Behavior

The test should pass.

It seems that the calls to apply (or some part of them) are being incorrectly memoized: as long as applier.apply is called with a different argument each time, the code works as expected, but any subsequent calls with an argument already used will have no effect.

Through bisecting, I've found that this regressed in 405ba26.

This might be further reducible, as there are quite a few moving parts in this example. I'll update this issue later if I'm able to remove any of them.

[Vexu]

According to your code apply cannot mutate comptime state but you mutate it either way. Removing a bunch of consts makes it work as expected so there should either be an error complaining about the mutation (#1487) or the compiler should detect the mutation and not memoize the call.

[mlugg]

I don't have access to a PC currently, but I'm 99% sure this is caused by the "can mutate comptime memory" check being faulty for constant pointers to comptime-mutable pointers. Fixing this ties into the plans for comptime memory as discussed in the 2023/10/28 compiler meeting.

[ianprime0509]

Thanks for the insight! This example seems to illustrate what you're suggesting (fails as written, but works if comptime is removed or the consts are removed):

const std = @import("std");

fn t() !void {
    var a: u32 = 0;
    const ptr = &a;
    const double_ptr = &ptr;
    setDoublePtr(double_ptr, 1);
    setDoublePtr(double_ptr, 2);
    setDoublePtr(double_ptr, 1);
    try std.testing.expect(a == 1);
}

fn setDoublePtr(ptr: *const *const u32, value: u32) void {
    setPtr(ptr.*, value);
}

fn setPtr(ptr: *const u32, value: u32) void {
    const mut_ptr: *u32 = @constCast(ptr);
    mut_ptr.* = value;
}

test {
    try comptime t();
}

In fact, this still fails even if setPtr is made to take a *u32 and setDoublePtr a *const *u32, so the @constCast is technically not needed to reproduce this.

This issue can also manifest when using std.io.Readers at comptime, since a similar pattern is used in the new type-erased reader implementation:

const std = @import("std");

fn t() !void {
    const buf = [_]u8{ 1, 2, 3, 4 };
    var buf_stream = std.io.fixedBufferStream(&buf);
    const reader = buf_stream.reader();
    const v1 = try reader.readIntBig(u16);
    const v2 = try reader.readIntBig(u16);
    try std.testing.expect(v1 == 0x0102);
    try std.testing.expect(v2 == 0x0304);
}

test {
    try comptime t();
}