Explicitly forbid `\x00` (null bytes) in `@""` identifier syntax

[andrewrk]

Some object formats, such as ELF, rely on null bytes to terminate strings, making it impossible to represent symbol names which contain null bytes. So there already exist at least some targets for which this must be rejected. This proposal is to unconditionally reject null bytes in identifiers for all targets.

Additionally, third party tools which attempt to integrate with Zig code or the Zig compiler may want to use null terminated strings in order to represent identifier names. By explicitly making it illegal to have null bytes in @"" identifier strings, this prevents possible bugs in the wild having to do with analysis of Zig code.

If this is accepted, it means the self-hosted compiler can sometimes use null termination to save bytes of memory in ZIR. For example an enum literal or field access could be a u32 index into a string table, rather than a index and a length. When the shrinkage from 8 bytes to 4 bytes allows an instruction to fit inside the 8 bytes allocated for every IR instruction rather than indexing into an extra_data array, this ends up saving an additional 4 bytes, for a total saving of 8 bytes per relevant instruction.

[daurnimator]

For example an enum literal or field access could be a u32 index into a string table, rather than a index and a length.

Have you considered e.g. a u20 index and a u12 length? (u12 because that's the common page size of 4096.... I wonder if symbols names are allowed to be larger than a page?)

[xackus]

I guess the alternative is mangling if the symbol is not exported?

[camconn]

This was... allowed? That's horrifying. I endorse this proposal to restrict @"name".

The PDB format suffers from this as well. The source file lines are correct, but @"test fn \x00 1234"() is truncated to @"test fn " in the corresponding .pdb file. PoC:

const std = @import("std");

pub fn main() anyerror!void {
    @"test fn 1234"();
    @"test fn \x00 1234"();
    @"test fn "();
}

fn @"test fn 1234"() void {
    std.log.info("test fn 1234", .{});
}

fn @"test fn \x00 1234"() void {
    std.log.info("test fn null 1234", .{});
}

fn @"test fn "() void {
    std.log.info("test fn space", .{});
}

[SpexGuy]

Another option would be to say that targets may set restrictions on @"" names, and disallow this specifically for file formats/backends that cannot represent it. This would avoid limiting platforms which could in theory support null bytes. Not sure if that's compelling though, I don't actually have a use case.

[andrewrk]

Mach-O does not support null bytes in symbols either. I think pretty much every target does not support them. I'm going to make the call to simplify things and unilaterally disallow null bytes in all zig identifiers. I think nobody will notice this change, and there will be some bugs (in third party code as well as ours) that will simply never manifest because of this decision.

Another option would be to say that targets may set restrictions on @"" names

We may need another proposal for this. As a prerequisite someone will have to do the work of figuring out what these restrictions should be per-target.

[Mouvedia]

I suppose this will cover \u{0000} as well?

[andrewrk]

\u{0000} is represented in UTF-8 as a null byte.

[Hadron67]

Maybe we should also disallow them in library name of extern function specifier, since almost all file system don't accept file names containing null bytes. Also current behaviour is a little buggy:

extern "asd\u{0}asd" fn test1() void;
comptime {
    _ = test1;
}

gives

./test.zig:37:9: error: dependency on dynamic library 'asd
    _ = test1;
        ^
./test.zig:36:10: note: referenced here
comptime {
         ^