proposal: do not pack SIMD vectors

[Snektron]

Currently, @Vector(N, T) is packed together. For example:

export fn square() u32 {
    return @bitSizeOf(@Vector(11, u3));
}

returns 33. This seems counter intuitive to me for several reasons:

• [N]T is not packed either.
• I see no good reason for vectors to be packed.
• The performance impact is highly questionable. For instance, consider

  export fn square(num: u32) u32 {
      return @reduce(.Add, @as(@Vector(11, u3), @bitCast(@as(u33, num))));
  }

  This generates an excessive amount of bit shifts

In my opinion, vectors should essentially be a bag of scalars that you want to perform the same operation on, and not provide any layout guarantees at all. This would enable compilers to lower @Vector(11, u3) to @Vector(11, u8), and omit these expensive shifts (and a whole lot of headaches).

An important edge case here are @Vector(N, bool) and @Vector(N, u1). I suspect the reason why the above are packed at all, is to provide the guarantee that those are backed by an integer (and that all operations on it are bitwise). This makes sense to me, and I don't think that we should remove that. I see three main paths forward:

• Make only @Vector(N, bool) be guaranteed to be backed by uN, and allow @bitcasting between these.
• Make only @Vector(N, bool) and @Vector(N, u1) be backed by uN, and allow @bitcasting between these.
• Add no such guarantee at all (leave it up to the backend), and provide utilities (functions or built-ins) to between uN and vectors.

In all cases, I think we only need to remove the capability to bitcast between vectors and integers.

[andrewrk]

@bitSizeOf reports how many bits of logical information there are. For example, i9 returns 9 for bit size of, while it actually takes up 2 bytes of actual memory (16 bits) in its representation. I can still make sense of your initial point however if we look at @sizeOf instead and make the same observation.

In my opinion, vectors should essentially be a bag of scalars that you want to perform the same operation on, and not provide any layout guarantees at all. This would enable compilers to lower @Vector(11, u3) to @Vector(11, u8), and omit these expensive shifts (and a whole lot of headaches).

This is status quo.

Add no such guarantee at all (leave it up to the backend), and provide utilities (functions or built-ins) to between uN and vectors.

Again this is status quo.

In all cases, I think we only need to remove the capability to bitcast between vectors and integers.

But that's the utility to convert between uN and vectors that you mentioned above.

[Snektron]

Alright, I guess I misunderstood the meaning of bitcast in this context. This doesn't change the fact that LLVM still reasons about it in a packed way, which I still believe should be corrected.

[andrewrk]

I agree, that should be corrected. Happy to make any clarifications to the lang ref or type up some spec text if you need it to get unblocked. Let me know how I can help. But it sounds like everything you want to do in the backend is already legal.

[mlugg]

Status quo vectors are a little bit of a mess - I discussed this with @andrewrk and @jacobly0 in a compiler meeting a while back. Some parts of the language (bitcasting) assume they are packed, and some (vector index in bit-pointers) do not.

If I remember correctly, Jacob proposed behavior something like this:

• @Vector(n, bool) has a well-defined layout and bit-packs; this is fine on any realistic platform, and can be useful for @bitCast shenanigans
• Any other vector has undefined layout; the backend can choose how best to lower it. This probably usually means they should be aligned up to the next power-of-two integer. (Since they don't have well-defined layout, @bitCasting to or from one is a compile error, and reinterpreting one through a pointer is UB.)

The only compiler change that would be needed here to match the language spec is to change Type.hasWellDefinedLayout, which currently just returns true for vectors.

Another problematic thing with vectors is the "vector index" field on bit-pointers. I don't love it in general - I think it would make more sense to return bit-pointers, even if per the above rules this makes the exact type returned backend-dependent (related: #16271) - but the bigger problem is that the vector index is permitted to be runtime-known (represented by writing it as a ?), which completely breaks how bit-pointers are supposed to work, and can cause completely incorrect behavior even today if you do &unaligned_vector[runtime_index]. (Fixing this basically would be #16271.)

[p7r0x7]

I've been thinking about this a lot, as my upcoming VP9 encoder project heavily relies on SIMD and @vector(). I would love to use arbitrary-bit-wide integers efficiently (because the compiler "reduces"/"promotes" them to 2^x bit sizes).

[alexrp]

While auditing volatile semantics in the LLVM backend, I noticed that we're in violation here:

zig/src/codegen/llvm.zig

Lines 8398 to 8419 in ab89af3

	fn airVectorStoreElem(self: *FuncGen, inst: Air.Inst.Index) !Builder.Value {
	const o = self.ng.object;
	const pt = o.pt;
	const zcu = pt.zcu;
	const data = self.air.instructions.items(.data)[@intFromEnum(inst)].vector_store_elem;
	const extra = self.air.extraData(Air.Bin, data.payload).data;
	const vector_ptr = try self.resolveInst(data.vector_ptr);
	const vector_ptr_ty = self.typeOf(data.vector_ptr);
	const index = try self.resolveInst(extra.lhs);
	const operand = try self.resolveInst(extra.rhs);
	const access_kind: Builder.MemoryAccessKind =
	if (vector_ptr_ty.isVolatilePtr(zcu)) .@"volatile" else .normal;
	const elem_llvm_ty = try o.lowerType(vector_ptr_ty.childType(zcu));
	const alignment = vector_ptr_ty.ptrAlignment(zcu).toLlvm();
	const loaded = try self.wip.load(access_kind, elem_llvm_ty, vector_ptr, alignment, "");
	const new_vector = try self.wip.insertElement(loaded, operand, index, "");
	_ = try self.store(vector_ptr, vector_ptr_ty, new_vector, .none);
	return .none;
	}

Note how an operation that looks like it should be a single store in code (volatile_vec_ptr[i] = x) turns into a load + store. We can fix this for some types of vectors, but not in the general case due to vectors being packed.

So I think I'm in favor of some form of this proposal.