Initializing array from data or element segment

[askeksa-google]

In various discussions, we have touched upon the subject of having a version of the array.init instruction which initializes the array from a data or element segment rather than from the stack. Most recently, @rossberg suggested some variations of the operation in #250 (comment).

Of the various proposed instructions for this purpose, I think the ones that create a new array are the most important, as these provide new capabilities for global initializers. These would be (staying with the init nomenclature):

array.init_from_data $a (data $d) : [i32 i32 (rtt $a)] -> [(ref $a)]
array.init_from_elem $a (elem $e) : [i32 i32 (rtt $a)] -> [(ref $a)]

each taking an offset (byte resp. element) and the array length from the stack.

I did an experiment in dart2wasm, replacing the use of array.init for string constants with array.init_from_data as specified above.

For a module with 34k of string data, it reduced the module size by 48k uncompressed. Compressed (zopfli), the size reduction was less dramatic: 300 bytes if the instruction takes the offset first, then the length, or 1200 bytes if it takes the length followed by the offset.

But this is not merely a matter of module size reduction. Due to the limitation of array.init that there is a maximum length it can accept, these instructions are essential to enable arrays of any length to be initialized in global initializers. This is important for the implementation of Dart constant expressions, for instance.

[tlively]

SGTM, and I think the names are appropriate as well. The memory.init instruction uses [dest_offset, source_offset, length] order for its operands, so it would be good to use [source_offset, length] in these instructions. Interesting that the compression was better the other way around, though.

[askeksa-google]

One thing to keep in mind is the interaction with data.drop and elem.drop.

Ordinarily, constant expressions are independent of all side effects, which means an engine is free to postpone initialization of a global until e.g. the first time a function using the global is compiled. However, if we specify that these instructions will trap when referring to a dropped data or element segment (which would be a natural thing to do), then such postponing becomes observable.

To restore the illusion that global initialization happens fully up front, the engine would need to perform at least a quick pass over the global section to identify references to data and element segments. It can then instantiate those arrays up front or keep the segments alive internally until all uses in initializers have been executed.

[rossberg]

Ah, I was actually assuming that these would be copying operations analogous to memory/table.init. I guess it also makes sense to have versions that are fused with allocation, but those I would probably name array.new_from_data/elem.

[askeksa-google]

Right, with other instructions with init in their name being copying instructions without allocation, it makes sense to distinguish by calling the allocating ones something with new. But then we should also rename array.init to array.new_something.

[rossberg]

Well, true, but array.init is not in the proposal yet. :)

[manoskouk]

I see a couple of issues here:

• We introduce circular dependencies between the global and element sections through global.get and array.init_from_elem.
• We introduce dependencies from any section that contains constant expressions to the data section. This could be resolved by somehow moving the data count section earlier in the module.
• We introduce dependencies between element segments. We could resolve this like we plan to do with globals, i.e. by only allowing an element segment to reference previous ones.

[askeksa-google]

Those are very good points. For the last two, I agree with your suggestions on how to solve them.

The first one is more tricky. One approach could be to let the global initialization allocate these arrays with uninitialized elements, and then fill them in when reading the element section. The engine would need to remember the element index and offset for each array.init_from_elem instruction along with a reference to the array to be initialized. To make it easier for the engine to allocate space for this auxiliary information, we should provide the number of array.init_from_elem instructions occurring in the globals section somewhere before that section.

We can make the process slightly easier for the engine by requiring that element indices occurring in array.init_from_elem instructions throughout the globals section are monotonically (though not necessarily strictly) increasing. However, given that this just saves a single sort by element index, it may not be worth the reduced flexibility. In any case, processing of the element section would walk through a queue of array.init_from_elem instructions, matching the element index at the head of the queue to each newly processed element.

If it's important to validate the array.init_from_elem instructions as they are encountered, we will also need an element count before the globals section. Alternatively, out-of-bounds element indices can be rejected during processing of the element section.

This approach allows the construction of cyclic constants. This can be considered a feature. If cyclic constants would cause trouble for some reason, we can avoid them by requiring any global.get occurring inside an element to refer to a global whose index is strictly lower than the one containing the next array.init_from_elem instruction in the queue. This will require streaming loaders to also remember which global contained each array.init_from_elem instruction. On the other hand, this (in conjunction with the requirement that element indices in array.init_from_elem instructions are monotonically increasing) would enable non-streaming loaders to avoid the auxiliary queue of array.init_from_elem instructions by processing the global and element sections side by side.

[manoskouk]

Another question for array.init_from_data is: How should integer values bigger than one byte be interpreted in the data section? We could interpret them as LEB128 to be consistent with other parts of the binary format, but that would prevent us from using memcpy to initialize arrays. I think we should interpret them as full-sized values.

[tlively]

Full-sized values based on the type of the array makes sense. So an i8 array would have an entry for each source byte and an i32 array would have an entry for each little-endian set of four bytes. Should the size be given in terms of array length rather than byte length to avoid having leftover bytes?

[askeksa-google]

For consistency with other instructions that specify an array length, I'd say specify the array length rather than the byte length.

[rossberg]

Mutual dependencies between sections are a reoccurring problem, esp wrt globals. Some might remember that that's also what caused us to introduce the declare segments for first-class references. As I argued at the time, that was a hack, and I would have preferred a cleaner and more general solution that scales to other instructions, like we have now.

The simplest way out would be requiring dependency ordering between sections, as you say. But that also requires allowing multiple occurrences of each section type to handle all cases. This keeps coming up with almost any interesting addition to the module scope, so perhaps we should just bite?

Another question for array.init_from_data is: How should integer values bigger than one byte be interpreted in the data section?

A data segment is a a sequence of bytes, so I don't think there is much leeway in interpreting it differently. And that instruction should amount to a memcopy anyway. So yes, the data segment has to lay out the values accordingly, as if they were read with the analogue of sequential load instructions of the right size.

[manoskouk]

Update: array.init_from_data is now available in V8. As usual, it is specified in our V8 spec document.
Note: For now we implemented the version which automatically includes the canonical rtt for the specified array type.

[titzer]

FWIW I implemented unrestricted section repetition in Wizard. It require some readjustment to module representations to keep a sane order for initialization, but otherwise I don't see a major blocker to doing this. For this proposal I suggest we at least consider repeating the type section as the mechanism to denote recursion groups for iso-recursive typing.

[rossberg]

For this proposal I suggest we at least consider repeating the type section as the mechanism to denote recursion groups for iso-recursive typing.

I thought about this, but turns out it's not quite that simple. For one, it would not be backwards-compatible to make the existing type section iso-recursive, since all existing types would then suddenly be reinterpreted as part of a recursive group (even if they aren't actually recursive), which drastically affects type equivalence. That would break all uses of call_indirect.

So, at a minimum, we'd need to introduce a new kind of recursive type section, that is distinct from the one we already have. So it's a new construct either way.

I'd also like a coherent story for sections in general. Currently, they have at least the following properties:

1. Sections can only refer to declarations from previous sections, but are not recursive themselves.
2. Concatenating sections preserves semantics of the contained definitions (modulo index shifts).

For this new type section we'd need to abandon these and make that section a special case. That might complicate some tooling. Now it's no longer just ordering of definitions that's relevant, but section boundaries themselves – two consecutive sections of the same kind would not be equivalent to their concatenation in this case. For example, a static linker would have to treat the new section differently when merging modules.

All this considered, it's not clear that using section boundaries to delimit recursion would be simpler overall. Being more explicit and uniform has benefits.