Suggestion: Combine opaque external references with explicit wasm-extern casts

[manoskouk]

In issue #293, we discussed the possibility that funcref not be a subtype of anyref, to avoid representation-change costs at the JS/Wasm boundary. It is reasonable to assume that other types might also require similar transformations in the future: i31ref will most probably require some form of range check, and it is not unlikely that structs and arrays will require some wrapping to function optimally in both worlds.
This creates a situation where passing a value to wasm as anyref carries significant cost at the boundary, since every object has to be checked against every Wasm type, in case a representation change has to be applied. The idea to apply these checks during downcasts from anyref will not work, as we would need to apply the reverse transformation when upcasting to anyref, breaking the fundamental assumption that upcasts are no-ops.
Therefore I suggest the following (assuming funcref <: anyref, but this does not matter for this discussion):

• Reintroduce an externref type which represents host references in the host's representation. This will be separate from the type hierarchy, i.e. not a subtype of anyref.
• Specify that anyref is not just a supertype of funcref, i31ref, and dataref, but in fact contains no values that do not classify as one of these three types. In other words, anyref is the type of all references that can be generated by a wasm module.
• Introduce explicit casts from anyref to externref and vice versa. These will not be free, and will perform the same representation change as at the Wasm/JS boundary. For performance, we will also introduce additional casts from externref to more specific types. The specifics of the representation change will of course depend on the module's host.

With this design

• an API that wants to pass opaque references to Wasm can pass them as externref and have the guarantee that there is no cost at the boundary. Such references might even be Wasm references (albeit in the host's representation); this way, another Wasm module can create objects internally and pass them to an externref interface.
• an API that wants to pass references of specific Wasm types can do so, accepting the cost of representation change. Note that this is likely to be quite minimal for more concrete types than anyref.

I understand that the premise of this post is not a given. However, I also do not see the downside: even if we decide that funcref </: anyref, we allow for more flexibility in the implementation of other (current and future) reference types, and allow Wasm modules to specify more cleanly whether they expose an opaque interface.

[jakobkummerow]

What I like about this idea is that it's a general solution that addresses several issues that came up in previous discussions (such as i31-vs-host-tagging-schemes, costs created by funcref <: anyref) by separating concerns and making certain costs pay-as-you-go (instead of imposing them on every case). Thereby it lets us define the core Wasm type hierarchy however we see fit without worrying too much about costs incurred when anyref is used at the module boundary. And I think it avoids all the pitfalls that plagued earlier suggestions (including my own) around possible relationships between anyref, externref, "opaqueref" etc.

[titzer]

+1 to reintroducing externref; I think we removed it prematurely.

From what I glean above, the anyref would in fact be a wasmref and the two types are not related, thus requiring explicit casts both directions. Did I get that right?

One concern that @rossberg has brought up in that context is a Wasm module should be virtualizable as a host module and vice versa. We'd need to be able to specify a type above these two, or have type imports, for that to work.

[fgmccabe]

Perhaps a different approach is to have type imports. An externref could then be a type import provided by the host.

[manoskouk]

From what I glean above, the anyref would in fact be a wasmref and the two types are not related, thus requiring explicit casts both directions. Did I get that right?

Correct. These would be the same "casts" that are happening in the boundary.

One concern that @rossberg has brought up in that context is a Wasm module should be virtualizable as a host module and vice versa. We'd need to be able to specify a type above these two, or have type imports, for that to work.

Would that not be achieved by creating references in the host, then casting them to externref (if needed) and passing them to the hosted module?

[titzer]

Correct. These would be the same "casts" that are happening in the boundary.

I agree that these casts are essentially hidden in the JS embedding, so making them explicit is attractive. That does have a downside in that they impose a cost on cross-module wasm -> wasm calls that use externref to communicate.

[lukewagner]

I also like the idea of externref </: anyref. E.g., this would allow SM to, if it wanted, represent externref with its existing 64-bit NaN-boxed JS value representation while representing anyref with a SMI/pointer word. So, along with funcref </: anyref, this would give us 3 "root" supertypes with 3 distinct representations which seems pragmatic.

Incidentally, I don't think we can emulate externref with type imports: the idea with type imports is that you import a subtype of some fixed supertype. Thus, you could only type-import a subtype of any, func, data or any other heaptype. This "can only import subtypes of a fixed heaptype" restriction is key in allowing us to compile type imports as a fixed blob of machine code without monomorphization (using the "universal representation" of the import's supertype).

[manoskouk]

That does have a downside in that they impose a cost on cross-module wasm -> wasm calls that use externref to communicate.

Right, but the downside compared the the current state would exist only in the case where they communicate directly though externref, and these values never cross the boundary. Modules could expose an anyref and an externref API if this starts becoming measurable.

[dcodeIO]

IIUC (please correct me if I'm wrong), in the presence of funcref <: anyref, one way to phrase the suggested hierarchy is

      may have casting cost
               |
     anyref   <->   externref
     /    \
dataref  funcref
   |
  ...

where anyref is kinda a typedref that can perhaps one day be obtained directly through type imports. Yet, this doesn't seem to solve the representation problem on the anyref/funcref edge - or does it?

If it does not, a likely (here: consolidated) alternative hierarchy with funcref </: anyref could become

      may have casting cost or may fail
             |               |
  dataref   <->   funcref   <->   externref
     |
    ...

yet this one seems odd on the background that a) from a producer's point of view (regardless of representation issues), the most viable/convenient/expected hierarchy would be the initially envisioned

     anyref
      /  \
dataref  funcref
   |
  ...

and b) given the former dataref/funcref/externref separation, if there is a JS import that accepts any JS/Wasm value (and would hence prefer to use the original anyref), the import needs to be duplicated twice to accept either a dataref, funcref or externref, offloading the representation change to the Wasm<->JS stub, unless (some of) these conversions are permitted implicitly to avoid the duplication, but doing either of these leads back to square one at the boundary anyway it seems.

What am I missing? :)

[manoskouk]

Casts from dataref or funcref to externref should always succeed, thus the import could accept externref.

[rossberg]

A few observations:

• We have identified 3 possible universes of representation:

  (1) external: references in host-prescribed representation
  (2) internal: references in native Wasm representation
  (3) function: function references in Wasm representation

• Internal includes both i31 and data refs – these must be in the same universe and have a shared representation and a common supertype, otherwise core use cases of i31ref would be broken. (But as long as extern is separate, there is no problem with that.)

• All external references ought to be convertible into internal references somehow (at least at the boundary), otherwise virtualisation would be broken in one direction. Furthermore, externref, if not fully internalisable, would always enforce extra boxing, even on platforms where it wouldn't be needed. Consequently, we need to allow that internal references can be formed from all external references, including those that were formed from functions (which does not imply that they are castable to functions or have the same representation).

Putting this together, there is the following conceptual hierarchy:

            ___________ any? _________          
           /             |            \
     extern            intern          func
    /      \          /  |   \
(host)  (exported) i31  data  (imported)
        /   |    \             /      \
 (e_i31) (e_data) (e_func)  (i_host) (i_func)

where the parenthesised nodes are types that we may not necessarily name in Wasm. Here, exported are the native Wasm values converted/wrapped to be represented at the host side, and dually, imported are host values converted/wrapped to be represented uniformly with native Wasm values (this may include wrapped Wasm functions).

If any does not exist, then explicit conversions will be required between these types (these are conversions, as opposed to casts, since they can involve representation changes):

extern.convert_intern : [internref] -> [externref]
intern.convert_extern : [externref] -> [internref]
extern.convert_func : [funcref] -> [externref]
func.convert_extern : [externref] -> [funcref]  ;; may trap

and for completeness, maybe also

intern.convert_func : [funcref] -> [internref]
func.convert_intern : [internref] -> [funcref]  ;; may trap

If any exists, then the diagram collapses imported = extern - func and exported = intern + func, necessarily (and e_* = * and i_* = *).

But if any does not exist (at least to unify intern and extern), then every module will have to pick a preferred representation between intern and extern. We need to be aware that this reduces composability. The same problem is created for type imports, as @lukewagner points out.

That is, every module will have to choose whether it is optimising for linking against the host or linking against another Wasm module. Note that this does not just affect exports, but internal choices like type definitions for data structures, so having two versions for every export function does not solve the problem (nor would it scale). Furthermore, the problem extends to exports like mutable tables and globals, which you cannot duplicate anyway.

So, while I think separating the types is possible, there is no free lunch, and this certainly creates its own set of complexities and problems (some of which may be difficult to predict).

[tlively]

At the last subgroup meeting we all agreed that figuring out the type hierarchy was a high priority, so it would be good to move this discussion forward as much as we can.

I was thinking about the top any type in @rossberg's diagram above and wondering whether we could get the benefit of having externref as a "lazy" internref even if we had that single top type. I think we can, but it would require representation changes on downcasts. Consider a smi that flows from JS into Wasm where an extern reference is expected. Since extern is supposed to be lazy, the engine does not check what kind of reference it is at this point. Later, the value flows to a location with type any and is then downcast to i31. Since it is a smi, the cast succeeds without representation changes. However, if a boxed number followed the same path, then this downcast would require a representation change to i31. (The end result in both cases needs to be the same since a smi and boxed number should not be distinguishable.)

In the OP, @manoskouk wrote:

The idea to apply these checks during downcasts from anyref will not work, as we would need to apply the reverse transformation when upcasting to anyref, breaking the fundamental assumption that upcasts are no-ops.

But I don't see why we would need to apply reverse transformations on upcasts to any. In the example above the i31 you get after the downcast can flow anywhere an any is expected with no further modifications. Is there another case where upcasts require representation changes that I'm missing?

Also, are representation changes on downcasts acceptable? I could imagine this could get hairy if the engine moved code around so that the value to be cast was materialized in multiple locations (e.g. in two different registers). Then the cast would have to update the representation in multiple locations.

[titzer]

The only case that I can think of a representation on downcast being a problem is if both the from and to type are both under eqref. IIUC the intention is that ref.eq is always a word compare.

As for multiple places to update representations, a cast produces a new value and leaves the old value unchanged.

[manoskouk]

Is there another case where upcasts require representation changes that I'm missing?

Other types may have different representations in JS and Wasm. In V8, these are currently functions, but it may expand in the future.
This means that if an upcast from funcref to anyref does not do the reverse transformation, you end up in a spot where an anyref value may have either representation. This complicates further downcasts, as well as crossing the boundary to JS.

[askeksa-google]

I very much like the framing of this as making boundary conversions optional and explicit. With this in mind, it makes sense to extend the conversions to any Wasm type, i.e. not just reference types. We can convert to/from any Wasm type at the boundary, and the explicit conversions would mirror that capability.

We would have two instructions (names not meant as suggestions):

• to_extern: any type -> externref
• from_extern *type*: externref -> type

This positions the feature as independent of the funcref <: anyref question. It also takes serious the observation (CMIIW) that, even for reference types, the engine can often do a better job of converting a value from externref to a known final type than converting to internref and then downcasting.

In this model, we might want to rename the externref type to reflect that it's not necessarily a reference type in the same sense as the other reference types (i.e. it's not part of the hierarchy and is not composed of a heap type and a nullability).