Add partial support for -fwasm-exceptions in Asyncify (#5343)

[caiiiycuk]

You can download binaries of wasm-opt that support fwasm-exceptions and Asyncify here.
Replace emsdk/upstream/bin/wasm-opt with downloaded version and then you can use -fwasm-exceptions with Asyncify.

--
Hi. This PR does not do lowering wasm exceptions to MVP as suggested in #5343, so I need to explain situation:

I am trying to port a dosbox/dosbox-x core as is to WebAssembly. The dosbox core is syncrhnous, it runs emulation in forever while loop, events are processed inside this loop. To make this loop asynchrous I use Asyncify, however the unmodified core uses exceptions in certain situations.

Asyncify does not support exceptions, this PR should change the situation in some cases including dosbox/x case.
The main idea of emulation loop is follow:

void emulation() {
  do {
    em_sleep(); // <-- **START_UNWIND**
    try {
      int ret = cpudecode();
      // rest logic
    } catch (PageFault&) {
      // load failed page 
    } catch (Some&) {
      // do some 
    }
  } while (ret);
}

int cpudecode() {
  // some logic that also can call **emulation** recursively
}

The catch blocks is used to recover core, or to jump out from some of deep cpudecode calls. There is no situation when emulation loop is called when catch block somewhere in stack:

void some() {
  while () {
    try {
      ...
      some(); // can call start_unwind
    } catch () {
      // never calls to start_unwind
    }
  }
}

This also means that catch block can't change the state, and so only try block can. But on other hand try block it self should works fine with current asyncify code. When we are doing rewind we are only visiting try blocks and never catch blocks, so it work out of the box.

For such type of synchronous code we can add exceptions support, we need only verify that unwind didn't start from a catch block (when catch is not in stack). We can validate this requirment and throw exception if it breaks or ignore the unwind call.

This is very small overhead and simple solution to bring exceptions support in Asyncify, more over for some codebases it should be simple to rewrite the synchronous loop to not call unwind from a catch blocks.

This PR is tested on dosbox-x, it works fine.
emscripten-discuss initial context

---

There is one thing in impelementation that probably is not correct:

            auto decIf = builder->makeIf(
              br->condition,
              builder->makeSequence(builder->makeDec(amount), br),
              br->value);
            br->condition = nullptr;
            replaceCurrent(decIf);

I have a global variable that I need to decrease on some amount just before br_if instruction,
but only when condition is true. I tried lot of other variants but no luck:

1. Adding new instruction into condition body:

            Index newLocal = builder->addVar(func, br->condition->type);
            auto setLocal = builder->makeLocalSet(newLocal, br->condition);
            auto getLocal =
              builder->makeLocalGet(newLocal, br->condition->type);
            
            br->condition = builder->makeBlock(
              {setLocal,
               builder->makeIf(getLocal, builder->makeDec(amount)),
               getLocal},
              br->condition->type);

Result:

Assertion failed: (!resultUsed || curr->type != Type::none), function optimize, file Vacuum.cpp, line 95.

>    // resultUsed only makes sense when the type is concrete
>    assert(!resultUsed || curr->type != Type::none);  <-- **ASSERT**
>    // If we actually need the result, then we must not change anything.
>    // TODO: maybe there is something clever though?
>    if (resultUsed) {
>      return curr;
>    }

2. Just wrap the br_if with new block:

            replaceCurrent(builder->makeSequence(
              builder->makeIf(br->condition, builder->makeDec(amount)), br));

Result:

Assertion failed: (values.size() == 1), function getSingleValue, file wasm-interpreter.h, line 67.

>    if (curr->condition) {
>      Flow conditionFlow = visit(curr->condition);
>      if (conditionFlow.breaking()) {
>        return conditionFlow;
>      }
>      condition = conditionFlow.getSingleValue().getInteger() != 0; // <-- **ASSERT**

3. Just wrap the br_if but save condition result into local var like in 1. Results is sames as in 2.

4. Completely replace br_if with if and br:

            auto decIf = builder->makeIf(
              br->condition,
              builder->makeSequence(builder->makeDec(amount), br));
            br->condition = nullptr;
            replaceCurrent(decIf);

Results:

Works fine on small wasm files
On big wasm file have this error:

[wasm-validator error in function CDROM_Interface_Image::LoadCueSheet\28char*\29] unexpected true: if without else must not return a value in body, on

>(if
> (i32.and
>  (i32.load8_u offset=16
>   (i32.add
>    (local.get $8)
>    (i32.load
>     (i32.sub
>      (i32.load
>       (local.get $8)
>      )
>      (i32.const 12)
>     )
>    )
>   )
>  )
>  (i32.const 5)
> )
> (block (result i32)
>  (global.set $__asyncify_catch_counter
>   (i32.sub
>    (global.get $__asyncify_catch_counter)
>    (i32.const 2)
>   )
>  )
>  (br $label$1
>   (i32.const 0)
>  )
> )
>)

OR

[wasm-validator error in function CDROM_Interface_Image::LoadCueSheet\28char*\29] unexpected false: can only drop a valid value, on

>(drop
> (if
>  (i32.and
>   (i32.load8_u offset=16
>    (i32.add
>     (local.get $8)
>     (i32.load
>      (i32.sub
>       (i32.load
>        (local.get $8)
>       )
>       (i32.const 12)
>      )
>     )
>    )
>   )
>   (i32.const 5)
>  )
>  (block (result i32)
>   (global.set $__asyncify_catch_counter
>    (i32.sub
>     (global.get $__asyncify_catch_counter)
>     (i32.const 2)
>    )
>   )
>   (br $label$1
>    (i32.const 0)
>   )
>  )
> )
>)

5. I thinking to not use replaceCurrent, find the parent block of br_if and insert builder->makeDec just before br, but seems even more cumbersome cause parent block can be Block, Loop, If, Drop, Try, etc. I don't have straightforward algorithm in my mind.

Thanks!

[kripken]

Interesting idea! I think this makes sense to do.

[caiiiycuk]

Refactored, now I change only top-level catch blocks. Also added lit tests, for me seems correct. Dosbox-x works fine.
Only here could be a problem:

// br_if case with condition and value

            Index newLocal = builder->addVar(func, br->value->type);
            auto setLocal = builder->makeLocalSet(newLocal, br->value);
            auto getLocal = builder->makeLocalGet(newLocal, br->value->type);
            auto condition = br->condition;
            br->condition = nullptr;
            br->value = getLocal;
            auto decIf =
              builder->makeIf(condition,
                              builder->makeSequence(builder->makeDec(), br),
                              getLocal);
            replaceCurrent(builder->makeSequence(setLocal, decIf));

Not sure how to deal if sideeffect will be in setLocal, currenlty it executes before condition....
@kripken please take a look

[allsey87]

@kripken is there anything preventing this PR from being merged?

[caiiiycuk]

I'm sorry, I want to finish this and add the changes suggested by @kripken, I'll try to find some time to finish it (not sure I can do it anytime soon). Btw I've used it many times and it works great in the case of js-dos.

[allsey87]

Excellent work @caiiiycuk! I look forward to seeing this merged!

[allsey87]

@caiiiycuk I am currently testing your branch and have updated the binaries in emsdk/upstream/bin with those from the build. However I am getting some errors when I try to build my test program:

em++: warning: ASYNCIFY=1 is not compatible with -fwasm-exceptions. Parts of the program that mix ASYNCIFY and exceptions will not compile. [-Wemcc]
error: DISABLE_EXCEPTION_THROWING was set (likely due to -fno-exceptions), which means no C++ exception throwing support code is linked in, but exception catching code appears. Either do not set DISABLE_EXCEPTION_THROWING (if you do want exception throwing) or compile all source files with -fno-except (so that no exceptions support code is required); also make sure DISABLE_EXCEPTION_CATCHING is set to the right value - if you want exceptions, it should be off, and vice versa.

I think the warning is fine. But I am confused as to why DISABLE_EXCEPTION_THROWING is being set. I have added -fwasm-exceptions to both the linker flags and compiler flags (both to the C++ and C flags (I guess the latter is not necessary)). Do have any suggestions on how to fix this? I am using Emscripten 3.1.32.

[caiiiycuk]

I just it like this

            -fwasm-exceptions
            "-sASYNCIFY=1"
            "-sASYNCIFY_IMPORTS=['syncSleep']"
            "-sASYNCIFY_WHITELIST=@${TARGETS_DIR}/dosbox-x-asyncify.txt"

Did you check, maybe you set flag -fno-exceptions somewhere.
Also you need to set -fwasm-exceptions both on compile and linking stages.

[caiiiycuk]

@kripken I apologize for the delay. I have made all the changes you requested. Please take a look and let me know if there's anything else to do.

[caiiiycuk]

@kripken friendly ping

[allsey87]

I have been using this for several weeks now and I haven't found any issues yet! Excellent work @caiiiycuk!

[caiiiycuk]

@allsey87 thanks, great to know

[kripken]

@caiiiycuk Sorry, I've been meaning to get to this but have been busy. Thanks for updating this!

There has recently been a suggestion to make some changes in how wasm exceptions works in the spec, see WebAssembly/exception-handling#280 - I have no idea what will happen there, but it might mean that if we land this we'd need to then refactor it significantly. On the positive side, they do mention it is a goal to make things like Asyncify easier, so perhaps this will become simpler actually. Either way, though, I think we should wait on this PR to see what the spec changes are first, to avoid wasted work for all of us.

[caiiiycuk]

@kripken No worries, all good. At least it works for my case. :)

[fs-eire]

Is there any chance that we can merge this? JSPI still need to take some time...

[kripken]

The new wasm EH proposal I mentioned above is in the process of being implemented now. The main component is this:

binaryen/src/wasm.h

Lines 1473 to 1474 in 3049fb8

	// 'try_table' from the new EH proposal
	class TryTable : public SpecificExpression<Expression::TryTableId> {

If we want to land this PR then targeting that would make sense, as the old EH will be removed. However, the new wasm EH is not done yet (e.g. parts of the optimizer don't work on it) so it's too early for that, I'm afraid.

[caiiiycuk]

@kripken

Please take a look at the last two commits. I attempted to implement the following algorithm:

1. We iterate over all function calls.
2. We determine whether the call is made inside a catch block.
3. If it is, we insert a check after the call to ensure that ASYNCIFY_STATE is still NORMAL.

More details on point 2, since I am not entirely sure the current implementation covers all cases—mainly because I may not be aware of every detail.

I use ExpressionStackWalker to locate the nearest parent Try block. Once it is found, I need to determine whether the call occurs inside the try body or inside one of the catch blocks. In practice, it is enough to check whether the current instruction belongs to the try.body. To do this, I simply take the first child of the Try block from the stack, and if that child is try->body, then the call is considered safe; otherwise, I insert the corresponding check.

I tested this code with js-dos, and so far I haven’t encountered any issues. However, I am still not completely confident in it overall.

[kripken]

Looks pretty good! I don't see anything incorrect.

[caiiiycuk]

@kripken PR updated, please check. If everything is ok, then I will squash into one commit.

[kripken]

Looks good!

No need to squash, we always do that when landing.

[kripken]

(It might be nice to also test call_indirect and call_ref, but I don't want to hold up landing on that, given the intended use cases. Based on feedback we can improve later.)

[kripken]

Oh, before I land, please provide the commit message. (Either update comment 0, or squash and put it in there, or here in a comment.)

[caiiiycuk]

Oh, before I land, please provide the commit message. (Either update comment 0, or squash and put it in there, or here in a comment.)

What about current one?
"Add partial support for -fwasm-exceptions in Asyncify"

[kripken]

Ok, sounds good. I'll elaborate a little in the text.

[hoodmane]

So to get this out we need a release of Binaryen and then a release of Emscripten that bumps the Binaryen version. What's the Binaryen release policy? Can we get a release for this?

[hoodmane]

(Also, thanks @caiiiycuk for finishing this two and a half years after the PR was opened!)

[sbc100]

So to get this out we need a release of Binaryen and then a release of Emscripten that bumps the Binaryen version. What's the Binaryen release policy? Can we get a release for this?

Not really answering you question, but you can always get the latest emscripten+binaryen with ./emsdk install tot (normally lags commits by about 2 hours)

[caiiiycuk]

(Also, thanks @caiiiycuk for finishing this two and a half years after the PR was opened!)

Thank you. Now I can use normal emscripten to build js-dos :)

[kripken]

PR up to prepare for a release: #8063

[caiiiycuk]

PR up to prepare for a release: #8063

Ahh, I forgot to ask. Is there any way to have a message when reaching "unreachable"? Currently it's not clear why we have an error in runtime, because error message is no sens (unreachable executed).

[kripken]

@caiiiycuk The best way is to call an import with a special name. E.g. the SafeHeap pass will call env.segfault when it segfaults (the JS import does a throw). But that requires changes on the JS side.

Otherwise the stack trace can be useful. But wasm sadly doesn't have a way to report different error types, or even do error logging...

[kripken]

@hoodmane Release 125 is complete.

[fs-eire]

@hoodmane Release 125 is complete.

Do we also need emscripten to upgrade the version to use emsdk with this change?

[hoodmane]

Tip of tree emscripten now has it: emscripten-core/emscripten#25834
If you want to use a released version of emscripten, you'll have to wait for 4.0.21 which will likely be about two weeks.