Broken canary tests

Canary tests are horribly broken since we parallelized our E2E tests.

This is probably due to multiple reasons:
- Running multiple `yarn install`s at the same time corrupts the yarn cache for some reason
- Our port assigning logic assigns the same ports for multiple canary tests
- Timeouts due to the runner being overwhelmed by too many processes being started