[cfe] Fail early on some subtype checks instead of crashing

Some subtype tests are crashing due to infinite recursion. It is a
known issue. An example of such test is FutureOr<T> against
FutureOr<S> where T and S are type variables declared as T extends
FutureOr<T> and S extends FutureOr<S>.

In some cases it's possible to fail early instead of crashing, for
exmaple, by checking Future-branch of the left-hand side first before
checking the type argumennt of FutureOr which would lead to the
infinite recursion. This CL does that, allowing some more type checks
to pass instead of crashing.

Change-Id: I075aa65522510db1a97ac2a8de7278a3ce3f400c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/227820
Reviewed-by: Johnni Winther <[email protected]>
Commit-Queue: Chloe Stefantsova <[email protected]>

Author: chloestefantsova

pkg/front_end/test/fasta/types/shared_type_tests.dart

@@ -426,6 +426,10 @@ abstract class SubtypeTest<T, E> {
         typeParameters: 'T extends Object');
     isObliviousSubtype('T & Future<int?>', 'FutureOr<num>?',
         typeParameters: 'T extends Object');
+    isNotSubtype('FutureOr<T?>?', 'FutureOr<Never?>?',
+        typeParameters: 'T extends FutureOr<T?>?');
+    isSubtype('FutureOr<Never?>?', 'FutureOr<T?>?',
+        typeParameters: 'T extends FutureOr<T?>?');
 
     if (!skipFutureOrPromotion) {
       isSubtype('T & FutureOr<int*>*', 'FutureOr<num*>*', typeParameters: 'T');

pkg/kernel/lib/src/types.dart

@@ -404,12 +404,11 @@ class IsInterfaceSubtypeOf extends TypeRelation<InterfaceType> {
   IsSubtypeOf isFutureOrRelated(FutureOrType s, InterfaceType t, Types types) {
     // Rules 7.1 and 7.2.
     return types
-        .performNullabilityAwareSubtypeCheck(s.typeArgument, t)
-        .andSubtypeCheckFor(
+        .performNullabilityAwareSubtypeCheck(
             new InterfaceType(types.hierarchy.coreTypes.futureClass,
                 Nullability.nonNullable, [s.typeArgument]),
-            t,
-            types)
+            t)
+        .andSubtypeCheckFor(s.typeArgument, t, types)
         .and(new IsSubtypeOf.basedSolelyOnNullabilities(s, t));
   }