-
Notifications
You must be signed in to change notification settings - Fork 6k
Added golden test to make sure that spawn engines work. #23066
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| // Copyright 2020 The Flutter Authors. All rights reserved. | ||
| // Use of this source code is governed by a BSD-style license that can be | ||
| // found in the LICENSE file. | ||
|
|
||
| #import <XCTest/XCTest.h> | ||
|
|
||
| NS_ASSUME_NONNULL_BEGIN | ||
|
|
||
| @interface SpawnEngineTest : XCTestCase | ||
| @property(nonatomic, strong) XCUIApplication* application; | ||
| @end | ||
|
|
||
| NS_ASSUME_NONNULL_END |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| // Copyright 2020 The Flutter Authors. All rights reserved. | ||
| // Use of this source code is governed by a BSD-style license that can be | ||
| // found in the LICENSE file. | ||
|
|
||
| #import "SpawnEngineTest.h" | ||
| #import "GoldenImage.h" | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. might be simpler if you used GoldenTestManager.h There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I had an eye out for this too. These tests are kind of a mess. You don't want to be in a situation where when you add tests you have to edit existing code, you should just be writing new code. That's why I don't like the GoldenTestManager. Anything that seemed to have references to platform view tests I tried to avoid, to help us try to get a perspective on what a test should really have. I didn't have a chance to refactor because my environment is broken. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. GoldenTestManager isn't related to platform view anymore. It was previously but I refactored it a bit a few weeks ago. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is specifically what I'm talking about: This has to be edited to be used. Once I saw "platform view" I avoided it. |
||
| @implementation SpawnEngineTest | ||
|
|
||
| - (void)setUp { | ||
| [super setUp]; | ||
| self.continueAfterFailure = NO; | ||
|
|
||
| self.application = [[XCUIApplication alloc] init]; | ||
| self.application.launchArguments = @[ @"--spawn-engine-works", @"--enable-software-rendering" ]; | ||
| [self.application launch]; | ||
| } | ||
|
|
||
| - (void)testSpawnEngineWorks { | ||
| NSString* prefix = @"golden_spawn_engine_works_"; | ||
| GoldenImage* golden = [[GoldenImage alloc] initWithGoldenNamePrefix:prefix]; | ||
| if (!golden.image) { | ||
| XCTFail(@"unable to find golden image for: %@", prefix); | ||
| } | ||
| XCUIScreenshot* screenshot = [[XCUIScreen mainScreen] screenshot]; | ||
| if (![golden compareGoldenToImage:screenshot.image]) { | ||
| XCTAttachment* screenshotAttachment = [XCTAttachment attachmentWithImage:screenshot.image]; | ||
| screenshotAttachment.name = [golden.goldenName stringByAppendingString:@"_actual"]; | ||
| screenshotAttachment.lifetime = XCTAttachmentLifetimeKeepAlways; | ||
| [self addAttachment:screenshotAttachment]; | ||
|
|
||
| XCTFail(@"Goldens do not match. Follow the steps in the " | ||
| @"README to update golden named %@ if needed.", | ||
| golden.goldenName); | ||
| } | ||
| } | ||
|
|
||
| @end | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -46,6 +46,7 @@ Map<String, ScenarioFactory> _scenarios = <String, ScenarioFactory>{ | |
| 'initial_route_reply': () => InitialRouteReply(PlatformDispatcher.instance), | ||
| 'platform_view_with_continuous_texture': () => PlatformViewWithContinuousTexture(PlatformDispatcher.instance, 'Platform View', id: _viewId++), | ||
| 'bogus_font_text': () => BogusFontText(PlatformDispatcher.instance), | ||
| 'spawn_engine_works' : () => BogusFontText(PlatformDispatcher.instance), | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You're compounding 2 tests. Maybe something generic like PoppableScreenScenario? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I planned on doing this. Unfortunately I spent hours yesterday trying to fix my scenario tests. I keep getting a bad kernel error. I can't figure it out so I can't work on this right now. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
| }; | ||
|
|
||
| Map<String, dynamic> _currentScenarioParams = <String, dynamic>{}; | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -36,6 +36,6 @@ fi | |
| cd ios/Scenarios | ||
| set -o pipefail && xcodebuild -sdk iphonesimulator \ | ||
| -scheme Scenarios \ | ||
| -destination 'platform=iOS Simulator,name=iPhone 8' \ | ||
| -destination 'platform=iOS Simulator,OS=13.0,name=iPhone 8' \ | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what was the issue? Was it behaving different on Xcode 12? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yea, on CI their default OS is 13, if you run it with a recent version of Xcode it will use iOS 14 which will cause all the tests to fail (minor differences in the golden images). |
||
| test \ | ||
| FLUTTER_ENGINE="$FLUTTER_ENGINE" | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was the root of all the problems I experienced the last couple of days. I don't think we need the ScopedTaskRunner. I got confused..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wait rly? Ha, awesome.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, so I was getting crashes inside the raster thread after the spawner was killed. It was a crash talking to the external view embedder. So, it appeared that the raster event queue had a queued event talking to that deleted rasterizer. It turns out the rasterizer that was involved in the crash was actually the spawned engines rasterizer. So, the event was correctly going forward with a live rasterizer but it ultimately was referencing something that had been deleted, the platformViewsController. It's easy to overlook that that is a shared_ptr, even then I didn't expect there to be a crash inside. The shared_ptr is kind of pointless if it doesn't actually maintain ownership of its children (or use weak ptrs).