Skip to content

Conversation

@abd99
Copy link
Contributor

@abd99 abd99 commented Dec 26, 2020

  • Migrated integration tests to use the new integration_test package
  • Removed old e2e and driver tests
  • Updated the testing guide

closes #622

/cc @domesticmouse @redbrogdon @johnpryan

PS - Feel free to take your time and review this PR after you're back from the holidays.

Copy link
Contributor

@domesticmouse domesticmouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but I'm deferring to @johnpryan on this one =)

@redbrogdon
Copy link
Contributor

@johnpryan, can you give this a look and merge it when you have a chance?

@abd99
Copy link
Contributor Author

abd99 commented Jan 14, 2021

@johnpryan, I would also like to know why did you suggest to follow this directory structure and keep the driver.dart file in the integration_test directory instead of the test_driver directory? I need to migrate the codelab once this PR lands and it would be really beneficial for me while doing it.

@johnpryan
Copy link
Contributor

It's okay to put the driver.dart file in the integration_test/ directory. This is what the counter template currently does:

integration_test/
  app_test.dart
  driver.dart

In the integration testing docs we recommend a test_driver directory, but either is okay since both files are passed to the flutter drive command:

flutter drive \
  --driver=test_driver/integration_test.dart \
  --target=integration_test/foo_test.dart \
  -d <DEVICE_ID>

for (var entry in data.entries) {
print('Writing ${entry.key} to the disk.');
// Default storage destination is the 'build' directory.
await writeResponseData(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs say that the default responseDataCallback is writeResponseData - is there a reason you are overriding it to write separate files?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the performance tests, I have recorded the timeline as well as its summary. If I don't manually separate the data into different files, it gets stored into one single file which becomes huge. My current approach also allows the timeline to be opened using Chrome's tracing tools to further evaluate the performance.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should be able to load a trace into chrome without any additional code here. @dnfield does this sound correct?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The raw trace should be loadable in chrome://tracing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason, it isn't working for me.
Here's what I did to my current integration-test branch:

  1. Removed everything passed to integrationDriver() and kept everything else as it is.
  2. Tried running the perf_test and then loading the integration_response_data.json file into chrome://tracing.

The file was loaded, but no charts or data appeared.

I also tried removing watchPerformance() to make the reported data contain only timelines and not timeline summaries.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the tracing tools aren't able to load multiple timelines at once. Am I missing something here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abd99 Can you file an issue in the Flutter issue tracker?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I'll create one shortly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@abd99 abd99 requested a review from johnpryan January 14, 2021 18:35
@abd99
Copy link
Contributor Author

abd99 commented Jan 14, 2021

In the integration testing docs we recommend a test_driver directory, but either is okay since both files are passed to the flutter drive command:

Great! I knew that it works both ways, but didn't know if there was a reason behind following one way or the other.
Thanks for the explanation!

@redbrogdon redbrogdon merged commit f63c465 into flutter:master Jan 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[testing_app] Migrate to integration_test package

5 participants