Skip to content

Conversation

@vaind
Copy link
Collaborator

@vaind vaind commented Jan 9, 2025

When working on replay I found myself needing to evaluate performance of possible solutions. This PR adds a scaffolding to run benchmarks as well as some benchmarks related to screenshot capture.

For reviewers: this is just flutter create microbenchmarks with updates to pubspec.yaml and lib/** + added the folder to the main package .pubignore

Current results from an old test device (Huawei Mate 20 Pro):

Images

Render object size: 800.0x600.0
Image size: 800x600

RenderRepaintBoundary.toImage()
          total runs:      2 537
          total time:     2.0006  s
         average run:        788 μs
         runs/second:    1 269.0
Image.toByteData(rawRgba)
          total runs:        463
          total time:     2.0003  s
         average run:     4.3200 ms
         runs/second:     231.48
Image.toByteData(rawStraightRgba)
          total runs:        206
          total time:     2.0002  s
         average run:      9.709 ms
         runs/second:     103.00
Image.toByteData(rawUnmodified)
          total runs:        506
          total time:     2.0004  s
         average run:     3.9530 ms
         runs/second:     252.97
Image.toByteData(rawExtendedRgba128)
          total runs:        161
          total time:     2.0031  s
         average run:     12.441 ms
         runs/second:     80.379
Image.toByteData(png)
          total runs:         54
          total time:     2.0203  s
         average run:     37.412 ms
         runs/second:     26.729
Image to Picture
          total runs:    305 508
          total time:     2.0000  s
         average run:          6 μs
         runs/second:    166 667
Picture.toImage()
          total runs:      1 865
          total time:     2.0000  s
         average run:     1.0720 ms
         runs/second:     932.84
Canvas.drawRect() x 100
          total runs:     19 291
          total time:     2.0000  s
         average run:        103 μs
         runs/second:    9 708.7

Memory

listEquals()
          total runs:      1 020
          total time:     2.0004  s
         average run:     1.9610 ms
         runs/second:     509.94
byteDataGetUint64()
          total runs:      1 111
          total time:     2.0007  s
         average run:     1.8000 ms
         runs/second:     555.56
uint64Lists()
          total runs:      8 032
          total time:     2.0000  s
         average run:        249 μs
         runs/second:    4 016.1
nativeMemcmp()
          total runs:     21 823
          total time:     2.0000  s
         average run:         91 μs
         runs/second:     10 989

#skip-changelog

@codecov
Copy link

codecov bot commented Jan 9, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.00%. Comparing base (0c08054) to head (56c8a38).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2565      +/-   ##
==========================================
+ Coverage   87.15%   92.00%   +4.84%     
==========================================
  Files         266       89     -177     
  Lines        9447     3102    -6345     
==========================================
- Hits         8234     2854    -5380     
+ Misses       1213      248     -965     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025

iOS Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1245.24 ms 1253.06 ms 7.82 ms
Size 8.42 MiB 9.88 MiB 1.46 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
c732386 1233.20 ms 1252.08 ms 18.88 ms
934f10e 1245.18 ms 1263.77 ms 18.59 ms
dd933d4 1238.73 ms 1252.43 ms 13.70 ms
e3d9076 1203.68 ms 1230.65 ms 26.97 ms
1b0c8a3 1259.30 ms 1282.78 ms 23.48 ms
02661eb 1244.45 ms 1252.37 ms 7.92 ms
1a93825 1257.25 ms 1261.55 ms 4.30 ms
00236a7 1250.06 ms 1274.00 ms 23.94 ms
8a10ab7 1226.49 ms 1250.52 ms 24.03 ms
3f23617 1261.93 ms 1286.10 ms 24.17 ms

App size

Revision Plain With Sentry Diff
c732386 8.28 MiB 9.33 MiB 1.05 MiB
934f10e 8.38 MiB 9.77 MiB 1.39 MiB
dd933d4 8.33 MiB 9.64 MiB 1.31 MiB
e3d9076 8.33 MiB 9.40 MiB 1.07 MiB
1b0c8a3 8.38 MiB 9.75 MiB 1.37 MiB
02661eb 8.42 MiB 9.86 MiB 1.44 MiB
1a93825 8.28 MiB 9.34 MiB 1.05 MiB
00236a7 8.38 MiB 9.77 MiB 1.39 MiB
8a10ab7 8.28 MiB 9.34 MiB 1.06 MiB
3f23617 8.16 MiB 9.17 MiB 1.01 MiB

Previous results on branch: chore/flutter-benchmarks

Startup times

Revision Plain With Sentry Diff
d3090bb 1257.50 ms 1275.60 ms 18.10 ms

App size

Revision Plain With Sentry Diff
d3090bb 8.42 MiB 9.88 MiB 1.46 MiB

@github-actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025

Android Performance metrics 🚀

  Plain With Sentry Diff
Startup time 477.30 ms 569.16 ms 91.86 ms
Size 6.46 MiB 7.48 MiB 1.01 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
4b943a1 348.17 ms 437.15 ms 88.98 ms
1596141 300.92 ms 368.94 ms 68.02 ms
afa6e2a 349.73 ms 428.48 ms 78.75 ms
9e7630d 449.43 ms 475.82 ms 26.39 ms
ee0ca56 355.35 ms 421.13 ms 65.78 ms
07cd9e8 427.71 ms 461.23 ms 33.52 ms
2e93bab 515.33 ms 558.79 ms 43.46 ms
9a5040f 490.15 ms 563.98 ms 73.83 ms
f735167 404.38 ms 412.57 ms 8.19 ms
3adbea9 395.16 ms 447.88 ms 52.71 ms

App size

Revision Plain With Sentry Diff
4b943a1 6.34 MiB 7.28 MiB 968.41 KiB
1596141 6.16 MiB 7.14 MiB 1003.98 KiB
afa6e2a 6.27 MiB 7.20 MiB 955.69 KiB
9e7630d 6.49 MiB 7.56 MiB 1.07 MiB
ee0ca56 6.33 MiB 7.30 MiB 992.52 KiB
07cd9e8 6.49 MiB 7.56 MiB 1.07 MiB
2e93bab 6.49 MiB 7.55 MiB 1.07 MiB
9a5040f 6.46 MiB 7.48 MiB 1.01 MiB
f735167 6.46 MiB 7.48 MiB 1.01 MiB
3adbea9 6.52 MiB 7.61 MiB 1.09 MiB

Previous results on branch: chore/flutter-benchmarks

Startup times

Revision Plain With Sentry Diff
d3090bb 470.84 ms 534.43 ms 63.60 ms

App size

Revision Plain With Sentry Diff
d3090bb 6.46 MiB 7.48 MiB 1.01 MiB

@vaind vaind marked this pull request as ready for review January 9, 2025 16:08
@vaind vaind merged commit ec50b21 into main Jan 13, 2025
56 checks passed
@vaind vaind deleted the chore/flutter-benchmarks branch January 13, 2025 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants