Consider adding a custom recording format

[liyuqian]

The custom recording format could be based on https://skia.googlesource.com/skia/+/refs/heads/android/10-release/src/core/SkLiteDL.h . We can use it to replace the SkPicture format to allow us to have more control. It could bring us

1. More performance optimization opportunities regarding picture raster caches
2. A potentially better way (than skp or sksl) to do shader warm-up
3. A potentially better way to capture and analyze frame than skp.

[liyuqian]

@chinmaygarde : I couldn't find such an issue in our database so I created this one. Please feel free to edit it.

[dnfield]

For memory usage, it would be helpful if this could support:

• Reusing objects that are the same/come from the same (unmutated) Dart object
• Efficiently determine if an expensive resource (such as an image) can be disposed of early
• Traceability for developers/devtools to know what code has instantiated or is retaining parts of a particular display list

[kf6gpe]

@chinmaygarde Are we actively working on this? I don't think so. I'm tempted to downgrade this to a P5 if we're not.

[yjbanov]

This doesn't feel like a P3, unless perhaps if we decide that this is a blocker for solving jank. FWIW on the web we implemented this in order to manage picture memory on Safari without weak refs: https://github.com/flutter/engine/blob/dcb57f003bd3f5f8fbb3c75cd94443e8096ade94/lib/web_ui/lib/src/engine/canvaskit/canvas.dart#L269

We don't use it anywhere except Safari though, and we don't get the benefits described here.

[dnfield]

This would solve a number of issues for us. I believe it's planned for the near term. Chinmay or @flar might know for sure

[flar]

It has been moving up the priority list. Some of the not so distant term issues it could solve:

• Collapsing opacity layers into picture layers: OpacityLayer causes sharp increases in memory usage in trivial apps #75697
• Better determination of when pictures actually change for DiffContext: Add DiffContext engine#21824

From there it could be a launching pad for other issues like:

• Being able to split and merge pictures at the framework level when nodes contributing to a shader picture start/stop animating without having to repaint everything.
• Minimize overhead of regenerating pictures when the contents are not changing by first comparing new operations to an existing picture and only forking it and creating a new picture when the contents change (this is an alternate solution to the previous issue, but with other benefits)

In addition to the list of benefits in the Description.