Deprecate ImageDecoderInit's premultiplyAlpha?

[ccameron-chromium]

This issue is to request removal of the premultiplyAlpha entry from ImageDecoderInit.

The primary reason is that it should have no observable effects on any of the consumers of the resulting VideoFrame. In particular:

• If the VideoFrame is drawn to a canvas, then the behavior is unaffected by whether or not the format is premultiplied.
• If the VideoFrame is uploaded to a WebGL texture via texImage2D, then whether or not the texture's pixel values are in premultiplied or non-premultiplied format depends on the UNPACK_PREMULTIPLY_ALPHA_WEBGL state (same as HTMLImageElement, HTMLCanvasElement, ImageData -- everything except ImageBitmap).
• If the VideoFrame is turned into an ImageBitmap (e.g, to upload to WebGPU), then the ImageBitmapOptions will specify premultiplication.
• If the VideoFrame is copied using CopyTo, and the output format is RGBA or BGRA, then the resulting pixels are to be in un-premultiplied format.

The secondary reason is that, if respected by an implementation, it can constrain the implementation in ways that may be suboptimal. For example:

• Many image formats are images internally represented as YUV(A). To specify that an VideoFrame is to be premultiplied is to force the implementation to convert it to RGBA, which often uses more memory, is often less efficient (because drawing it uses more bandwidth), and loses some precision
• Image decode is almost always done on the CPU. Requiring conversion to be done at decode pushes implementations towards doing pixel format conversion operations on the CPU, which is generally less efficient than doing them on the GPU.
• Relatedly, if the VideoFrame is to be copied to a WebGL or WebGPU texture, and is indeed desired to be premultiplied, on which specific GPU to do the conversion isn't known at decode time.

Unlike the ImageBitmapOptions' premultiplyAlpha parameter that it was based on, it can be removed without breaking any compatibility.

• WebGL specifies that, for ImageBitmaps, the value of ImageBitmapOption's premultiplyAlpha parameter is to be used instead of UNPACK_PREMULTIPLY_ALPHA_WEBGL (see 6.10 Pixel Storage Parameters).
• WebGL does not specify this behavior (or the similar UNPACK_FLIP_Y_WEBGL behavior) for VideoFrame.
• If ImageBitmapOptions does not set premultiplyAlpha (leaves it as "default"), then the result is uploaded to a WebGL texture, then the result is technically undefined. In practice, the result is premultiplied, and that behavior would be inverted were that parameter deprecated, which would likely break many applications.

On a related topic, the effect of the parameter colorSpaceConversion is to "ignore any embedded color profile, and assume all specified pixels are sRGB". The name does not convey this behavior clearly, but does match the name and behavior of ImageBitmapOptions and UNPACK_COLORSPACE_CONVERSION_WEBGL.

[dalecurtis]

@aboba @padenot @chcunningham

As the original author I support this deprecation. In retrospect the parameter doesn't make sense and shouldn't have been brought over from ImageBitmapOptions. Clients wishing for this functionality can still call createImageBitmap() on the resultant VideoFrame to get premultiplied behavior.

[aboba]

Agree that the premultiplyAlpha entry from ImageDecoderInit appears unnecessary at best. Should we remove it, or move it to a section documenting deprecated functionality? Also, I'm curious whether there is any usage.

[dalecurtis]

I don't know of anyone using it at this time. The major ImageDecoder users are Google Docs and Flutter. We'll add a UseCounter as part of the intent to deprecate process. I suspect we can just remove it given no current usage.

[dalecurtis]

Ping @padenot - does this seem reasonable? We're seeing almost no usage of this feature, so are planning to deprecate.

[dalecurtis]

Ping @padenot , have you had a chance to consider this?

[padenot]

Yeah, we can probably remove this, especially if the only implementation sees no usage. Sorry for the delay.

[dalecurtis]

Great, closing this as resolved and will send intent to deprecate for Chromium.