[Camera] Add lens type information (iOS)

[lenzpaul]

This PR adds lens type information. The goal is to identify whether the lens type is wide, ultra-wide, telephoto, etc., as discussed in flutter/flutter#119908. The iOS implementation is complete, so lens data will now be populated on iOS.

• Introduces a new CameraLensType enum to provide lens type information about the camera (e.g.: ultra-wide, telephoto, ...)
• Adds lensType in the PlatformCameraDescription and CameraDescription classes
• Implements utility functions to convert between PlatformCameraLensType and CameraLensType.
• Updates auto-generated code (using Pigeon) to reflect these changes.

Current CameraDescription for iPhone 11

[
    CameraDescription(
        com.apple.avfoundation.avcapturedevice.built-in_video:0,
        CameraLensDirection.back,
        90
    ),
    CameraDescription(
        com.apple.avfoundation.avcapturedevice.built-in_video:1,
        CameraLensDirection.front,
        90
    ),
    CameraDescription(
        com.apple.avfoundation.avcapturedevice.built-in_video:5,
        CameraLensDirection.back,
        90
    )
]

New CameraDescription for iPhone 11

[
    CameraDescription(
        com.apple.avfoundation.avcapturedevice.built-in_video:0,
        CameraLensDirection.back,
        90,
        CameraLensType.wide
    ),
    CameraDescription(
        com.apple.avfoundation.avcapturedevice.built-in_video:1,
        CameraLensDirection.front,
        90,
        CameraLensType.wide
    ),
    CameraDescription(
        com.apple.avfoundation.avcapturedevice.built-in_video:5,
        CameraLensDirection.back,
        90,
        CameraLensType.ultraWide
    )
]

Fixes flutter/flutter#174390

Pre-launch Checklist

• I read the Contributor Guide and followed the process outlined there for submitting PRs.
• I read the Tree Hygiene page, which explains my responsibilities.
• I read and followed the relevant style guides and ran the auto-formatter. (Unlike the flutter/flutter repo, the flutter/packages repo does use dart format.)
• I signed the CLA.
• The title of the PR starts with the name of the package surrounded by square brackets, e.g. [shared_preferences]
• I linked to at least one issue that this PR fixes in the description above.
• I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy, or this PR is exempt from version changes.
• I updated CHANGELOG.md to add a description of the change, following repository CHANGELOG style, or this PR is exempt from CHANGELOG changes.
• I updated/added relevant documentation (doc comments with ///).
• I added new tests to check the change I am making, or this PR is test-exempt.
• All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

[stuartmorgan-g]

Please see https://github.com/flutter/flutter/blob/master/docs/ecosystem/contributing/README.md#changing-federated-plugins for information on how to make this PR pass CI checks, which is important for review.

[lenzpaul]

Please see https://github.com/flutter/flutter/blob/master/docs/ecosystem/contributing/README.md#changing-federated-plugins for information on how to make this PR pass CI checks, which is important for review.

I made changes as specified by the docs, please let me if there's anything else I need to do

[stuartmorgan-g]

• All existing and new tests are passing.

This PR is currently failing a substantial number of our CI checks, so this checkbox state is not accurate. If you click through the details of each CI task failure you can see the specific issues that will need to be fixed before this would be ready for review.

[lenzpaul]

Hi @stuartmorgan, I fixed tests and most CI errors.

One of the remaining errors is related to versions and the changelog not being updated. But I think that some packages don't require version bumps since there are no changes. Could you confirm if this assumption is correct?

• camera_avfoundation, camera_platform_interface: Version already bumped and CHANGELOG.md updated.
• camera, camera_android, camera_android_camerax, camera_web, camera_windows: No changes, no version bump needed.

The other issue is with formatting. I ran the format command using Flutter Plugin Tools, but there's still an error in camera/camera_avfoundation/ios/camera_avfoundation/Sources/camera_avfoundation/include/camera_avfoundation/QueueUtils.h. Should I manually update the formatting for this file?

[stuartmorgan-g]

But I think that some packages don't require version bumps since there are no changes. Could you confirm if this assumption is correct?

Yes; for packages that don't require any changes at all, you can just revert the dependency_override changes in those packages, at which point the CI will stop flagging them as issue.

The other issue is with formatting. I ran the format command using Flutter Plugin Tools, but there's still an error in camera/camera_avfoundation/ios/camera_avfoundation/Sources/camera_avfoundation/include/camera_avfoundation/QueueUtils.h. Should I manually update the formatting for this file?

This is probably due to a clang-format difference between your local tools and the CI; the CI is the authoritative version, so you'll need to match it by hand in this case, yes.

[stuartmorgan-g]

As a high level review note: as discussed here, we don't model platform interface APIs directly on one platform's APIs. A comparison of lens type information across at least iOS, Android, and web will be a necessary part of evaluating whether the values of this new enum are the ones we want.

[lenzpaul]

What is the best way to discuss this more in-depth? Is discord the best channel ?

[stuartmorgan-g]

Discord or the GitHub issue are both options for discussing API design. For something like this, I think the best place to start would be a comparison table showing how each platform's lens information does or doesn't align with other platforms.

[lenzpaul]

Here is a comparison table @stuartmorgan

Lens Type	iOS	Android	Web	Windows	macOS	Linux
General Support	Available via AVCaptureDeviceType API	Lens type info can be determined using Camera2 API. LENS_FACING for front/back/external, and field of view calculated using focal length (LENS_INFO_AVAILABLE_FOCAL_LENGTHS) and sensor size (SENSOR_INFO_PHYSICAL_SIZE). CameraX does not expose this level of detail.	Lens type info not available. Web APIs (MediaDevices) allow access to cameras but without detailed hardware info on lens types.	Lens type info not available. No native API for detailed lens types, standard webcam info available via DirectShow/MediaFoundation APIs	Lens type info available via AVCaptureDevice API. If the attached camera (e.g., with telephoto or wide lens) supports detailed lens info, the API can retrieve it. However, most built-in or external webcams do not expose multiple lens types	Lens type info not available. No native API for detailed lens types, standard webcam info available via v4l2 (Video4Linux2)
Ultra-Wide	Supported	Supported	No lens type info	No lens type info	Supported	No lens type info
Telephoto	Supported	Supported	No lens type info	No lens type info	Supported	No lens type info
Wide	Supported (default)	Supported (default)	No lens type info	No lens type info	Supported (default)	No lens type info

• iOS avcapturedevicetype
• Android Camera 2 CameraCharacteristics
• On Web, Windows, Linux, and MacOS, there is camera access but no native support to access specific lens types or hardware details.

[stuartmorgan-g]

Thanks; what about the other values in your enum? Are dual, dual-wide, and triple supported on Android? What does "continuity" mean conceptually as a lens type, and is there some more abstract definition of it that could apply to other platforms?

(The issue might be a better place for this, as the table is pretty hard to read in the review comment UI.)

[lenzpaul]

Hi @stuartmorgan , dual, dualWide, triple are not directly supported on Android. But it is possible to get the info, e.g. from the number of physical lenses via the Camera2 API CameraCharacteristics.LOGICAL_MULTI_CAMERA_PHYSICAL_IDS for dual and triple. As for "continuity", I don't think we should include it here after all. It refers to a iPhone camera used as a webcam for a Mac.

I was thinking to change this to support only these three - Ultra-Wide, Wide, Telephoto - for simplicity, and because it covers most common use-cases, without handling more complex multi-lens configurations (e.g., dual, triple).

[stuartmorgan-g]

I was thinking to change this to support only these three - Ultra-Wide, Wide, Telephoto - for simplicity, and because it covers most common use-cases, without handling more complex multi-lens configurations (e.g., dual, triple).

This seems like a reasonable starting place to me; we can always add more options in later PRs if there are compelling use cases and we can do something reasonable across platforms. @bparrishMines what do you think?

[bparrishMines]

Just to clarify, this PR adds CameraLensType to CameraDescription. And this value will only ever be set on iOS because none of the other platforms have a comparable distinction?

This is fine with me for the current architecture of the camera plugin. I typically don't like adding an API to the platform interface that will seemingly only ever be set by one platform because this leads to a lot of confusion as more info is added to the class. I would prefer that camera_avfoundation return a subclass of CameraDescription with the value, but I don't want to set the precedent of importing camera_avfoundation without a more in-depth look at the entire plugin structure.

[lenzpaul]

@bparrishMines Yes at this time I have only made the changes for iOS. The additional info will not be returned on any other platform. Although this sets the table so that we can expand support to Android or other platforms in the future without significant changes to the architecture.

[stuartmorgan-g]

I typically don't like adding an API to the platform interface that will seemingly only ever be set by one platform

It seems plausible that a community PR would add this to camera_android in the future, given the comments in the table that Camera2 supports it. I believe we still expect camera_android to exist for the foreseeable future even though our efforts will be focused on camera_android_camerax, because some complex use cases can't be supported via CameraX.

[stuartmorgan-g]

From triage: @lenzpaul Are you still planning on the discussion related to the enum values?

[lenzpaul]

Note that there seems to be an error in the CI related to Package.resolved. I'm not sure how to resolve this. Could this maybe be related to the XCode version on the build machine? XCode 16.2 was just released...

[hellohuanlin]

LGTM after addressing comments

[lenzpaul]

Hi @stuartmorgan, this is now approved by @hellohuanlin. What are the next steps to merge it? And when would this be available for us to use? Thanks!

[stuartmorgan-g]

From triage: based on Discord discussion, updating this for the camera Swift conversion is in progress

[stuartmorgan-g]

From triage: the camera Swift conversion churn should be nearly complete; we will revisit this after that.

[Piinks]

Greetings from stale PR triage! 👋
Is this still waiting on Swift conversion churn?

[stuartmorgan-g]

The relevant code has been converted to Swift, so this just needs to be reconciled against the current source and then can be landed.

[stuartmorgan-g]

I reconciled this with the changes that have landed in the meantime, most notably the native code file that was changed to Swift and the introduction of the protocols around the SDK types. @hellohuanlin Could you review my conversion of the CameraPlugin changes to Swift?

[hellohuanlin]

re-reviewed the swift conversion part

[hellohuanlin]

nit: can just infer the type with .back

[stuartmorgan-g]

Switched to inferred types for both helpers.

[hellohuanlin]

do we wanna add a dualWide type too?

[stuartmorgan-g]

#7653 (comment)

This was the functionality of the Obj-C code that was already reviewed, and my goal here is simply to update the already-approved code so that it can be landed despite the unrelated churn that had blocked it, since I don't want all the work done by @lenzpaul here to be lost just because it happened to collide with the Swift migration.

[stuartmorgan-g]

Thanks again @lenzpaul ! Sorry that the timing of this PR and the Swift migration ended up causing this to take a lot of extra time to land.