Hardware keyboard: Web, embedder, and dart:ui

[dkwingsmt]

Description

This PR is part of Hardware Keyboard project, specifically the embedder API, dart:ui API, and Web engine parts. The embedder and dart:ui parts are the foundation to other engine changes, while Web is required to change simultaneously for the API mirroring check.

The hardware keyboard API requires each platform to convert native events to "key data" that meets Flutter's specification and send them via the embedder API. The framework is then able to receive ui.KeyDatas in a regular, unified form, and perform the simplest 1-to-1 conversion into KeyEvent (to be added) and assertions.

The key event model is described at here. Simply put,

• Key events are categorized into down, up, and repeat
• A key press always starts with a down, goes through zero or more repeats, then ends with an up.
• All events throughout the press contains the same physical key and logical key.
• Missing native down or up events will be synthesized to guarantee the regularity.

Public API changes

This PR adds the following APIs to dart:ui:

• enum KeyChange
• class KeyData
• typedef KeyDataCallback
• property PlatformDispatcher.onKeyData
• property SingletonFlutterWindow.onKeyData

This PR adds the following APIs to embedder.h:

• enum FlutterKeyEventKind
• struct FlutterKeyEvent
• function FlutterEngineSendKeyEvent
• typedef FlutterEngineSendKeyEventFnPtr
• property FlutterEngineProcTable.SendKeyEvent

Although the key mapping file claims to be auto-generated, the codegen script is not checked in until flutter/flutter#73440.

Related Issues

• Umbrella: ☂️ Device-specific keyboard events should happen in the embedder flutter#44918

Tests

I added the following tests:

For web engine:

• Single key press, repeat, and release
• Release modifier during a repeated sequence
• Distinguish between left and right modifiers
• Distinguish between normal and numpad digits
• Prevents default when pressing Tab
• Dead keys are distinguishable
• Duplicate down is ignored
• Duplicate ups are skipped
• Conflict from multiple keyboards do not crash
• CapsLock down synthesizes an immediate cancel on macOS
• CapsLock behaves normally on non-macOS
• Key guards: key down events are guarded
• Key guards: key repeated down events refreshes guards
• Key guards: cleared by keyups
• Lock flags of other keys
• Deduce modifier key cancel from modifier field

For embedder:

• KeyDataIsCorrectlySerialized
• KeyDataResponseIsCorrectlyInvoked

Checklist

Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

• I read the contributor guide and followed the process outlined there for submitting PRs.
• I signed the CLA.
• I read and followed the C++, Objective-C, Java style guides for the engine.
• I read the tree hygiene wiki page, which explains my responsibilities.
• I updated/added relevant documentation.
• All existing and new tests are passing.
• I am willing to follow-up on review comments in a timely manner.

Reviewer Checklist

• I have submitted any presubmit flakes in this PR using the engine presubmit flakes form before re-triggering the failure.

Breaking Change

Did any tests fail when you ran them? Please read handling breaking changes.

• No, no existing tests failed, so this is not a breaking change.
• Yes, this is a breaking change. If not, delete the remainder of this section.
  • I wrote a design doc: https://flutter.dev/go/template Replace this with a link to your design doc's short link
  • I got input from the developer relations team, specifically from: Replace with the names of who gave advice
  • I wrote a migration guide: https://flutter.dev/go/breaking-changes-template Replace this with a link to a pull request that adds the migration guide to https://flutter.dev/docs/release/breaking-changes

[dkwingsmt]

cc @gspencergoog

[stuartmorgan-g]

I would expect to see tests for the new embedder APIs (i.e., changes in embedder/test/) as part of this.

[stuartmorgan-g]

The comment should be very clear about the requirements here. E.g., is using a monotonic clock a requirement, or is it okay for later events to have earlier timestamps when there's a time change?

[dkwingsmt]

I see the time stamp of key events are defined as FlutterEngineGetCurrentTime. Currently I'm using as much time from native as possible in my PRs, wondering if it might provide extra precision or information to applications with certain requirement. Do you recommend me to switch to FlutterEngineGetCurrentTime?

[stuartmorgan-g]

Not if we don't need it; I just think we should be explicit about what is (or isn't required). If we're okay with native event time, we should say that.

Is it the case that Flutter itself will be fine with non-monotonic event times?

[stuartmorgan-g]

It's not obvious what this means exactly; what is the definitive source of numbers for logical keys?

Since this API surface is intended for use by third-party developers as well, it's important not to assume too much context in the comments.

[mdebbar]

Is this callback called synchronously? I'm worried that _onKeyData may return before the callback gets a chance to update result.

[dkwingsmt]

It is supposed to. If the code is returned before the handler is called, result will be null and the code will throw on the ! of the return statement. Let me add a comment to explain it.

[stuartmorgan-g]

The changes all LGTM!