screen sharing for desktop.

[cloudwebrtc]

Currently, flutter-webrtc solves macOS/Windows screen/window enumeration and can select specific sources for sharing.
The following issues still need to be resolved

Capture screen

• macOS
• Windows

Capture window

• macOS
• Windows

Watch window changes to refresh the selection dialog, if some windows are closed, or the title changes, refresh the list.

• macOS
• Windows

Monitor the status of the window/screen, for example, if we close the window that is in capturing, or unplug the second display HDMI cable, we need to send a notification, and the client SDK needs to automatically close the sharing, remove the video source track.

• macOS
• Windows

Another question, should we move the example/lib/widgets/screen_select_dialog.dart file inside the SDK? this In this case, we need to pass in a BuildContext to build the Widget when starting the SDK.

• Add capture source selection dialog to SDK.

related PRs:
flutter-webrtc/flutter-webrtc#1015
webrtc-sdk/libwebrtc#36
webrtc-sdk/webrtc#24

[jdde]

Hi @cloudwebrtc when it's possible to handover a deviceId (window selection) to the ScreenShareCaptureOptions I can definitely test the implementation :) just let me know, I could directly use this branch in our develop version & test it against a production system - happy to help!

[cloudwebrtc]

@jdde I'm almost done with screen sharing for windows and mac, and we have a desktop source selection dialog, but I still need a few days to improve the code and testing, which will land on the Livekit flutter SDK soon.

flutter-webrtc/flutter-webrtc#1015

[cloudwebrtc]

when it's possible to handover a deviceId (window selection) to the ScreenShareCaptureOptions I can definitely test the implementation :)

The idea is to integrate the screen selection dialog directly inside the SDK, when we use localParticipant.selectDesktopSource(BuildContext context);
In a similar way, the SDK will ask you to choose a screen or window, and you can share your screen

[jdde]

Ah nicy!
But on our side we will definitely integrate a custom selection screen. So maybe it makes sense to split that story on your side and take an iterative approach to first adjust the api in data layer to make it possible to implement such a view and then make another change, which includes this dialog?
With that approach, we could already provide earlier feedback :)

[cloudwebrtc]

@jdde Yes, we can construct LocalVideoTrack by passing in MediaStream or VideoTrack through ScreenShareCaptureOptions.

In this case, you will be able to use
https://github.com/flutter-webrtc/flutter-webrtc/blob/feat/screen-capture-event-listener-for-win/lib/src/desktop_capturer.dart#L50
to create a custom select box. And create a MediaStream through getDiaplayMedia.

Both methods will work at the same time

[cloudwebrtc]

@davidliu @hiroshihorie @davidzhao

I think this PR can be reviewed. CI still has an error because the git repo is referenced as a test, when flutter-webrtc is merged I will change to the version on pub.

There are also several larger PRs related to this issue: xD
flutter-webrtc/flutter-webrtc#1015
webrtc-sdk/libwebrtc#36
webrtc-sdk/webrtc#24

[jdde]

@cloudwebrtc I've already tested our current feature branch for screensharing on macOS with this branch as a dependency.
We're using a custom selector and hand over the sourceId.
Works really well! didn't noticed any problems. awesome work!!!

[jdde]

@cloudwebrtc okay, maybe there's one problem - but I'm not yet sure if it's connected, I updated the dependencies again to the newest flutter-webrtc version and now I'm not getting the system popups anymore, where macOS asks for permission to access camera, microphone.
Cleaned pods, cleaned flutter, fresh install. Will investigate deeper, and make sure it's because of that dependency change.

• Update: nevermind. built a release version, it's still there.

[cloudwebrtc]

hey @jdde, when your own project depends on the latest version of flutter-webrtc or livekit-sdk, after running flutter run -d macos for the first time in macOS, you can try open macos/Runner.xcworkspace, and then check the following options

In macOS also make sure the following items are added to Info.plist
https://github.com/flutter-webrtc/flutter-webrtc#ios

<key>NSCameraUsageDescription</key>
<string>$(PRODUCT_NAME) Camera Usage!</string>
<key>NSMicrophoneUsageDescription</key>
<string>$(PRODUCT_NAME) Microphone Usage!</string>

[jdde]

Yess, I know, it's already configured in our project since months, and it's also already build with those permissions. All entries are there (also for debug & production targets), but just since this morning, macOS is not opening the popup anymore (in debug).

Anyway, looks like the issue is not connected to this pull-request :)

• Update: restarting my system solved it...

[cloudwebrtc]

yeah, I have also encountered a similar situation, it is easy to appear in the development environment, or m1 mac, you can try the following sqlite command to directly replace or add permission info.

sudo sqlite3 ~/Library/Application\ Support/com.apple.TCC/TCC.db "INSERT or REPLACE INTO access VALUES('kTCCServiceMicrophone','com.your.app.bundle.id',0,0,4,1,NULL,NULL,0,'UNUSED',NULL,0,1622199671);"
sudo sqlite3 ~/Library/Application\ Support/com.apple.TCC/TCC.db "INSERT or REPLACE INTO access VALUES('kTCCServiceCamera','com.your.app.bundle.id',0,0,4,1,NULL,NULL,0,'UNUSED',NULL,0,1622199671);"
sudo sqlite3 ~/Library/Application\ Support/com.apple.TCC/TCC.db "INSERT or REPLACE INTO access VALUES('kTCCServiceScreenCapture','com.your.app.bundle.id',0,0,4,1,NULL,NULL,0,'UNUSED',NULL,0,1622199671);"

and replace com.your.app.bundle.id with your app's id

[jdde]

Awesome! Thanks a lot for this hint! Will directly integrate it in our documentation. Never had this case before oO And yes, I'm using M1 - obviously a problem with macOS.

[davidzhao]

lgtm!