diff --git a/docs/apis/imaging.md b/docs/apis/imaging.md index d437d707..d1ff233b 100644 --- a/docs/apis/imaging.md +++ b/docs/apis/imaging.md @@ -15,6 +15,7 @@ Imaging features in Windows AI Foundry support the following capabilities: - [**Image Super Resolution**](#what-can-i-do-with-image-super-resolution): scaling and sharpening an image. - [**Image Description**](#what-can-i-do-with-image-description): generating text that describes an image. - [**Image Segmentation**](#what-can-i-do-with-image-segmentation): identifying objects within an image. +- [**Image Foreground Extractor**](#what-can-i-do-with-object-erase): foreground/background segmentation on an input image - [**Object Erase**](#what-can-i-do-with-object-erase): removing objects from an image. For **API details**, see [API ref for AI imaging features](/windows/windows-app-sdk/api/winrt/microsoft.windows.ai.imaging). @@ -353,6 +354,64 @@ ImageObjectExtractorHint hint( ); ``` +## What can I do with Image Foreground Extractor ? + +Image foreground extractor can be used to perform foreground or background segmentation on an input image. This API can be used for tasks like background removal, sticker generation, etc. + +The returned mask is in grayscale-8 format with the pixels of the mask for the foreground having a value of 255 (background pixels having a value of 0). + +### Generating a Mask from a SoftwareBitmap + +1. Ensure the `ImageForegroundExtractor` is ready by calling `GetReadyState()` and waiting for the EnsureReadyAsync method to return successfully. +2. Once the model is ready, you can create an instance of `ImageForegroundExtractor` using the `CreateAsync` method. +3. Pass the input image to ImageForegroundExtractor.GetMaskFromSoftwareBitmap() to obtain the foreground mask. + +```csharp +using Microsoft.Windows.AI.Imaging; +using Microsoft.Windows.AI; + +if (ImageForegroundExtractor::GetReadyState() == AIFeatureReadyState.EnsureNeeded) +{ + var result = await ImageObjectRemover.EnsureReadyAsync(); + if (result.Status != PackageDeploymentStatus.CompletedSuccess) + { + throw result.ExtendedError; + } +} + +var model = await ImageForegroundExtractor.CreateAsync(); + +// Insert your own softwareBitmap here +var foregroundMask = model.GetMaskFromSoftwareBitmap(softwareBitmap); +``` + +```cpp +#include +#include +#include +#include +using namespace winrt::Microsoft::Graphics::Imaging; +using namespace winrt::Microsoft::Windows::AI.Imaging; +using namespace winrt::Windows::Graphics::Imaging; +using namespace winrt::Windows::Foundation; + +if (ImageForegroundExtractor::GetReadyState() == AIFeatureReadyState::NotReady) +{ + auto loadResult = ImageForegroundExtractor::EnsureReadyAsync().get(); + + if (loadResult.Status() != AIFeatureReadyResultState::Success) + { + throw winrt::hresult_error(loadResult.ExtendedError()); + } +} + +auto model = co_await ImageForegroundExtractor::CreateAsync(); + +// Insert your own softwareBitmap here +auto foregroundMask = model.GetMaskFromSoftwareBitmap(softwareBitmap); + +``` + ## What can I do with Object Erase? Object Erase can be used to remove objects from images. The model takes both an image and a greyscale mask indicating the object to be removed, erases the masked area from the image, and replaces the erased area with the image background.