Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions docs/apis/imaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Imaging features in Windows AI Foundry support the following capabilities:
- [**Image Super Resolution**](#what-can-i-do-with-image-super-resolution): scaling and sharpening an image.
- [**Image Description**](#what-can-i-do-with-image-description): generating text that describes an image.
- [**Image Segmentation**](#what-can-i-do-with-image-segmentation): identifying objects within an image.
- [**Image Foreground Extractor**](#what-can-i-do-with-object-erase): foreground/background segmentation on an input image
- [**Object Erase**](#what-can-i-do-with-object-erase): removing objects from an image.

For **API details**, see [API ref for AI imaging features](/windows/windows-app-sdk/api/winrt/microsoft.windows.ai.imaging).
Expand Down Expand Up @@ -353,6 +354,64 @@ ImageObjectExtractorHint hint(
);
```

## What can I do with Image Foreground Extractor ?

Image foreground extractor can be used to perform foreground or background segmentation on an input image. This API can be used for tasks like background removal, sticker generation, etc.

The returned mask is in grayscale-8 format with the pixels of the mask for the foreground having a value of 255 (background pixels having a value of 0).

### Generating a Mask from a SoftwareBitmap

1. Ensure the `ImageForegroundExtractor` is ready by calling `GetReadyState()` and waiting for the EnsureReadyAsync method to return successfully.
2. Once the model is ready, you can create an instance of `ImageForegroundExtractor` using the `CreateAsync` method.
3. Pass the input image to ImageForegroundExtractor.GetMaskFromSoftwareBitmap() to obtain the foreground mask.

```csharp
using Microsoft.Windows.AI.Imaging;
using Microsoft.Windows.AI;

if (ImageForegroundExtractor::GetReadyState() == AIFeatureReadyState.EnsureNeeded)
{
var result = await ImageObjectRemover.EnsureReadyAsync();
if (result.Status != PackageDeploymentStatus.CompletedSuccess)
{
throw result.ExtendedError;
}
}

var model = await ImageForegroundExtractor.CreateAsync();

// Insert your own softwareBitmap here
var foregroundMask = model.GetMaskFromSoftwareBitmap(softwareBitmap);
```

```cpp
#include <winrt/Microsoft.Graphics.Imaging.h>
#include <winrt/Microsoft.Windows.AI.Imaging.h>
#include <winrt/Windows.Graphics.Imaging.h>
#include <winrt/Windows.Foundation.h>
using namespace winrt::Microsoft::Graphics::Imaging;
using namespace winrt::Microsoft::Windows::AI.Imaging;
using namespace winrt::Windows::Graphics::Imaging;
using namespace winrt::Windows::Foundation;

if (ImageForegroundExtractor::GetReadyState() == AIFeatureReadyState::NotReady)
{
auto loadResult = ImageForegroundExtractor::EnsureReadyAsync().get();

if (loadResult.Status() != AIFeatureReadyResultState::Success)
{
throw winrt::hresult_error(loadResult.ExtendedError());
}
}

auto model = co_await ImageForegroundExtractor::CreateAsync();

// Insert your own softwareBitmap here
auto foregroundMask = model.GetMaskFromSoftwareBitmap(softwareBitmap);

```

## What can I do with Object Erase?

Object Erase can be used to remove objects from images. The model takes both an image and a greyscale mask indicating the object to be removed, erases the masked area from the image, and replaces the erased area with the image background.
Expand Down