Skip to content

Commit 6e81ac5

Browse files
piEspositopiEsposito
committed
Merge branch 'main' of github.com:piEsposito/diffusers into main
2 parents 965dfe1 + 9af3535 commit 6e81ac5

33 files changed

+5044
-209
lines changed

‎.github/actions/setup-miniconda/action.yml‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
name: Set up conda environment for testing
2+
3+
description: Sets up miniconda in your ${RUNNER_TEMP} environment and gives you the ${CONDA_RUN} environment variable so you don't have to worry about polluting non-empeheral runners anymore
4+
5+
inputs:
6+
python-version:
7+
description: If set to any value, dont use sudo to clean the workspace
8+
required: false
9+
type: string
10+
default: "3.9"
11+
miniconda-version:
12+
description: Miniconda version to install
13+
required: false
14+
type: string
15+
default: "4.12.0"
16+
environment-file:
17+
description: Environment file to install dependencies from
18+
required: false
19+
type: string
20+
default: ""
21+
22+
runs:
23+
using: composite
24+
steps:
25+
# Use the same trick from https://github.com/marketplace/actions/setup-miniconda
26+
# to refresh the cache daily. This is kind of optional though
27+
- name: Get date
28+
id: get-date
29+
shell: bash
30+
run: echo "::set-output name=today::$(/bin/date -u '+%Y%m%d')d"
31+
- name: Setup miniconda cache
32+
id: miniconda-cache
33+
uses: actions/cache@v2
34+
with:
35+
path: ${{ runner.temp }}/miniconda
36+
key: miniconda-${{ runner.os }}-${{ runner.arch }}-${{ inputs.python-version }}-${{ steps.get-date.outputs.today }}
37+
- name: Install miniconda (${{ inputs.miniconda-version }})
38+
if: steps.miniconda-cache.outputs.cache-hit != 'true'
39+
env:
40+
MINICONDA_VERSION: ${{ inputs.miniconda-version }}
41+
shell: bash -l {0}
42+
run: |
43+
MINICONDA_INSTALL_PATH="${RUNNER_TEMP}/miniconda"
44+
mkdir -p "${MINICONDA_INSTALL_PATH}"
45+
case ${RUNNER_OS}-${RUNNER_ARCH} in
46+
Linux-X64)
47+
MINICONDA_ARCH="Linux-x86_64"
48+
;;
49+
macOS-ARM64)
50+
MINICONDA_ARCH="MacOSX-arm64"
51+
;;
52+
macOS-X64)
53+
MINICONDA_ARCH="MacOSX-x86_64"
54+
;;
55+
*)
56+
echo "::error::Platform ${RUNNER_OS}-${RUNNER_ARCH} currently unsupported using this action"
57+
exit 1
58+
;;
59+
esac
60+
MINICONDA_URL="https://repo.anaconda.com/miniconda/Miniconda3-py39_${MINICONDA_VERSION}-${MINICONDA_ARCH}.sh"
61+
curl -fsSL "${MINICONDA_URL}" -o "${MINICONDA_INSTALL_PATH}/miniconda.sh"
62+
bash "${MINICONDA_INSTALL_PATH}/miniconda.sh" -b -u -p "${MINICONDA_INSTALL_PATH}"
63+
rm -rf "${MINICONDA_INSTALL_PATH}/miniconda.sh"
64+
- name: Update GitHub path to include miniconda install
65+
shell: bash
66+
run: |
67+
MINICONDA_INSTALL_PATH="${RUNNER_TEMP}/miniconda"
68+
echo "${MINICONDA_INSTALL_PATH}/bin" >> $GITHUB_PATH
69+
- name: Setup miniconda env cache (with env file)
70+
id: miniconda-env-cache-env-file
71+
if: ${{ runner.os }} == 'macOS' && ${{ inputs.environment-file }} != ''
72+
uses: actions/cache@v2
73+
with:
74+
path: ${{ runner.temp }}/conda-python-${{ inputs.python-version }}
75+
key: miniconda-env-${{ runner.os }}-${{ runner.arch }}-${{ inputs.python-version }}-${{ steps.get-date.outputs.today }}-${{ hashFiles(inputs.environment-file) }}
76+
- name: Setup miniconda env cache (without env file)
77+
id: miniconda-env-cache
78+
if: ${{ runner.os }} == 'macOS' && ${{ inputs.environment-file }} == ''
79+
uses: actions/cache@v2
80+
with:
81+
path: ${{ runner.temp }}/conda-python-${{ inputs.python-version }}
82+
key: miniconda-env-${{ runner.os }}-${{ runner.arch }}-${{ inputs.python-version }}-${{ steps.get-date.outputs.today }}
83+
- name: Setup conda environment with python (v${{ inputs.python-version }})
84+
if: steps.miniconda-env-cache-env-file.outputs.cache-hit != 'true' && steps.miniconda-env-cache.outputs.cache-hit != 'true'
85+
shell: bash
86+
env:
87+
PYTHON_VERSION: ${{ inputs.python-version }}
88+
ENV_FILE: ${{ inputs.environment-file }}
89+
run: |
90+
CONDA_BASE_ENV="${RUNNER_TEMP}/conda-python-${PYTHON_VERSION}"
91+
ENV_FILE_FLAG=""
92+
if [[ -f "${ENV_FILE}" ]]; then
93+
ENV_FILE_FLAG="--file ${ENV_FILE}"
94+
elif [[ -n "${ENV_FILE}" ]]; then
95+
echo "::warning::Specified env file (${ENV_FILE}) not found, not going to include it"
96+
fi
97+
conda create \
98+
--yes \
99+
--prefix "${CONDA_BASE_ENV}" \
100+
"python=${PYTHON_VERSION}" \
101+
${ENV_FILE_FLAG} \
102+
cmake=3.22 \
103+
conda-build=3.21 \
104+
ninja=1.10 \
105+
pkg-config=0.29 \
106+
wheel=0.37
107+
- name: Clone the base conda environment and update GitHub env
108+
shell: bash
109+
env:
110+
PYTHON_VERSION: ${{ inputs.python-version }}
111+
CONDA_BASE_ENV: ${{ runner.temp }}/conda-python-${{ inputs.python-version }}
112+
run: |
113+
CONDA_ENV="${RUNNER_TEMP}/conda_environment_${GITHUB_RUN_ID}"
114+
conda create \
115+
--yes \
116+
--prefix "${CONDA_ENV}" \
117+
--clone "${CONDA_BASE_ENV}"
118+
# TODO: conda-build could not be cloned because it hardcodes the path, so it
119+
# could not be cached
120+
conda install --yes -p ${CONDA_ENV} conda-build=3.21
121+
echo "CONDA_ENV=${CONDA_ENV}" >> "${GITHUB_ENV}"
122+
echo "CONDA_RUN=conda run -p ${CONDA_ENV} --no-capture-output" >> "${GITHUB_ENV}"
123+
echo "CONDA_BUILD=conda run -p ${CONDA_ENV} conda-build" >> "${GITHUB_ENV}"
124+
echo "CONDA_INSTALL=conda install -p ${CONDA_ENV}" >> "${GITHUB_ENV}"
125+
- name: Get disk space usage and throw an error for low disk space
126+
shell: bash
127+
run: |
128+
echo "Print the available disk space for manual inspection"
129+
df -h
130+
# Set the minimum requirement space to 4GB
131+
MINIMUM_AVAILABLE_SPACE_IN_GB=4
132+
MINIMUM_AVAILABLE_SPACE_IN_KB=$(($MINIMUM_AVAILABLE_SPACE_IN_GB * 1024 * 1024))
133+
# Use KB to avoid floating point warning like 3.1GB
134+
df -k | tr -s ' ' | cut -d' ' -f 4,9 | while read -r LINE;
135+
do
136+
AVAIL=$(echo $LINE | cut -f1 -d' ')
137+
MOUNT=$(echo $LINE | cut -f2 -d' ')
138+
if [ "$MOUNT" = "/" ]; then
139+
if [ "$AVAIL" -lt "$MINIMUM_AVAILABLE_SPACE_IN_KB" ]; then
140+
echo "There is only ${AVAIL}KB free space left in $MOUNT, which is less than the minimum requirement of ${MINIMUM_AVAILABLE_SPACE_IN_KB}KB. Please help create an issue to PyTorch Release Engineering via https://github.com/pytorch/test-infra/issues and provide the link to the workflow run."
141+
exit 1;
142+
else
143+
echo "There is ${AVAIL}KB free space left in $MOUNT, continue"
144+
fi
145+
fi
146+
done

‎.github/workflows/pr_tests.yml‎

Lines changed: 53 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: Run non-slow tests
1+
name: Run fast tests
22

33
on:
44
pull_request:
@@ -10,14 +10,14 @@ concurrency:
1010
cancel-in-progress: true
1111

1212
env:
13-
HF_HOME: /mnt/cache
1413
OMP_NUM_THREADS: 8
1514
MKL_NUM_THREADS: 8
1615
PYTEST_TIMEOUT: 60
16+
MPS_TORCH_VERSION: 1.13.0
1717

1818
jobs:
1919
run_tests_cpu:
20-
name: Diffusers tests
20+
name: CPU tests on Ubuntu
2121
runs-on: [ self-hosted, docker-gpu ]
2222
container:
2323
image: python:3.7
@@ -39,7 +39,7 @@ jobs:
3939
run: |
4040
python utils/print_env.py
4141
42-
- name: Run all non-slow selected tests on CPU
42+
- name: Run all fast tests on CPU
4343
run: |
4444
python -m pytest -n 2 --max-worker-restart=0 --dist=loadfile -s -v --make-reports=tests_torch_cpu tests/
4545
@@ -51,5 +51,53 @@ jobs:
5151
if: ${{ always() }}
5252
uses: actions/upload-artifact@v2
5353
with:
54-
name: pr_torch_test_reports
54+
name: pr_torch_cpu_test_reports
55+
path: reports
56+
57+
run_tests_apple_m1:
58+
name: MPS tests on Apple M1
59+
runs-on: [ self-hosted, apple-m1 ]
60+
61+
steps:
62+
- name: Checkout diffusers
63+
uses: actions/checkout@v3
64+
with:
65+
fetch-depth: 2
66+
67+
- name: Clean checkout
68+
shell: arch -arch arm64 bash {0}
69+
run: |
70+
git clean -fxd
71+
72+
- name: Setup miniconda
73+
uses: ./.github/actions/setup-miniconda
74+
with:
75+
python-version: 3.9
76+
77+
- name: Install dependencies
78+
shell: arch -arch arm64 bash {0}
79+
run: |
80+
${CONDA_RUN} python -m pip install --upgrade pip
81+
${CONDA_RUN} python -m pip install -e .[quality,test]
82+
${CONDA_RUN} python -m pip install --pre torch==${MPS_TORCH_VERSION} --extra-index-url https://download.pytorch.org/whl/test/cpu
83+
84+
- name: Environment
85+
shell: arch -arch arm64 bash {0}
86+
run: |
87+
${CONDA_RUN} python utils/print_env.py
88+
89+
- name: Run all fast tests on MPS
90+
shell: arch -arch arm64 bash {0}
91+
run: |
92+
${CONDA_RUN} python -m pytest -n 4 --max-worker-restart=0 --dist=loadfile -s -v --make-reports=tests_torch_mps tests/
93+
94+
- name: Failure short reports
95+
if: ${{ failure() }}
96+
run: cat reports/tests_torch_mps_failures_short.txt
97+
98+
- name: Test suite reports artifacts
99+
if: ${{ always() }}
100+
uses: actions/upload-artifact@v2
101+
with:
102+
name: pr_torch_mps_test_reports
55103
path: reports

‎README.md‎

Lines changed: 12 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -210,14 +210,16 @@ You can also run this example on colab [![Open In Colab](https://colab.research.
210210

211211
### In-painting using Stable Diffusion
212212

213-
The `StableDiffusionInpaintPipeline` lets you edit specific parts of an image by providing a mask and text prompt.
213+
The `StableDiffusionInpaintPipeline` lets you edit specific parts of an image by providing a mask and a text prompt. It uses a model optimized for this particular task, whose license you need to accept before use.
214214

215-
```python
216-
from io import BytesIO
215+
Please, visit the [model card](https://huggingface.co/runwayml/stable-diffusion-inpainting), read the license carefully and tick the checkbox if you agree. Note that this is an additional license, you need to accept it even if you accepted the text-to-image Stable Diffusion license in the past. You have to be a registered user in 🤗 Hugging Face Hub, and you'll also need to use an access token for the code to work. For more information on access tokens, please refer to [this section](https://huggingface.co/docs/hub/security-tokens) of the documentation.
217216

218-
import torch
219-
import requests
217+
218+
```python
220219
import PIL
220+
import requests
221+
import torch
222+
from io import BytesIO
221223

222224
from diffusers import StableDiffusionInpaintPipeline
223225

@@ -231,21 +233,15 @@ mask_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data
231233
init_image = download_image(img_url).resize((512, 512))
232234
mask_image = download_image(mask_url).resize((512, 512))
233235

234-
device = "cuda"
235-
model_id_or_path = "CompVis/stable-diffusion-v1-4"
236236
pipe = StableDiffusionInpaintPipeline.from_pretrained(
237-
model_id_or_path,
238-
revision="fp16",
237+
"runwayml/stable-diffusion-inpainting",
238+
revision="fp16",
239239
torch_dtype=torch.float16,
240240
)
241-
# or download via git clone https://huggingface.co/CompVis/stable-diffusion-v1-4
242-
# and pass `model_id_or_path="./stable-diffusion-v1-4"`.
243-
pipe = pipe.to(device)
244-
245-
prompt = "a cat sitting on a bench"
246-
images = pipe(prompt=prompt, init_image=init_image, mask_image=mask_image, strength=0.75).images
241+
pipe = pipe.to("cuda")
247242

248-
images[0].save("cat_on_bench.png")
243+
prompt = "Face of a yellow cat, high resolution, sitting on a park bench"
244+
image = pipe(prompt=prompt, image=init_image, mask_image=mask_image).images[0]
249245
```
250246

251247
### Tweak prompts reusing seeds and latents

‎docs/source/api/pipelines/overview.mdx‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -151,10 +151,10 @@ You can generate your own latents to reproduce results, or tweak your prompt on
151151
The `StableDiffusionInpaintPipeline` lets you edit specific parts of an image by providing a mask and text prompt.
152152

153153
```python
154-
from io import BytesIO
155-
156-
import requests
157154
import PIL
155+
import requests
156+
import torch
157+
from io import BytesIO
158158

159159
from diffusers import StableDiffusionInpaintPipeline
160160

@@ -170,15 +170,15 @@ mask_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data
170170
init_image = download_image(img_url).resize((512, 512))
171171
mask_image = download_image(mask_url).resize((512, 512))
172172

173-
device = "cuda"
174173
pipe = StableDiffusionInpaintPipeline.from_pretrained(
175-
"CompVis/stable-diffusion-v1-4", revision="fp16", torch_dtype=torch.float16
176-
).to(device)
177-
178-
prompt = "a cat sitting on a bench"
179-
images = pipe(prompt=prompt, init_image=init_image, mask_image=mask_image, strength=0.75).images
174+
"runwayml/stable-diffusion-inpainting",
175+
revision="fp16",
176+
torch_dtype=torch.float16,
177+
)
178+
pipe = pipe.to("cuda")
180179

181-
images[0].save("cat_on_bench.png")
180+
prompt = "Face of a yellow cat, high resolution, sitting on a park bench"
181+
image = pipe(prompt=prompt, image=init_image, mask_image=mask_image).images[0]
182182
```
183183

184184
You can also run this example on colab [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/in_painting_with_stable_diffusion_using_diffusers.ipynb)

‎docs/source/using-diffusers/inpaint.mdx‎

Lines changed: 26 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -12,13 +12,19 @@ specific language governing permissions and limitations under the License.
1212

1313
# Text-Guided Image-Inpainting
1414

15-
The [`StableDiffusionInpaintPipeline`] lets you edit specific parts of an image by providing a mask and text prompt.
15+
The [`StableDiffusionInpaintPipeline`] lets you edit specific parts of an image by providing a mask and a text prompt. It uses a version of Stable Diffusion specifically trained for in-painting tasks.
1616

17-
```python
18-
from io import BytesIO
17+
<Tip warning={true}>
18+
Note that this model is distributed separately from the regular Stable Diffusion model, so you have to accept its license even if you accepted the Stable Diffusion one in the past.
1919

20-
import requests
20+
Please, visit the [model card](https://huggingface.co/runwayml/stable-diffusion-inpainting), read the license carefully and tick the checkbox if you agree. You have to be a registered user in 🤗 Hugging Face Hub, and you'll also need to use an access token for the code to work. For more information on access tokens, please refer to [this section](https://huggingface.co/docs/hub/security-tokens) of the documentation.
21+
</Tip>
22+
23+
```python
2124
import PIL
25+
import requests
26+
import torch
27+
from io import BytesIO
2228

2329
from diffusers import StableDiffusionInpaintPipeline
2430

@@ -34,15 +40,24 @@ mask_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data
3440
init_image = download_image(img_url).resize((512, 512))
3541
mask_image = download_image(mask_url).resize((512, 512))
3642

37-
device = "cuda"
3843
pipe = StableDiffusionInpaintPipeline.from_pretrained(
39-
"CompVis/stable-diffusion-v1-4", revision="fp16", torch_dtype=torch.float16
40-
).to(device)
44+
"runwayml/stable-diffusion-inpainting",
45+
revision="fp16",
46+
torch_dtype=torch.float16,
47+
)
48+
pipe = pipe.to("cuda")
49+
50+
prompt = "Face of a yellow cat, high resolution, sitting on a park bench"
51+
image = pipe(prompt=prompt, image=init_image, mask_image=mask_image).images[0]
52+
```
4153

42-
prompt = "a cat sitting on a bench"
43-
images = pipe(prompt=prompt, init_image=init_image, mask_image=mask_image, strength=0.75).images
54+
`image` | `mask_image` | `prompt` | **Output** |
55+
:-------------------------:|:-------------------------:|:-------------------------:|-------------------------:|
56+
<img src="https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png" alt="drawing" width="250"/> | <img src="https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png" alt="drawing" width="250"/> | ***Face of a yellow cat, high resolution, sitting on a park bench*** | <img src="https://huggingface.co/datasets/patrickvonplaten/images/resolve/main/test.png" alt="drawing" width="250"/> |
4457

45-
images[0].save("cat_on_bench.png")
46-
```
4758

4859
You can also run this example on colab [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/in_painting_with_stable_diffusion_using_diffusers.ipynb)
60+
61+
<Tip warning={true}>
62+
A previous experimental implementation of in-painting used a different, lower-quality process. To ensure backwards compatibility, loading a pretrained pipeline that doesn't contain the new model will still apply the old in-painting method.
63+
</Tip>

0 commit comments

Comments
 (0)