Skip to content

Conversation

@piEsposito
Copy link
Contributor

@piEsposito piEsposito commented Oct 31, 2022

Follow up of #1024

Adds enable_sequential_cpu_offload method to StableDiffusionInpaintPipeline and StableDiffusionImg2ImgPipeline alongs with proper tests.

Also renames the test on StableDiffusionPipeline.enable_sequential_cpu_offload to test_stable_diffusion_pipeline_with_sequential_cpu_offloading for clarity.

@patrickvonplaten here I'm just adding the enable_sequential_cpu_offload method to the img2img and inpainting pipelines for Stable Diffusion.

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Oct 31, 2022

The documentation is not available anymore as the PR was closed or merged.

@piEsposito piEsposito marked this pull request as ready for review October 31, 2022 17:48
@piEsposito
Copy link
Contributor Author

piEsposito commented Oct 31, 2022

@patrickvonplaten the tests that are not passing seem to be related to the runwayml/stable-diffusion-v1-5, which you can see I did not edit at all.

Same happened with #1010 btw.

Edit: nevermind you fixed it lightning fast.

@piEsposito
Copy link
Contributor Author

piEsposito commented Nov 2, 2022

@patrickvonplaten I've added @pedrogengo to d6fcb91 because he found the bug that this commit solves and helped me find the solution.

@pedrogengo
Copy link
Contributor

@piEsposito After line 402 of inpaint pipeline file you should add:

masked_image_latents = masked_image_latents.to(device=self.device, dtype=text_embeddings.dtype)

To make sure that all tensors are on the same device.
Before add this line the inference was throwing an error for me (RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument tensors in method wrapper_cat)). However, after add this, everything worked fine!

Unfortunately, I could not comment direct on the file.

@piEsposito
Copy link
Contributor Author

@patrickvonplaten thank you for the kindness of addreessing your review.

@patrickvonplaten patrickvonplaten merged commit 1172c96 into huggingface:main Nov 4, 2022
@patrickvonplaten
Copy link
Contributor

Thanks a lot the PR :-)

@anton-l anton-l mentioned this pull request Nov 7, 2022
PhaneeshB pushed a commit to nod-ai/diffusers that referenced this pull request Mar 1, 2023
* make some paths windows friendly (huggingface#1066)

* add csv output to builder script

and reduce number of models tested
yoonseokjin pushed a commit to yoonseokjin/diffusers that referenced this pull request Dec 25, 2023
…nes (huggingface#1085)

* add enable sequential cpu offloading to other stable diffusion pipelines

* trigger ci

* fix styling

* interpolate before converting to device to avoid breking when cpu_offload is enabled with fp16

Co-authored-by: Pedro Gengo  <[email protected]>

* style again I need to stop forgething this thing

* fix inpainting bug that could cause device misalignment

Co-authored-by: Pedro Gengo  <[email protected]>

* Apply suggestions from code review

Co-authored-by: Pedro Gengo  <[email protected]>
Co-authored-by: Patrick von Platen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants