Stable Diffusion 2 doesn't work with `enable_attention_slicing()`

[patrickvonplaten]

Describe the bug

As pointed out here: #1484 (comment) attention slicing doesn't work with SD 2.

Reproduction

import torch
from diffusers import StableDiffusionPipeline, EulerDiscreteScheduler

model_id = "stabilityai/stable-diffusion-2"

# Use the Euler scheduler here instead
scheduler = EulerDiscreteScheduler.from_pretrained(model_id, subfolder="scheduler")
pipe = StableDiffusionPipeline.from_pretrained(model_id, scheduler=scheduler, revision="fp16", torch_dtype=torch.float16)
pipe = pipe.to("cuda")
# pipe.enable_attention_slicing() # seems to not affect this behavior either way

prompt = "a photo of an astronaut riding a horse on mars"

# orig example size, works fine
# image = pipe(prompt, height=768, width=768).images[0]

# widescreen version was what I wanted, and this works fine too
# image = pipe(prompt, height=512, width=1080).images[0]

# but at the "1080p" res, a black image every time (I think)
image = pipe(prompt, height=1080, width=1920).images[0]
    
image.save("astronaut_rides_horse.png")

Logs

OOM which shouldn't be

System Info

- `diffusers` version: 0.10.0.dev0
- Platform: Linux-4.19.0-22-cloud-amd64-x86_64-with-debian-10.13
- Python version: 3.7.12
- PyTorch version (GPU?): 1.12.1+cu113 (True)
- Huggingface_hub version: 0.11.0.dev0
- Transformers version: 4.25.0.dev0
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

[patrickvonplaten]

Solved by: #1532