AttributeError: 'numpy.ndarray' object has no attribute 'device' [OnnxPipeline] [Schedulers DDPM, Euler, Euler-A]

[averad]

Describe the bug

PR 636 and PR 944 added the Euler and Euler-A schedulers. When using the DDPM, Euler or Euler-A schedulers with the Onnx pipeline w/DirectML the following error occurs:

AttributeError: 'numpy.ndarray' object has no attribute 'device'

Reproduction

Test environment setup following Stable Diffusion for AMD GPUs on Windows using DirectML

from diffusers import OnnxStableDiffusionPipeline, DDPMScheduler, EulerDiscreteScheduler, EulerAncestralDiscreteScheduler
height=512
width=512
num_inference_steps=30
guidance_scale=7.5
prompt = "a photo of an astronaut riding a horse on mars"
negative_prompt="bad hands, blurry"
#scheduler = DDPMScheduler()
#scheduler = EulerDiscreteScheduler()
scheduler = EulerAncestralDiscreteScheduler()
pipe = OnnxStableDiffusionPipeline.from_pretrained("./models/stable_diffusion_v1-5-vae_mse_onnx", provider="DmlExecutionProvider", scheduler=scheduler, safety_checker=None)
image = pipe(prompt, height, width, num_inference_steps, guidance_scale, negative_prompt).images[0] 
image.save("astronaut_rides_horse.png")

Logs

Traceback (most recent call last):
  File "D:\ai\1.py", line 19, in <module>
    image = pipe(prompt, height, width, num_inference_steps, guidance_scale, negative_prompt).images[0]
  File "D:\ai\sd_env\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_onnx_stable_diffusion.py", line 262, in __call__
    latents = self.scheduler.step(noise_pred, t, torch.from_numpy(latents), **extra_step_kwargs).prev_sample
  File "D:\ai\sd_env\lib\site-packages\diffusers\schedulers\scheduling_ddpm.py", line 295, in step
    device = model_output.device
AttributeError: 'numpy.ndarray' object has no attribute 'device'

Traceback (most recent call last):
  File "D:\ai\1.py", line 17, in <module>
    image = pipe(prompt, height, width, num_inference_steps, guidance_scale, negative_prompt).images[0]
  File "D:\ai\sd_env\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_onnx_stable_diffusion.py", line 262, in __call__
    latents = self.scheduler.step(noise_pred, t, torch.from_numpy(latents), **extra_step_kwargs).prev_sample
  File "D:\ai\sd_env\lib\site-packages\diffusers\schedulers\scheduling_euler_discrete.py", line 221, in step
    device = model_output.device
AttributeError: 'numpy.ndarray' object has no attribute 'device'

Traceback (most recent call last):
  File "D:\ai\1.py", line 14, in <module>
    image = pipe(prompt, height, width, num_inference_steps, guidance_scale, negative_prompt).images[0]
  File "D:\ai\sd_env\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_onnx_stable_diffusion.py", line 262, in __call__
    latents = self.scheduler.step(noise_pred, t, torch.from_numpy(latents), **extra_step_kwargs).prev_sample
  File "D:\ai\sd_env\lib\site-packages\diffusers\schedulers\scheduling_euler_ancestral_discrete.py", line 224, in step
    device = model_output.device
AttributeError: 'numpy.ndarray' object has no attribute 'device'

System Info

• diffusers version: 0.8.0.dev0
• Platform: Windows-11
• Python version: 3.10.8
• PyTorch version (GPU?): 1.13.0+cpu (False)
• Huggingface_hub version: 0.10.1
• Transformers version: 4.24.0
• Using GPU in script?: RX580

[patrickvonplaten]

Gently pinging @anton-l here

[anton-l]

Thank you for reporting @averad! Adding yet another numpy-> torch cast to support those schedulers: #1328

@echarlaix and I are planning to introduce IO binding to the pipelines to avoid all of this copying and casting of the tensors, which will considerably speed up GPU inference as well :)

[averad]

@anton-l tested #1328 and verified the reported issue is fixed. Thank you!