The first image generated with LMSScheduler is completely green

[allo-]

Describe the bug

The first image generated with LMSScheduler is just a green image.

Reproduction

Run with:

lms = LMSDiscreteScheduler(beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear"
pipe = StableDiffusionPipeline.from_pretrained(..., scheduler=lms)

instead of using the default scheduler. Then the first image is completely green. The workaround I am using is to run one step before the actual diffuse function:

if isinstance(pipe.scheduler, LMSDiscreteScheduler):
    # Prevent an empty first frame
    with autocast("cuda"):
        image = diffuse(pipe, text_embeddings, init1, 1, settings["guidance_scale"], eta)

Afterward I compute images similarly, but using e.g. 50 steps instead of one and everything is fine. The default scheduler also seems to work. I did not test other schedulers yet.

System Info

Diffusers from git, python 3.10.6, CUDA with a RTX 3060 card

[patrickvonplaten]

cc @anton-l

[patrickvonplaten]

Probably related to: #336

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[patrickvonplaten]

@allo- is this issue still relevant?

[allo-]

I removed the workaround and cannot reproduce the issue. I guess I close here for now and if my scripts without the workarounds get the problem again I reopen or make a new issue.

[anton-l]

Yes, the issue was caused by improper LMS timesteps initialization and should be fixed now :)