allow multiple generations per prompt

[patil-suraj]

To generate multiple images for a prompt currently we need to repeat the prompts before calling the pipeline.

pipe = StableDiffusionPipeline.from_pretrained(
    "CompVis/stable-diffusion-v1-4", 
    revision="fp16", 
    torch_dtype=torch.float16,
)
pipe = pipe.to("cuda")

prompt = "a photo of an astronaut riding a horse on mars"
images = pipe([prompt] * 2).images

Because of this the text embeddings and uncond embeddings are computed multiple times for the same prompt.

This PR adds num_images_per_prompt argument to stable diffusion pipelines to allow returning multiple images per prompt without repeating them. With this the text embeddings and uncond embeddings for each prompt are computed once and repeated according to the value of num_images_per_prompt.

prompt = "a photo of an astronaut riding a horse on mars"
images = pipe(prompt, num_images_per_prompt=2).images
assert len(images) == 2

Thanks @NouamaneTazi !

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[pcuenca]

Looks good!

We should also do it in the flax version for faster inference, but we can do so later.

[NouamaneTazi]

Amazing PR, thank you for taking care of this 🙏🏼

[NouamaneTazi]

Last time I checked I had issues with repeat_interleave and ONNX. We probably just need to require a specific opset version though.
Can you please double-check @anton-l?

[patil-suraj]

Thanks, will check that. But this PR doesn't modify onnx pipeline, so it should be fine.

[patrickvonplaten]

Nice!

[rvorias]

In the current implementation, num_images_per_prompt is just a batch_size scaler.

e.g. if you want 9 images, the current implementation multiplies the batch size by 9. Good luck with fitting that on a consumer GPU 😁

When I saw this arg I was expecting it to have sequential pipeline runs (with just the same prompt). A lot of other repos have it like this.

Extra kudos: when num_images_per_prompt > 1, also allow a list of seeds intake.

[patrickvonplaten]

We need indeed some good warning / error throwing system here

[rvorias]

We need indeed some good warning / error throwing system here

Hmm I'm not sure this has anything to do with warning or errors. Just pointing out that -from a user perspective- batch_size and num_images_per_prompt are the same.

[patrickvonplaten]

Yeah but batch_size cannot be passed in stable diffusion pipeline as an input argument no?

[patil-suraj]

Thanks for the comment @rvorias ! It would be bit complicated to run pipe in sequence inside the pipe, the users can do it very easily, by just calling the pipeline in a loop, this way you as a users have full control over it. We'll document this better in examples, docs :)