Incremental Diffusion

[deepdiffuser]

Is your feature request related to a problem? Please describe.

An interesting feature for consumers of the diffusers API may be incremental diffusion, that is, the ability to parse diffusion results at every n steps during the pipeline run.

Describe the solution you'd like
One way of implementing this is through a callback as an optional kwarg to the diffusion pipeline, as here (deepdiffuser@af6ebda), but I'm open to other suggestions.

Describe alternatives you've considered
I've considered just unrolling the diffusion loop into the caller, and this works, but it might still be interesting to have this as a feature in the pipeline.

Additional context

• My attempt at implementing this: deepdiffuser@af6ebda

[patrickvonplaten]

Hey @deepdiffuser, I think we could easily add such a pipeline under the https://github.com/huggingface/diffusers/tree/main/examples/community folder. Would you like to give it a try? Would love to advertise it!

[jamestiotio]

I am a proponent of adding this feature to the official examples/pipelines.

Looking through the original Stable Diffusion repository, it seems that their implementation provides the parameters callback and img_callback here, here, and here. Adding this feature would further align the available API between diffusers and the original Stable Diffusion repository.

Furthermore, exposing such callbacks in the official examples would allow for further customization by consumers of diffusers and follow the easy-to-tweak philosophy defined in this README. One use case for this feature might include updating a user interface to show the current status of inference.

I believe that adding this to the official examples would benefit users of the diffusers library significantly. I also don't mind creating a PR for this feature if the maintainers are okay with this!

[patrickvonplaten]

A PR would be great @jamestiotio ! Thanks a lot for offering to do this.

Think then we could also see better what the best place is :-) Note that we follow this philosophy for diffusion pipelines: https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines#contribution

[ayan4m1]

Just tagging on to this, I don't think I solved the problem the same way you're all discussing, but this is how I got that functionality: ayan4m1@fc2d3dd