[Community Pipeline] Mix Prompting pipeline?

[daspartho]

Idea

I've been playing with this concept of mixed embeddings. The premise is simple: the pipeline takes a list of prompts and generates images based on a mix of embeddings of the prompts. It produces some interesting mixed-up outcomes.

Example

Prompt 1: a mountain, cinematic angle, studio Ghibli, cinematic lighting, detailed oil painting, hyperrealistic, 8k
Prompt 2: A long and winding beach, tropical, bright, simple, by Studio Ghibli and Greg Rutkowski, artstation

Prompt 1 only

Prompt 2 only

Prompt 1 + Prompt 2

Implementation

I've put up a working implementation for this here. I could open up a PR for a community pipeline for this if that's something folks would like to be added. #841

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[Randolph-zeng]

There is a ICLR 2022 paper "PROMPT-TO-PROMPT IMAGE EDITING WITH CROSS-ATTENTION CONTROL" that does similar but more things into the pipeline. There are also open sourced code in the web, would be interesting to add them to Diffusers too @patrickvonplaten https://github.com/google/prompt-to-prompt

[patrickvonplaten]

Hey @Randolph-zeng, we are working on enabling exactly this use case: #1639

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.