allow passing components to connected pipelines when use the combined pipeline

[yiyixuxu]

As I'm working on the fine tune training script for Kandinsky, I noticed when we use the combined pipeline to load both prior and decoder pipelines together, we can't pass the prior pipeline's components, e.g. we will not be able to do this

pipe = AutoPipelineForText2Image.from_pretrained(
    "kandinsky-community/kandinsky-2-2-decoder", 
    torch_dtype=torch.float16,
    prior_image_encoder = image_encoder)

Below test currently fails, but will be fixed with this PR

from diffusers import AutoPipelineForText2Image, DDPMScheduler, UnCLIPScheduler
import torch

pipe = AutoPipelineForText2Image.from_pretrained(
    "kandinsky-community/kandinsky-2-2-decoder", 
    torch_dtype=torch.float16)

# confirm that the default prior_scheduler is unclip
assert pipe.prior_scheduler.__class__.__name__ == "UnCLIPScheduler"

scheduler = DDPMScheduler.from_pretrained("kandinsky-community/kandinsky-2-2-decoder", subfolder="scheduler")

pipe = AutoPipelineForText2Image.from_pretrained(
    "kandinsky-community/kandinsky-2-2-decoder", 
    torch_dtype=torch.float16,
    prior_scheduler = scheduler)
pipe = pipe.to("cuda")

# should update the prior_scheduler to ddpm
assert pipe.prior_scheduler.__class__.__name__ == "DDPMScheduler"

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[patrickvonplaten]

Very cool! I was actually thinking about the exact same design yesterday since we need it for #3849 as well!

Could you maybe add some tests and then more than happy to merge this one :-)

[patrickvonplaten]

Good to merge for me - thanks!

[sayakpaul]

Clean and simple!

Let's fix the failing CI before merging, though.

Hmm, the failing test actually seems to be flaky as mentioned in #4842.