Use pipelines without vae

[hlky]

What does this PR do?

As in #10439 and #10440 this PR guards some checks involving vae in a pipeline's __init__ that prevent it from being used without vae i.e. for text encoder only.

This is mainly vae_scale_factor, the default is set to 8 as this is the most common value. vae_scale_factor effects several parts of the pipeline, default height/width calculation, prepare_latents, check_inputs etc. Pipelines are generally designed to allow use without loading vae with output_type="latents" for example, so it will be important to identify the pipelines where vae_scale_factor is not 8.

In some Flux pipelines we add a guard for self.vae.config.latent_channels used in mask_processor with default 16, this value is taken from Flux configs.

Checking issues with pipeline's __call__ and output_type="latents" will be covered in another PR. Example in FluxFillPipeline and the fix already present in FluxPipeline:

diffusers/src/diffusers/pipelines/flux/pipeline_flux_fill.py

Line 843 in c28db0a

num_channels_latents = self.vae.config.latent_channels

diffusers/src/diffusers/pipelines/flux/pipeline_flux.py

Line 810 in c28db0a

num_channels_latents = self.transformer.config.in_channels // 4

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sayakpaul @yiyixuxu

[sayakpaul]

Suggested change

	self.vae_scale_factor = (
	2 ** (len(self.vae.config.block_out_channels) - 1) if hasattr(self, "vae") and self.vae is not None else 8
	)
	self.vae_scale_factor = (
	2 ** (len(self.vae.config.block_out_channels) - 1) if getattr(self, "vae", None) is not None else 8
	)

leaner.

[hlky]

Thanks, I've applied this everywhere.

[sayakpaul]

Single command. Thanks for this!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[yiyixuxu]

thanks! very nice initiative