[diffusers]: model management (configure and pre-load)

[keturn]

write config first and then download everything in config,
or download first and write config for downloads?

The diffusers API is happy to load model weights on demand from the huggingface.co model hub. But we'd still like the configure script to download model weights ahead of time because (1) the web UI has no "download in progress" indicator, and (2) we want invoke.py to be able work offline even on its first run.

In its current state, #1583 has two model download functions:

• one that takes some keys defined in INITIAL_MODELS, does some downloads, and then adds the successfully downloaded ones to the local models.yaml.
• another for diffusers, that looks for models in the local models.yaml and loads them to make sure they've been downloaded.

Having both those approaches is a sign of internal confusion and makes it hard to figure out how to specify which things to load.

Since the diffusers API can download models on-demand, I think we should assume it will be used this way at least part of the time, so whatever pre-loading method the configure script uses should be compatible with that.

The most straightforward thing would be to do what download_diffusers_in_config does now:

InvokeAI/scripts/configure_invokeai.py

Lines 428 to 430 in 23eb80b

	print(f" * Downloading diffusers {model_name}...")
	cache.get_model(model_name)
	cache.offload_model(model_name)

so there's never any difference in the download-cache-load code between the ModelCache and the configure script, but I'm not sure whether that offers the same set of functionality as configure_invokeai's current download functions regarding progress indicators, behavior with interrupted downloads, etc.

There's still some uncertainty about what Invoke will do with ldm-flavored model files after our #1583 diffusers integration, but for new installations I think the configure process should get the diffusers flavored files.

There is a version of INITIAL_MODELS that's diffusers-friendly here: 6e4dad6 (#1583)

(I didn't find a hosted version of the Trinart Characters models, so I left those out.)

[hipsterusername]

I'll also note, I think there is value in being able to boot the WebUI without a model. This would allow users to skip configure, download the support models, and then boot the WebUI to scan a folder and upload.

As far as ldm vs diffusers models, well... you know my opinion there.

[keturn]

done, configure script now gets diffusers-format models.