Skip to content

Commit 80d56ce

Browse files
shungcpshungcp
authored
Merge pull request #1 from qihqi/main
Add wan_tx
2 parents f0a3f27 + 9df4b1f commit 80d56ce

File tree

7 files changed

+372
-230
lines changed

7 files changed

+372
-230
lines changed

‎README.md‎

Lines changed: 19 additions & 227 deletions
Original file line numberDiff line numberDiff line change
@@ -1,239 +1,31 @@
1-
<!---
2-
Copyright 2022 - The HuggingFace Team. All rights reserved.
1+
Original readme moved to README_original.md
32

4-
Licensed under the Apache License, Version 2.0 (the "License");
5-
you may not use this file except in compliance with the License.
6-
You may obtain a copy of the License at
7-
8-
http://www.apache.org/licenses/LICENSE-2.0
9-
10-
Unless required by applicable law or agreed to in writing, software
11-
distributed under the License is distributed on an "AS IS" BASIS,
12-
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13-
See the License for the specific language governing permissions and
14-
limitations under the License.
15-
-->
16-
17-
<p align="center">
18-
<br>
19-
<img src="https://raw.githubusercontent.com/huggingface/diffusers/main/docs/source/en/imgs/diffusers_library.jpg" width="400"/>
20-
<br>
21-
<p>
22-
<p align="center">
23-
<a href="https://github.com/huggingface/diffusers/blob/main/LICENSE"><img alt="GitHub" src="https://img.shields.io/github/license/huggingface/datasets.svg?color=blue"></a>
24-
<a href="https://github.com/huggingface/diffusers/releases"><img alt="GitHub release" src="https://img.shields.io/github/release/huggingface/diffusers.svg"></a>
25-
<a href="https://pepy.tech/project/diffusers"><img alt="GitHub release" src="https://static.pepy.tech/badge/diffusers/month"></a>
26-
<a href="CODE_OF_CONDUCT.md"><img alt="Contributor Covenant" src="https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg"></a>
27-
<a href="https://twitter.com/diffuserslib"><img alt="X account" src="https://img.shields.io/twitter/url/https/twitter.com/diffuserslib.svg?style=social&label=Follow%20%40diffuserslib"></a>
28-
</p>
29-
30-
🤗 Diffusers is the go-to library for state-of-the-art pretrained diffusion models for generating images, audio, and even 3D structures of molecules. Whether you're looking for a simple inference solution or training your own diffusion models, 🤗 Diffusers is a modular toolbox that supports both. Our library is designed with a focus on [usability over performance](https://huggingface.co/docs/diffusers/conceptual/philosophy#usability-over-performance), [simple over easy](https://huggingface.co/docs/diffusers/conceptual/philosophy#simple-over-easy), and [customizability over abstractions](https://huggingface.co/docs/diffusers/conceptual/philosophy#tweakable-contributorfriendly-over-abstraction).
31-
32-
🤗 Diffusers offers three core components:
33-
34-
- State-of-the-art [diffusion pipelines](https://huggingface.co/docs/diffusers/api/pipelines/overview) that can be run in inference with just a few lines of code.
35-
- Interchangeable noise [schedulers](https://huggingface.co/docs/diffusers/api/schedulers/overview) for different diffusion speeds and output quality.
36-
- Pretrained [models](https://huggingface.co/docs/diffusers/api/models/overview) that can be used as building blocks, and combined with schedulers, for creating your own end-to-end diffusion systems.
37-
38-
## Installation
39-
40-
We recommend installing 🤗 Diffusers in a virtual environment from PyPI or Conda. For more details about installing [PyTorch](https://pytorch.org/get-started/locally/) and [Flax](https://flax.readthedocs.io/en/latest/#installation), please refer to their official documentation.
41-
42-
### PyTorch
43-
44-
With `pip` (official package):
45-
46-
```bash
47-
pip install --upgrade diffusers[torch]
3+
# install
484
```
5+
pip install -e .
6+
pip install transformers accelerate
497
50-
With `conda` (maintained by the community):
51-
52-
```sh
53-
conda install -c conda-forge diffusers
54-
```
55-
56-
### Flax
57-
58-
With `pip` (official package):
59-
60-
```bash
61-
pip install --upgrade diffusers[flax]
8+
# install torchax
9+
pip install git+https://github.com/pytorch/xla.git@hanq_wan_changes
10+
pip install torch --index-url https://download.pytorch.org/whl/cpu
11+
pip install jax[tpu]
6212
```
6313

64-
### Apple Silicon (M1/M2) support
65-
66-
Please refer to the [How to use Stable Diffusion in Apple Silicon](https://huggingface.co/docs/diffusers/optimization/mps) guide.
67-
68-
## Quickstart
69-
70-
Generating outputs is super easy with 🤗 Diffusers. To generate an image from text, use the `from_pretrained` method to load any pretrained diffusion model (browse the [Hub](https://huggingface.co/models?library=diffusers&sort=downloads) for 30,000+ checkpoints):
71-
72-
```python
73-
from diffusers import DiffusionPipeline
74-
import torch
14+
To run:
7515

76-
pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16)
77-
pipeline.to("cuda")
78-
pipeline("An image of a squirrel in Picasso style").images[0]
7916
```
80-
81-
You can also dig into the models and schedulers toolbox to build your own diffusion system:
82-
83-
```python
84-
from diffusers import DDPMScheduler, UNet2DModel
85-
from PIL import Image
86-
import torch
87-
88-
scheduler = DDPMScheduler.from_pretrained("google/ddpm-cat-256")
89-
model = UNet2DModel.from_pretrained("google/ddpm-cat-256").to("cuda")
90-
scheduler.set_timesteps(50)
91-
92-
sample_size = model.config.sample_size
93-
noise = torch.randn((1, 3, sample_size, sample_size), device="cuda")
94-
input = noise
95-
96-
for t in scheduler.timesteps:
97-
with torch.no_grad():
98-
noisy_residual = model(input, t).sample
99-
prev_noisy_sample = scheduler.step(noisy_residual, t, input).prev_sample
100-
input = prev_noisy_sample
101-
102-
image = (input / 2 + 0.5).clamp(0, 1)
103-
image = image.cpu().permute(0, 2, 3, 1).numpy()[0]
104-
image = Image.fromarray((image * 255).round().astype("uint8"))
105-
image
17+
python wan_tx.py
10618
```
10719

108-
Check out the [Quickstart](https://huggingface.co/docs/diffusers/quicktour) to launch your diffusion journey today!
109-
110-
## How to navigate the documentation
111-
112-
| **Documentation** | **What can I learn?** |
113-
|---------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
114-
| [Tutorial](https://huggingface.co/docs/diffusers/tutorials/tutorial_overview) | A basic crash course for learning how to use the library's most important features like using models and schedulers to build your own diffusion system, and training your own diffusion model. |
115-
| [Loading](https://huggingface.co/docs/diffusers/using-diffusers/loading) | Guides for how to load and configure all the components (pipelines, models, and schedulers) of the library, as well as how to use different schedulers. |
116-
| [Pipelines for inference](https://huggingface.co/docs/diffusers/using-diffusers/overview_techniques) | Guides for how to use pipelines for different inference tasks, batched generation, controlling generated outputs and randomness, and how to contribute a pipeline to the library. |
117-
| [Optimization](https://huggingface.co/docs/diffusers/optimization/fp16) | Guides for how to optimize your diffusion model to run faster and consume less memory. |
118-
| [Training](https://huggingface.co/docs/diffusers/training/overview) | Guides for how to train a diffusion model for different tasks with different training techniques. |
119-
## Contribution
120-
121-
We ❤️ contributions from the open-source community!
122-
If you want to contribute to this library, please check out our [Contribution guide](https://github.com/huggingface/diffusers/blob/main/CONTRIBUTING.md).
123-
You can look out for [issues](https://github.com/huggingface/diffusers/issues) you'd like to tackle to contribute to the library.
124-
- See [Good first issues](https://github.com/huggingface/diffusers/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22) for general opportunities to contribute
125-
- See [New model/pipeline](https://github.com/huggingface/diffusers/issues?q=is%3Aopen+is%3Aissue+label%3A%22New+pipeline%2Fmodel%22) to contribute exciting new diffusion models / diffusion pipelines
126-
- See [New scheduler](https://github.com/huggingface/diffusers/issues?q=is%3Aopen+is%3Aissue+label%3A%22New+scheduler%22)
127-
128-
Also, say 👋 in our public Discord channel <a href="https://discord.gg/G7tWnz98XR"><img alt="Join us on Discord" src="https://img.shields.io/discord/823813159592001537?color=5865F2&logo=discord&logoColor=white"></a>. We discuss the hottest trends about diffusion models, help each other with contributions, personal projects or just hang out ☕.
12920

21+
## sizes:
22+
### wan 1.3B:
23+
text_encoder 12.537574768066406 G
24+
transformer 2.64891254901886 G
25+
vae 0.23635575734078884 G
13026

131-
## Popular Tasks & Pipelines
27+
### wan 14B
28+
text_encoder 12.537574768066406 G
29+
transformer 26.66874897480011 G
30+
vae 0.23635575734078884 G
13231

133-
<table>
134-
<tr>
135-
<th>Task</th>
136-
<th>Pipeline</th>
137-
<th>🤗 Hub</th>
138-
</tr>
139-
<tr style="border-top: 2px solid black">
140-
<td>Unconditional Image Generation</td>
141-
<td><a href="https://huggingface.co/docs/diffusers/api/pipelines/ddpm"> DDPM </a></td>
142-
<td><a href="https://huggingface.co/google/ddpm-ema-church-256"> google/ddpm-ema-church-256 </a></td>
143-
</tr>
144-
<tr style="border-top: 2px solid black">
145-
<td>Text-to-Image</td>
146-
<td><a href="https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/text2img">Stable Diffusion Text-to-Image</a></td>
147-
<td><a href="https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5"> stable-diffusion-v1-5/stable-diffusion-v1-5 </a></td>
148-
</tr>
149-
<tr>
150-
<td>Text-to-Image</td>
151-
<td><a href="https://huggingface.co/docs/diffusers/api/pipelines/unclip">unCLIP</a></td>
152-
<td><a href="https://huggingface.co/kakaobrain/karlo-v1-alpha"> kakaobrain/karlo-v1-alpha </a></td>
153-
</tr>
154-
<tr>
155-
<td>Text-to-Image</td>
156-
<td><a href="https://huggingface.co/docs/diffusers/api/pipelines/deepfloyd_if">DeepFloyd IF</a></td>
157-
<td><a href="https://huggingface.co/DeepFloyd/IF-I-XL-v1.0"> DeepFloyd/IF-I-XL-v1.0 </a></td>
158-
</tr>
159-
<tr>
160-
<td>Text-to-Image</td>
161-
<td><a href="https://huggingface.co/docs/diffusers/api/pipelines/kandinsky">Kandinsky</a></td>
162-
<td><a href="https://huggingface.co/kandinsky-community/kandinsky-2-2-decoder"> kandinsky-community/kandinsky-2-2-decoder </a></td>
163-
</tr>
164-
<tr style="border-top: 2px solid black">
165-
<td>Text-guided Image-to-Image</td>
166-
<td><a href="https://huggingface.co/docs/diffusers/api/pipelines/controlnet">ControlNet</a></td>
167-
<td><a href="https://huggingface.co/lllyasviel/sd-controlnet-canny"> lllyasviel/sd-controlnet-canny </a></td>
168-
</tr>
169-
<tr>
170-
<td>Text-guided Image-to-Image</td>
171-
<td><a href="https://huggingface.co/docs/diffusers/api/pipelines/pix2pix">InstructPix2Pix</a></td>
172-
<td><a href="https://huggingface.co/timbrooks/instruct-pix2pix"> timbrooks/instruct-pix2pix </a></td>
173-
</tr>
174-
<tr>
175-
<td>Text-guided Image-to-Image</td>
176-
<td><a href="https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/img2img">Stable Diffusion Image-to-Image</a></td>
177-
<td><a href="https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5"> stable-diffusion-v1-5/stable-diffusion-v1-5 </a></td>
178-
</tr>
179-
<tr style="border-top: 2px solid black">
180-
<td>Text-guided Image Inpainting</td>
181-
<td><a href="https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/inpaint">Stable Diffusion Inpainting</a></td>
182-
<td><a href="https://huggingface.co/runwayml/stable-diffusion-inpainting"> runwayml/stable-diffusion-inpainting </a></td>
183-
</tr>
184-
<tr style="border-top: 2px solid black">
185-
<td>Image Variation</td>
186-
<td><a href="https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/image_variation">Stable Diffusion Image Variation</a></td>
187-
<td><a href="https://huggingface.co/lambdalabs/sd-image-variations-diffusers"> lambdalabs/sd-image-variations-diffusers </a></td>
188-
</tr>
189-
<tr style="border-top: 2px solid black">
190-
<td>Super Resolution</td>
191-
<td><a href="https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/upscale">Stable Diffusion Upscale</a></td>
192-
<td><a href="https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler"> stabilityai/stable-diffusion-x4-upscaler </a></td>
193-
</tr>
194-
<tr>
195-
<td>Super Resolution</td>
196-
<td><a href="https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/latent_upscale">Stable Diffusion Latent Upscale</a></td>
197-
<td><a href="https://huggingface.co/stabilityai/sd-x2-latent-upscaler"> stabilityai/sd-x2-latent-upscaler </a></td>
198-
</tr>
199-
</table>
200-
201-
## Popular libraries using 🧨 Diffusers
202-
203-
- https://github.com/microsoft/TaskMatrix
204-
- https://github.com/invoke-ai/InvokeAI
205-
- https://github.com/InstantID/InstantID
206-
- https://github.com/apple/ml-stable-diffusion
207-
- https://github.com/Sanster/lama-cleaner
208-
- https://github.com/IDEA-Research/Grounded-Segment-Anything
209-
- https://github.com/ashawkey/stable-dreamfusion
210-
- https://github.com/deep-floyd/IF
211-
- https://github.com/bentoml/BentoML
212-
- https://github.com/bmaltais/kohya_ss
213-
- +14,000 other amazing GitHub repositories 💪
214-
215-
Thank you for using us ❤️.
216-
217-
## Credits
218-
219-
This library concretizes previous work by many different authors and would not have been possible without their great research and implementations. We'd like to thank, in particular, the following implementations which have helped us in our development and without which the API could not have been as polished today:
220-
221-
- @CompVis' latent diffusion models library, available [here](https://github.com/CompVis/latent-diffusion)
222-
- @hojonathanho original DDPM implementation, available [here](https://github.com/hojonathanho/diffusion) as well as the extremely useful translation into PyTorch by @pesser, available [here](https://github.com/pesser/pytorch_diffusion)
223-
- @ermongroup's DDIM implementation, available [here](https://github.com/ermongroup/ddim)
224-
- @yang-song's Score-VE and Score-VP implementations, available [here](https://github.com/yang-song/score_sde_pytorch)
225-
226-
We also want to thank @heejkoo for the very helpful overview of papers, code and resources on diffusion models, available [here](https://github.com/heejkoo/Awesome-Diffusion-Models) as well as @crowsonkb and @rromb for useful discussions and insights.
227-
228-
## Citation
229-
230-
```bibtex
231-
@misc{von-platen-etal-2022-diffusers,
232-
author = {Patrick von Platen and Suraj Patil and Anton Lozhkov and Pedro Cuenca and Nathan Lambert and Kashif Rasul and Mishig Davaadorj and Dhruv Nair and Sayak Paul and William Berman and Yiyi Xu and Steven Liu and Thomas Wolf},
233-
title = {Diffusers: State-of-the-art diffusion models},
234-
year = {2022},
235-
publisher = {GitHub},
236-
journal = {GitHub repository},
237-
howpublished = {\url{https://github.com/huggingface/diffusers}}
238-
}
239-
```

0 commit comments

Comments
 (0)