[Community effort] Type Hints

[patrickvonplaten]

Is your feature request related to a problem? Please describe.

Many important functions and classes are missing type hints

Describe the solution you'd like

It would be great if the following functions could have better type hints to understand what input is expected.

Models

1. • diffusers/src/diffusers/models/unet_2d.py

     Line 14 in 7e1b202

     def __init__(

2. • diffusers/src/diffusers/models/unet_2d_condition.py

     Line 14 in 7e1b202

     def __init__(

3. • diffusers/src/diffusers/models/vae.py

     Line 394 in 7e1b202

     def __init__(

4. • diffusers/src/diffusers/models/vae.py

     Line 443 in 7e1b202

     def forward(self, sample, sample_posterior=False):

5. • diffusers/src/diffusers/models/vae.py

     Line 432 in 7e1b202

     def encode(self, x):

6. • diffusers/src/diffusers/models/vae.py

     Line 438 in 7e1b202

     def decode(self, z):

7. • diffusers/src/diffusers/models/vae.py

     Line 328 in 7e1b202

     def __init__(

8. • diffusers/src/diffusers/models/vae.py

     Line 370 in 7e1b202

     def encode(self, x):

9. • diffusers/src/diffusers/models/vae.py

     Line 375 in 7e1b202

     def decode(self, h, force_not_quantize=False):

10. • diffusers/src/diffusers/models/vae.py

      Line 385 in 7e1b202

      def forward(self, sample):

11. • diffusers/src/diffusers/models/vae.py

      Line 296 in 7e1b202

      def sample(self, generator=None):

Schedulers

12. • diffusers/src/diffusers/schedulers/scheduling_ddim.py

      Line 53 in 7e1b202

      def __init__(

13. • diffusers/src/diffusers/schedulers/scheduling_ddim.py

      Line 103 in 7e1b202

      def set_timesteps(self, num_inference_steps, offset=0):

14. • diffusers/src/diffusers/schedulers/scheduling_ddim.py

      Line 179 in 7e1b202

      def add_noise(self, original_samples, noise, timesteps):

15. • diffusers/src/diffusers/schedulers/scheduling_ddpm.py

      Line 52 in 7e1b202

      def __init__(

16. • diffusers/src/diffusers/schedulers/scheduling_ddpm.py

      Line 90 in 7e1b202

      def set_timesteps(self, num_inference_steps):

17. • diffusers/src/diffusers/schedulers/scheduling_ddpm.py

      Line 182 in 7e1b202

      def add_noise(self, original_samples, noise, timesteps):

18. • diffusers/src/diffusers/schedulers/scheduling_karras_ve.py

      Line 36 in 7e1b202

      def __init__(

19. • diffusers/src/diffusers/schedulers/scheduling_karras_ve.py

      Line 71 in 7e1b202

      def set_timesteps(self, num_inference_steps):

20. • diffusers/src/diffusers/schedulers/scheduling_karras_ve.py

      Line 82 in 7e1b202

      def add_noise_to_input(self, sample, sigma, generator=None):

21. • diffusers/src/diffusers/schedulers/scheduling_lms_discrete.py

      Line 28 in 7e1b202

      def __init__(

22. • diffusers/src/diffusers/schedulers/scheduling_lms_discrete.py

      Line 82 in 7e1b202

      def set_timesteps(self, num_inference_steps):

23. • diffusers/src/diffusers/schedulers/scheduling_lms_discrete.py

      Line 126 in 7e1b202

      def add_noise(self, original_samples, noise, timesteps):

24. • diffusers/src/diffusers/schedulers/scheduling_pndm.py

      Line 52 in 7e1b202

      def __init__(

25. • diffusers/src/diffusers/schedulers/scheduling_pndm.py

      Line 100 in 7e1b202

      def set_timesteps(self, num_inference_steps, offset=0):

26. • diffusers/src/diffusers/schedulers/scheduling_pndm.py

      Line 263 in 7e1b202

      def add_noise(self, original_samples, noise, timesteps):

27. • diffusers/src/diffusers/schedulers/scheduling_sde_ve.py

      Line 40 in 7e1b202

      def __init__(

28. • diffusers/src/diffusers/schedulers/scheduling_sde_ve.py

      Line 62 in 7e1b202

      def set_timesteps(self, num_inference_steps, sampling_eps=None):

29. • diffusers/src/diffusers/schedulers/scheduling_sde_ve.py

      Line 72 in 7e1b202

      def set_sigmas(self, num_inference_steps, sigma_min=None, sigma_max=None, sampling_eps=None):

Pipelines

30. • diffusers/src/diffusers/pipelines/ddim/pipeline_ddim.py

      Line 31 in 7e1b202

      def __call__(self, batch_size=1, generator=None, eta=0.0, num_inference_steps=50, output_type="pil", **kwargs):

31. • diffusers/src/diffusers/pipelines/ddpm/pipeline_ddpm.py

      Line 31 in 7e1b202

      def __call__(self, batch_size=1, generator=None, output_type="pil", **kwargs):

32. • diffusers/src/diffusers/pipelines/latent_diffusion/pipeline_latent_diffusion.py

      Line 19 in 7e1b202

      def __init__(self, vqvae, bert, tokenizer, unet, scheduler):

33. • diffusers/src/diffusers/pipelines/latent_diffusion_uncond/pipeline_latent_diffusion_uncond.py

      Line 10 in 7e1b202

      def __init__(self, vqvae, unet, scheduler):

34. • diffusers/src/diffusers/pipelines/latent_diffusion_uncond/pipeline_latent_diffusion_uncond.py

      Line 16 in 7e1b202

      def __call__(self, batch_size=1, generator=None, eta=0.0, num_inference_steps=50, output_type="pil", **kwargs):

35. • diffusers/src/diffusers/pipelines/pndm/pipeline_pndm.py

      Line 31 in 7e1b202

      def __call__(self, batch_size=1, generator=None, num_inference_steps=50, output_type="pil", **kwargs):

36. • diffusers/src/diffusers/pipelines/pndm/pipeline_pndm.py

      Line 25 in 7e1b202

      def __init__(self, unet, scheduler):

37. • diffusers/src/diffusers/pipelines/score_sde_ve/pipeline_score_sde_ve.py

      Line 10 in 7e1b202

      def __init__(self, unet, scheduler):

38. • diffusers/src/diffusers/pipelines/score_sde_ve/pipeline_score_sde_ve.py

      Line 15 in 7e1b202

      def __call__(self, batch_size=1, num_inference_steps=2000, generator=None, output_type="pil", **kwargs):

39. • diffusers/src/diffusers/pipelines/stochatic_karras_ve/pipeline_stochastic_karras_ve.py

      Line 24 in 7e1b202

      def __init__(self, unet, scheduler):

40. • diffusers/src/diffusers/pipelines/stochatic_karras_ve/pipeline_stochastic_karras_ve.py

      Line 30 in 7e1b202

      def __call__(self, batch_size=1, num_inference_steps=50, generator=None, output_type="pil", **kwargs):

[patrickvonplaten]

Lots of beginner good first issues here for a first contribution. The way it works is easy! Pick some numbers by posting them here as a message here, open a PR to add type hints, ping @anton-l @patil-suraj @pcuenca or @patrickvonplaten on the PR to merge it and we'll tick your numbers of :-)

Example:

I'm taking numbers 39. and 40. and will attach a PR below

Note: Don't forget to run make style after you've finished your PR 🙂

[leszekhanusz]

Maybe that would also be a good idea to make a GitHub action to check that mypy is happy in all the PRs.

[patrickvonplaten]

Usually we're happy with just black, isort and flake8 - are there any advantages of mypy over flake8 ? mypy is a bit too restrictive to me personally

[leszekhanusz]

Well it catches typing errors... That would be a good idea when you add type hints.

[patrickvonplaten]

What do you think here @anton-l @patil-suraj @pcuenca - note that we don't have it in Transformers and might require some more setting up . Not sure if it's worth it from my point of view, but happy to go for it if others think it's a good idea.

Also cc @LysandreJik @sgugger would love to hear your opinion on mypy

[sgugger]

I do have some opinions on mypy ;-) It usually ends up requiring the maintainer to write some horrible type hints that no one reading it will understand.

IMO you should focus on having type hints which make your code better documented and are readable and understandable, but enabling mypy is a step ahead as it will send errors if you use one function with a type that doesn't strictly match the type hint (and Python is dynamically typed so this tends to happen a lot for good reasons. For instance in Transformers, we had some issues AutoModel).

[santiviquez]

Hey, if there is no problem. I'm taking numbers 37. and 38. and will attach a PR below.

[daspartho]

I'll be taking numbers 35. and 36. and will attach a PR below.