Skip to content

Conversation

@awaelchli
Copy link
Contributor

@awaelchli awaelchli commented Sep 22, 2022

What does this PR do?

Requested by @carmocca in #14792

The changes can't be carried out just in Lite, because PL imports the precision classes and requires them to be consistent in the types.

Fixes #16011

Before submitting

  • Was this discussed/approved via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

I made sure I had fun coding 🙃

cc @Borda @carmocca @justusschock @awaelchli

@github-actions github-actions bot added the pl Generic label for PyTorch Lightning package label Sep 22, 2022
@awaelchli awaelchli added precision: apex (removed) NVIDIA/apex precision precision: double Double precision precision: amp Automatic Mixed Precision and removed pl Generic label for PyTorch Lightning package labels Sep 22, 2022
@github-actions github-actions bot added the pl Generic label for PyTorch Lightning package label Sep 22, 2022
@stale
Copy link

stale bot commented Oct 15, 2022

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If you need further help see our docs: https://pytorch-lightning.readthedocs.io/en/latest/generated/CONTRIBUTING.html#pull-request or ask the assistance of a core contributor here or on Slack. Thank you for your contributions.

@stale stale bot added the won't fix This will not be worked on label Oct 15, 2022
@awaelchli awaelchli modified the milestones: future, v1.9 Oct 16, 2022
@awaelchli awaelchli removed the won't fix This will not be worked on label Oct 16, 2022
@awaelchli awaelchli added the fun Staff contributions outside working hours - to differentiate from the "community" label label Dec 26, 2022
@awaelchli awaelchli marked this pull request as ready for review January 3, 2023 09:42
@github-actions
Copy link
Contributor

github-actions bot commented Jan 3, 2023

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow
Check ID Status
pl-cpu (macOS-11, pytorch, 3.8, 1.11) success
pl-cpu (macOS-11, pytorch, 3.9, 1.12) success
pl-cpu (macOS-11, pytorch, 3.10, 1.13) success
pl-cpu (macOS-11, pytorch, 3.8, 1.10, oldest) success
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.10) success
pl-cpu (ubuntu-20.04, pytorch, 3.9, 1.11) success
pl-cpu (ubuntu-20.04, pytorch, 3.10, 1.12) success
pl-cpu (ubuntu-20.04, pytorch, 3.10, 1.13) success
pl-cpu (ubuntu-20.04, pytorch, 3.7, 1.10, oldest) success
pl-cpu (windows-2022, pytorch, 3.9, 1.11) success
pl-cpu (windows-2022, pytorch, 3.10, 1.12) success
pl-cpu (windows-2022, pytorch, 3.10, 1.13) success
pl-cpu (windows-2022, pytorch, 3.7, 1.10, oldest) success
pl-cpu (slow, macOS-11, pytorch, 3.7, 1.11) success
pl-cpu (slow, ubuntu-20.04, pytorch, 3.7, 1.11) success
pl-cpu (slow, windows-2022, pytorch, 3.7, 1.11) success
pl-cpu (macOS-11, lightning, 3.8, 1.13) success
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.13) success
pl-cpu (windows-2022, lightning, 3.8, 1.13) success

These checks are required after the changes to src/lightning_fabric/connector.py, src/lightning_fabric/plugins/precision/deepspeed.py, src/lightning_fabric/plugins/precision/double.py, src/lightning_fabric/plugins/precision/fsdp.py, src/lightning_fabric/plugins/precision/native_amp.py, src/lightning_fabric/plugins/precision/precision.py, src/lightning_fabric/plugins/precision/tpu_bf16.py, src/lightning_fabric/strategies/deepspeed.py, src/lightning_fabric/strategies/fairscale.py, src/lightning_fabric/utilities/enums.py, src/pytorch_lightning/lite/lite.py, src/pytorch_lightning/plugins/precision/colossalai.py, src/pytorch_lightning/plugins/precision/deepspeed.py, src/pytorch_lightning/plugins/precision/double.py, src/pytorch_lightning/plugins/precision/fsdp_native_native_amp.py, src/pytorch_lightning/plugins/precision/hpu.py, src/pytorch_lightning/plugins/precision/ipu.py, src/pytorch_lightning/plugins/precision/native_amp.py, src/pytorch_lightning/plugins/precision/precision_plugin.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/plugins/precision/tpu_bf16.py, src/pytorch_lightning/strategies/colossalai.py, src/pytorch_lightning/strategies/deepspeed.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/hivemind.py, src/pytorch_lightning/strategies/utils.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py, src/pytorch_lightning/trainer/connectors/data_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/utilities/argparse.py, src/pytorch_lightning/utilities/enums.py, tests/tests_pytorch/accelerators/test_ipu.py, tests/tests_pytorch/deprecated_api/test_remove_1-10.py, tests/tests_pytorch/helpers/deterministic_model.py, tests/tests_pytorch/plugins/precision/hpu/test_hpu.py, tests/tests_pytorch/strategies/test_deepspeed_strategy.py, tests/tests_pytorch/trainer/connectors/test_accelerator_connector.py.

🟢 pytorch_lightning: Azure GPU
Check ID Status
pytorch-lightning (GPUs) success

These checks are required after the changes to src/pytorch_lightning/lite/lite.py, src/pytorch_lightning/plugins/precision/colossalai.py, src/pytorch_lightning/plugins/precision/deepspeed.py, src/pytorch_lightning/plugins/precision/double.py, src/pytorch_lightning/plugins/precision/fsdp_native_native_amp.py, src/pytorch_lightning/plugins/precision/hpu.py, src/pytorch_lightning/plugins/precision/ipu.py, src/pytorch_lightning/plugins/precision/native_amp.py, src/pytorch_lightning/plugins/precision/precision_plugin.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/plugins/precision/tpu_bf16.py, src/pytorch_lightning/strategies/colossalai.py, src/pytorch_lightning/strategies/deepspeed.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/hivemind.py, src/pytorch_lightning/strategies/utils.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py, src/pytorch_lightning/trainer/connectors/data_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/utilities/argparse.py, src/pytorch_lightning/utilities/enums.py, tests/tests_pytorch/accelerators/test_ipu.py, tests/tests_pytorch/deprecated_api/test_remove_1-10.py, tests/tests_pytorch/helpers/deterministic_model.py, tests/tests_pytorch/plugins/precision/hpu/test_hpu.py, tests/tests_pytorch/strategies/test_deepspeed_strategy.py, tests/tests_pytorch/trainer/connectors/test_accelerator_connector.py, src/lightning_fabric/connector.py, src/lightning_fabric/plugins/precision/deepspeed.py, src/lightning_fabric/plugins/precision/double.py, src/lightning_fabric/plugins/precision/fsdp.py, src/lightning_fabric/plugins/precision/native_amp.py, src/lightning_fabric/plugins/precision/precision.py, src/lightning_fabric/plugins/precision/tpu_bf16.py, src/lightning_fabric/strategies/deepspeed.py, src/lightning_fabric/strategies/fairscale.py, src/lightning_fabric/utilities/enums.py.

🟢 pytorch_lightning: Azure HPU
Check ID Status
pytorch-lightning (HPUs) success

These checks are required after the changes to src/lightning_fabric/connector.py, src/lightning_fabric/plugins/precision/deepspeed.py, src/lightning_fabric/plugins/precision/double.py, src/lightning_fabric/plugins/precision/fsdp.py, src/lightning_fabric/plugins/precision/native_amp.py, src/lightning_fabric/plugins/precision/precision.py, src/lightning_fabric/plugins/precision/tpu_bf16.py, src/lightning_fabric/strategies/deepspeed.py, src/lightning_fabric/strategies/fairscale.py, src/lightning_fabric/utilities/enums.py, src/pytorch_lightning/lite/lite.py, src/pytorch_lightning/plugins/precision/colossalai.py, src/pytorch_lightning/plugins/precision/deepspeed.py, src/pytorch_lightning/plugins/precision/double.py, src/pytorch_lightning/plugins/precision/fsdp_native_native_amp.py, src/pytorch_lightning/plugins/precision/hpu.py, src/pytorch_lightning/plugins/precision/ipu.py, src/pytorch_lightning/plugins/precision/native_amp.py, src/pytorch_lightning/plugins/precision/precision_plugin.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/plugins/precision/tpu_bf16.py, src/pytorch_lightning/strategies/colossalai.py, src/pytorch_lightning/strategies/deepspeed.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/hivemind.py, src/pytorch_lightning/strategies/utils.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py, src/pytorch_lightning/trainer/connectors/data_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/utilities/argparse.py, src/pytorch_lightning/utilities/enums.py, tests/tests_pytorch/accelerators/test_ipu.py, tests/tests_pytorch/deprecated_api/test_remove_1-10.py, tests/tests_pytorch/helpers/deterministic_model.py, tests/tests_pytorch/plugins/precision/hpu/test_hpu.py, tests/tests_pytorch/strategies/test_deepspeed_strategy.py, tests/tests_pytorch/trainer/connectors/test_accelerator_connector.py.

🟢 pytorch_lightning: Azure IPU
Check ID Status
pytorch-lightning (IPUs) success

These checks are required after the changes to src/lightning_fabric/connector.py, src/lightning_fabric/plugins/precision/deepspeed.py, src/lightning_fabric/plugins/precision/double.py, src/lightning_fabric/plugins/precision/fsdp.py, src/lightning_fabric/plugins/precision/native_amp.py, src/lightning_fabric/plugins/precision/precision.py, src/lightning_fabric/plugins/precision/tpu_bf16.py, src/lightning_fabric/strategies/deepspeed.py, src/lightning_fabric/strategies/fairscale.py, src/lightning_fabric/utilities/enums.py, src/pytorch_lightning/lite/lite.py, src/pytorch_lightning/plugins/precision/colossalai.py, src/pytorch_lightning/plugins/precision/deepspeed.py, src/pytorch_lightning/plugins/precision/double.py, src/pytorch_lightning/plugins/precision/fsdp_native_native_amp.py, src/pytorch_lightning/plugins/precision/hpu.py, src/pytorch_lightning/plugins/precision/ipu.py, src/pytorch_lightning/plugins/precision/native_amp.py, src/pytorch_lightning/plugins/precision/precision_plugin.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/plugins/precision/tpu_bf16.py, src/pytorch_lightning/strategies/colossalai.py, src/pytorch_lightning/strategies/deepspeed.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/hivemind.py, src/pytorch_lightning/strategies/utils.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py, src/pytorch_lightning/trainer/connectors/data_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/utilities/argparse.py, src/pytorch_lightning/utilities/enums.py, tests/tests_pytorch/accelerators/test_ipu.py, tests/tests_pytorch/deprecated_api/test_remove_1-10.py, tests/tests_pytorch/helpers/deterministic_model.py, tests/tests_pytorch/plugins/precision/hpu/test_hpu.py, tests/tests_pytorch/strategies/test_deepspeed_strategy.py, tests/tests_pytorch/trainer/connectors/test_accelerator_connector.py.

🟢 pytorch_lightning: Docs
Check ID Status
make-doctest (pytorch) success
make-html (pytorch) success

These checks are required after the changes to src/pytorch_lightning/lite/lite.py, src/pytorch_lightning/plugins/precision/colossalai.py, src/pytorch_lightning/plugins/precision/deepspeed.py, src/pytorch_lightning/plugins/precision/double.py, src/pytorch_lightning/plugins/precision/fsdp_native_native_amp.py, src/pytorch_lightning/plugins/precision/hpu.py, src/pytorch_lightning/plugins/precision/ipu.py, src/pytorch_lightning/plugins/precision/native_amp.py, src/pytorch_lightning/plugins/precision/precision_plugin.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/plugins/precision/tpu_bf16.py, src/pytorch_lightning/strategies/colossalai.py, src/pytorch_lightning/strategies/deepspeed.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/hivemind.py, src/pytorch_lightning/strategies/utils.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py, src/pytorch_lightning/trainer/connectors/data_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/utilities/argparse.py, src/pytorch_lightning/utilities/enums.py.

🟢 lightning_fabric: CPU workflow
Check ID Status
fabric-cpu (macOS-11, fabric, 3.8, 1.11) success
fabric-cpu (macOS-11, fabric, 3.9, 1.12) success
fabric-cpu (macOS-11, fabric, 3.10, 1.13) success
fabric-cpu (macOS-11, fabric, 3.7, 1.10, oldest) success
fabric-cpu (ubuntu-20.04, fabric, 3.8, 1.10) success
fabric-cpu (ubuntu-20.04, fabric, 3.9, 1.11) success
fabric-cpu (ubuntu-20.04, fabric, 3.10, 1.12) success
fabric-cpu (ubuntu-20.04, fabric, 3.10, 1.13) success
fabric-cpu (ubuntu-20.04, fabric, 3.7, 1.10, oldest) success
fabric-cpu (windows-2022, fabric, 3.9, 1.11) success
fabric-cpu (windows-2022, fabric, 3.10, 1.12) success
fabric-cpu (windows-2022, fabric, 3.10, 1.13) success
fabric-cpu (windows-2022, fabric, 3.7, 1.10, oldest) success
fabric-cpu (macOS-11, lightning, 3.8, 1.13) success
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.13) success
fabric-cpu (windows-2022, lightning, 3.8, 1.13) success

These checks are required after the changes to src/lightning_fabric/connector.py, src/lightning_fabric/plugins/precision/deepspeed.py, src/lightning_fabric/plugins/precision/double.py, src/lightning_fabric/plugins/precision/fsdp.py, src/lightning_fabric/plugins/precision/native_amp.py, src/lightning_fabric/plugins/precision/precision.py, src/lightning_fabric/plugins/precision/tpu_bf16.py, src/lightning_fabric/strategies/deepspeed.py, src/lightning_fabric/strategies/fairscale.py, src/lightning_fabric/utilities/enums.py, tests/tests_fabric/plugins/precision/test_deepspeed_integration.py, tests/tests_fabric/strategies/test_fsdp_integration.py, tests/tests_fabric/utilities/test_enums.py.

🟢 lightning_fabric: Azure GPU
Check ID Status
lightning-fabric (GPUs) success

These checks are required after the changes to src/lightning_fabric/connector.py, src/lightning_fabric/plugins/precision/deepspeed.py, src/lightning_fabric/plugins/precision/double.py, src/lightning_fabric/plugins/precision/fsdp.py, src/lightning_fabric/plugins/precision/native_amp.py, src/lightning_fabric/plugins/precision/precision.py, src/lightning_fabric/plugins/precision/tpu_bf16.py, src/lightning_fabric/strategies/deepspeed.py, src/lightning_fabric/strategies/fairscale.py, src/lightning_fabric/utilities/enums.py, tests/tests_fabric/plugins/precision/test_deepspeed_integration.py, tests/tests_fabric/strategies/test_fsdp_integration.py, tests/tests_fabric/utilities/test_enums.py.

🟢 mypy
Check ID Status
mypy success

These checks are required after the changes to src/lightning_fabric/connector.py, src/lightning_fabric/plugins/precision/deepspeed.py, src/lightning_fabric/plugins/precision/double.py, src/lightning_fabric/plugins/precision/fsdp.py, src/lightning_fabric/plugins/precision/native_amp.py, src/lightning_fabric/plugins/precision/precision.py, src/lightning_fabric/plugins/precision/tpu_bf16.py, src/lightning_fabric/strategies/deepspeed.py, src/lightning_fabric/strategies/fairscale.py, src/lightning_fabric/utilities/enums.py, src/pytorch_lightning/lite/lite.py, src/pytorch_lightning/plugins/precision/colossalai.py, src/pytorch_lightning/plugins/precision/deepspeed.py, src/pytorch_lightning/plugins/precision/double.py, src/pytorch_lightning/plugins/precision/fsdp_native_native_amp.py, src/pytorch_lightning/plugins/precision/hpu.py, src/pytorch_lightning/plugins/precision/ipu.py, src/pytorch_lightning/plugins/precision/native_amp.py, src/pytorch_lightning/plugins/precision/precision_plugin.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/plugins/precision/tpu_bf16.py, src/pytorch_lightning/strategies/colossalai.py, src/pytorch_lightning/strategies/deepspeed.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/hivemind.py, src/pytorch_lightning/strategies/utils.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py, src/pytorch_lightning/trainer/connectors/data_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/utilities/argparse.py, src/pytorch_lightning/utilities/enums.py.

🟢 install
Check ID Status
install-pkg (ubuntu-22.04, app, 3.7) success
install-pkg (ubuntu-22.04, app, 3.10) success
install-pkg (ubuntu-22.04, fabric, 3.7) success
install-pkg (ubuntu-22.04, fabric, 3.10) success
install-pkg (ubuntu-22.04, pytorch, 3.7) success
install-pkg (ubuntu-22.04, pytorch, 3.10) success
install-pkg (ubuntu-22.04, lightning, 3.7) success
install-pkg (ubuntu-22.04, lightning, 3.10) success
install-pkg (ubuntu-22.04, notset, 3.7) success
install-pkg (ubuntu-22.04, notset, 3.10) success
install-pkg (macOS-12, app, 3.7) success
install-pkg (macOS-12, app, 3.10) success
install-pkg (macOS-12, fabric, 3.7) success
install-pkg (macOS-12, fabric, 3.10) success
install-pkg (macOS-12, pytorch, 3.7) success
install-pkg (macOS-12, pytorch, 3.10) success
install-pkg (macOS-12, lightning, 3.7) success
install-pkg (macOS-12, lightning, 3.10) success
install-pkg (macOS-12, notset, 3.7) success
install-pkg (macOS-12, notset, 3.10) success
install-pkg (windows-2022, app, 3.7) success
install-pkg (windows-2022, app, 3.10) success
install-pkg (windows-2022, fabric, 3.7) success
install-pkg (windows-2022, fabric, 3.10) success
install-pkg (windows-2022, pytorch, 3.7) success
install-pkg (windows-2022, pytorch, 3.10) success
install-pkg (windows-2022, lightning, 3.7) success
install-pkg (windows-2022, lightning, 3.10) success
install-pkg (windows-2022, notset, 3.7) success
install-pkg (windows-2022, notset, 3.10) success

These checks are required after the changes to src/lightning_fabric/connector.py, src/lightning_fabric/plugins/precision/deepspeed.py, src/lightning_fabric/plugins/precision/double.py, src/lightning_fabric/plugins/precision/fsdp.py, src/lightning_fabric/plugins/precision/native_amp.py, src/lightning_fabric/plugins/precision/precision.py, src/lightning_fabric/plugins/precision/tpu_bf16.py, src/lightning_fabric/strategies/deepspeed.py, src/lightning_fabric/strategies/fairscale.py, src/lightning_fabric/utilities/enums.py, src/pytorch_lightning/lite/lite.py, src/pytorch_lightning/plugins/precision/colossalai.py, src/pytorch_lightning/plugins/precision/deepspeed.py, src/pytorch_lightning/plugins/precision/double.py, src/pytorch_lightning/plugins/precision/fsdp_native_native_amp.py, src/pytorch_lightning/plugins/precision/hpu.py, src/pytorch_lightning/plugins/precision/ipu.py, src/pytorch_lightning/plugins/precision/native_amp.py, src/pytorch_lightning/plugins/precision/precision_plugin.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/plugins/precision/tpu_bf16.py, src/pytorch_lightning/strategies/colossalai.py, src/pytorch_lightning/strategies/deepspeed.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/hivemind.py, src/pytorch_lightning/strategies/utils.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py, src/pytorch_lightning/trainer/connectors/data_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/utilities/argparse.py, src/pytorch_lightning/utilities/enums.py.


Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

@mergify mergify bot added the has conflicts label Jan 4, 2023
@carmocca carmocca force-pushed the lite/precision-str-int branch from f23c07d to 9049485 Compare January 5, 2023 15:34
@carmocca carmocca force-pushed the lite/precision-str-int branch from 9049485 to 0715ea4 Compare January 5, 2023 16:10
@awaelchli awaelchli requested a review from Borda as a code owner January 6, 2023 16:05
@mergify mergify bot added the ready PRs ready to be merged label Jan 6, 2023
@awaelchli awaelchli enabled auto-merge (squash) January 6, 2023 17:31
@awaelchli awaelchli force-pushed the lite/precision-str-int branch from 8413194 to 29c8931 Compare January 6, 2023 18:52
@awaelchli awaelchli disabled auto-merge January 6, 2023 19:09
@awaelchli awaelchli force-pushed the lite/precision-str-int branch from d844c6a to 72dd301 Compare January 6, 2023 19:18
@awaelchli awaelchli enabled auto-merge (squash) January 6, 2023 19:19
@awaelchli awaelchli merged commit 4c3ce60 into master Jan 6, 2023
@awaelchli awaelchli deleted the lite/precision-str-int branch January 6, 2023 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

code quality fun Staff contributions outside working hours - to differentiate from the "community" label pl Generic label for PyTorch Lightning package precision: amp Automatic Mixed Precision precision: apex (removed) NVIDIA/apex precision precision: double Double precision ready PRs ready to be merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

precision parameter in Trainer does not accept "16" passed as str

4 participants