Add Float64 support

[richardk53]

Float64 precision is currently not supported.
Why? This should be pretty straightforward, raising an Exception if the device or other configurations are not compatible with it.
Are you planning to add support soon?
What is the best workaround currently? Will it work if I do it manually via model.to(torch.float64) and similar for each dataloader or are there some caveats?

[github-actions]

Hi! thanks for your contribution!, great first issue!

[williamFalcon]

i didn’t know it wasn’t supported haha. what’s special about 64 that it doesn’t work?

[richardk53]

I see ;-)
I thought that the argument precision=64 in the Trainer would take care of casting both the model and the batches to the desired precision, similarly to how it takes care of moving the model and data to the correct device(s).

However, taking Step 1 and Step 2 from the Quickstart in the docs and replacing the trainer by trainer = Trainer(gpus=1, precision=64, use_amp=False) does not make a difference, everything is still FP32.
Also, the documentation states that precision should be 16 or 32.

For this example I can simply do model = model.to(torch.float64) and in the train_dataloader something like dataset = MNIST(os.getcwd(), train=True, download=True, transform=lambda x: transforms.ToTensor()(x).to(torch.float64)) to make it work.

But I think it would be a nice feature to have this done by the Trainer using the precision argument. What do you think?

[williamFalcon]

oooohhh love the idea...
Want to take a stab at the PR? we'll help you finish it :)

[richardk53]

sure :)

[Borda]

@richardk53 how is it going?

[richardk53]

Hey, sorry, haven’t had the time for this yet.