Standardize on either GiB/MiB or GB/MB in logging

[awgu]

In my understanding, gibibyte (GiB) is $1024^3$ bytes, and mebibyte (MiB) is $1024^2$ bytes. However, gigabyte (GB) is $1000^3$ bytes, and megabyte (MB) is $1000$ bytes.

I think we should standardize on one or the other, maybe GiB since the memory profiler went that direction: pytorch/pytorch#120172 In that case, I think we mainly need to change "GB" to "GiB" in our logging.

[lessw2020]

good catch. Right now we are actually calculating GiB in the logging but calling it (incorrectly) GB, so we are 7% off.

That said, I think we should stay consistent with how Nvidia specs their GPU's and atm it's in GB rather than GiB, so I would vote for correcting the calcs to 1000^3 rather than current 1024^3.

Here's H100 specs for example:

[awgu]

For some reason I cannot see the screenshot. Interestingly, in pytorch/pytorch#120172, the user suggested that Nvidia labels it as GB but actually means GiB 😆 (pytorch/pytorch#120172 (comment))

[lessw2020]

oh I think the comment that Nvidia labels things as GB in specs but means GiB is correct - nvidia-smi reports in MiB/GiB actually:

So agree with your initial assessment, lets' update it to GiB labelling (and MiB) and then we'll be both correct label wise and in synch with Nividia-smi and mem profiler.

Thanks for flagging this - I'll make a very simple PR right now to fix.