Fixes to torchao blog

Author: msaroufim

_posts/2024-09-26-pytorch-native-architecture-optimization.md

@@ -30,37 +30,37 @@ Below we'll walk through some of the techniques available in torchao you can app
 
 ## Inference
 
-[Our inference quantization algorithms](https://github.com/pytorch/ao/tree/main/torchao/quantization) work over arbitrary PyTorch models that contain nn.Linear layers. Weight only and dynamic activation quantization for various dtypes and sparse layouts can be chosen using our top level quantize\_ api
+[Our inference quantization algorithms](https://github.com/pytorch/ao/tree/main/torchao/quantization) work over arbitrary PyTorch models that contain nn.Linear layers. Weight only and dynamic activation quantization for various dtypes and sparse layouts can be chosen using our top level `quantize_` api
 
-```py
+```python
 from torchao.quantization import (  
-    quantize\_,  
-    int4\_weight\_only,  
+    quantize_,  
+    int4_weight_only,  
 )  
-quantize\_(model, int4\_weight\_only())
+quantize_(model, int4_weight_only())
 ```
 
 Sometimes quantizing a layer can make it slower because of overhead so if you’d rather we just pick how to quantize each layer in a model for you then you can instead run
 
-```py
-model \= torchao.autoquant(torch.compile(model, mode='max-autotune'))
+```python
+model = torchao.autoquant(torch.compile(model, mode='max-autotune'))
 ```
 
-quantize\_ API has a few different options depending on whether your model is compute bound or memory bound.
+`quantize_` API has a few different options depending on whether your model is compute bound or memory bound.
 
-```py
+```python
 from torchao.quantization import (  
     # Memory bound models  
-    int4\_weight\_only,  
-    int8\_weight\_only,
+    int4_weight_only,  
+    int8_weight_only,
 
     # Compute bound models  
-    int8\_dynamic\_activation\_int8\_semi\_sparse\_weight,  
-    int8\_dynamic\_activation\_int8\_weight,  
+    int8_dynamic_activation_int8_semi_sparse_weight,  
+    int8_dynamic_activation_int8_weight,  
 
     # Device capability 8.9+  
-    float8\_weight\_only,  
-    float8\_dynamic\_activation\_float8\_weight,  
+    float8_weight_only,  
+    float8_dynamic_activation_float8_weight,  
 )
 ```
 
@@ -86,8 +86,10 @@ Post training quantization, especially at less than 4 bit can suffer from seriou
 
 torchao provides easy to use e2e workflows for reducing the precision of training compute and distributed communications, starting with float8 for \`torch.nn.Linear\` layers.Here is a one-liner to convert the compute gemms of your training run to float8:
 
-from torchao.float8 import convert\_to\_float8\_training  
-convert\_to\_float8\_training(model)
+```python
+from torchao.float8 import convert_to_float8_training  
+convert_to_float8_training(model)
+```
 
 For an e2e example of how to speed up LLaMa 3 70B pretraining by up to **1.5x** with float8, see our [README](https://github.com/pytorch/ao/tree/main/torchao/float8), and torchtitan's [blog](https://dev-discuss.pytorch.org/t/enabling-float8-all-gather-in-fsdp2/2359) and [float8 recipe](https://github.com/pytorch/torchtitan/blob/main/docs/float8.md).
 
@@ -106,8 +108,11 @@ We are expanding our training workflows to more dtypes and layouts
 
 Inspired by Bits and Bytes we’ve also added prototype support for 8 and 4 bit optimizers as a drop in replacement for AdamW.
 
-from torchao.prototype.low\_bit\_optim import AdamW8bit, AdamW4bit  
-optim \= AdamW8bit(model.parameters())  
+```python
+from torchao.prototype.low_bit_optim import AdamW8bit, AdamW4bit  
+optim = AdamW8bit(model.parameters())
+```
+ 
 ![](/assets/images/Figure_5.png){:style="width:100%"}
 
 # Integrations
@@ -129,16 +134,16 @@ pip install torchao
 
 There are a lot of things we’re excited about next ranging from going lower than 4 bit, performant kernels for high-throughput inference, expanding to more layers, scaling types or granularities, MX hardware support and supporting more hardware backends. If any of the above sounds exciting you can follow our progress at: [https://github.com/pytorch/ao](https://github.com/pytorch/ao)
 
-If you’re interested in working on torchao, we’ve created a [contributors guide](https://github.com/pytorch/ao/issues/391), and if you have any questions we hang out on the \#torchao channel on [discord.gg/cudamode](http://discord.gg/cudamode)
+If you’re interested in working on torchao, we’ve created a [contributors guide](https://github.com/pytorch/ao/issues/391), and if you have any questions we hang out on the `#torchao` channel on [discord.gg/gpumode](http://discord.gg/gpumode)
 
 ## Acknowledgements
 
-We are fortunate to stand on the shoulders of giants and collaborate with some of the best people in open source. Thank you\!
+We are fortunate to stand on the shoulders of giants and collaborate with some of the best people in open source. Thank you!
 
 1. Bits and Bytes for pioneering work in low bit optimizers and QLoRA  
 2. Answer.ai for their engineering work to get FSDP and QLoRA composing  
 3. Mobius Labs for the lovely back and forths on quantization algorithms and low bit kernels  
 4. HuggingFace transformers for their help in battle testing and integrating our work  
 5. HuggingFace diffusers for our collaboration on extensive benchmarks and best practices  
 6. torch.compile so we could write our algorithms in pure PyTorch  
-7. CUDA MODE for most of our early contributors
+7. GPU MODE for most of our early contributors