README: Add a model customization guide

[Jack-Khuu]

This adds an initial index for model customization options accessible from the README.

The introduced a "model_customization.md" that will be iterated and expanded upon over time

Note that some of the content is extracted/inspired by quantization.md and the ADVANCED_USERS docs, which are marked as outdated/unstable.

README's
https://github.com/pytorch/torchchat/blob/readme-model-customization-guide/README.md
https://github.com/pytorch/torchchat/blob/readme-model-customization-guide/docs/model_customization.md

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/962

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 2 Unrelated Failures

As of commit a340f0c with merge base 900b6d4 ():

NEW FAILURE - The following job has failed:

• pull / torchchat-command-load-test (macos-14) (gh)
  Process completed with exit code 1.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / compile-gguf (macos-14) (gh) (trunk failure)
  RuntimeError: MPS backend out of memory (MPS allocated: 1.04 GB, other allocations: 0 bytes, max allowed: 7.93 GB). Tried to allocate 256 bytes on shared pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
• Run the README instructions - with stories - on MPS/MacOS / test-readme-mps-macos / macos-job (gh) (trunk failure)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[digantdesai]

Love it. For compile, it is not really a model customization, if we can, to be more complete, extend that category of this doc to encompass that to (et | aoti | eager | compile) it might be more comprehensive / useful.

Also It would be great if you have a "support matrix" view of these four knobs for a flagship model for torchchat which would be llama3-8b I suppose.

[digantdesai]

helpful to be specific about unquantized d-type i.e. fp32 or fp16 or bf16?

[Jack-Khuu]

It's the checkpoint dtype, so it'll vary

[digantdesai]

This doesn't fit very well IMHO in this otherwise awesome high level, almost orthogonal optimization knob categorization. It could be just me.

[Jack-Khuu]

Agreed, it's not an optimization knob, it's a model customization knob

Luckily Quant/optimization gets it's own page for that