Define hf_config explisitly for convert_hf_mpt_legacy

[bloodeagle40234]

Otherwise, the convert_hf_mpt_legacy call may fail when the function is called in other than global scope because the function refers hf_config in the global scope right now.

[nv-guomingz]

Hi @bloodeagle40234 could u please share us the reproduce steps when u met the issues?

[bloodeagle40234]

Hi @nv-guomingz

My minimum snippet is

from convert_checkpoint import convert_hf_mpt_legacy
from tensorrt_llm import Mapping
from transformers import AutoModelForCausalLM
hf_model = AutoModelForCausalLM.from_pretrained("mosaicml/mpt-7b")

convert_hf_mpt_legacy(hf_model, Mapping())

This snippet can be located within /TensorRT-LLM/examples/mpt dir.
After running the snippet via python command, we will get the following error.

Traceback (most recent call last):
  File "/TensorRT-LLM/examples/mpt/snippet.py", line 6, in <module>
    convert_hf_mpt_legacy(hf_model, Mapping())
  File "/TensorRT-LLM/examples/mpt/convert_checkpoint.py", line 666, in convert_hf_mpt_legacy
    num_key_value_heads = hf_config.attn_config['kv_n_heads'] if 'kv_n_heads' in hf_config.attn_config \
NameError: name 'hf_config' is not defined

The reason why calling convert_checkpoint doesn't hit this problem is because hf_config is defined in __main__ condition in the global module scope. Then, the global scoped variable is referred even though hf_config is not defined in the local scope of convert_hf_mpt_legacy

As you may notice it, convert_hf_mpt funciton already has hf_config argument to resolve the issue.

[nv-guomingz]

Hi @bloodeagle40234 , thanks for your explanation. Let me cherry-pick your mr into our internal code repo firstly.