[Inference] Add validated models for Gaudi

[Deegue]

Model list:

bloom-7b1 single card without template
Falcon-7b single card without template
Falcon-40b multiple cards without template
Gemma-2b single card without template
Llama3-7b single card unknow
Llama3-70b multiple cards unknow
Mistral-7b single card without template
Mixtral-8x7B-Instruct-v0.1 single card with template
llama-2-7b single card unknow
llama-2-70b multiple cards unknow
CodeLlama single card unknow
GPT2 single card without template
GPT-J single card without template
MPT-7b single card without template
Qwen1.5-110B single card with template

[carsonwang]

@Deegue, although the result showed pass but they actually failed. Please check the run result. So can you please check the response of the model for the tests.

[Deegue]

All CI passed. Gentle ping @carsonwang for review, thanks!

[carsonwang]

@kira-lin is helping to review this. For qwen, can you please update to use Qwen/Qwen2-7B-Instruct?

[Deegue]

@kira-lin is helping to review this. For qwen, can you please update to use Qwen/Qwen2-7B-Instruct?

Added Qwen1.5-7B-Chat and Qwen2-7B-Instruct.

[kira-lin]

Qwen 110 not tested.
Many models output weird things, including mpt mistral gpt2 gemma falcon 7b/40b bloom.
Compare them with CPU/GPU to see if it's normal behavior. For example, codellama is outputting some markup language, but I think it's tuned that way. Also pay attention to temperature, which might lead to random result, and it might be different for different models

[kira-lin]

For falcon and qwen, @KepingYan can look into this.

[KepingYan]

llm-on-ray/llm_on_ray/inference/predictors/hpu_predictor.py

Line 340 in cdce225

if self.model.config.model_type == "llama":

Let's modify this line according to:
https://github.com/huggingface/optimum-habana/blob/595cc3e4ec219b1ce469b323cf94e994c5c5d8f3/examples/text-generation/utils.py#L311-L312

[Deegue]

llm-on-ray/llm_on_ray/inference/predictors/hpu_predictor.py

Line 340 in cdce225

if self.model.config.model_type == "llama":

Let's modify this line according to: https://github.com/huggingface/optimum-habana/blob/595cc3e4ec219b1ce469b323cf94e994c5c5d8f3/examples/text-generation/utils.py#L311-L312

Updated, thanks for comment. Btw, will it still be any place to be changed since I found some other places specially handled through model_type == llama?