[feature request]Blip2 T5 support request

[yuanze1024]

Hi, I fount that you have supported T5 now, but Blip2's examples only contains OPT pipeline. As for the Blip2 on T5 gets a better performance, I wonder if you can support Blip2 on FlanT5xxl? I've tried a long time, and found it's too difficult for me(who just knew TRT, and can not understand how to get ITensor's real value)...

[ncomly-nvidia]

Thanks for the request & feedback @yuanze1024 ! We are starting to expand more into enc-dec & multi-modal models & will put this pairing on the list.

[symphonylyh]

@yuanze1024 thanks! internally we have enabled BLIP2 on Flan-T5-XXL and several other LLM models. Stay tuned once we have a release for that

[yuanze1024]

@yuanze1024 thanks! internally we have enabled BLIP2 on Flan-T5-XXL and several other LLM models. Stay tuned once we have a release for that

@symphonylyh Hi, is there any progress?

[symphonylyh]

@yuanze1024 it was a little slow moving during the US holidays. But I'm expecting the release in mid-Jan will contain this (i.e., in about 2 weeks)

[symphonylyh]

@yuanze1024 right on time! we now released the multi-modal support for BLIP w/ OPT or T5, and LLaVA. Please take a look :)
Annoucement: #847

Feel free to re-open this if you have encountered any issue. But since this issue is a feature request, would better file a new issue regarding any specific problem.