multimodal support

[Osilly]

Great work! I want to know if this framework can integrate the image tokens into the rollout trajectory, the base model can use these image tokens to generate reasoning for the next step, i.e., think with images. This implement has been implemented in here and here.

[Osilly]

Furthermore, I would like to know whether this framework supports Multimodal LLMs, like Qwen3-VL. Alternatively, could you provide some examples of training an MLLM in rllm? Thank you!

[jeffreysijuntan]

There is an ongoing effort to support multimodal training in rLLM (here). We plan to get it merged soon. Please stay tuned.

[Osilly]

I'm really looking forward to seeing this support!

[Osilly]

@jeffreysijuntan Excuse me, but I'd like to know the timelines for supporting multimodal, as I am looking for a framework to finish one work involving multimodal ReAct. Do you plan to support them in the near future?

[nancyjlau]

Ok, I tried to update rllm in my branch above to use the more updated verl code and enabled processor support for training. The multimodal Qwen2.5-VL models seem to work, I have tested my branch for Qwen/Qwen2.5-VL-3B-Instruct and was able to load the Qwen2_5_VLProcessor processor.

Unfortunately for Qwen3-VL, when testing Qwen/Qwen3-VL-8B-Instruct with the current transformers version (from rLLM's install script), AutoProcessor returns Qwen2TokenizerFast instead of a processor class. verl's hf_processor function rejects anything without "Processor" in the class name.

However, Qwen3-VL is fully supported by VERL as of October 6, 2025, and their training scripts recommend pip install -U transformers. The issue is that Qwen3VLProcessor requires transformers >= 4.57.0, but rllm/scripts/install_verl.sh currently specifies >=4.51.0

Updating the transformers dependency from >=4.51.0 to >=4.57.0 should enable Qwen3-VL support in rLLM, but I will do further testing