Error finetuning SmolVLM (idefics3 template) with batch size > 1 #6419
Description
When fine-tuning SmolVLM with batch_size > 1, this error occured: RuntimeError: Sizes of tensors must match except in dimension 0. Expected size 13 but got size 17 for tensor number 2 in the list.