Error in modeling_videollama3.py
#4
by
aj70
- opened
Hi, I found a bug in your code where a call to self._compress_visual_tokens() in prepare_inputs_labels_for_multimodal() is made with the wrong order of accepted inputs. The current order is (compression_mask, mm_features, input_ids, attention_mask, labels, position_ids) where the correct order should instead be (compression_mask, mm_features, input_ids, attention_mask, position_ids, labels).
This bug is causing the position IDs to be set as labels and, as a result, throwing off the training. Please fix it.