I would like to thank my intern mentor Lawrence Chen from Meta, and all other peers Tong Xiao, Rui Wang, Guan Pang, and Pengchuan Zhang. Big thanks to my lab mate @billzhu.bsky.social for valuable discussions and my advisor @robinjia.bsky.social for thoughtful inputs.

Finally, token-level annotations given by TLDR model could speedup human annotators to fix image captions that are slightly off. In fact, it can speed up human annotation by 3 times!

Next, there is something interesting. After finishing training the TLDR model, one can simply remove the reward model head and re-attach the original language model head, to, obviously, become a new vision-language model. It's shown that these new models become better.

TLDR has rich usefulness. First, it can serve as a hallucination rate evaluation metric. As shown in the table, GPT-4o is still the best vision language model in the token level while open-weight models such as Llama-3.2-90B is catching up in the sentence and response level.

TLDR is trained on synthetic hard negatives generated via a perturbation-based method. The architecture is very simple. Instead of applying the reward model head to the last token, as many RMs are doing, TLDR applies the reward model head to every token.

I think it may come from pretraining data and how numbers are presented by humans. We are still investigating how/why these features emerge from LLMs and will keep you updated with any new findings!

we have a very much similar results in NeurIPS 2024: arxiv.org/abs/2406.03445

Can add add me please? Thanks!

Thanks for making this pack. Can you add me please? Thank you!

🙌