Yapei Chang
banner
yapeichang.bsky.social
Yapei Chang
@yapeichang.bsky.social
☁️ phd in progress @ UMD | 🔗 https://lilakk.github.io/
Paper: arxiv.org/pdf/2505.11080
Code: github.com/lilakk/BLEUB... (coming soon)

Work done with the amazing @yekyung.bsky.social from UMD, Michael Krumdick from Kensho, Amir Zadeh and Chuan Li from LambdaAI ,
@chriswtanner.bsky.social from Kensho, and @miyyer.bsky.social from UMD
arxiv.org
May 20, 2025 at 4:25 PM
Beyond benchmarks, human annotators rate BLEUBERI outputs as comparable to those from GRPO-RM models.
May 20, 2025 at 4:25 PM
Qualitatively, BLEUBERI models produce more factually grounded outputs, as measured by VeriScore on three diverse datasets. VeriScore extracts verifiable claims from responses and checks each one against Google Search.
May 20, 2025 at 4:25 PM
The surprising effectiveness of BLEU extends to training. BLEUBERI first selects 5K low-BLEU examples, then trains LLMs with GRPO using BLEU as the reward. BLEUBERI models are competitive as those trained with GRPO-RM (8B) and SFT across 4 benchmarks.
May 20, 2025 at 4:25 PM
When BLEU agrees with humans on a pair of model outputs, what n-grams contribute to this decision? Below is an example where it captures both format (the “Ukrainian” and “English” headers) and factuality (the number 6.1).
May 20, 2025 at 4:25 PM
BLEU is often dismissed for weak human correlation in generation tasks. But on general instruction following, using BLEU to rank pairs of Chatbot Arena outputs—scored against references from strong LLMs—matches 8B & 27B reward models in human agreement, especially with more refs.
May 20, 2025 at 4:25 PM
BLEU is widely used for machine translation (MT) eval. Given a reference and a generation, it computes modified n-gram precision (1–4 grams) and applies a brevity penalty to penalize short outputs. If given multiple references, it takes the max match per n-gram.
May 20, 2025 at 4:25 PM
i've been using this one: repo2txt.simplebasedomain.com it also lets you filter by file type and supports private/local repos
GitHub to Plain Text Converter
Convert GitHub repositories to plain text files easily. Transform code into a single formatted text file.
repo2txt.simplebasedomain.com
December 8, 2024 at 2:55 AM
🙋🏻‍♀️
November 23, 2024 at 10:19 PM
i also got 10/10! the ones that rhyme too well feel very AI to me..
November 21, 2024 at 4:51 PM