Alham Fikri Aji
@afaji.bsky.social
Faculty @MBZUAI, visiting scientist @Google
We also explored other benchmark datasets and different models.
If you're interested in learning more, check out our paper, Data Laundering: arxiv.org/pdf/2412.15255
If you're interested in learning more, check out our paper, Data Laundering: arxiv.org/pdf/2412.15255
arxiv.org
December 27, 2024 at 10:42 AM
We also explored other benchmark datasets and different models.
If you're interested in learning more, check out our paper, Data Laundering: arxiv.org/pdf/2412.15255
If you're interested in learning more, check out our paper, Data Laundering: arxiv.org/pdf/2412.15255
We discovered that the (illegal) knowledge of GPQA was leaked through the distillation loss, even though it was never explicitly trained on during the distillation stage.
We also repeated the distillation process multiple times and found that the performance was maintained
We also repeated the distillation process multiple times and found that the performance was maintained
December 27, 2024 at 10:42 AM
We discovered that the (illegal) knowledge of GPQA was leaked through the distillation loss, even though it was never explicitly trained on during the distillation stage.
We also repeated the distillation process multiple times and found that the performance was maintained
We also repeated the distillation process multiple times and found that the performance was maintained
Data Laundering
We first train a model on the GPQA test data, which obviously made this model achieve 100% performance. But hey, don’t many LLMs train on test data anyway?🙈
Then, we train a new model on another (fair) data, but with a distillation loss from the cheating model
We first train a model on the GPQA test data, which obviously made this model achieve 100% performance. But hey, don’t many LLMs train on test data anyway?🙈
Then, we train a new model on another (fair) data, but with a distillation loss from the cheating model
December 27, 2024 at 10:42 AM
Data Laundering
We first train a model on the GPQA test data, which obviously made this model achieve 100% performance. But hey, don’t many LLMs train on test data anyway?🙈
Then, we train a new model on another (fair) data, but with a distillation loss from the cheating model
We first train a model on the GPQA test data, which obviously made this model achieve 100% performance. But hey, don’t many LLMs train on test data anyway?🙈
Then, we train a new model on another (fair) data, but with a distillation loss from the cheating model