6/6 For more details, see:

Paper: arxiv.org/pdf/2502.09969
Code: github.com/agarwalishik...

Thank you so much to @dilekh.bsky.social and @convai-uiuc.bsky.social for their guidance and support during this project 🎉🎉

5/6 Finally, using our influence values, we pick a small subset & fine-tune the model. In our evaluation, we use 4 SOTA influence functions -- NN-CIFT achieves the same performance while using a model 34,000x smaller!

4/6 Second, we train the InfluenceNetwork using basic mini-batch gradient descent, then let it estimate the influence for the remaining data. It has a very low error of 0.067!

3/6 First, the neural network (called the “InfluenceNetwork”) needs to be trained. We compute influence values using existing methods -- but only for a tiny fraction of data (just 0.25%-5%).

2/6 Estimating the value of data is expensive.

Past works use LLMs to estimate the influence of data -- we use small neural networks to *learn to estimate* influence, instead. This reduces costs and adapts to new data without heavy recomputation.

Here’s how it works:

Thank you Guneet! Would love to hear more about these stress tests :)

👋

Hey! Would love to be added :)

For more details, see:
Paper: arxiv.org/pdf/2411.04425
Code: github.com/agarwalishik...

Thank you so much to Krishnateja, Lucian, and Marina for their help, mentorship, and guidance during this project! 🎉🎉

3. Continual fine-tuning: given a fine-tuned model, enabling it to integrate new and complementary information while mitigating catastrophic forgetting. We find that reducing the dataset helps remove samples that hinder performance, surpassing the performance of the full dataset.

2. Task-specific fine-tuning: given an instruction-tuned model, refining the LLM's expertise in specific domains. We find that pruning the dataset removes noise and keeps relevant examples, achieving better performance than fine-tuning on the full dataset.

1. Instruction tuning: given a base model, fine-tuning a model to follow general instructions. We find that performance drops are minimal when reducing the dataset by 70%.

DELIFT quantifies the information present in a sample wrt an LLM's capabilities. Using submodular functions, DELIFT can automatically adapt the chosen subset based on the objectives in the 3 stages of language model fine-tuning:

TreeInstruct is preferred 78.43% of the time. It solves 14.09% more bugs across all settings, and our questions are 14.18% better at addressing bugs, maintaining relevance, and ensuring logical conversation flow. TreeInstruct also adapts to human students of varying backgrounds.

TreeInstruct estimates the knowledge a student needs to debug their code and devises a conversation plan. It then dynamically constructs a question tree based on its interactions with the student, navigating the knowledge state space till the student comprehends & fixes all bugs.

github.com/agarwalishik...
We apply TreeInstruct to code debugging. Prior works directly give away bugs/fixes, assume single-turn conversations, and only work for one bug. We create a realistic, multi-bug dataset, where the bugs are mutually dependent.

I'd love to be added - thank you!!