Thanks!

8/
- Decima (www.biorxiv.org/content/10.1...)
- grelu (www.biorxiv.org/content/10.1...)
- scooby (www.biorxiv.org/content/10.1...)
- flashzoi (www.biorxiv.org/content/10.1...)
We hope this community continues to grow and drive innovation in scalable, data-efficient regulatory modeling!

7/ Finally, our PEFT-based Borzoi transfer approach aim to contribute to a growing ecosystem to make large input DNA models like Borzoi more accessible for personalized regulatory genomics research, alongside tools like:

6/ Additionally, we characterize the tradeoffs of adapter insertion strategies and have some interesting observations. E.g., as we try to adapt conv layers, intermediate activations, rather than trainable parameters, become the practical bottleneck for efficient memory usage.

5/ The transferred models:
- Run on a single GPU with less than 20GB
- Accurately predict gene expression and specificity
- Identify key regulatory factors driving differential expression
- Predict cell-type specific genetic variant effects

4/ Parameter-efficient fine-tuning (PEFT) achieves efficient transfer by updating a small subset of parameters and maintains strong performance. We implemented multiple PEFT modules for both attention and conv layers in the TensorFlow Borzoi codebase and applied to both bulk and scRNA-seq datasets.

3/ Borzoi is a large pre-trained model that learns the general rules of gene regulation. But transferring the model to a custom dataset remains challenging.
- Training from scratch is slow.
- Updating only the final layer underperforms.
- Updating all parameters (full fine-tuning) is expensive.

2/ Code: github.com/calico/baske...
Tutorial: github.com/calico/baske...

yep, nice to see more contents here! thanks! the credit goes to Dave and Johannes. hope you find the model useful!

likewise! sorry i'm bouncing back and forth b/w twitter and bluesky nowadays and sometimes lose track of threads!