Thank you for contributing to bioicons! Sorry I forgot to add to acknowledgements, I will in the final version!

Thank you Remi!

TraitGym is available on HuggingFace, including a Colab notebook to eval a model in few minutes:
huggingface.co/datasets/son...

Check out the paper for more details, including stratification by consequence trait, and eQTL:
www.biorxiv.org/content/10.1...

Scaling is probably part of the solution, but data curation might be the major bottleneck. The vast majority of bases in mammalian genomes lack evolutionary constraint which is precisely the signal leveraged by self-supervision.

Alignment-free DNA language models are not yet competitive. The best among them, our GPN-Promoter and SpeciesLM from @gagneurlab.bsky.social , are not the largest in number of parameters or context. Their key feature is having been trained only on functional regions of the genome.

Conservation-aware CADD and GPN-MSA do better on Mendelian trait variants, expected to be under strong purifying selection. On complex trait variants, especially for non-disease traits, functional-genomics models Enformer and Borzoi tend to do better. However, ensembling helps:

We evaluate models zero-shot (unsupervised) and with linear probing (logistic regression on top of extracted features):

We evaluate a wide range of models with up to 7B parameters and 500K context size. Do these numbers matter? 🤔

We collect putative causal variants from OMIM and UKBB with carefully matched controls.

I still believe in alignment-free gLMs with better data curation and loss functions, I've been seeing advances but still tough.

*An exception are alignment-based gLMs which do improve (non-trivially) over conservation scores.

A simple bar is: do you surpass conservation scores in identifying functional mutations? This bar was easily passed by pLMs and plant gLMs but not yet by human gLMs* even after 5 years.

Thank you Jo!