Prior-data fitted networks (PFNs) do just that!

The PFN idea is to use a prior, e.g. a bayesian neural network (BNN) prior, sample datasets from that prior, and then train to predict the hold-out labels of these datasets. (no training on real-world data) 2/n

I first played easy mode (see below), where I got two answers from each model and need to match them.
I used 20 interactions in the easy mode to learn the models' behaviors.
In hard mode (see prev post), you need to match three responses to the LLM name.

Second, employees are very likely able to tell models apart based on their gut feeling.
To figure out if this is the case, I created a game with two modes.
The game is about identifying which answer was provided by which LLM.

The new TabPFN even outperforms Autogluon, which is a tool that mixes the best already existing methods (e.g. boosted trees and random forests). See plot (c) for classification results and (d) for regression results.

How did we do this?
We train a neural network that natively handles tables, using attention across rows and columns, on millions of artificial tabular datasets from a meticulously designed data generator. It then performs in-context learning to make predictions on unseen data.

This might be the first time after 10 years that boosted trees are not the best default choice when working with data in tables.
Instead a pre-trained neural network is, the new TabPFN, as we just published in Nature 🎉