People say the instruct models feel “lobotomised” compared to base models after instruct tuning.

Wondering if this still would be an issue if instruct-training was done with extra layers on top of a frozen base model?

made this little thing that colors a world map by the distance between vector embedding of a phrase vs the embedding of the name of the country

Personal robot which plugs in your phone when you’re sleeping when it’s running low on battery

Good time to buy probably

Converting raw sensor data like a gyroscope in ur earbuds directly to model’s embedding space to give it a digital cochlea instead of a json in prompt like they did with vision LMs.

Actually here all the diagrams I drew while reimplementing the 2017 transformer

How I picture transformers in my head