on the next page it says all chat bots must answer to the name “big brother”

i tried doing a startup doing exactly this with KGs, it did not pan out

ya that’s where i’m at too. it feels strange watching someone use a sub-par mode

my suspicion is that Google doesn’t do this and that they might be the only ones that don’t

the longer i think about it, the more i suspect they’re doing something at an even higher level

like instead of dynamic batch sizing, maybe they do constant and just have very smart load balancers that keep load saturated

probably balancing training & serving utilization

it didn’t occur to me until now, but monad is in the range of HRM but generalized

honestly surprised he didn’t do that years ago

ya that was my thought too

maybe, but the even “twice” makes me think it was something that more directly adds to 2x perf

so sweet

@dorialexander.bsky.social i wish you blessings in the form of billions of euros in funding

i’m surprised! i expected them to train in fp32, but no, they went with a legit bf16

while being the most French model yet, they had to rationalize why it wasn’t trained on French

but fr imagine being able to do ablations on THE ENTIRE end-to-end training process. you’d learn so much

lol

Google will also budget rewrites into their timelines, or so i've heard