I assume that good distillation queries follow the same distribution as user queries, therefore it’s hard for public APIs to tell them apart if done in a distributed way.
And when distilling an offline LLM, it doesn’t have memory (beyond context), it simply doesn’t know what questions came before.
January 29, 2025 at 9:41 AM
I assume that good distillation queries follow the same distribution as user queries, therefore it’s hard for public APIs to tell them apart if done in a distributed way.
And when distilling an offline LLM, it doesn’t have memory (beyond context), it simply doesn’t know what questions came before.
It’s a remarkable arc for a company to go in 25 years from revolutionizing search by using PageRank to prioritize results from influential and trusted sites, to grabbing any old random bullshit from other scammers and offering that up as a search result.
January 6, 2025 at 4:26 PM
It’s a remarkable arc for a company to go in 25 years from revolutionizing search by using PageRank to prioritize results from influential and trusted sites, to grabbing any old random bullshit from other scammers and offering that up as a search result.