Ross Taylor
@rosstaylor90.bsky.social
Building new thing. Same handle at the other place
Reposted by Ross Taylor
“And yet it reasons”
I honestly believe all LLM results should come with a disclaimer reminding people that the thing doesn’t (and absolutely cannot) know any facts or do any reasoning; it is simply designed to *sound* like it knows facts and frequently sweeps some up accidentally in the process.
I really think many if not most of the LLM fail cases are people trying to use it as a search engine. It is the opposite of a search engine.
December 5, 2024 at 7:05 AM
“And yet it reasons”
(🎥45m) The Hitchhiker's Guide to Reasoning
A talk about LLM reasoning, covering various methods, core problems, and future research directions!
www.youtube.com/watch?v=S5l5...
A talk about LLM reasoning, covering various methods, core problems, and future research directions!
www.youtube.com/watch?v=S5l5...
The Hitchhiker's Guide to Reasoning
YouTube video by Ross Taylor
www.youtube.com
December 2, 2024 at 11:33 AM
(🎥45m) The Hitchhiker's Guide to Reasoning
A talk about LLM reasoning, covering various methods, core problems, and future research directions!
www.youtube.com/watch?v=S5l5...
A talk about LLM reasoning, covering various methods, core problems, and future research directions!
www.youtube.com/watch?v=S5l5...
Reposted by Ross Taylor
IMO VQGAN is why GANs deserve the NeurIPS test of time award. Suddenly our image representations were an order of magnitude more compact. Absolute game changer for generative modelling at scale, and the basis for latent diffusion models.
Taming Transformers for High-Resolution Image Synthesis
Designed to learn long-range interactions on sequential data, transformers continue to show state-of-the-art results on a wide variety of tasks. In contrast to CNNs, they contain no inductive bias tha...
arxiv.org
November 28, 2024 at 12:09 AM
IMO VQGAN is why GANs deserve the NeurIPS test of time award. Suddenly our image representations were an order of magnitude more compact. Absolute game changer for generative modelling at scale, and the basis for latent diffusion models.
The toxicity towards that HF employee for sharing that Bluesky post dataset was really bad, and reminded me of 2022 anti-LLM sentiment.
Not a great look for 🦋.
Not a great look for 🦋.
November 28, 2024 at 6:26 AM
The toxicity towards that HF employee for sharing that Bluesky post dataset was really bad, and reminded me of 2022 anti-LLM sentiment.
Not a great look for 🦋.
Not a great look for 🦋.
Reposted by Ross Taylor
I personally vote text is self supervised learning (SSL) for the same reason image is SSL.
Text tokenization is a modification but so are patches in vision. If you want you can go all the way down to bits or raw audio samples. We know text bits work as (again modded) it's used in LLM compression.
Text tokenization is a modification but so are patches in vision. If you want you can go all the way down to bits or raw audio samples. We know text bits work as (again modded) it's used in LLM compression.
November 19, 2024 at 9:20 AM
I personally vote text is self supervised learning (SSL) for the same reason image is SSL.
Text tokenization is a modification but so are patches in vision. If you want you can go all the way down to bits or raw audio samples. We know text bits work as (again modded) it's used in LLM compression.
Text tokenization is a modification but so are patches in vision. If you want you can go all the way down to bits or raw audio samples. We know text bits work as (again modded) it's used in LLM compression.
Is it right to call large-scale pre-training on text self-supervised learning?
Seems more like large-scale, multi-task supervised learning. Labels are present as semantic units (tokens) in the corpus, unlike SSL for vision and other modalities?
Seems more like large-scale, multi-task supervised learning. Labels are present as semantic units (tokens) in the corpus, unlike SSL for vision and other modalities?
November 18, 2024 at 9:01 AM
Is it right to call large-scale pre-training on text self-supervised learning?
Seems more like large-scale, multi-task supervised learning. Labels are present as semantic units (tokens) in the corpus, unlike SSL for vision and other modalities?
Seems more like large-scale, multi-task supervised learning. Labels are present as semantic units (tokens) in the corpus, unlike SSL for vision and other modalities?
Giving this place a go…
November 17, 2024 at 5:57 PM
Giving this place a go…