Javier Cancela
LightNews LightNews
banner
javiercancela.com
Javier Cancela
@javiercancela.com
98 followers 680 following 55 posts
Interested in data and AI (mostly LLMs).
Mainly in English, but sometimes in Spanish or Galician.
Director of Data Analytics @ Joor
Posts Replies Media Videos
javiercancela.com
Oops
Percentage of people 65 or older in the Iberian Peninsula. In Galicia we are f*cked.
December 13, 2024 at 6:57 PM
javiercancela.com
Speaking of this, the original MapReduce paper contains what I imagine are the first versions of Borg, Jupiter, and Colossus.
(1) Machines are typically dual-processor x86 processors running Linux, with 2-4 GB of memory per machine. > (2) Commodity networking hardware is used – typically either 100 megabits/second or 1 gigabit/second at the machine level, but averaging considerably less in overall bisection bandwidth. > (3) A cluster consists of hundreds or thousands of machines, and therefore machine failures are common. > (4) Storage is provided by inexpensive IDE disks attached directly to individual machines. A distributed file system [8] developed in-house is used to manage the data stored on these disks. The file system uses replication to provide availability and reliability on top of unreliable hardware. > (5) Users submit jobs to a scheduling system. Each job consists of a set of tasks, and is mapped by the scheduler to a set of available machines within a cluster.
December 1, 2024 at 5:13 PM
javiercancela.com
I especially liked this answer.
Question from Leo on Twitter: It doesn’t interpolate, does it? If I ask “What color is a Gropy?”, and we had 100 labellers say it’s blue and 100 labellers say it’s yellow, it’s going to randomly say blue or yellow - but never “It’s a debated question, some say blue, some say yellow”. Right? Answer from Andrej: Excellent question and yes exactly, it responds with blue or yellow with 50% probability. Saying “It’s a debated question, some say blue, some say yellow” is just a sequence of tokens that would be super unlikely, it doesn't match the statistics of the training data at all.
November 29, 2024 at 8:55 PM
javiercancela.com
Andrej is one of the reasons why I still check Twitter.
Text from Andrej Karpathy in Twitter: People have too inflated sense of what it means to "ask an AI" about something. The AI are language models trained basically by imitation on data from human labelers. Instead of the mysticism of "asking an AI", think of it more as "asking the average data labeler" on the internet. Few caveats apply because e.g. in many domains (e.g. code, math, creative writing) the companies hire skilled data labelers (so think of it as asking them instead), and this is not 100% true when reinforcement learning is involved, though I have an earlier rant on how RLHF is just barely RL, and "actual RL" is still too early and/or constrained to domains that offer easy reward functions (math etc.). But roughly speaking (and today), you're not asking some magical AI. You're asking a human data labeler. Whose average essence was lossily distilled into statistical token tumblers that are LLMs. This can still be super useful ofc ourse. Post triggered by someone suggesting we ask an AI how to run the government etc. TLDR you're not asking an AI, you're asking some mashup spirit of its average data labeler.
November 29, 2024 at 8:54 PM
javiercancela.com
Ouch!
November 24, 2024 at 9:38 AM
javiercancela.com
I like this take on AI art.
www.astralcodexten.com/p/how-did-yo...
Extract from the link
November 21, 2024 at 4:42 PM