mikelpz.bsky.social
LightNews LightNews
mikelpz.bsky.social
@mikelpz.bsky.social
39 followers 690 following 1 posts
Posts Replies Media Videos
Reposted by mikelpz.bsky.social
manuelrico.bsky.social
Hoy hace 1.709 días que el Gobierno Ayuso aprobó el Protocolo que impedía trasladar al hospital a los residentes más vulnerables. 7.291 murieron sin recibir atención médica (5.795 con covid). @fiscal_es no investiga. Ellos pasan, nosotros #NoOlvidamos. ¿Se lo recordamos a diario?
November 22, 2024 at 7:59 AM
Reposted by mikelpz.bsky.social
manuferre.bsky.social
Me está faltando euskalmet aquí
November 18, 2024 at 6:48 PM
Reposted by mikelpz.bsky.social
christophmolnar.bsky.social
Just realized BlueSky allows sharing valuable stuff cause it doesn't punish links. 🤩

Let's start with "What are embeddings" by @vickiboykis.com

The book is a great summary of embeddings, from history to modern approaches.

The best part: it's free.

Link: vickiboykis.com/what_are_emb...
Book outline Over the past decade, embeddings — numerical representations of machine learning features used as input to deep learning models — have become a foundational data structure in industrial machine learning systems. TF-IDF, PCA, and one-hot encoding have always been key tools in machine learning systems as ways to compress and make sense of large amounts of textual data. However, traditional approaches were limited in the amount of context they could reason about with increasing amounts of data. As the volume, velocity, and variety of data captured by modern applications has exploded, creating approaches specifically tailored to scale has become increasingly important. Google’s Word2Vec paper made an important step in moving from simple statistical representations to semantic meaning of words. The subsequent rise of the Transformer architecture and transfer learning, as well as the latest surge in generative methods has enabled the growth of embeddings as a foundational machine learning data structure. This survey paper aims to provide a deep dive into what embeddings are, their history, and usage patterns in industry. Cover image
November 22, 2024 at 11:13 AM
Reposted by mikelpz.bsky.social
jmhessel.bsky.social
LLMs generate novel word sequences not contained in their pretraining data. However, compared to humans, models generate significantly fewer novel n-grams.

RLHF = 30% *more* copying than base!

Awesome work from the awesome Ximing Lu (gloriaximinglu.github.io) et al. 🤩

arxiv.org/pdf/2410.04265
A screenshot from the linked paper's figure 1. The figure is a pretty-complicated three column figure, but --- in essence, it sketches out how the authors compare llm sequences to the pretraining data / human authors to the pretraining data. Humans write more novel n-gram sequences.
November 22, 2024 at 6:14 AM