Smerity
@smerity.bsky.social
Always pondering startups, ML, Rust, Python, and 3D printing.
Independent ML researcher consulting on LMs + data.
Previously: Salesforce Research, MetaMind, CommonCrawl, Harvard. 🇦🇺 in SF. He/him.
Personal blog: https://state.smerity.com
Independent ML researcher consulting on LMs + data.
Previously: Salesforce Research, MetaMind, CommonCrawl, Harvard. 🇦🇺 in SF. He/him.
Personal blog: https://state.smerity.com
Pinned
Smerity
@smerity.bsky.social
· Nov 22
Smerity.com: Python isn't just glue, it's an implicit JIT ecosystem
The ecosystem wears in the desire paths, and when slow Python becomes a problem, the ecosystem doesn't optimize the Python, it paves new roads beneath
state.smerity.com
Writing more Rust recently led me to a revelation about Python.
Rust was vital to my original task, but only a few simplifications away, the shorter Python version leapt to almost as fast. I'd stumbled from a cold path to a hot path, part of a vast network of optimized trails in Python's ecosystem.
Rust was vital to my original task, but only a few simplifications away, the shorter Python version leapt to almost as fast. I'd stumbled from a cold path to a hot path, part of a vast network of optimized trails in Python's ecosystem.
Brainstorming with an LLM that's glazing you a tad is helpful when you're under glazing by default 🤔
September 12, 2025 at 9:17 PM
Brainstorming with an LLM that's glazing you a tad is helpful when you're under glazing by default 🤔
Reposted by Smerity
einstein sent this to curie in 1911 when she was being harassed by tabloids. it contains everything you’d want in such a letter:
(1) your haters are trash
(2) you’re a baller, a true queen
(3) i have determined the statistical law of motion of the diatomic molecule in planck’s radiation field 🧪⚛️
(1) your haters are trash
(2) you’re a baller, a true queen
(3) i have determined the statistical law of motion of the diatomic molecule in planck’s radiation field 🧪⚛️
June 27, 2024 at 2:17 PM
einstein sent this to curie in 1911 when she was being harassed by tabloids. it contains everything you’d want in such a letter:
(1) your haters are trash
(2) you’re a baller, a true queen
(3) i have determined the statistical law of motion of the diatomic molecule in planck’s radiation field 🧪⚛️
(1) your haters are trash
(2) you’re a baller, a true queen
(3) i have determined the statistical law of motion of the diatomic molecule in planck’s radiation field 🧪⚛️
Reposted by Smerity
Very few people see a full video on YouTube apparently, even if it lasts just 10-15 minutes.
January 4, 2025 at 11:02 AM
Very few people see a full video on YouTube apparently, even if it lasts just 10-15 minutes.
My contribution to a discussion at hiddenstates.org on explorables/user interfaces for controlling ML tools
> We're trying to rig soundboards to control LLMs thinking there's a well defined interface underneath when it's actually a button that drops fertilizer into the river of a complex ecosystem.
> We're trying to rig soundboards to control LLMs thinking there's a well defined interface underneath when it's actually a button that drops fertilizer into the river of a complex ecosystem.
December 3, 2024 at 8:34 PM
My contribution to a discussion at hiddenstates.org on explorables/user interfaces for controlling ML tools
> We're trying to rig soundboards to control LLMs thinking there's a well defined interface underneath when it's actually a button that drops fertilizer into the river of a complex ecosystem.
> We're trying to rig soundboards to control LLMs thinking there's a well defined interface underneath when it's actually a button that drops fertilizer into the river of a complex ecosystem.
Reposted by Smerity
The fact that most wouldn't pay money for passive consumption "there" (reels, feed, ...) but do actively pay with the tick tock of our mind (hours spent, enriching the content locally, ...) is fascinating. If this is psychological obliteration we're very much an active participant in it.
November 29, 2024 at 1:24 PM
The fact that most wouldn't pay money for passive consumption "there" (reels, feed, ...) but do actively pay with the tick tock of our mind (hours spent, enriching the content locally, ...) is fascinating. If this is psychological obliteration we're very much an active participant in it.
Reposted by Smerity
A new-to-me perspective on passive consumption of media
(Also: I'm gifting you some psychological obliteration here!)
(Also: I'm gifting you some psychological obliteration here!)
November 29, 2024 at 5:06 AM
A new-to-me perspective on passive consumption of media
(Also: I'm gifting you some psychological obliteration here!)
(Also: I'm gifting you some psychological obliteration here!)
Reposted by Smerity
congratulations, @ian-goodfellow.bsky.social, for the test-of-time award at @neuripsconf.bsky.social!
this award reminds me of how GAN started with this one email ian sent to the Mila (then Lisa) lab mailing list in May 2014. super insightful and amazing execution!
this award reminds me of how GAN started with this one email ian sent to the Mila (then Lisa) lab mailing list in May 2014. super insightful and amazing execution!
November 27, 2024 at 6:31 PM
congratulations, @ian-goodfellow.bsky.social, for the test-of-time award at @neuripsconf.bsky.social!
this award reminds me of how GAN started with this one email ian sent to the Mila (then Lisa) lab mailing list in May 2014. super insightful and amazing execution!
this award reminds me of how GAN started with this one email ian sent to the Mila (then Lisa) lab mailing list in May 2014. super insightful and amazing execution!
Who is playing around with Bluesky at an API / data level?
I've used the API to {read a bot's DMs, collect your likes, analyze your following / followers} (with heavy `time.sleep` for latter as I've not found bulk queries).
Leaping into "Bluesky as a dataset" by exploring Jetstream.
Hints?
I've used the API to {read a bot's DMs, collect your likes, analyze your following / followers} (with heavy `time.sleep` for latter as I've not found bulk queries).
Leaping into "Bluesky as a dataset" by exploring Jetstream.
Hints?
November 26, 2024 at 1:38 AM
Who is playing around with Bluesky at an API / data level?
I've used the API to {read a bot's DMs, collect your likes, analyze your following / followers} (with heavy `time.sleep` for latter as I've not found bulk queries).
Leaping into "Bluesky as a dataset" by exploring Jetstream.
Hints?
I've used the API to {read a bot's DMs, collect your likes, analyze your following / followers} (with heavy `time.sleep` for latter as I've not found bulk queries).
Leaping into "Bluesky as a dataset" by exploring Jetstream.
Hints?
Writing more Rust recently led me to a revelation about Python.
Rust was vital to my original task, but only a few simplifications away, the shorter Python version leapt to almost as fast. I'd stumbled from a cold path to a hot path, part of a vast network of optimized trails in Python's ecosystem.
Rust was vital to my original task, but only a few simplifications away, the shorter Python version leapt to almost as fast. I'd stumbled from a cold path to a hot path, part of a vast network of optimized trails in Python's ecosystem.
Smerity.com: Python isn't just glue, it's an implicit JIT ecosystem
The ecosystem wears in the desire paths, and when slow Python becomes a problem, the ecosystem doesn't optimize the Python, it paves new roads beneath
state.smerity.com
November 22, 2024 at 9:35 PM
Writing more Rust recently led me to a revelation about Python.
Rust was vital to my original task, but only a few simplifications away, the shorter Python version leapt to almost as fast. I'd stumbled from a cold path to a hot path, part of a vast network of optimized trails in Python's ecosystem.
Rust was vital to my original task, but only a few simplifications away, the shorter Python version leapt to almost as fast. I'd stumbled from a cold path to a hot path, part of a vast network of optimized trails in Python's ecosystem.
PSA: Are your Bluesky notifications a tad too loud? You can set Bluesky to only replies and quotes.
Go to the Notifications tab (web or mobile) and click on the ⚙️ in the top right.
Note: This option is not found under Settings
Go to the Notifications tab (web or mobile) and click on the ⚙️ in the top right.
Note: This option is not found under Settings
![Notification Settings
Notification filters
Enable priority notifications [Toggle box]
Experimental: When this preference is enabled, you'll only receive reply and quote notifications from users you follow. We'll continue to add more controls here over time.](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:xbgkcznj7mmvmdlbdze52jka/bafkreib4q7gy35stembonhf4u2viox6xifhwivdhoo6cfewvaorsvtkj4u@jpeg)
November 20, 2024 at 11:12 AM
PSA: Are your Bluesky notifications a tad too loud? You can set Bluesky to only replies and quotes.
Go to the Notifications tab (web or mobile) and click on the ⚙️ in the top right.
Note: This option is not found under Settings
Go to the Notifications tab (web or mobile) and click on the ⚙️ in the top right.
Note: This option is not found under Settings
I still had muscle memory of typing "tw + Enter" to go to X, even a year and a half after the rename.
Browser extensions exist redirecting Twitter / X to Bluesky but those were too extreme - I wrote one that _just_ redirects twitter.com to bsky.com so I could take advantage of that muscle memory 🤣
Browser extensions exist redirecting Twitter / X to Bluesky but those were too extreme - I wrote one that _just_ redirects twitter.com to bsky.com so I could take advantage of that muscle memory 🤣
November 20, 2024 at 8:58 AM
I still had muscle memory of typing "tw + Enter" to go to X, even a year and a half after the rename.
Browser extensions exist redirecting Twitter / X to Bluesky but those were too extreme - I wrote one that _just_ redirects twitter.com to bsky.com so I could take advantage of that muscle memory 🤣
Browser extensions exist redirecting Twitter / X to Bluesky but those were too extreme - I wrote one that _just_ redirects twitter.com to bsky.com so I could take advantage of that muscle memory 🤣
I'm wondering what the world would look like if there was a
🔴Big Red Button🔴
on each page of the documentation for {libraries, APIs, books, ...}.
It doesn't even need to be complex.
A heat map of pain is by itself such a strong signal given we lack insight even as simple as churn rate for non-use.
🔴Big Red Button🔴
on each page of the documentation for {libraries, APIs, books, ...}.
It doesn't even need to be complex.
A heat map of pain is by itself such a strong signal given we lack insight even as simple as churn rate for non-use.
November 19, 2024 at 3:50 PM
I'm wondering what the world would look like if there was a
🔴Big Red Button🔴
on each page of the documentation for {libraries, APIs, books, ...}.
It doesn't even need to be complex.
A heat map of pain is by itself such a strong signal given we lack insight even as simple as churn rate for non-use.
🔴Big Red Button🔴
on each page of the documentation for {libraries, APIs, books, ...}.
It doesn't even need to be complex.
A heat map of pain is by itself such a strong signal given we lack insight even as simple as churn rate for non-use.
Reposted by Smerity
Hi, just sharing an updated version of the PyTorch 2 Internals slides: drive.google.com/file/d/18YZV.... Content: basics, jit, dynamo, Inductor, export path and executorch. This is focused on internals so you will need a bit of C/C++. I show how you can export and run a model on a Pixel Watch too.
November 19, 2024 at 11:05 AM
Hi, just sharing an updated version of the PyTorch 2 Internals slides: drive.google.com/file/d/18YZV.... Content: basics, jit, dynamo, Inductor, export path and executorch. This is focused on internals so you will need a bit of C/C++. I show how you can export and run a model on a Pixel Watch too.
Reposted by Smerity
Ever wanted to train your own 13B Llama2 model from scratch on a 24GB GPU? Or fine-tune one without compromising performance compared to full training? 🦙
You now can, with LoQT: Low-Rank Adapters for Quantized Pretaining! arxiv.org/abs/2405.16528
1/4
You now can, with LoQT: Low-Rank Adapters for Quantized Pretaining! arxiv.org/abs/2405.16528
1/4
November 18, 2024 at 9:29 AM
Ever wanted to train your own 13B Llama2 model from scratch on a 24GB GPU? Or fine-tune one without compromising performance compared to full training? 🦙
You now can, with LoQT: Low-Rank Adapters for Quantized Pretaining! arxiv.org/abs/2405.16528
1/4
You now can, with LoQT: Low-Rank Adapters for Quantized Pretaining! arxiv.org/abs/2405.16528
1/4
Reposted by Smerity
Have you ever done a dense grid search over neural network hyperparameters? Like a *really dense* grid search? It looks like this (!!). Blueish colors correspond to hyperparameters for which training converges, redish to those for which training diverges.
Even better, a video: vimeo.com/903855670
Even better, a video: vimeo.com/903855670
February 12, 2024 at 5:38 AM
Have you ever done a dense grid search over neural network hyperparameters? Like a *really dense* grid search? It looks like this (!!). Blueish colors correspond to hyperparameters for which training converges, redish to those for which training diverges.
Even better, a video: vimeo.com/903855670
Even better, a video: vimeo.com/903855670
I love that Bluesky has App Passwords.
Even better that Sky Follower Bridge, likely one of the first apps a new user will log in to, explicitly nudges you towards using an app password.
There are still full blown APIs (and platforms / banks -_-) that don't allow per-app keys/passwords 😱
Even better that Sky Follower Bridge, likely one of the first apps a new user will log in to, explicitly nudges you towards using an app password.
There are still full blown APIs (and platforms / banks -_-) that don't allow per-app keys/passwords 😱
github.com
November 19, 2024 at 2:06 PM
I love that Bluesky has App Passwords.
Even better that Sky Follower Bridge, likely one of the first apps a new user will log in to, explicitly nudges you towards using an app password.
There are still full blown APIs (and platforms / banks -_-) that don't allow per-app keys/passwords 😱
Even better that Sky Follower Bridge, likely one of the first apps a new user will log in to, explicitly nudges you towards using an app password.
There are still full blown APIs (and platforms / banks -_-) that don't allow per-app keys/passwords 😱
Reposted by Smerity
Game Engine Black Book: DOOM v1.2
Game Engine Black Book: WOLFENSTEIN 3D v2.2
The Book of CP-System v1.0
Free pdfs, high-quality 80g premium color prints, PlayStore, and source code are now available.
fabiensanglard.net/three_books_...
Game Engine Black Book: WOLFENSTEIN 3D v2.2
The Book of CP-System v1.0
Free pdfs, high-quality 80g premium color prints, PlayStore, and source code are now available.
fabiensanglard.net/three_books_...
November 17, 2024 at 6:14 AM
Game Engine Black Book: DOOM v1.2
Game Engine Black Book: WOLFENSTEIN 3D v2.2
The Book of CP-System v1.0
Free pdfs, high-quality 80g premium color prints, PlayStore, and source code are now available.
fabiensanglard.net/three_books_...
Game Engine Black Book: WOLFENSTEIN 3D v2.2
The Book of CP-System v1.0
Free pdfs, high-quality 80g premium color prints, PlayStore, and source code are now available.
fabiensanglard.net/three_books_...
If we were mutuals on Twitter ping me and I'll follow you here :)
I found ~800 of the 2.5k follows I have on Twitter / X, a starting point!
Auto-migration had false positives but the feed's core (ML / computing) and flavour (art, science, graphics, ...) transferred ^_^
www.sky-follower-bridge.dev
I found ~800 of the 2.5k follows I have on Twitter / X, a starting point!
Auto-migration had false positives but the feed's core (ML / computing) and flavour (art, science, graphics, ...) transferred ^_^
www.sky-follower-bridge.dev
Sky Follower Bridge
Instantly find and follow the same users from your 𝕏 followers on Bluesky
www.sky-follower-bridge.dev
November 19, 2024 at 8:53 AM
If we were mutuals on Twitter ping me and I'll follow you here :)
I found ~800 of the 2.5k follows I have on Twitter / X, a starting point!
Auto-migration had false positives but the feed's core (ML / computing) and flavour (art, science, graphics, ...) transferred ^_^
www.sky-follower-bridge.dev
I found ~800 of the 2.5k follows I have on Twitter / X, a starting point!
Auto-migration had false positives but the feed's core (ML / computing) and flavour (art, science, graphics, ...) transferred ^_^
www.sky-follower-bridge.dev
Reposted by Smerity
🚨New dataset + challenge🚨
We release ASL STEM Wiki: the first signing dataset of STEM articles!
📰 254 Wikipedia articles
📹 ~300 hours of ASL interpretations
👋 New task: automatic sign suggestion to make STEM education more accessible
microsoft.com/en-us/resear...
🧵 #EMNLP2024
We release ASL STEM Wiki: the first signing dataset of STEM articles!
📰 254 Wikipedia articles
📹 ~300 hours of ASL interpretations
👋 New task: automatic sign suggestion to make STEM education more accessible
microsoft.com/en-us/resear...
🧵 #EMNLP2024
November 19, 2024 at 12:19 AM
🚨New dataset + challenge🚨
We release ASL STEM Wiki: the first signing dataset of STEM articles!
📰 254 Wikipedia articles
📹 ~300 hours of ASL interpretations
👋 New task: automatic sign suggestion to make STEM education more accessible
microsoft.com/en-us/resear...
🧵 #EMNLP2024
We release ASL STEM Wiki: the first signing dataset of STEM articles!
📰 254 Wikipedia articles
📹 ~300 hours of ASL interpretations
👋 New task: automatic sign suggestion to make STEM education more accessible
microsoft.com/en-us/resear...
🧵 #EMNLP2024
PSA: By default, only people you follow can send you DMs.
If you want your profile to be open to serendipitous DMs from new possible friends it's an explicit option you need to select :)
If you want your profile to be open to serendipitous DMs from new possible friends it's an explicit option you need to select :)
November 19, 2024 at 6:55 AM
PSA: By default, only people you follow can send you DMs.
If you want your profile to be open to serendipitous DMs from new possible friends it's an explicit option you need to select :)
If you want your profile to be open to serendipitous DMs from new possible friends it's an explicit option you need to select :)
Reposted by Smerity
On how Bluesky starter packs might flatten the typical rich-get-richer power law dynamics of social media connections... For a time, at least...
Gotta love a good complexity science meta-post!
Gotta love a good complexity science meta-post!
@lhd.bsky.social wrote a model back in 2011 that I think applies here journals.aps.org/prl/abstract....
Growth is currently driven by cliques (starter packs) and this leads to different scaling relationships.
Growth is currently driven by cliques (starter packs) and this leads to different scaling relationships.
Structural Preferential Attachment: Network Organization beyond the Link
We introduce a mechanism which models the emergence of the universal properties of complex networks, such as scale independence, modularity and self-similarity, and unifies them under a scale-free org...
journals.aps.org
November 18, 2024 at 2:27 AM
On how Bluesky starter packs might flatten the typical rich-get-richer power law dynamics of social media connections... For a time, at least...
Gotta love a good complexity science meta-post!
Gotta love a good complexity science meta-post!
Reposted by Smerity
New here? Interested in AI/ML? Check out these great starter packs!
AI: go.bsky.app/SipA7it
RL: go.bsky.app/3WPHcHg
Women in AI: go.bsky.app/LaGDpqg
NLP: go.bsky.app/SngwGeS
AI and news: go.bsky.app/5sFqVNS
You can also search all starter packs here: blueskydirectory.com/starter-pack...
AI: go.bsky.app/SipA7it
RL: go.bsky.app/3WPHcHg
Women in AI: go.bsky.app/LaGDpqg
NLP: go.bsky.app/SngwGeS
AI and news: go.bsky.app/5sFqVNS
You can also search all starter packs here: blueskydirectory.com/starter-pack...
November 9, 2024 at 9:13 AM
New here? Interested in AI/ML? Check out these great starter packs!
AI: go.bsky.app/SipA7it
RL: go.bsky.app/3WPHcHg
Women in AI: go.bsky.app/LaGDpqg
NLP: go.bsky.app/SngwGeS
AI and news: go.bsky.app/5sFqVNS
You can also search all starter packs here: blueskydirectory.com/starter-pack...
AI: go.bsky.app/SipA7it
RL: go.bsky.app/3WPHcHg
Women in AI: go.bsky.app/LaGDpqg
NLP: go.bsky.app/SngwGeS
AI and news: go.bsky.app/5sFqVNS
You can also search all starter packs here: blueskydirectory.com/starter-pack...
Reposted by Smerity
New post up! ✨
Exploring AT Protocol with Python to visualize the #databs social graph!
davidgasquez.com/exploring-at...
Took less than 1 hour to get the data and plot it. Amazing what you can do with open APIs and great SDKs!
Exploring AT Protocol with Python to visualize the #databs social graph!
davidgasquez.com/exploring-at...
Took less than 1 hour to get the data and plot it. Amazing what you can do with open APIs and great SDKs!
November 14, 2024 at 1:39 PM
New post up! ✨
Exploring AT Protocol with Python to visualize the #databs social graph!
davidgasquez.com/exploring-at...
Took less than 1 hour to get the data and plot it. Amazing what you can do with open APIs and great SDKs!
Exploring AT Protocol with Python to visualize the #databs social graph!
davidgasquez.com/exploring-at...
Took less than 1 hour to get the data and plot it. Amazing what you can do with open APIs and great SDKs!
Reposted by Smerity
This Bluesky follow finder is super neat. It finds folks you don’t follow who follow folks you follow. So it helps fill in your network. It’s the type of cool stuff enabled by an open ecosystem. bsky-follow-finder.theo.io
Bluesky Network Analyzer
Find accounts that you don't follow (yet) but are followed by lots of accounts that you do follow.
bsky-follow-finder.theo.io
November 16, 2024 at 12:51 PM
This Bluesky follow finder is super neat. It finds folks you don’t follow who follow folks you follow. So it helps fill in your network. It’s the type of cool stuff enabled by an open ecosystem. bsky-follow-finder.theo.io
My joke paper that uses the non-linear non-sensical pricing of chicken nuggets in fast food to power a novel activation function for ML models had:
(a) A citation
(b) For a definition I'm proud of and want better known
(c) In a paper + thesis about using ML for typhoon satellite imagery
Happy days!
(a) A citation
(b) For a definition I'm proud of and want better known
(c) In a paper + thesis about using ML for typhoon satellite imagery
Happy days!
![Dropout
Another powerful regularisation technique is dropout [52], which consists of randomly setting some activations to zero. It aims to prevent specific units (neurons) to co-adapt too much. A very concise yet simple definition of dropout has been recently given by [53]:
[...]. Dropout, simply described, is the concept that if you can
learn how to do a task repeatedly whilst drunk, you should be
able to do the task even better when sober. [...]](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:xbgkcznj7mmvmdlbdze52jka/bafkreifcugcsmeuxgsuvflotkc3pyjpwtenhhau7vpavl3fho3ri3wzelq@jpeg)
November 16, 2024 at 11:04 AM
My joke paper that uses the non-linear non-sensical pricing of chicken nuggets in fast food to power a novel activation function for ML models had:
(a) A citation
(b) For a definition I'm proud of and want better known
(c) In a paper + thesis about using ML for typhoon satellite imagery
Happy days!
(a) A citation
(b) For a definition I'm proud of and want better known
(c) In a paper + thesis about using ML for typhoon satellite imagery
Happy days!