in Claude Code, which models can read a skill file with 3 simple criteria to follow a specific code style?
Sonnet 4.5 can for sure. But haiku 4.5 cannot consistently.
Sonnet 4.5 can for sure. But haiku 4.5 cannot consistently.
December 27, 2025 at 1:18 AM
in Claude Code, which models can read a skill file with 3 simple criteria to follow a specific code style?
Sonnet 4.5 can for sure. But haiku 4.5 cannot consistently.
Sonnet 4.5 can for sure. But haiku 4.5 cannot consistently.
One day into building my eval driven python skills for claude code and I am already running into instruction following issues (haiku 4.5).
There's only 3 instructions and it can't follow them consistently.
My whole skill can fit in a post, so I'll attach it in the replies.
There's only 3 instructions and it can't follow them consistently.
My whole skill can fit in a post, so I'll attach it in the replies.
December 24, 2025 at 1:39 PM
One day into building my eval driven python skills for claude code and I am already running into instruction following issues (haiku 4.5).
There's only 3 instructions and it can't follow them consistently.
My whole skill can fit in a post, so I'll attach it in the replies.
There's only 3 instructions and it can't follow them consistently.
My whole skill can fit in a post, so I'll attach it in the replies.
"it's a gamechanger, trust me bro" in AI coding circles is like a sickness, and @scottspence.dev writing a real evaluation for his svelte claude code plugin is my medicine. Thank you for this!
I'm making python plugins using this strategy because data driven is the way!
github.com/jack-michaud...
I'm making python plugins using this strategy because data driven is the way!
github.com/jack-michaud...
GitHub - jack-michaud/faire
Contribute to jack-michaud/faire development by creating an account on GitHub.
github.com
December 23, 2025 at 1:00 PM
"it's a gamechanger, trust me bro" in AI coding circles is like a sickness, and @scottspence.dev writing a real evaluation for his svelte claude code plugin is my medicine. Thank you for this!
I'm making python plugins using this strategy because data driven is the way!
github.com/jack-michaud...
I'm making python plugins using this strategy because data driven is the way!
github.com/jack-michaud...
Reposted by jm
New from Microsoft Research & Harvard Business School colleagues
"Shifting Work Patterns with Generative AI" by Eleanor Wiske Dillon, Sonia Jaffe, Nicole Immorlica, Christopher T. Stanton
arxiv.org/abs/2504.11436
"Shifting Work Patterns with Generative AI" by Eleanor Wiske Dillon, Sonia Jaffe, Nicole Immorlica, Christopher T. Stanton
arxiv.org/abs/2504.11436
April 19, 2025 at 10:13 PM
New from Microsoft Research & Harvard Business School colleagues
"Shifting Work Patterns with Generative AI" by Eleanor Wiske Dillon, Sonia Jaffe, Nicole Immorlica, Christopher T. Stanton
arxiv.org/abs/2504.11436
"Shifting Work Patterns with Generative AI" by Eleanor Wiske Dillon, Sonia Jaffe, Nicole Immorlica, Christopher T. Stanton
arxiv.org/abs/2504.11436
Reposted by jm
1/🚨 𝗡𝗲𝘄 𝗽𝗮𝗽𝗲𝗿 𝗮𝗹𝗲𝗿𝘁 🚨
RAG systems excel on academic benchmarks - but are they robust to variations in linguistic style?
We find RAG systems are brittle. Small shifts in phrasing trigger cascading errors, driven by the complexity of the RAG pipeline 🧵
RAG systems excel on academic benchmarks - but are they robust to variations in linguistic style?
We find RAG systems are brittle. Small shifts in phrasing trigger cascading errors, driven by the complexity of the RAG pipeline 🧵
April 17, 2025 at 7:55 PM
1/🚨 𝗡𝗲𝘄 𝗽𝗮𝗽𝗲𝗿 𝗮𝗹𝗲𝗿𝘁 🚨
RAG systems excel on academic benchmarks - but are they robust to variations in linguistic style?
We find RAG systems are brittle. Small shifts in phrasing trigger cascading errors, driven by the complexity of the RAG pipeline 🧵
RAG systems excel on academic benchmarks - but are they robust to variations in linguistic style?
We find RAG systems are brittle. Small shifts in phrasing trigger cascading errors, driven by the complexity of the RAG pipeline 🧵
When a new language model conquers a benchmark, it's because that benchmark exposed an "adversarial case" in the model's function - then they stuff in a trillion tokens of data augmentation to fix that case.
But this highlights something fundamental about today's language model architectures...
But this highlights something fundamental about today's language model architectures...
arxiv.org/abs/2406.12843
"Our results suggest that building robust AI systems is challenging even with extremely superhuman systems in some of the most tractable settings, and highlight two key gaps: efficient generalization in defenses, and diversity in training."
"Our results suggest that building robust AI systems is challenging even with extremely superhuman systems in some of the most tractable settings, and highlight two key gaps: efficient generalization in defenses, and diversity in training."
Can Go AIs be adversarially robust?
Prior work found that superhuman Go AIs can be defeated by simple adversarial strategies, especially "cyclic" attacks. In this paper, we study whether adding natural countermeasures can achieve robust...
arxiv.org
December 24, 2024 at 7:56 PM
When a new language model conquers a benchmark, it's because that benchmark exposed an "adversarial case" in the model's function - then they stuff in a trillion tokens of data augmentation to fix that case.
But this highlights something fundamental about today's language model architectures...
But this highlights something fundamental about today's language model architectures...
arxiv.org/abs/2406.12843
"Our results suggest that building robust AI systems is challenging even with extremely superhuman systems in some of the most tractable settings, and highlight two key gaps: efficient generalization in defenses, and diversity in training."
"Our results suggest that building robust AI systems is challenging even with extremely superhuman systems in some of the most tractable settings, and highlight two key gaps: efficient generalization in defenses, and diversity in training."
Can Go AIs be adversarially robust?
Prior work found that superhuman Go AIs can be defeated by simple adversarial strategies, especially "cyclic" attacks. In this paper, we study whether adding natural countermeasures can achieve robust...
arxiv.org
December 24, 2024 at 7:48 PM
arxiv.org/abs/2406.12843
"Our results suggest that building robust AI systems is challenging even with extremely superhuman systems in some of the most tractable settings, and highlight two key gaps: efficient generalization in defenses, and diversity in training."
"Our results suggest that building robust AI systems is challenging even with extremely superhuman systems in some of the most tractable settings, and highlight two key gaps: efficient generalization in defenses, and diversity in training."
Still waiting for the GSM-Symbolic benchmarks on o3.. if you can pay $1000 for a model that gets something right between 80-99% of the time, is that still valuable?
December 20, 2024 at 7:57 PM
Still waiting for the GSM-Symbolic benchmarks on o3.. if you can pay $1000 for a model that gets something right between 80-99% of the time, is that still valuable?
Reposted by jm
"Wrong people listened to on puberty blockers ban".
The National in Scotland continuing to do some good work.
The National in Scotland continuing to do some good work.
December 17, 2024 at 9:44 AM
"Wrong people listened to on puberty blockers ban".
The National in Scotland continuing to do some good work.
The National in Scotland continuing to do some good work.
Reposted by jm
I've trained a new handwriting flux lora.
It can do many styles.
Prompt it with HWRIT keyword, give it some short text, a handwriting style and some ink and paper types.
More examples and download links in 🧵
It can do many styles.
Prompt it with HWRIT keyword, give it some short text, a handwriting style and some ink and paper types.
More examples and download links in 🧵
December 14, 2024 at 1:39 PM
I've trained a new handwriting flux lora.
It can do many styles.
Prompt it with HWRIT keyword, give it some short text, a handwriting style and some ink and paper types.
More examples and download links in 🧵
It can do many styles.
Prompt it with HWRIT keyword, give it some short text, a handwriting style and some ink and paper types.
More examples and download links in 🧵
Reposted by jm
Want to clear up some misconceptions, pydantic is actually short for Pytholomew Daniel Ticonderoga, the inventor and patent-holder of the first disposable pencil that also forced you to write only on paper with dotted lines
December 3, 2024 at 12:13 AM
Want to clear up some misconceptions, pydantic is actually short for Pytholomew Daniel Ticonderoga, the inventor and patent-holder of the first disposable pencil that also forced you to write only on paper with dotted lines
Reposted by jm
can I code fast? no. but can I code well? also no. but does my code work? alas, no
November 30, 2024 at 9:39 PM
can I code fast? no. but can I code well? also no. but does my code work? alas, no
Reposted by jm
New paper: Do social media algorithms shape affective polarization?
We ran a field experiment on X/Twitter (N=1,256) using LLMs to rerank content in real-time, adjusting exposure to polarizing posts. Result: Algorithmic ranking impacts feelings toward the political outgroup! 🧵⬇️
We ran a field experiment on X/Twitter (N=1,256) using LLMs to rerank content in real-time, adjusting exposure to polarizing posts. Result: Algorithmic ranking impacts feelings toward the political outgroup! 🧵⬇️
November 25, 2024 at 8:32 PM
New paper: Do social media algorithms shape affective polarization?
We ran a field experiment on X/Twitter (N=1,256) using LLMs to rerank content in real-time, adjusting exposure to polarizing posts. Result: Algorithmic ranking impacts feelings toward the political outgroup! 🧵⬇️
We ran a field experiment on X/Twitter (N=1,256) using LLMs to rerank content in real-time, adjusting exposure to polarizing posts. Result: Algorithmic ranking impacts feelings toward the political outgroup! 🧵⬇️
Reposted by jm
Doing good science is 90% finding a science buddy to constantly talk to about the project.
November 9, 2024 at 10:53 PM
Doing good science is 90% finding a science buddy to constantly talk to about the project.
Reposted by jm
posting "Gentle reminder that it's okay to unplug today and take care of yourself. Pass it on." immediately before commencing a 19-hour, 277-post meltdown
November 5, 2024 at 3:05 PM
posting "Gentle reminder that it's okay to unplug today and take care of yourself. Pass it on." immediately before commencing a 19-hour, 277-post meltdown
I miss DJ Filthy K
A notable cultural shift over the last 40 years is that DJs used to have names like DJ Funky Paul and now DJs all have names like DJ Adult Circumcision
September 21, 2023 at 11:20 AM
I miss DJ Filthy K
Reposted by jm
A notable cultural shift over the last 40 years is that DJs used to have names like DJ Funky Paul and now DJs all have names like DJ Adult Circumcision
September 21, 2023 at 10:24 AM
A notable cultural shift over the last 40 years is that DJs used to have names like DJ Funky Paul and now DJs all have names like DJ Adult Circumcision
Reposted by jm
"Create a problem to solve for pay," this dog was the first tech bro.
September 20, 2023 at 11:57 PM
"Create a problem to solve for pay," this dog was the first tech bro.
wondering how possible it is to automate a system to add people to a mute list. we could make a handle to @ which allows suggestions to happen. then there's a language model that is really good at classifying toxic content called wormgpt that could be used to check submissions. fairly possible
September 21, 2023 at 1:51 AM
wondering how possible it is to automate a system to add people to a mute list. we could make a handle to @ which allows suggestions to happen. then there's a language model that is really good at classifying toxic content called wormgpt that could be used to check submissions. fairly possible
santa's head comes off www.youtube.com/watch?v=Wecb...
adding to my list of enthralling asmr channels
adding to my list of enthralling asmr channels
I Applied HIGH VOLTAGE to Electric Toys! #5 (DANGEROUS)
I Applied HIGH VOLTAGE to Electric Toys! #5 (DANGEROUS)In this video, I connected some electric toys to the WANPTEK DC power supply using wires. I continuous...
www.youtube.com
September 21, 2023 at 12:18 AM
santa's head comes off www.youtube.com/watch?v=Wecb...
adding to my list of enthralling asmr channels
adding to my list of enthralling asmr channels
i'm addicted to this youtube channel. he plays switch sports golf. it's like asmr for kens youtu.be/AtuAhtRMtJg?...
September 19, 2023 at 11:34 PM
i'm addicted to this youtube channel. he plays switch sports golf. it's like asmr for kens youtu.be/AtuAhtRMtJg?...