@wololooo.bsky.social
Reposted
Blogpost to read today: strong argument that excessive focus on the first tokens is not something learned from data distribution (like model should naturally "care" about the start of the text to grasp the rest) but a fundamental feature of attention graph. publish.obsidian.md/the-tensor-t...
August 24, 2025 at 4:33 PM
Blogpost to read today: strong argument that excessive focus on the first tokens is not something learned from data distribution (like model should naturally "care" about the start of the text to grasp the rest) but a fundamental feature of attention graph. publish.obsidian.md/the-tensor-t...
Reposted
In a horrifying abuse of modern computers, here's a shell one-liner you can run on macOS to calculate the number of days between two dates in the most complex way imaginable - using uv and Deno and the Pydantic AI Python sandbox MCP server simonwillison.net/2025/Apr/18/...
![ANTHROPIC_API_KEY="sk-ant-..." \
uv run --with pydantic-ai python -c '
import asyncio
from pydantic_ai import Agent
from pydantic_ai.mcp import MCPServerStdio
server = MCPServerStdio(
"deno",
args=[
"run",
"-N",
"-R=node_modules",
"-W=node_modules",
"--node-modules-dir=auto",
"jsr:@pydantic/mcp-run-python",
"stdio",
],
)
agent = Agent("claude-3-5-haiku-latest", mcp_servers=[server])
async def main():
async with agent.run_mcp_servers():
result = await agent.run("How many days between 2000-01-01 and 2025-03-18?")
print(result.output)
asyncio.run(main())'](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:kft6lu4trxowqmter2b6vg6z/bafkreifooagn44gyz6vnte4xljdlfngex4gfig5h4pa6f3yj5xeqlhaoii@jpeg)
April 18, 2025 at 4:54 AM
In a horrifying abuse of modern computers, here's a shell one-liner you can run on macOS to calculate the number of days between two dates in the most complex way imaginable - using uv and Deno and the Pydantic AI Python sandbox MCP server simonwillison.net/2025/Apr/18/...
Regex free Tokenizer
github.com/exploringwei...
github.com/exploringwei...
GitHub - exploringweirdmachines/Entropy-based-Tokenizer: A memory-efficient tokenizer that uses entropy calculations to identify natural boundaries in text and create tokens.
A memory-efficient tokenizer that uses entropy calculations to identify natural boundaries in text and create tokens. - GitHub - exploringweirdmachines/Entropy-based-Tokenizer: A memory-efficient ...
github.com
December 30, 2024 at 10:08 PM
Regex free Tokenizer
github.com/exploringwei...
github.com/exploringwei...
Entropy based text chunker.
github.com/exploringwei...
github.com/exploringwei...
GitHub - exploringweirdmachines/EntropyTextMerger
Contribute to exploringweirdmachines/EntropyTextMerger development by creating an account on GitHub.
github.com
December 22, 2024 at 11:31 AM
Entropy based text chunker.
github.com/exploringwei...
github.com/exploringwei...
Reposted
This is a neat new variant on RAG - no vectors, not even full-text search, instead showing the model a header hierarchy and giving it a tool to read the relevant sections
My notes here: simonwillison.net/2024/Dec/6/r...
My notes here: simonwillison.net/2024/Dec/6/r...
December 6, 2024 at 3:04 AM
This is a neat new variant on RAG - no vectors, not even full-text search, instead showing the model a header hierarchy and giving it a tool to read the relevant sections
My notes here: simonwillison.net/2024/Dec/6/r...
My notes here: simonwillison.net/2024/Dec/6/r...
Reposted
i wrote a custom llm sampler for llama-3.1-8b so it could only say words that are in the bible
github.com/vgel/biblica...
github.com/vgel/biblica...
December 6, 2024 at 3:22 AM
i wrote a custom llm sampler for llama-3.1-8b so it could only say words that are in the bible
github.com/vgel/biblica...
github.com/vgel/biblica...
Reposted
I always dreamed of a model that simultaneously
1. optimizes NLL of raw pixel data,
2. generates competitive high-res. natural images,
3. is practical.
But it seemed too good to be true. Until today!
Our new JetFormer model (arxiv.org/abs/2411.19722) ticks on all of these.
🧵
1. optimizes NLL of raw pixel data,
2. generates competitive high-res. natural images,
3. is practical.
But it seemed too good to be true. Until today!
Our new JetFormer model (arxiv.org/abs/2411.19722) ticks on all of these.
🧵
December 2, 2024 at 5:19 PM
I always dreamed of a model that simultaneously
1. optimizes NLL of raw pixel data,
2. generates competitive high-res. natural images,
3. is practical.
But it seemed too good to be true. Until today!
Our new JetFormer model (arxiv.org/abs/2411.19722) ticks on all of these.
🧵
1. optimizes NLL of raw pixel data,
2. generates competitive high-res. natural images,
3. is practical.
But it seemed too good to be true. Until today!
Our new JetFormer model (arxiv.org/abs/2411.19722) ticks on all of these.
🧵
Reposted
In a strange turn of events, George Hotz has written an article advocating for expedited nuclear war, and has also deleted all of his tweets
November 30, 2024 at 5:03 AM
In a strange turn of events, George Hotz has written an article advocating for expedited nuclear war, and has also deleted all of his tweets