SocioTechnica
@sociotechnica.org
Creating at the intersection of people, systems, & technology.
A @jessmart.in and @danversfleury.bsky.social project.
A @jessmart.in and @danversfleury.bsky.social project.
Reposted by SocioTechnica
What's next?
We're building a AI intern in this new environment and planning a bakeoff against Pat Sharpe's approach.
Stay tuned for results and more technical deep-dives...
Read the full update here: sociotechnica.org/q1-2025-update
We're building a AI intern in this new environment and planning a bakeoff against Pat Sharpe's approach.
Stay tuned for results and more technical deep-dives...
Read the full update here: sociotechnica.org/q1-2025-update
SocioTechnica Q1 2025 Update
SocioTechnica Q1 2025 Update
sociotechnica.org
April 17, 2025 at 2:07 PM
What's next?
We're building a AI intern in this new environment and planning a bakeoff against Pat Sharpe's approach.
Stay tuned for results and more technical deep-dives...
Read the full update here: sociotechnica.org/q1-2025-update
We're building a AI intern in this new environment and planning a bakeoff against Pat Sharpe's approach.
Stay tuned for results and more technical deep-dives...
Read the full update here: sociotechnica.org/q1-2025-update
What might Pat's replacement be like?
An invention of necessity:
👁️ Environmental Awareness: A shared data environment where the AI sees context & remembers past actions
⚒️ Tool Autonomy: Agents discover, request & create their own tools
🤝 True Partnership: Genuine human-AI collaboration
An invention of necessity:
👁️ Environmental Awareness: A shared data environment where the AI sees context & remembers past actions
⚒️ Tool Autonomy: Agents discover, request & create their own tools
🤝 True Partnership: Genuine human-AI collaboration
April 17, 2025 at 2:07 PM
What might Pat's replacement be like?
An invention of necessity:
👁️ Environmental Awareness: A shared data environment where the AI sees context & remembers past actions
⚒️ Tool Autonomy: Agents discover, request & create their own tools
🤝 True Partnership: Genuine human-AI collaboration
An invention of necessity:
👁️ Environmental Awareness: A shared data environment where the AI sees context & remembers past actions
⚒️ Tool Autonomy: Agents discover, request & create their own tools
🤝 True Partnership: Genuine human-AI collaboration
Let's look at Pat Sharpe's 3-month performance review:
❎ Needs microscopic task breakdowns
❎ Forgets everything after each task
❎ Never learns from mistakes
❎ Can't do basic math
❎ Works in a windowless room with no awareness
❎ Needs everything translated to simplified formats
You're fired. 🙅
❎ Needs microscopic task breakdowns
❎ Forgets everything after each task
❎ Never learns from mistakes
❎ Can't do basic math
❎ Works in a windowless room with no awareness
❎ Needs everything translated to simplified formats
You're fired. 🙅
April 17, 2025 at 2:07 PM
Let's look at Pat Sharpe's 3-month performance review:
❎ Needs microscopic task breakdowns
❎ Forgets everything after each task
❎ Never learns from mistakes
❎ Can't do basic math
❎ Works in a windowless room with no awareness
❎ Needs everything translated to simplified formats
You're fired. 🙅
❎ Needs microscopic task breakdowns
❎ Forgets everything after each task
❎ Never learns from mistakes
❎ Can't do basic math
❎ Works in a windowless room with no awareness
❎ Needs everything translated to simplified formats
You're fired. 🙅
The results: $4.33/day average over two weeks ✅
We (almost) hit our $5/day target! But what we built was still fairly dumb—essentially just fancy scripts that don't truly leverage the reasoning capabilities that make agents interesting.
We (almost) hit our $5/day target! But what we built was still fairly dumb—essentially just fancy scripts that don't truly leverage the reasoning capabilities that make agents interesting.
April 17, 2025 at 2:07 PM
The results: $4.33/day average over two weeks ✅
We (almost) hit our $5/day target! But what we built was still fairly dumb—essentially just fancy scripts that don't truly leverage the reasoning capabilities that make agents interesting.
We (almost) hit our $5/day target! But what we built was still fairly dumb—essentially just fancy scripts that don't truly leverage the reasoning capabilities that make agents interesting.
What worked well: Communication!
Our LLM kept us updated through Discord with helpful and fun pirate-themed messages. No need to specify exact text—the AI knew how to be engaging and clear when reporting back.
Our LLM kept us updated through Discord with helpful and fun pirate-themed messages. No need to specify exact text—the AI knew how to be engaging and clear when reporting back.
April 17, 2025 at 2:07 PM
What worked well: Communication!
Our LLM kept us updated through Discord with helpful and fun pirate-themed messages. No need to specify exact text—the AI knew how to be engaging and clear when reporting back.
Our LLM kept us updated through Discord with helpful and fun pirate-themed messages. No need to specify exact text—the AI knew how to be engaging and clear when reporting back.
Problem 3: LLMs are bad at math 🧮
We saw this coming.
Calculating profit margins is crucial, and AI consistently makes elementary errors. Even giving it a calculator didn't help—calculators require knowing how to use them properly!
We ended up writing custom calculation tools instead.
We saw this coming.
Calculating profit margins is crucial, and AI consistently makes elementary errors. Even giving it a calculator didn't help—calculators require knowing how to use them properly!
We ended up writing custom calculation tools instead.
April 17, 2025 at 2:07 PM
Problem 3: LLMs are bad at math 🧮
We saw this coming.
Calculating profit margins is crucial, and AI consistently makes elementary errors. Even giving it a calculator didn't help—calculators require knowing how to use them properly!
We ended up writing custom calculation tools instead.
We saw this coming.
Calculating profit margins is crucial, and AI consistently makes elementary errors. Even giving it a calculator didn't help—calculators require knowing how to use them properly!
We ended up writing custom calculation tools instead.
Problem 2: Big context windows ≠ smarter
LLMs often drew incorrect conclusions or overlooked details when parsing the entire HTML for the page.
We had to transform everything into simplified, structured formats like CSV with filtered columns.
LLMs often drew incorrect conclusions or overlooked details when parsing the entire HTML for the page.
We had to transform everything into simplified, structured formats like CSV with filtered columns.
April 17, 2025 at 2:07 PM
Problem 2: Big context windows ≠ smarter
LLMs often drew incorrect conclusions or overlooked details when parsing the entire HTML for the page.
We had to transform everything into simplified, structured formats like CSV with filtered columns.
LLMs often drew incorrect conclusions or overlooked details when parsing the entire HTML for the page.
We had to transform everything into simplified, structured formats like CSV with filtered columns.
Problem #1: Claude has a vision problem 👀
Our AI confused browser UI with website content, misread digits, and struggled with navigation.
An LLM with vision issues + your credit card = danger! We had to shift to browser automation scripts instead.
Our AI confused browser UI with website content, misread digits, and struggled with navigation.
An LLM with vision issues + your credit card = danger! We had to shift to browser automation scripts instead.
April 17, 2025 at 2:07 PM
Problem #1: Claude has a vision problem 👀
Our AI confused browser UI with website content, misread digits, and struggled with navigation.
An LLM with vision issues + your credit card = danger! We had to shift to browser automation scripts instead.
Our AI confused browser UI with website content, misread digits, and struggled with navigation.
An LLM with vision issues + your credit card = danger! We had to shift to browser automation scripts instead.
Meet Pat Sharpe, our pirate-talking AI intern! 🏴☠️🏀
We built a simple trading system with clear rules:
- Buy low, sell high-ish
- Target uninjured stars on playoff teams
- Make offers below market
- Post purchases for resale
- Talk like a pirate, arrr!
We built a simple trading system with clear rules:
- Buy low, sell high-ish
- Target uninjured stars on playoff teams
- Make offers below market
- Post purchases for resale
- Talk like a pirate, arrr!
April 17, 2025 at 2:07 PM
Meet Pat Sharpe, our pirate-talking AI intern! 🏴☠️🏀
We built a simple trading system with clear rules:
- Buy low, sell high-ish
- Target uninjured stars on playoff teams
- Make offers below market
- Post purchases for resale
- Talk like a pirate, arrr!
We built a simple trading system with clear rules:
- Buy low, sell high-ish
- Target uninjured stars on playoff teams
- Make offers below market
- Post purchases for resale
- Talk like a pirate, arrr!
Our Q1 goal: build a system where AI agents could make us $5/day in a marketplace—a stepping stone to economically-valuable AI agents.
We chose NBA Top Shot as our testing ground. Why a dying market? Because we know it well, and handing AI your credit card is weird enough without extra variables.
We chose NBA Top Shot as our testing ground. Why a dying market? Because we know it well, and handing AI your credit card is weird enough without extra variables.
April 17, 2025 at 2:07 PM
Our Q1 goal: build a system where AI agents could make us $5/day in a marketplace—a stepping stone to economically-valuable AI agents.
We chose NBA Top Shot as our testing ground. Why a dying market? Because we know it well, and handing AI your credit card is weird enough without extra variables.
We chose NBA Top Shot as our testing ground. Why a dying market? Because we know it well, and handing AI your credit card is weird enough without extra variables.