Valentin Klotzbücher
@valentink.bsky.social
Postdoc @uni-freiburg.de | https://valentink.quarto.pub/
Reposted by Valentin Klotzbücher
At least for academics, we often have a good comparison point, which is human Research Assistants, whose failure rates are… not always zero.
An LLM doesn’t have to be perfect to be useful to provide comparable work. Freeing up RAs to more interesting things than mini lit reviews or hand coding data
An LLM doesn’t have to be perfect to be useful to provide comparable work. Freeing up RAs to more interesting things than mini lit reviews or hand coding data
Yes! I recommend validating LLMs against a task you're actually interested in rather than trying to guess whether these failure modes indicate that it will or won't be capable.
So much this. Letter counting and state names are gimmicks, not primary use cases. Frontier models are already remarkably capable for many quite difficult tasks despite still, of course, failing in important ways. We should be able to recognize this fact without uncritically accepting industry hype.
August 9, 2025 at 4:37 PM
At least for academics, we often have a good comparison point, which is human Research Assistants, whose failure rates are… not always zero.
An LLM doesn’t have to be perfect to be useful to provide comparable work. Freeing up RAs to more interesting things than mini lit reviews or hand coding data
An LLM doesn’t have to be perfect to be useful to provide comparable work. Freeing up RAs to more interesting things than mini lit reviews or hand coding data
Reposted by Valentin Klotzbücher
![Comic. [block quote] “Far better an approximate answer to the *right* question, which is often vague, than an *exact* answer to the wrong question, which can always be made precise.” -John W. Tukey, The Future of Data Analysis (1962) [caption] Happy Approximate Birthday to John Tukey, author of my favorite statistics quote, who was born 110.000 years ago sometime this week.](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:cz73r7iyiqn26upot4jtjdhk/bafkreid4vie3df7ltmh7capumhhqcpbejyr7ya7y762fasdeq46ud3gyva@jpeg)
Reposted by Valentin Klotzbücher
Zimbabwe grandmas giving 45 minutes of therapy– among the most effective ways to boost wellbeing, says HLI meta-analysis/review/CEA.
bit.ly/44cvnh0 Both evaluators had ~confidence in main results. Also critiques & suggestions; authors responded in detail. ->
bit.ly/44cvnh0 Both evaluators had ~confidence in main results. Also critiques & suggestions; authors responded in detail. ->
June 20, 2025 at 8:43 PM
Zimbabwe grandmas giving 45 minutes of therapy– among the most effective ways to boost wellbeing, says HLI meta-analysis/review/CEA.
bit.ly/44cvnh0 Both evaluators had ~confidence in main results. Also critiques & suggestions; authors responded in detail. ->
bit.ly/44cvnh0 Both evaluators had ~confidence in main results. Also critiques & suggestions; authors responded in detail. ->
Reposted by Valentin Klotzbücher
This is false and due to a bad math error in an academic paper that you can spot by just asking AI.
Prompting: “carefully check the math in this paper” when this paper came out (so this info was not yet in training data), o1 got it in a single shot. Worth using it to doublecheck claims.
Prompting: “carefully check the math in this paper” when this paper came out (so this info was not yet in training data), o1 got it in a single shot. Worth using it to doublecheck claims.
April 26, 2025 at 5:33 AM
This is false and due to a bad math error in an academic paper that you can spot by just asking AI.
Prompting: “carefully check the math in this paper” when this paper came out (so this info was not yet in training data), o1 got it in a single shot. Worth using it to doublecheck claims.
Prompting: “carefully check the math in this paper” when this paper came out (so this info was not yet in training data), o1 got it in a single shot. Worth using it to doublecheck claims.
Reposted by Valentin Klotzbücher
After a long wait, the working paper for the Many-Economists Project: The Sources of Researcher Variation in Economics. We had 146 teams perform the same research three times, each time with less freedom. What source of freedom leads to different choices and results? papers.ssrn.com/sol3/papers....
The Sources of Researcher Variation in Economics
We use a rigorous three-stage many-analysts design to assess how different researcher decisions—specifically data cleaning, research design, and the interpretat
papers.ssrn.com
February 25, 2025 at 7:17 PM
After a long wait, the working paper for the Many-Economists Project: The Sources of Researcher Variation in Economics. We had 146 teams perform the same research three times, each time with less freedom. What source of freedom leads to different choices and results? papers.ssrn.com/sol3/papers....
Reposted by Valentin Klotzbücher
Listening to headphones on the massive drugdealing goldfinch
February 22, 2025 at 6:59 AM
Listening to headphones on the massive drugdealing goldfinch
Reposted by Valentin Klotzbücher
I used AI to help write a paper and got it published. All in record time. What does this mean for research? My latest post. open.substack.com/pub/joshuaga...
What will AI do to (p)research?
AI makes doing and communicating research much easier. Will there be any point to it?
open.substack.com
February 5, 2025 at 2:24 AM
I used AI to help write a paper and got it published. All in record time. What does this mean for research? My latest post. open.substack.com/pub/joshuaga...
Reposted by Valentin Klotzbücher
When my colleague Sue-Lin first told me about this podcast idea, and how the global scam industry works, I thought she was joking. It is a lot darker and scarier than I thought.
Highly recommended.
www.economist.com/audio/podcas...
Highly recommended.
www.economist.com/audio/podcas...
Scam Inc. from Economist Podcasts+
Uncover a predatory, multi-billion-dollar industry emerging from the shadows.
www.economist.com
February 3, 2025 at 1:45 PM
When my colleague Sue-Lin first told me about this podcast idea, and how the global scam industry works, I thought she was joking. It is a lot darker and scarier than I thought.
Highly recommended.
www.economist.com/audio/podcas...
Highly recommended.
www.economist.com/audio/podcas...
Reposted by Valentin Klotzbücher
New research alert! Our study investigates the effectiveness of human-only, AI-assisted, and AI-led teams in assessing the reproducibility of quantitative social science research. We've got some surprising findings!
January 22, 2025 at 2:23 AM
New research alert! Our study investigates the effectiveness of human-only, AI-assisted, and AI-led teams in assessing the reproducibility of quantitative social science research. We've got some surprising findings!
Reposted by Valentin Klotzbücher
Now accepting applications for "Unjournal Research Affiliates" (URAs).
Help:
- Improve how research is evaluated.
- Prioritize work with potential global impact.
- Promote open, robust research in quant. social science & economics.
More info & application link: https://bit.ly/3WFInZJ
Help:
- Improve how research is evaluated.
- Prioritize work with potential global impact.
- Promote open, robust research in quant. social science & economics.
More info & application link: https://bit.ly/3WFInZJ
Organizational roles and responsibilities | The Unjournal: project and communication space
More information on The Unjournal's roles and how to apply
bit.ly
January 13, 2025 at 9:45 PM
Now accepting applications for "Unjournal Research Affiliates" (URAs).
Help:
- Improve how research is evaluated.
- Prioritize work with potential global impact.
- Promote open, robust research in quant. social science & economics.
More info & application link: https://bit.ly/3WFInZJ
Help:
- Improve how research is evaluated.
- Prioritize work with potential global impact.
- Promote open, robust research in quant. social science & economics.
More info & application link: https://bit.ly/3WFInZJ
Reposted by Valentin Klotzbücher
"Bar is raised because gravity is lower" was a fun sentence to write
January 13, 2025 at 9:36 PM
"Bar is raised because gravity is lower" was a fun sentence to write
Reposted by Valentin Klotzbücher
One could say that Berlin is a city perpetually at war with itself, its identity, its grasp of history, its language. And once a year, this spritual war manifests in the streets. Ancient Chinese weapons, from a more civilized age, are deployed. The goal is only chaos. www.youtube.com/watch?v=16sR...
FIREWORK MADNESS & CHAOS in BERLIN 2025 NEW YEAR EVE
YouTube video by Off The Rails
www.youtube.com
January 1, 2025 at 8:22 AM
One could say that Berlin is a city perpetually at war with itself, its identity, its grasp of history, its language. And once a year, this spritual war manifests in the streets. Ancient Chinese weapons, from a more civilized age, are deployed. The goal is only chaos. www.youtube.com/watch?v=16sR...
Reposted by Valentin Klotzbücher
https://bit.ly/3ZXgOx1 #researchhub
#Unjournal.org agrees: pay people for work, even peer reviewers.
We're committed to paying $450 in average compensation per evaluator, including prizes.
#450movement.
Consider joining our evaluator pool: https://bit.ly/3ZVcYEl
#Unjournal.org agrees: pay people for work, even peer reviewers.
We're committed to paying $450 in average compensation per evaluator, including prizes.
#450movement.
Consider joining our evaluator pool: https://bit.ly/3ZVcYEl
‘Getting paid to review is justice’: journal pays peer reviewers in cryptocurrency
ResearchHub Journal launches latest attempt to compensate referees for their labour.
bit.ly
December 16, 2024 at 9:10 PM
https://bit.ly/3ZXgOx1 #researchhub
#Unjournal.org agrees: pay people for work, even peer reviewers.
We're committed to paying $450 in average compensation per evaluator, including prizes.
#450movement.
Consider joining our evaluator pool: https://bit.ly/3ZVcYEl
#Unjournal.org agrees: pay people for work, even peer reviewers.
We're committed to paying $450 in average compensation per evaluator, including prizes.
#450movement.
Consider joining our evaluator pool: https://bit.ly/3ZVcYEl
Reposted by Valentin Klotzbücher
Once again reminded of @rmcelreath.bsky.social's "Science as Amateur Software Development" (www.youtube.com/watch?v=8qzV...). I know that time is a finite resource, but better coding would definitely be worth it -- it's a genuinely useful skill, outside of academia as well.
December 12, 2024 at 8:15 AM
Once again reminded of @rmcelreath.bsky.social's "Science as Amateur Software Development" (www.youtube.com/watch?v=8qzV...). I know that time is a finite resource, but better coding would definitely be worth it -- it's a genuinely useful skill, outside of academia as well.
Reposted by Valentin Klotzbücher
Barricades, beatings and an interview with Georgia's president who - for now - is refusing to step down. Our latest report from Tbilisi for BBC News. Camera: Anton Chicherov Producer @bentavener.bsky.social
Tbilisi, Georgia - our report on a third night of protests
YouTube video by Steve Rosenberg
youtu.be
December 1, 2024 at 4:23 AM
Barricades, beatings and an interview with Georgia's president who - for now - is refusing to step down. Our latest report from Tbilisi for BBC News. Camera: Anton Chicherov Producer @bentavener.bsky.social
Reposted by Valentin Klotzbücher
Time for a reminder.
November 20, 2024 at 1:19 AM
Time for a reminder.