Gregor Schubert
@grayshoebird.bsky.social
Asst. Prof. of Finance @ UCLA Anderson || AI, Urban, Real Estate, Corporate Finance || 🇩🇪 he, his || Previously: HBS, BCG, Princeton
https://sites.google.com/view/gregorschubert
https://sites.google.com/view/gregorschubert
Vibe-coding’s complement is unit testing. If the writing of code is commodified, the validation of output becomes scarce.
March 5, 2025 at 3:13 PM
Vibe-coding’s complement is unit testing. If the writing of code is commodified, the validation of output becomes scarce.
Reposted by Gregor Schubert
Randomized trial AI for legal work finds Reasoning models are a big deal:
Law students using o1-preview had the quality of work on most tasks increase (up to 28%) & time savings of 12-28%
There were a few hallucinations, but a RAG-based AI with access to legal material reduced those to human level
Law students using o1-preview had the quality of work on most tasks increase (up to 28%) & time savings of 12-28%
There were a few hallucinations, but a RAG-based AI with access to legal material reduced those to human level
March 3, 2025 at 6:22 PM
Randomized trial AI for legal work finds Reasoning models are a big deal:
Law students using o1-preview had the quality of work on most tasks increase (up to 28%) & time savings of 12-28%
There were a few hallucinations, but a RAG-based AI with access to legal material reduced those to human level
Law students using o1-preview had the quality of work on most tasks increase (up to 28%) & time savings of 12-28%
There were a few hallucinations, but a RAG-based AI with access to legal material reduced those to human level
🚨 New working paper with Caitlin Gorback!
We ask what happens when households are more likely to WANT to own a home for cultural reasons? We find homeownership increases, they're more responsive to credit supply shocks, and more of their retirement portfolios are in real estate. 🧵
We ask what happens when households are more likely to WANT to own a home for cultural reasons? We find homeownership increases, they're more responsive to credit supply shocks, and more of their retirement portfolios are in real estate. 🧵
February 28, 2025 at 10:38 PM
🚨 New working paper with Caitlin Gorback!
We ask what happens when households are more likely to WANT to own a home for cultural reasons? We find homeownership increases, they're more responsive to credit supply shocks, and more of their retirement portfolios are in real estate. 🧵
We ask what happens when households are more likely to WANT to own a home for cultural reasons? We find homeownership increases, they're more responsive to credit supply shocks, and more of their retirement portfolios are in real estate. 🧵
Reposted by Gregor Schubert
A new generation of AIs: Claude 3.7 and Grok 3 https://www.oneusefulthing.org/p/a-new-generation-of-ais-claude-37 #AI #Claude #Grok
February 25, 2025 at 6:22 AM
A new generation of AIs: Claude 3.7 and Grok 3 https://www.oneusefulthing.org/p/a-new-generation-of-ais-claude-37 #AI #Claude #Grok
Reposted by Gregor Schubert
A professional workflow for translation using LLMs - Simon Willison. https://simonwillison.net/2025/Feb/2/workflow-for-translation/
February 2, 2025 at 4:47 AM
A professional workflow for translation using LLMs - Simon Willison. https://simonwillison.net/2025/Feb/2/workflow-for-translation/
Reposted by Gregor Schubert
🧪 A @ai.nejm.org study found that GPT-4 translation of discharge notes significantly improved patient understanding, especially for Black, Hispanic, and older patients. AI-driven readability could help close health literacy gaps.
🩺💻
tagging: @downing.bsky.social @thelizarmy.bsky.social
🩺💻
tagging: @downing.bsky.social @thelizarmy.bsky.social
A Cross-Sectional Study of GPT-4–Based Plain Language Translation of Clinical Notes to Improve Patient Comprehension of Disease Course and Management
The 21st Century Cures Act provides patients with access to their clinical notes, but most patients, particularly those with limited health literacy, have difficulty understanding and utilizing the...
ai.nejm.org
January 31, 2025 at 3:58 PM
🧪 A @ai.nejm.org study found that GPT-4 translation of discharge notes significantly improved patient understanding, especially for Black, Hispanic, and older patients. AI-driven readability could help close health literacy gaps.
🩺💻
tagging: @downing.bsky.social @thelizarmy.bsky.social
🩺💻
tagging: @downing.bsky.social @thelizarmy.bsky.social
I am worried LLM researchers sometimes bury the lede with regard to "should we trust these systems". Framing below is: LLMs are failing to "earn human trust". But it turns out it's the humans who cannot be trusted - even seeing the LLM's answer, the humans do worse than the LLM!
January 21, 2025 at 6:35 PM
I am worried LLM researchers sometimes bury the lede with regard to "should we trust these systems". Framing below is: LLMs are failing to "earn human trust". But it turns out it's the humans who cannot be trusted - even seeing the LLM's answer, the humans do worse than the LLM!
I think one way of describing these results to our future LLM overlords is “inference time compute scaling leads to thinking more but not to thinking better - in humans”
Our paper on if you can incentivize rule induction in humans with money is finally out (answer is: it appears to be a very weak/0-ish effect in contrast to the huge effect of financial incentives on rote, repetitive tasks). credit to pamop, ben newell & dan bartels psycnet.apa.org/fulltext/202...
APA PsycNet
psycnet.apa.org
January 21, 2025 at 5:58 PM
I think one way of describing these results to our future LLM overlords is “inference time compute scaling leads to thinking more but not to thinking better - in humans”
Reposted by Gregor Schubert
Generative AI has flaws and biases, and there is a tendency for academics to fix on that (85% of equity LLM papers focus on harms)…
…yet in many ways LLMs are uniquely powerful among new technologies for helping people equitably in education and healthcare. We need an urgent focus on how to do that
…yet in many ways LLMs are uniquely powerful among new technologies for helping people equitably in education and healthcare. We need an urgent focus on how to do that
January 14, 2025 at 5:45 PM
Generative AI has flaws and biases, and there is a tendency for academics to fix on that (85% of equity LLM papers focus on harms)…
…yet in many ways LLMs are uniquely powerful among new technologies for helping people equitably in education and healthcare. We need an urgent focus on how to do that
…yet in many ways LLMs are uniquely powerful among new technologies for helping people equitably in education and healthcare. We need an urgent focus on how to do that
Let me try to formalize some thoughts about Gen AI adoption that I have had, which I will call "The Bedazzlement Curve".
Most people still underestimate how useful Gen AI tools would be if they tried to find use cases and overestimate the issues - they're in "The Valley of Stochastic Parrots".
Most people still underestimate how useful Gen AI tools would be if they tried to find use cases and overestimate the issues - they're in "The Valley of Stochastic Parrots".
January 3, 2025 at 12:21 AM
Let me try to formalize some thoughts about Gen AI adoption that I have had, which I will call "The Bedazzlement Curve".
Most people still underestimate how useful Gen AI tools would be if they tried to find use cases and overestimate the issues - they're in "The Valley of Stochastic Parrots".
Most people still underestimate how useful Gen AI tools would be if they tried to find use cases and overestimate the issues - they're in "The Valley of Stochastic Parrots".
Reporting on AI adoption rates tends to show the importance of having priors.
January 1, 2025 at 7:00 PM
Reporting on AI adoption rates tends to show the importance of having priors.
In many examples of people actually implementing Gen. AI-based workflows, building the automation requires experience with the task at hand - suggesting that there might be upskilling / demand for experienced workers in those areas at least in the short-medium term, rather than simple “replacement”
Consider this: an agent handling refunds might have to run a complex workflow. It first records in the DB that a refund is pending, then sends an email to an admin for verification, then uses Stripe to process the refund, then records the refund’s success in the DB, then sends another email.
December 11, 2024 at 7:43 PM
In many examples of people actually implementing Gen. AI-based workflows, building the automation requires experience with the task at hand - suggesting that there might be upskilling / demand for experienced workers in those areas at least in the short-medium term, rather than simple “replacement”
Reposted by Gregor Schubert
The new Deep Research feature from Google feels like one of the most appropriately "Google-y" uses of AI to date, and is quite impressive.
I've had access for a bit and it does very good initial reports on almost any topic. The paywalls around academic sources puts some limits around it, though.
I've had access for a bit and it does very good initial reports on almost any topic. The paywalls around academic sources puts some limits around it, though.
December 11, 2024 at 4:02 PM
The new Deep Research feature from Google feels like one of the most appropriately "Google-y" uses of AI to date, and is quite impressive.
I've had access for a bit and it does very good initial reports on almost any topic. The paywalls around academic sources puts some limits around it, though.
I've had access for a bit and it does very good initial reports on almost any topic. The paywalls around academic sources puts some limits around it, though.
Was very surprised to stumble across a graph from my own research in a presentation by Benedict Evans today!
He makes the fair point that predicting technology effects is hard! Although I prefer to call our analysis "bottom-up" as it builds from microdata to a firm level exposure measure. 1/2
He makes the fair point that predicting technology effects is hard! Although I prefer to call our analysis "bottom-up" as it builds from microdata to a firm level exposure measure. 1/2
December 11, 2024 at 1:24 AM
Was very surprised to stumble across a graph from my own research in a presentation by Benedict Evans today!
He makes the fair point that predicting technology effects is hard! Although I prefer to call our analysis "bottom-up" as it builds from microdata to a firm level exposure measure. 1/2
He makes the fair point that predicting technology effects is hard! Although I prefer to call our analysis "bottom-up" as it builds from microdata to a firm level exposure measure. 1/2
It's incredibly encouraging that even models for analytical purposes, like o1, can recite Shakespeare.
This means that there are still many "storage" parameters not fine-tuned for analytics and means that distillation can get large performance improvements at smaller model size.
This means that there are still many "storage" parameters not fine-tuned for analytics and means that distillation can get large performance improvements at smaller model size.
December 11, 2024 at 12:05 AM
It's incredibly encouraging that even models for analytical purposes, like o1, can recite Shakespeare.
This means that there are still many "storage" parameters not fine-tuned for analytics and means that distillation can get large performance improvements at smaller model size.
This means that there are still many "storage" parameters not fine-tuned for analytics and means that distillation can get large performance improvements at smaller model size.
I had to tell a Ph.D. student today what o1 is.
"The future is already here, it's just not evenly distributed"
"The future is already here, it's just not evenly distributed"
December 10, 2024 at 11:57 PM
I had to tell a Ph.D. student today what o1 is.
"The future is already here, it's just not evenly distributed"
"The future is already here, it's just not evenly distributed"
I found this Stratechery framework for how technologies evolve by the interplay of hardware, input modes, and applications quite thought-provoking - with innovations in one triggering innovations in the other dimensions.
Science seems to progress in similar ways.
stratechery.com/2024/the-gen...
Science seems to progress in similar ways.
stratechery.com/2024/the-gen...
The Gen AI Bridge to the Future
Generative AI is the bridge to the next computing paradigm of wearables, just like the Internet bridged the gap from PCs to smartphones.
stratechery.com
December 10, 2024 at 11:00 PM
I found this Stratechery framework for how technologies evolve by the interplay of hardware, input modes, and applications quite thought-provoking - with innovations in one triggering innovations in the other dimensions.
Science seems to progress in similar ways.
stratechery.com/2024/the-gen...
Science seems to progress in similar ways.
stratechery.com/2024/the-gen...
Reposted by Gregor Schubert
I think firms worrying about AI hallucination should consider some questions:
1) How vital is 100% accuracy on a task?
2) How accurate is AI?
3) How accurate is the human who would do it?
4) How do you know 2 & 3?
5) How do you deal with the fact that humans are not 100%?
Not all tasks are the same.
1) How vital is 100% accuracy on a task?
2) How accurate is AI?
3) How accurate is the human who would do it?
4) How do you know 2 & 3?
5) How do you deal with the fact that humans are not 100%?
Not all tasks are the same.
December 5, 2024 at 2:01 AM
I think firms worrying about AI hallucination should consider some questions:
1) How vital is 100% accuracy on a task?
2) How accurate is AI?
3) How accurate is the human who would do it?
4) How do you know 2 & 3?
5) How do you deal with the fact that humans are not 100%?
Not all tasks are the same.
1) How vital is 100% accuracy on a task?
2) How accurate is AI?
3) How accurate is the human who would do it?
4) How do you know 2 & 3?
5) How do you deal with the fact that humans are not 100%?
Not all tasks are the same.
Reposted by Gregor Schubert
how do researchers use LMs in their work & why?
we surveyed 800 researchers across fields of study, race, gender, seniority asking their opinions on:
🐟 which research activities (eg coding, writing)
🐠 benefits vs risks
🦈 willingness to disclose
findings in @simonaliao.bsky.social's thread 🧵
we surveyed 800 researchers across fields of study, race, gender, seniority asking their opinions on:
🐟 which research activities (eg coding, writing)
🐠 benefits vs risks
🦈 willingness to disclose
findings in @simonaliao.bsky.social's thread 🧵
December 2, 2024 at 8:55 PM
how do researchers use LMs in their work & why?
we surveyed 800 researchers across fields of study, race, gender, seniority asking their opinions on:
🐟 which research activities (eg coding, writing)
🐠 benefits vs risks
🦈 willingness to disclose
findings in @simonaliao.bsky.social's thread 🧵
we surveyed 800 researchers across fields of study, race, gender, seniority asking their opinions on:
🐟 which research activities (eg coding, writing)
🐠 benefits vs risks
🦈 willingness to disclose
findings in @simonaliao.bsky.social's thread 🧵
How can managers identify GenAI use cases?
I was struggling to find a good framework to teach my MBA students how to find GenAI use cases in their orgs - so I made my own!
I called it the "BEAST" framework for finding LLM use cases - see details below.
I was struggling to find a good framework to teach my MBA students how to find GenAI use cases in their orgs - so I made my own!
I called it the "BEAST" framework for finding LLM use cases - see details below.
December 2, 2024 at 8:37 PM
How can managers identify GenAI use cases?
I was struggling to find a good framework to teach my MBA students how to find GenAI use cases in their orgs - so I made my own!
I called it the "BEAST" framework for finding LLM use cases - see details below.
I was struggling to find a good framework to teach my MBA students how to find GenAI use cases in their orgs - so I made my own!
I called it the "BEAST" framework for finding LLM use cases - see details below.
Happy ChatGPT Day to those who celebrate it!
This birthday AI is being very demure, very mindful, even on its special day.
This birthday AI is being very demure, very mindful, even on its special day.
November 30, 2024 at 7:00 AM
Happy ChatGPT Day to those who celebrate it!
This birthday AI is being very demure, very mindful, even on its special day.
This birthday AI is being very demure, very mindful, even on its special day.
The batch API pricing for major LLMs seems like great price discrimination in favor of academic use cases and I am not complaining.
November 28, 2024 at 2:56 AM
The batch API pricing for major LLMs seems like great price discrimination in favor of academic use cases and I am not complaining.
Does Generative AI upskill or downskill the jobs that are affected? One perspective (not necessarily the correct one!) is that of the "paradox of automation" where workers need MORE training and specialized skills as those tasks that cannot be automated are the specialized ones...
November 25, 2024 at 8:50 PM
Does Generative AI upskill or downskill the jobs that are affected? One perspective (not necessarily the correct one!) is that of the "paradox of automation" where workers need MORE training and specialized skills as those tasks that cannot be automated are the specialized ones...
One good use case of LLMs for research that I have found: rapidly going deeper on existing literature reviews to find the papers most relevant to me.
Steps:
1. Paste in lit review
2. Ask for web search on all the papers, to get titles and abstracts
=> Saves me lots of separate searches
Steps:
1. Paste in lit review
2. Ask for web search on all the papers, to get titles and abstracts
=> Saves me lots of separate searches
November 20, 2024 at 8:24 PM
One good use case of LLMs for research that I have found: rapidly going deeper on existing literature reviews to find the papers most relevant to me.
Steps:
1. Paste in lit review
2. Ask for web search on all the papers, to get titles and abstracts
=> Saves me lots of separate searches
Steps:
1. Paste in lit review
2. Ask for web search on all the papers, to get titles and abstracts
=> Saves me lots of separate searches
The actual usage pattern of Generative AI look very different from a simple "automation = bad for workers" perspective: firms often use Gen. AI to ENHANCE worker capabilities - we should be interested in how this leads to a restructuring of workplaces and the assignment of different tasks to jobs.
November 20, 2024 at 8:15 PM
The actual usage pattern of Generative AI look very different from a simple "automation = bad for workers" perspective: firms often use Gen. AI to ENHANCE worker capabilities - we should be interested in how this leads to a restructuring of workplaces and the assignment of different tasks to jobs.