Jacob Rosenthal
@jacobrosenth.al
AI + medicine. MD-PhD student at Weill Cornell / Cornell Tech.
Interests: computer vision, LLMs, clinical trials, medical AI governance, radiology, python, chess, running, birds, cortados
Interests: computer vision, LLMs, clinical trials, medical AI governance, radiology, python, chess, running, birds, cortados
Have been reading about Test-Time Training after seeing it in the ARC prize technical report. Very interesting ideas and tied with some things I have been thinking about recently re: AI in medicine. AI models deployed in Clinical trials should continue learning at test-time (i.e. in the real world)!
December 9, 2024 at 7:09 PM
Have been reading about Test-Time Training after seeing it in the ARC prize technical report. Very interesting ideas and tied with some things I have been thinking about recently re: AI in medicine. AI models deployed in Clinical trials should continue learning at test-time (i.e. in the real world)!
What benchmarks should we be using to evaluate medical AI models? Answer: all benchmarks have limitations. Need to instead use real-world metrics that reflect what actually matters to patients. Huge opportunity for AI models fully integrated into EHR.
Ayers et al, JAMA 2024
Ayers et al, JAMA 2024
December 8, 2024 at 1:16 AM
What benchmarks should we be using to evaluate medical AI models? Answer: all benchmarks have limitations. Need to instead use real-world metrics that reflect what actually matters to patients. Huge opportunity for AI models fully integrated into EHR.
Ayers et al, JAMA 2024
Ayers et al, JAMA 2024
Tech Debt in ML Systems (Sculley et al, NeurIPS 2015)
Takeaways:
1. SYSTEMS are what matter most, not individual models
2. Training new models is not the only way to conduct research and/or have impact
3. Models are useless w/o infrastructure for data ingestion, serving predictions, and monitoring
Takeaways:
1. SYSTEMS are what matter most, not individual models
2. Training new models is not the only way to conduct research and/or have impact
3. Models are useless w/o infrastructure for data ingestion, serving predictions, and monitoring
December 7, 2024 at 4:48 PM
Tech Debt in ML Systems (Sculley et al, NeurIPS 2015)
Takeaways:
1. SYSTEMS are what matter most, not individual models
2. Training new models is not the only way to conduct research and/or have impact
3. Models are useless w/o infrastructure for data ingestion, serving predictions, and monitoring
Takeaways:
1. SYSTEMS are what matter most, not individual models
2. Training new models is not the only way to conduct research and/or have impact
3. Models are useless w/o infrastructure for data ingestion, serving predictions, and monitoring
I love this city 🗽
November 11, 2024 at 12:04 AM
I love this city 🗽