Ali Siahkoohi
@siahkoohi.bsky.social
Pinned
Ali Siahkoohi
@siahkoohi.bsky.social
· Nov 21
A statistical approach to model evaluations
A research paper from Anthropic on how to apply statistics to improve language model evaluations
www.anthropic.com
Great to see Anthropic highlighting the importance of uncertainty quantification (UQ) in AI.
Having spent my PhD and postdoc developing scalable UQ methods for real-world, high-dimensional problems, I’m currently on the academic job market—feel free to reach out!
Having spent my PhD and postdoc developing scalable UQ methods for real-world, high-dimensional problems, I’m currently on the academic job market—feel free to reach out!
Reposted by Ali Siahkoohi
The review debate has entered 🦋. Let me re-propose my favorite idea so far: We reduce everything to a single point of entry with rolling review that only checks for scientific correctness (like TMLR). Once a paper is published there, you can apply for it to be presented at an upcoming conference.
We need to abolish peer-review conferences, at the very minimum. Everyone knows it’s a charade, yet they still keep playing the game.
I'm old enough to remember when mailing in a review and disappearing was all that was expected. 🙃
We keep expecting more from reviewers, but somehow the results seem to be getting worse. 🤔
We keep expecting more from reviewers, but somehow the results seem to be getting worse. 🤔
November 24, 2024 at 9:58 AM
The review debate has entered 🦋. Let me re-propose my favorite idea so far: We reduce everything to a single point of entry with rolling review that only checks for scientific correctness (like TMLR). Once a paper is published there, you can apply for it to be presented at an upcoming conference.
Great to see Anthropic highlighting the importance of uncertainty quantification (UQ) in AI.
Having spent my PhD and postdoc developing scalable UQ methods for real-world, high-dimensional problems, I’m currently on the academic job market—feel free to reach out!
Having spent my PhD and postdoc developing scalable UQ methods for real-world, high-dimensional problems, I’m currently on the academic job market—feel free to reach out!
A statistical approach to model evaluations
A research paper from Anthropic on how to apply statistics to improve language model evaluations
www.anthropic.com
November 21, 2024 at 8:04 PM
Great to see Anthropic highlighting the importance of uncertainty quantification (UQ) in AI.
Having spent my PhD and postdoc developing scalable UQ methods for real-world, high-dimensional problems, I’m currently on the academic job market—feel free to reach out!
Having spent my PhD and postdoc developing scalable UQ methods for real-world, high-dimensional problems, I’m currently on the academic job market—feel free to reach out!