Fully open reproduction of DeepSeek-R1. Contribute to huggingface/open-r1 development by creating an account on GitHub.

25 January 2025 (

Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities. In response, we introduce Humanity's Last Exam, a multi-modal benchmark at the frontier of human knowledge, designed to be the final closed-ended academic benchmark of its kind with broad subject coverage. The dataset consists of 3,000 challenging questions across over a hundred subjects. We publicly release these questions, while maintaining a private test set of held out questions to assess model overfitting.

Roughly 6,000 readers answered our poll, with many declaring that Bluesky was nicer, kinder and less antagonistic to science than X.

There is growing interest in the reproducibility of research and ways to enhance transparency. This study uses an international survey of more than 1900 biomedical researchers to reveal the perceived ...

CDC and other agencies were told to halt public communications, and the NIH grant process appears to be on hold.

Abstract Recent clinical trials on slowing dementia progression have led to renewed focus on finding safer, more effective treatments. One approach to identify plausible candidates is to assess whet...