Don’t miss #MLCommons Endpoints in San Diego, Dec 1–2!
Learn, connect, and shape the future of AI with top experts at Qualcomm Hall.
🗓 Dec 1–2 | 🎟 Free tickets available now!

www.eventbrite.com/e/mlcommons-...

#AI #MachineLearning #SanDiego

🚨 NEW: We tested 39 AI models for security vulnerabilities.

Not a single one was as secure as it was "safe."

Today, we're releasing the industry's first standardized jailbreak benchmark. Here's what we found 🧵1/6

mlcommons.org/2025/10/ailu...

MLPerf Inference v5.1 results are live!
Record 27 organizations submitted 1,472 performance results across new and established AI workloads.
Three new benchmarks debut:

Reasoning with Deepseek R1
Speech to text with Whisper
Small LLM with Llama 3.1 8B

Read More: mlcommons.org/2025/09/mlpe...

MLCommons just launched MLPerf Mobile on the Google Play Store! 📱
Benchmark your Android device’s AI performance on real-world ML tasks with this free, open-source app.
Try it now: play.google.com/store/apps/d...

Call for Submissions!

#MLCommons & @AVCConsortium are accepting submissions for the #MLPerf Automotive Benchmark Suite! Help drive fair comparisons & optimize AI systems in vehicles. Focus is on camera sensor perception.

📅 Submissions close June 13th, 2025

Join: mlcommons.org/community/su...

MLCommons is partnering with Nasscom to develop globally recognized AI reliability benchmarks, including India-specific, Hindi-language evaluations. Together, we are advancing trustworthy AI.
🔗 mlcommons.org/2025/05/nass...

#AIForAll #IndiaAI #ResponsibleAI #Nasscom #MLCommons

1/ MLCommons announces the release of MLPerf Client v0.6, the first open benchmark to support NPU and GPU acceleration on consumer AI PCs.
Read more: mlcommons.org/2025/04/mlpe...

MLCommons introduced a new Automotive PointPainting benchmark in its recent MLPerf Inference v5.0 release. The benchmark is based on established industry methods such as PointPainting, DeepLab3+, and the Waymo Open Dataset. For more details, visit mlcommons.org/2025/03/auto....

1/ MLPerf Inference benchmark v5.0 includes a new GNN benchmark providing a standardized way to assess the performance of GNN implementations and hardware accelerators.

1/ We are excited to announce the latest MLCommons MLPerf Inference v5.0 benchmark suite results. This round featured robust participation from 23 submitting organizations delivering over 17,000 performance results!
mlcommons.org/2025/04/mlperf-inference-v5-0-results/

MedPerf integrates smart contracts & private data objects to improve accountability & integrity.
#MedPerf is an open framework for benchmarking medical AI using real-world private datasets to ensure transparency and privacy.
mlcommons.org/2025/03/medp...

#MedicalAI #smartcontracts

Submissions are open for the MLPerf Training v5.0 Benchmark.

We are excited to add a new pretraining benchmark, llama3.1 405B, to showcase the latest innovations in AI.

To participate, join the Working Group
mlcommons.org/working-grou...

Croissant, a metadata format for ML-ready datasets, has gained traction since its launch in March 2024. Major platforms like Kaggle and HuggingFace now support it.

The MLCommons Croissant working group co-chairs shared insights on its rapid adoption and future plans.
mlcommons.org/2025/02/croi...

MLCommons, in partnership with the AI Verify Foundation, released the AILuminate v1.1, incorporating new French language capabilities into its first-of-its-kind AI safety benchmark.

Learn more: mlcommons.org/2025/02/ailu...

#ailuminate #parisaiactionsummit #aiverifyfoundation

@mlcommons.org releases Unsupervised People's Speech dataset:
1M+ hours of multilingual audio
821K+ hours of detected speech
89 languages
48+ TB of data

Empowering research in:
✅ Speech recognition
✅ Language ID
✅ Global communication tech

Learn more: mlcommons.org/2025/01/new-...

#nlp #datasets

Announcing the release of AILuminate, a first-of-its kind benchmark to measure the safety of LLMs. The AILuminate v1.0 benchmark offers a comprehensive set of safety grades for today's most prevalent #LLMs.
mlcommons.org/2024/12/mlco...
(1/4)