The next generation of AI won't just be innovative—it'll be resilient.

Access the benchmark and full findings: mlcommons.org/ailuminate/j...

Join the conversation!
6/6
#AIRiskandReliability #AISecurity

Why this matters:
→ Developers get standardized metrics to find and fix vulnerabilities
→ Policymakers get transparent, reproducible data
→ Users get systems they can actually trust

We're making hidden risks visible and measurable.
5/6

The Jailbreak Benchmark v0.5 tests AI resilience across:
-Text-to-text scenarios
-Multimodal scenarios
-12 hazard categories (violent crimes, CBRNE, child exploitation, suicide/self-harm, and more)

Built on our AILuminate safety benchmark methodology.
4/6

What is jailbreaking?

It's when users manipulate AI systems to bypass safety filters and produce harmful, unintended, or policy-violating content.

It's not theoretical. It's happening now.
3/6

The gap between AI safety and security is real—and dangerous.

89% of models showed degraded safety performance when exposed to common jailbreak techniques.

As AI powers healthcare, finance, and critical infrastructure, this vulnerability can't be ignored.
2/6

Nebius, NVIDIA, Oracle, Quanta Cloud Technology, Red Hat, Supermicro, TheStage.AI, University of Florida, Vultr

Results:
Datacenter: mlcommons.org/benchmarks/i...
Edge: mlcommons.org/benchmarks/i...
#MLPerf

Llama 2 70B shows remarkable progress - best systems now 5x faster than v4.0.
Thanks to all submitters AMD, Amitash Nanda, ASUSTeK, Broadcom, Cisco, CoreWeave, Dell, GATEOverflow, GigaComputing, Google, HPE, Intel, KRAI, Lambda, Lenovo, MangoBoost, Microsoft Azure, MiTAC,

6/6
Congrats to all contributors and working group members for advancing industry benchmarking! #MLPerf #Automotive #ADAS #AutonomousVehicles #AI #Cognata #Motional #NVIDIA #AVCC #MLCommons

5/6
The results are designed to help OEMs, suppliers, and the whole ecosystem make informed decisions for next-generation, safety-critical automotive AI systems. See results: mlcommons.org/benchmarks/mlperf-automotive/

4/6
MLPerf Automotive v0.5 covers 2D object recognition & segmentation and 3D object recognition using high-res datasets from Cognata (8-megapixel imagery) and Motional (nuScenes).

3/6
Special thanks to submitters GateOverflow and NVIDIA, and dataset partners Cognata_Ltd and Motional for making these benchmarks possible.

2/6
This milestone is powered by collaboration across Ambarella, ARM, Bosch, C-Tuning Foundation, CeCaS, Cognata, Motional, NVIDIA, Qualcomm, Red Hat, Samsung, Siemens EDA, UC Davis, and ZF Group.

Simplyblock, TTA, UBIX, IBM, WDC, and YanRong.
Check the results here:
mlcommons.org/benchmarks/s...

#MLPerf #AI #Storage #Benchmarking #MachineLearning #MLCommons

5/ Congratulations and thanks to all submitters!
Alluxio, Argonne National Lab, DDN, ExponTech, FarmGPU, H3C, Hammerspace, HPE, JNIST/Huawei, Juicedata, Kingston, KIOXIA, Lightbits Labs, MangoBoost, Micron, Nutanix, Oracle, Quanta Cloud Technology, Samsung, Sandisk,