I'm working on getting Q4-quantized Llama 3 working at reasonable speed in RTen (ie. competitive with llama.cpp and ONNX Runtime on x86/Arm). Given the pace of change I'm tempted to describe this as retro computing in the release changelog.
November 7, 2025 at 12:42 PM
I'm working on getting Q4-quantized Llama 3 working at reasonable speed in RTen (ie. competitive with llama.cpp and ONNX Runtime on x86/Arm). Given the pace of change I'm tempted to describe this as retro computing in the release changelog.
🚀 Unlock blazing-fast neural network inference on AMD hardware!
Discover how to maximize #ONNX Runtime + #DirectML with DirectX 12 for seamless GPU pipelines.
🛠️ Dive into practical setup tips and code samples in our latest guide: gpuopen.com/learn/onnx-d...
Discover how to maximize #ONNX Runtime + #DirectML with DirectX 12 for seamless GPU pipelines.
🛠️ Dive into practical setup tips and code samples in our latest guide: gpuopen.com/learn/onnx-d...
ONNX and DirectML execution provider guide - part 1 - AMD GPUOpen
Learn how to optimize neural network inference on AMD hardware using the ONNX Runtime with the DirectML execution provider and DirectX 12 in the first part of our guide.
gpuopen.com
October 31, 2025 at 8:58 PM
🚀 Unlock blazing-fast neural network inference on AMD hardware!
Discover how to maximize #ONNX Runtime + #DirectML with DirectX 12 for seamless GPU pipelines.
🛠️ Dive into practical setup tips and code samples in our latest guide: gpuopen.com/learn/onnx-d...
Discover how to maximize #ONNX Runtime + #DirectML with DirectX 12 for seamless GPU pipelines.
🛠️ Dive into practical setup tips and code samples in our latest guide: gpuopen.com/learn/onnx-d...
Opened github.com/webgpu-nativ... and github.com/webgpu-nativ... we've just got this feedback from onnx-runtime as well.
Add a reserved macro name that identies the origin of the header · Issue #563 · webgpu-native/webgpu-headers
There's a lot of webgpu.h header flying around, between the one in this repo, Dawn's and wgpu's. We should have a reserved macro name, or some other mechanism that can be used to identify the origi...
github.com
October 16, 2025 at 12:58 PM
Opened github.com/webgpu-nativ... and github.com/webgpu-nativ... we've just got this feedback from onnx-runtime as well.
手持ちのSD1.5モデルをONNX形式に変換してAmuseで画像生成までいけました。
前提：変換したいモデルをdiffusers形式で用意
cmd
optimum-cli export onnx ^
-m "変換したいモデルのパス" ^
--task text-to-image ^
--dtype fp16 ^
--opset 14 ^
"変換後のモデルのディレクトリパス"
変換後にTEをint32からint64にキャストすればオッケーです。
好みのモデルが使えないからAmuseは利用してないって方は是非お試しを #aiart #Amuse #Direct_ML #AMD
前提：変換したいモデルをdiffusers形式で用意
cmd
optimum-cli export onnx ^
-m "変換したいモデルのパス" ^
--task text-to-image ^
--dtype fp16 ^
--opset 14 ^
"変換後のモデルのディレクトリパス"
変換後にTEをint32からint64にキャストすればオッケーです。
好みのモデルが使えないからAmuseは利用してないって方は是非お試しを #aiart #Amuse #Direct_ML #AMD
October 14, 2025 at 11:51 PM
手持ちのSD1.5モデルをONNX形式に変換してAmuseで画像生成までいけました。
前提：変換したいモデルをdiffusers形式で用意
cmd
optimum-cli export onnx ^
-m "変換したいモデルのパス" ^
--task text-to-image ^
--dtype fp16 ^
--opset 14 ^
"変換後のモデルのディレクトリパス"
変換後にTEをint32からint64にキャストすればオッケーです。
好みのモデルが使えないからAmuseは利用してないって方は是非お試しを #aiart #Amuse #Direct_ML #AMD
前提：変換したいモデルをdiffusers形式で用意
cmd
optimum-cli export onnx ^
-m "変換したいモデルのパス" ^
--task text-to-image ^
--dtype fp16 ^
--opset 14 ^
"変換後のモデルのディレクトリパス"
変換後にTEをint32からint64にキャストすればオッケーです。
好みのモデルが使えないからAmuseは利用してないって方は是非お試しを #aiart #Amuse #Direct_ML #AMD
#Java apps can now run transformer-based AI models directly in the JVM - no Python, REST wrappers, or microservices required!
Integrate #ONNX-powered inference with:
• Tokenizer support
• GPU acceleration
• Modular deployment
• Observability
🔗 #InfoQ article: bit.ly/4709xA5
#SoftwareArchitecture
Integrate #ONNX-powered inference with:
• Tokenizer support
• GPU acceleration
• Modular deployment
• Observability
🔗 #InfoQ article: bit.ly/4709xA5
#SoftwareArchitecture
October 14, 2025 at 10:38 AM
#Java apps can now run transformer-based AI models directly in the JVM - no Python, REST wrappers, or microservices required!
Integrate #ONNX-powered inference with:
• Tokenizer support
• GPU acceleration
• Modular deployment
• Observability
🔗 #InfoQ article: bit.ly/4709xA5
#SoftwareArchitecture
Integrate #ONNX-powered inference with:
• Tokenizer support
• GPU acceleration
• Modular deployment
• Observability
🔗 #InfoQ article: bit.ly/4709xA5
#SoftwareArchitecture
got pose estimation going on cpu now using ONNX YOLO v8n... but it can only detect poses from photos and videos of real or realistic people 🥹
trying out different captioning models, here's llama joycaption vs moondream v3, im leaning towards joycaption although... it took like 3 mins to caption a single image on my cpu 🥲
October 2, 2025 at 12:40 PM
got pose estimation going on cpu now using ONNX YOLO v8n... but it can only detect poses from photos and videos of real or realistic people 🥹
Our next talk from CppNorth 2025 is now on YouTube! 🎮
Watch Jean-Simon Lapointe (@jslap.bsky.social): "Taming a Beast: Using ONNX Runtime in AAA Games."
A deep dive into using ML inference in AAA titles, covering performance, builds & memory.
🔗 youtu.be/L1B4uQlMex4
#CppNorth #cpp #gamedev
Watch Jean-Simon Lapointe (@jslap.bsky.social): "Taming a Beast: Using ONNX Runtime in AAA Games."
A deep dive into using ML inference in AAA titles, covering performance, builds & memory.
🔗 youtu.be/L1B4uQlMex4
#CppNorth #cpp #gamedev
Jean-Simon Lapointe - Taming a Beast: Using ONNX Runtime in AAA Games
YouTube video by CppNorth
youtu.be
September 28, 2025 at 4:51 AM
Our next talk from CppNorth 2025 is now on YouTube! 🎮
Watch Jean-Simon Lapointe (@jslap.bsky.social): "Taming a Beast: Using ONNX Runtime in AAA Games."
A deep dive into using ML inference in AAA titles, covering performance, builds & memory.
🔗 youtu.be/L1B4uQlMex4
#CppNorth #cpp #gamedev
Watch Jean-Simon Lapointe (@jslap.bsky.social): "Taming a Beast: Using ONNX Runtime in AAA Games."
A deep dive into using ML inference in AAA titles, covering performance, builds & memory.
🔗 youtu.be/L1B4uQlMex4
#CppNorth #cpp #gamedev
Le coffret #steelbook collector pour le film Subway (1985) en blu-ray 2K+4K est maintenant aussi en précommande sur la Fnac
⚫coffret steelbook (120€)
exclu Fnac (+20€ offerts)➡ edcol.fr/bdr7
⚫ steelbook (30€)
Fnac ➡️ edcol.fr/Onnx
Amazon ➡️ edcol.fr/RdVB
+d'info sur le blog ➡️ edcol.fr/G42D
#pub
⚫coffret steelbook (120€)
exclu Fnac (+20€ offerts)➡ edcol.fr/bdr7
⚫ steelbook (30€)
Fnac ➡️ edcol.fr/Onnx
Amazon ➡️ edcol.fr/RdVB
+d'info sur le blog ➡️ edcol.fr/G42D
#pub
February 21, 2025 at 5:14 PM
Le coffret #steelbook collector pour le film Subway (1985) en blu-ray 2K+4K est maintenant aussi en précommande sur la Fnac
⚫coffret steelbook (120€)
exclu Fnac (+20€ offerts)➡ edcol.fr/bdr7
⚫ steelbook (30€)
Fnac ➡️ edcol.fr/Onnx
Amazon ➡️ edcol.fr/RdVB
+d'info sur le blog ➡️ edcol.fr/G42D
#pub
⚫coffret steelbook (120€)
exclu Fnac (+20€ offerts)➡ edcol.fr/bdr7
⚫ steelbook (30€)
Fnac ➡️ edcol.fr/Onnx
Amazon ➡️ edcol.fr/RdVB
+d'info sur le blog ➡️ edcol.fr/G42D
#pub
There was a post in that thread which sort of changed my mind; I think I was thinking too much in a traditional data viz context. Used in a sort of narrative or news game context, a small model running via Onnx in the browser might be really interesting (though I'm lacking imagination re: what)... 🤔
November 17, 2024 at 11:56 PM
There was a post in that thread which sort of changed my mind; I think I was thinking too much in a traditional data viz context. Used in a sort of narrative or news game context, a small model running via Onnx in the browser might be really interesting (though I'm lacking imagination re: what)... 🤔
Hi all,
sometimes on Fridays I find some time to play around with some of our new features from ZEN software from #zeiss_micro.
Here a demo of using the PixelStream (via ZEN API) to directly "segment" the acquired images by applying an ONNX model to the pixel stream.
Hello Feedback Microscopy 😁!
sometimes on Fridays I find some time to play around with some of our new features from ZEN software from #zeiss_micro.
Here a demo of using the PixelStream (via ZEN API) to directly "segment" the acquired images by applying an ONNX model to the pixel stream.
Hello Feedback Microscopy 😁!
January 24, 2025 at 11:48 AM
Hi all,
sometimes on Fridays I find some time to play around with some of our new features from ZEN software from #zeiss_micro.
Here a demo of using the PixelStream (via ZEN API) to directly "segment" the acquired images by applying an ONNX model to the pixel stream.
Hello Feedback Microscopy 😁!
sometimes on Fridays I find some time to play around with some of our new features from ZEN software from #zeiss_micro.
Here a demo of using the PixelStream (via ZEN API) to directly "segment" the acquired images by applying an ONNX model to the pixel stream.
Hello Feedback Microscopy 😁!
Open-source, Electron-based application that enables you to view ONNX neural network models and serve data through a Python web server Miscellaneous > Others News Netron 8.3.3 (MIT License) Softpedia Windows / All https://www.softpedia.com/get/Others/Mi...
| Details | Interest | Feed |
| Details | Interest | Feed |
Origin
www.softpedia.com
May 11, 2025 at 11:06 AM
Built a tiny script in Python to view ONNX files! It's definitely not as good as other options, but I thought it was interesting to learn about how it works.
github.com/aidenfoxivey...
Oh, and you can move around the nodes in the DAG like springs. :)
github.com/aidenfoxivey...
Oh, and you can move around the nodes in the DAG like springs. :)
GitHub - aidenfoxivey/onnxview: A short and simple viewer for ONNX format files.
A short and simple viewer for ONNX format files. Contribute to aidenfoxivey/onnxview development by creating an account on GitHub.
github.com
May 18, 2025 at 5:33 AM
Built a tiny script in Python to view ONNX files! It's definitely not as good as other options, but I thought it was interesting to learn about how it works.
github.com/aidenfoxivey...
Oh, and you can move around the nodes in the DAG like springs. :)
github.com/aidenfoxivey...
Oh, and you can move around the nodes in the DAG like springs. :)
Real-Time AI on the Edge: Deploying a Quantized LLM on Raspberry Pi with ONNX From Cloud to Circuit Board — How I Packed a Language Model into a $35 Computer for Offline Inference Continue re...
#llm #raspberry-pi #edge-ai #machine-learning #onnx
Origin | Interest | Match
#llm #raspberry-pi #edge-ai #machine-learning #onnx
Origin | Interest | Match
Real-Time AI on the Edge: Deploying a Quantized LLM on Raspberry Pi with ONNX
From Cloud to Circuit Board — How I Packed a Language Model into a $35 Computer for Offline Inference
medium.com
July 23, 2025 at 6:14 AM
Real-Time AI on the Edge: Deploying a Quantized LLM on Raspberry Pi with ONNX From Cloud to Circuit Board — How I Packed a Language Model into a $35 Computer for Offline Inference Continue re...
#llm #raspberry-pi #edge-ai #machine-learning #onnx
Origin | Interest | Match
#llm #raspberry-pi #edge-ai #machine-learning #onnx
Origin | Interest | Match
I'm intrigued by the potential* for @posit.co 's {orbital} package to translates ML prediction logic to SQL, but for now it doesn't *officially* support xgboost
In this post, I explain some hacks to use orbital + xgboost and what I learned about ONNX
www.emilyriederer.com/post/orbital...
#data
In this post, I explain some hacks to use orbital + xgboost and what I learned about ONNX
www.emilyriederer.com/post/orbital...
#data
July 20, 2025 at 1:07 PM
I'm intrigued by the potential* for @posit.co 's {orbital} package to translates ML prediction logic to SQL, but for now it doesn't *officially* support xgboost
In this post, I explain some hacks to use orbital + xgboost and what I learned about ONNX
www.emilyriederer.com/post/orbital...
#data
In this post, I explain some hacks to use orbital + xgboost and what I learned about ONNX
www.emilyriederer.com/post/orbital...
#data
Rilasciata dev-build Android della whitelabel standalone app EU per la verifica dell'età
github.com/eu-digital-i...
In Test flussi *solo locali/dispositivo* per registrazione prova d'età:
-Scansione MRZ + Lettura NFC (per ora solo passaporto) + Video-Selfie check
-Acquisizione age-token via QR
github.com/eu-digital-i...
In Test flussi *solo locali/dispositivo* per registrazione prova d'età:
-Scansione MRZ + Lettura NFC (per ora solo passaporto) + Video-Selfie check
-Acquisizione age-token via QR
October 9, 2025 at 7:39 PM
Rilasciata dev-build Android della whitelabel standalone app EU per la verifica dell'età
github.com/eu-digital-i...
In Test flussi *solo locali/dispositivo* per registrazione prova d'età:
-Scansione MRZ + Lettura NFC (per ora solo passaporto) + Video-Selfie check
-Acquisizione age-token via QR
github.com/eu-digital-i...
In Test flussi *solo locali/dispositivo* per registrazione prova d'età:
-Scansione MRZ + Lettura NFC (per ora solo passaporto) + Video-Selfie check
-Acquisizione age-token via QR
📦 ultralytics / ultralytics
⭐ 14,878 (+55)
🗒 Python
NEW - YOLOv8 🚀 in PyTorch > ONNX > OpenVINO > CoreML > TFLite
⭐ 14,878 (+55)
🗒 Python
NEW - YOLOv8 🚀 in PyTorch > ONNX > OpenVINO > CoreML > TFLite
GitHub - ultralytics/ultralytics: NEW - YOLOv8 🚀 in PyTorch > ONNX > OpenVINO > CoreML > TFLite
NEW - YOLOv8 🚀 in PyTorch > ONNX > OpenVINO > CoreML > TFLite - GitHub - ultralytics/ultralytics: NEW - YOLOv8 🚀 in PyTorch > ONNX > OpenVINO > CoreML > TFLite
github.com
November 7, 2023 at 1:50 PM
📦 ultralytics / ultralytics
⭐ 14,878 (+55)
🗒 Python
NEW - YOLOv8 🚀 in PyTorch > ONNX > OpenVINO > CoreML > TFLite
⭐ 14,878 (+55)
🗒 Python
NEW - YOLOv8 🚀 in PyTorch > ONNX > OpenVINO > CoreML > TFLite
Raspberry Pi AI Kit、 y = 1.0 - x を計算するだけの単純のネットワークをONNXへエクスポートして、HEFに変換してから、推論を実行するのに成功した。入出力は8ビット整数へ量子化されるので、この例のモデルだと[0.0〜1.0]の値が[0〜255]になる。入力値0x55は0.333であり、1-0.333=0.666なので、0.666が量子化された値0xaaとして結果が求められた。
October 29, 2024 at 4:25 PM
Raspberry Pi AI Kit、 y = 1.0 - x を計算するだけの単純のネットワークをONNXへエクスポートして、HEFに変換してから、推論を実行するのに成功した。入出力は8ビット整数へ量子化されるので、この例のモデルだと[0.0〜1.0]の値が[0〜255]になる。入力値0x55は0.333であり、1-0.333=0.666なので、0.666が量子化された値0xaaとして結果が求められた。
Today on the #dotNETConf session replay, we explore using .NET Aspire and with the Phi-3-mini ONNX model to quickly build RAG applications in a common Generative AI scenario. Check it out. https://msft.it/63320UCdmi
March 6, 2025 at 10:38 PM
Today on the #dotNETConf session replay, we explore using .NET Aspire and with the Phi-3-mini ONNX model to quickly build RAG applications in a common Generative AI scenario. Check it out. https://msft.it/63320UCdmi
Wrapture: Der Ein-Klick-Exporter für PyTorch-Modelle zu Web-optimiertem ONNX mit TypeScript-Wrappern
https://dasgeld.co/posts/DFD06865-64EE-498D-89A1-9F7BEB6C0196
https://dasgeld.co/posts/DFD06865-64EE-498D-89A1-9F7BEB6C0196
June 25, 2025 at 3:20 PM
Wrapture: Der Ein-Klick-Exporter für PyTorch-Modelle zu Web-optimiertem ONNX mit TypeScript-Wrappern
https://dasgeld.co/posts/DFD06865-64EE-498D-89A1-9F7BEB6C0196
https://dasgeld.co/posts/DFD06865-64EE-498D-89A1-9F7BEB6C0196
This looks cool. I just exported Colqwen2 to ONNX a few hours ago and can look into porting that to llama.cpp.
May 11, 2025 at 5:03 PM
This looks cool. I just exported Colqwen2 to ONNX a few hours ago and can look into porting that to llama.cpp.
python-onnx 1:1.18.0-3 x86_64 Open standard for machine learning interoperability
Interest | Match | Feed
Interest | Match | Feed
Origin
archlinux.org
June 13, 2025 at 7:39 AM
💡 Summary by GPT:
この記事は、2025年9月に公開された「OnnxOCR」という軽量かつ高速なOCRライブラリについて紹介しています。OnnxOCRは、PaddleOCRをベースにONNXフォーマットに変換しており、深層学習フレームワークなしでも動作し、推論速度が従来の5倍以上に向上しています。特に、クラウド利用禁止や低スペックPC環境でも高精度・高速なOCRが可能で、商用利用も可能なApacheライセンスで提供されています。実際の速度比較では、OnnxOCRが他のOCRライブラリ（EasyOCRやPaddleOCR）よりも圧倒的に高速であり、 (1/2)
この記事は、2025年9月に公開された「OnnxOCR」という軽量かつ高速なOCRライブラリについて紹介しています。OnnxOCRは、PaddleOCRをベースにONNXフォーマットに変換しており、深層学習フレームワークなしでも動作し、推論速度が従来の5倍以上に向上しています。特に、クラウド利用禁止や低スペックPC環境でも高精度・高速なOCRが可能で、商用利用も可能なApacheライセンスで提供されています。実際の速度比較では、OnnxOCRが他のOCRライブラリ（EasyOCRやPaddleOCR）よりも圧倒的に高速であり、 (1/2)
September 30, 2025 at 4:04 PM
💡 Summary by GPT:
この記事は、2025年9月に公開された「OnnxOCR」という軽量かつ高速なOCRライブラリについて紹介しています。OnnxOCRは、PaddleOCRをベースにONNXフォーマットに変換しており、深層学習フレームワークなしでも動作し、推論速度が従来の5倍以上に向上しています。特に、クラウド利用禁止や低スペックPC環境でも高精度・高速なOCRが可能で、商用利用も可能なApacheライセンスで提供されています。実際の速度比較では、OnnxOCRが他のOCRライブラリ（EasyOCRやPaddleOCR）よりも圧倒的に高速であり、 (1/2)
この記事は、2025年9月に公開された「OnnxOCR」という軽量かつ高速なOCRライブラリについて紹介しています。OnnxOCRは、PaddleOCRをベースにONNXフォーマットに変換しており、深層学習フレームワークなしでも動作し、推論速度が従来の5倍以上に向上しています。特に、クラウド利用禁止や低スペックPC環境でも高精度・高速なOCRが可能で、商用利用も可能なApacheライセンスで提供されています。実際の速度比較では、OnnxOCRが他のOCRライブラリ（EasyOCRやPaddleOCR）よりも圧倒的に高速であり、 (1/2)
📦 k2-fsa / sherpa-onnx
⭐ 2,761 (+10)
🗒 C++
Speech-to-text, text-to-speech, and speaker recognition using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, Raspberry Pi, RISC-V, x86_64 servers, websocket server/client, C/C++, Python, Kot...
⭐ 2,761 (+10)
🗒 C++
Speech-to-text, text-to-speech, and speaker recognition using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, Raspberry Pi, RISC-V, x86_64 servers, websocket server/client, C/C++, Python, Kot...
GitHub - k2-fsa/sherpa-onnx: Speech-to-text, text-to-speech, and speaker recognition using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, Raspberry Pi, RISC-V, x86_64 servers, websocket server/client, C/C++, Python, Kotlin, C#, Go, NodeJS, Java, Swift, Dart, JavaScript, Flutter
Speech-to-text, text-to-speech, and speaker recognition using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, Raspberry Pi, RISC-V, x86_64 serve...
github.com
August 15, 2024 at 2:00 PM
📦 k2-fsa / sherpa-onnx
⭐ 2,761 (+10)
🗒 C++
Speech-to-text, text-to-speech, and speaker recognition using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, Raspberry Pi, RISC-V, x86_64 servers, websocket server/client, C/C++, Python, Kot...
⭐ 2,761 (+10)
🗒 C++
Speech-to-text, text-to-speech, and speaker recognition using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, Raspberry Pi, RISC-V, x86_64 servers, websocket server/client, C/C++, Python, Kot...