Ajeet Singh Raina
@ajeetraina.bsky.social
👣 Follow me for Docker 🐳 Kubernetes, Cloud-Native, LLM and GenAI stuffs | Developer Advocate at Docker | @Collabnix | Distinguished Arm Ambassador
During this active 5-minute window, llama.cpp's KV cache functionality is fully operational, reusing cached prompt tokens automatically across requests with similar prefixes.
October 23, 2025 at 4:42 PM
During this active 5-minute window, llama.cpp's KV cache functionality is fully operational, reusing cached prompt tokens automatically across requests with similar prefixes.
Docker Model Runner uses llama.cpp as the inference engine running as a native host process, loading the requested model on demand and performing inference on received requests Docker. Models are loaded into memory on demand and unloaded after 5 minutes of inactivity.
October 23, 2025 at 4:42 PM
Docker Model Runner uses llama.cpp as the inference engine running as a native host process, loading the requested model on demand and performing inference on received requests Docker. Models are loaded into memory on demand and unloaded after 5 minutes of inactivity.
Yes, Token caching through llama.cpp's KV cache works automatically in Docker Model Runner - no configuration needed!
October 23, 2025 at 4:41 PM
Yes, Token caching through llama.cpp's KV cache works automatically in Docker Model Runner - no configuration needed!
Here’s the complete write-up collabnix.com/running-dock...
Running Docker Desktop on NVIDIA Jetson Orin Nano Super for the first time – Collabnix
I’ve been eyeing the NVIDIA Jetson lineup for ages, and when the Orin Nano Super was released, I knew I had to get my hands on one. After weeks of hunting—and honestly, some desperate emails to NVIDIA...
collabnix.com
April 4, 2025 at 3:13 AM
Here’s the complete write-up collabnix.com/running-dock...