🔧 Supports W4A8 / W4A16 / W4AX / W8A8 for Mamba1 and Mamba2
🚀 Achieves 4× memory reduction and 3× generation speedup
⚡️ Enables 8B model inference on Orin Nano 8G at 13 tokens/sec
🔥 Outperforms W4A8KV4 Llama3-8B in both speed and quality
Main Link | Techmeme Permalink
Please @standupforscience.bsky.social March 7, links enclosed.
open.substack.com/pub/garymarc...
Please @standupforscience.bsky.social March 7, links enclosed.
open.substack.com/pub/garymarc...
Check it out: cra.org/crn/2025/02/...
#CRA #ComputingResearchNews #ComputingResearch
Check it out: cra.org/crn/2025/02/...
#CRA #ComputingResearchNews #ComputingResearch
cockrell.utexas.edu/news/archive...
cockrell.utexas.edu/news/archive...
Swipe through and tap to learn how these neurons can also keep working for a long time without losing efficiency, a major challenge.
Government workers are 12% more productive when randomly assigned to work from home. They're more efficient where it's quiet.
Most people aren't shirking from home. They're escaping distractions and long commutes.
Government workers are 12% more productive when randomly assigned to work from home. They're more efficient where it's quiet.
Most people aren't shirking from home. They're escaping distractions and long commutes.
People may be impressed by the volume you produce, but impact depends on the value you create.
Success is not about getting more things done. It's about doing more worthwhile things well.
People may be impressed by the volume you produce, but impact depends on the value you create.
Success is not about getting more things done. It's about doing more worthwhile things well.