Scott Bleackley 🇨🇦
@folkstone.bsky.social
Complex Systems, AI, ML,
Social Implications of Emerging Technology
Heliox Podcast: Where Evidence Meets Empathy
@folkstone on twitter in transit
Social Implications of Emerging Technology
Heliox Podcast: Where Evidence Meets Empathy
@folkstone on twitter in transit
🚀 Introducing NSA: A Hardware-Aligned and Natively Trainable Sparse Attention mechanism for ultra-fast long-context training & inference!
Core components of NSA:
• Dynamic hierarchical sparse strategy
• Coarse-grained token compression
• Fine-grained token selection
📖 arxiv.org/abs/2502.11089
Core components of NSA:
• Dynamic hierarchical sparse strategy
• Coarse-grained token compression
• Fine-grained token selection
📖 arxiv.org/abs/2502.11089
February 18, 2025 at 6:24 PM
🚀 Introducing NSA: A Hardware-Aligned and Natively Trainable Sparse Attention mechanism for ultra-fast long-context training & inference!
Core components of NSA:
• Dynamic hierarchical sparse strategy
• Coarse-grained token compression
• Fine-grained token selection
📖 arxiv.org/abs/2502.11089
Core components of NSA:
• Dynamic hierarchical sparse strategy
• Coarse-grained token compression
• Fine-grained token selection
📖 arxiv.org/abs/2502.11089