I like to play dumb. Make them pronounce it and explain the definition. Finally, sigh and say; "there sure is a lot of nuance to fucking kids"
I like to play dumb. Make them pronounce it and explain the definition. Finally, sigh and say; "there sure is a lot of nuance to fucking kids"
• Theory: excess risk = √r/√N + r·2⁻²ᵇ σ² → derive optimal bits b* ≈ ½ log₂(rN)
• DialoGPT tests:
8-bit ≈ FP16 for ranks ≤16
4-bit OK to rank 8, breaks after
grad-var ∝ r·2⁻²ᵇ
RoT: higher LoRA rank ⇒ use higher precision. #LoRA #LLM #AI
We tested non-uniform LoRA rank allocation (linear decay, attention-heavy, empirical) on DialoGPT.
Findings:
• Fixed-rank LoRA (r=16) still wins (PPL: 134.0)
• Linear decay is stable but weaker (PPL: 165.1)
• Complex strategies = unstable
🎯 Adaptive LoRA ≠ magic—yet.
We tested non-uniform LoRA rank allocation (linear decay, attention-heavy, empirical) on DialoGPT.
Findings:
• Fixed-rank LoRA (r=16) still wins (PPL: 134.0)
• Linear decay is stable but weaker (PPL: 165.1)
• Complex strategies = unstable
🎯 Adaptive LoRA ≠ magic—yet.
But with GOP leadership, I don’t think that’s true anymore. Just look at what happened with our DNA with Ancestry…
But with GOP leadership, I don’t think that’s true anymore. Just look at what happened with our DNA with Ancestry…
“Be prepared to fight to get the Dems out of the fed government.”
“Be prepared to fight to get the Dems out of the fed government.”
apnews.com/article/trum...
apnews.com/article/trum...