big "model distillation" wants you to believe that you can

bring back the hackintosh

Here's an article about using QLoRA on Llama 2 and Mistral using a 3090. Although the tricky thing with R1 is that it's MoE, so I think you'll have to load all 671M params into memory for training. It might be easier to fine-tune one of the Llama-distilled versions.

medium.com/@geronimo7/f...

since i’m just starting to mess around with MoE, the use of dynamic biases is really interesting to me. a super cool intuition!