Gift article: www.theatlantic.com/politics/arc...
Gift article: www.theatlantic.com/politics/arc...
I downloaded the Q4_K_L quantization (as usual) from Bartowski (good past experience) and this is the first time I experienced a broken model/quantization:
$ llm 'Why did the chicken cross the road?'
of * if! (true,,)
#llm
I downloaded the Q4_K_L quantization (as usual) from Bartowski (good past experience) and this is the first time I experienced a broken model/quantization:
$ llm 'Why did the chicken cross the road?'
of * if! (true,,)
#llm
🗓️ Thursday 17th 11:30 AM EDT
🎯 A chill livestream unpacking LLM #Quantization: #vllm vs #ollama. Learn about the What & How.
🔥Dope guest stars:
#bartowski from arcee.ai & Eldar Kurtic from #RedHat
🔗Stream on YouTube & Linkedin:
www.youtube.com/watch?v=XTE0...
🗓️ Thursday 17th 11:30 AM EDT
🎯 A chill livestream unpacking LLM #Quantization: #vllm vs #ollama. Learn about the What & How.
🔥Dope guest stars:
#bartowski from arcee.ai & Eldar Kurtic from #RedHat
🔗Stream on YouTube & Linkedin:
www.youtube.com/watch?v=XTE0...
"Chuck Bartowski becomes a Superhero.."
"Chuck Bartowski becomes a Superhero.."
huggingface.co/bartowski/o...
huggingface.co/bartowski/o...
llama-server.exe -hf bartowski/google_gemma-3-12b-it-qat-GGUF -c 40960
Then, access 127.0.0.1
Since it is a long text, the context length is increased with -c
llama-server.exe -hf bartowski/google_gemma-3-12b-it-qat-GGUF -c 40960
Then, access 127.0.0.1
Since it is a long text, the context length is increased with -c
Click "Use this model", select 👋 Jan.
Thanks to Hugging Face folks, GG's llama.cpp and the quant crew; Bartowski, Unsloth, and others.
Click "Use this model", select 👋 Jan.
Thanks to Hugging Face folks, GG's llama.cpp and the quant crew; Bartowski, Unsloth, and others.
Big thanks to Bartowski for jumping on it early and releasing the GGUF version: huggingface.co/bartowski/M...
Big thanks to Bartowski for jumping on it early and releasing the GGUF version: huggingface.co/bartowski/M...
Thanks to Meta for the Llama 3.2B base, Unsloth for AutoDidact (the framework we built on), and Bartowski for quantizing the model!
Thanks to Meta for the Llama 3.2B base, Unsloth for AutoDidact (the framework we built on), and Bartowski for quantizing the model!
It enables full-document, multi-document, or long-reasoning tasks with no chunking.
To run it locally: click "Use this model" on Hugging Face and select Jan: huggingface.co/bartowski/n...
It enables full-document, multi-document, or long-reasoning tasks with no chunking.
To run it locally: click "Use this model" on Hugging Face and select Jan: huggingface.co/bartowski/n...
Shows 50%+ improvement over the base R1-Distill-1.5B, achieving 37.91% pass@1 on GPQA-Diamond.
To run it locally: click "Use this model"on @huggingface and select Jan. huggingface.co/bartowski/Z...
Shows 50%+ improvement over the base R1-Distill-1.5B, achieving 37.91% pass@1 on GPQA-Diamond.
To run it locally: click "Use this model"on @huggingface and select Jan. huggingface.co/bartowski/Z...
Achieves 66.1% on GPQA Diamond vs 65.2% for DeepSeek-R1-Llama-70B.
To run it locally, click Use this model on Hugging Face and select Jan: huggingface.co/bartowski/L...
Achieves 66.1% on GPQA Diamond vs 65.2% for DeepSeek-R1-Llama-70B.
To run it locally, click Use this model on Hugging Face and select Jan: huggingface.co/bartowski/L...
Fine-tuned via SFT on competitive coding (Codeforces). Thanks @ben_burtenshaw!
To run it locally, click Use this model on @huggingface and select Jan: huggingface.co/bartowski/b...
Fine-tuned via SFT on competitive coding (Codeforces). Thanks @ben_burtenshaw!
To run it locally, click Use this model on @huggingface and select Jan: huggingface.co/bartowski/b...
Connect Bluesky
Enter your Bluesky handle and app password to unlock posting, likes, and your Following feed.
Need an app password? Open Bluesky, go to Settings > App passwords, and create a new one.
Connect with Bluesky
Sign in with your Bluesky account to unlock posting, likes, and your Following feed.