Dagfinn Parnas
@elsewhat.bsky.social
Architecture and emerging technologies.
Soft spot for local llms and multi-agent scenarios
Soft spot for local llms and multi-agent scenarios
Nice, Silkworm next?
February 10, 2025 at 9:57 PM
Nice, Silkworm next?
Yes, the error is on the ollama setup of the model as far as I can see. In some cases ollama issues can also have a root in llama.cpp
December 6, 2024 at 1:05 PM
Yes, the error is on the ollama setup of the model as far as I can see. In some cases ollama issues can also have a root in llama.cpp
Created bug report to ollama now
github.com/ollama/ollam...
github.com/ollama/ollam...
Add stop word <|endoftext|> to qwq models · Issue #7967 · ollama/ollama
What is the issue? The qwq models currently go into an infinite loop. The reasons for this appears that the model outputs <|endoftext|> at the end of its response, but ollama does not handle this a...
github.com
December 6, 2024 at 11:05 AM
Created bug report to ollama now
github.com/ollama/ollam...
github.com/ollama/ollam...
Same thing I saw. Basically Ollama doesn't stop the llm when the model indicates it's done through the <|endoftext|> token.
Fixed for me through the custom model file link to above (which can be imported through ollama create qwk-fix-stop:latest -f qwq-fix-stop-modelfile.md
FROM qwq:latest)
Fixed for me through the custom model file link to above (which can be imported through ollama create qwk-fix-stop:latest -f qwq-fix-stop-modelfile.md
FROM qwq:latest)
December 5, 2024 at 9:32 PM
Same thing I saw. Basically Ollama doesn't stop the llm when the model indicates it's done through the <|endoftext|> token.
Fixed for me through the custom model file link to above (which can be imported through ollama create qwk-fix-stop:latest -f qwq-fix-stop-modelfile.md
FROM qwq:latest)
Fixed for me through the custom model file link to above (which can be imported through ollama create qwk-fix-stop:latest -f qwq-fix-stop-modelfile.md
FROM qwq:latest)
Think the ollama modelfile for qwq is missing a stopword for <|endoftext|>
See of this helps
github.com/elsewhat/adv...
See of this helps
github.com/elsewhat/adv...
github.com
December 5, 2024 at 9:17 PM
Think the ollama modelfile for qwq is missing a stopword for <|endoftext|>
See of this helps
github.com/elsewhat/adv...
See of this helps
github.com/elsewhat/adv...