threads.net/@herval
banner
herval.bsky.social
threads.net/@herval
@herval.bsky.social
Just another Geriatric Millenial
agora ele tem uma oportunidade de virar influencer de turismo com milhoes de followers
February 16, 2025 at 6:23 PM
Sharing the code as well, in case anyone has suggestions on how to make this work!

gist.github.com/herval/e341d...
Function calling comparison
Function calling comparison. GitHub Gist: instantly share code, notes, and snippets.
gist.github.com
February 14, 2025 at 1:20 AM
I actually tried the same thing with multiple frameworks (crewai, langchain & dspy), and the results were similar with them too (some of the frameworks, like crewai, try to inject chain-of-thoughts to make the models get out of their loop, but it doesn't seem to work)
February 14, 2025 at 1:20 AM
The results:
- Claude got it right every time
- GPT got the response wrong
- tinyllama couldn't figure out how to call the function at all
- mistral, qwen2.5, llama3.3 and deepseek-r1 hallucinated functions that don't exist
- llama3.2 get in an infinite loop, calling the same function forever
February 14, 2025 at 1:20 AM
I'm using litellm since it helps calling ollama + remote models with the same interface. It also automatically adds instructions around function calling to the system prompt.
February 14, 2025 at 1:20 AM
Here's a fun benchmark showing the performance of 2 online models and 5 lines of local models, when trying to use function calling.

The test is simple:
- there's a function for listing files in a directory
- the question is simply how many files exist in the current folder + its parent
February 14, 2025 at 1:20 AM
É preguiça demais..
February 1, 2025 at 2:34 PM
bets sao o novo PIB do Brazil
January 10, 2025 at 8:47 PM
Nao necessariamente. Muitas vezes é só contrarianismo mesmo. E convenhamos, tem crença que é mais facil de aceitar que outras - homeopatia é bem menos absurdo que o velhinho de barba branca que mora no céu
December 28, 2024 at 4:30 AM
This dude was never an entrepreneur in any capacity, shape or form…
December 28, 2024 at 4:26 AM
Ateismo não é garantia de inteligência
December 28, 2024 at 4:18 AM
Please post more here, I still get to twitter from time to time just to read a select few posts. It’s like sticking your hand in a cesspool to fish for a nugget of gold
December 22, 2024 at 9:59 PM
Summarization and text edits on iOS work pretty well
November 29, 2024 at 1:18 AM
Destruição presente, tem engarrafamento ate na calçada, nao cabe mais gente na cidade
November 23, 2024 at 10:59 PM
Come try Clio! would love to hear what you think 👉 https://clio.so/maker
April 26, 2023 at 9:59 AM