Issue with tool calling
#1
by
bambusek
- opened
Hello, I get template error when I try to tool call some MCP server or any other tools. I tried different templates, but none works, maybe it’s issue with quantisation so tool calling doesn’t work?
Thanks in advance for help.
I would try using a different prompt template. I used the standard one, try the one from Unsloth.
Also, for the similar quant from Air I hear that Cline works great with tools on this model.
If you have the RAM, try also the qx53gx, that has 5 bit embeddings, and a bit higher fidelity. There is no other difference between this quant and the one that works, but I will double-check the template and if it's different I will fix it :)
bambusek
changed discussion status to
closed