fix: function calling formatting in chat template
The updated one tested on llama.cpp doesn't use all tools properly, Gemini fixed it a bit: https://pastebin.com/raw/hnPGq0ht. Works better somehow?
The updated one tested on llama.cpp doesn't use all tools properly, Gemini fixed it a bit: https://pastebin.com/raw/hnPGq0ht. Works better somehow?
Error rendering prompt with jinja template: "Unknown test: sequence".
The updated one tested on llama.cpp doesn't use all tools properly, Gemini fixed it a bit: https://pastebin.com/raw/hnPGq0ht. Works better somehow?
Error rendering prompt with jinja template: "Unknown test: sequence".
maybe you are using lm studio, try: https://pastebin.com/raw/qc1FTAcG
When an assistant message contains content and tool_calls, the generated template ends with <turn|>\n but doesn't add <|turn>model\n even if add_generation_prompt = true.
Example ran on https://huggingface.co/spaces/huggingfacejs/chat-template-playground?modelId=google%2Fgemma-4-26B-A4B-it
Parameters
{
"messages": [
{
"role": "user",
"content": "What's the weather like in Austin today?"
},
{
"role": "assistant",
"content": "Let me call the weather tool",
"reasoning_content": "The user is asking about the weather in Austin today. I should check the available tools to see if there's a way to get weather information. The `get_weather` tool seems appropriate for this task. It requires a `location` parameter. The user provided \"Austin\", which I can use. I'll assume the default unit is Fahrenheit unless specified otherwise, which is fine.\n\nPlan:\n1. Call `get_weather` with `location='Austin, TX'`.\n2. Present the result to the user.",
"tool_calls": [
{
"type": "function",
"function": {
"name": "get_weather",
"arguments": "{\"location\":\"Austin, TX\"}"
},
"id": "6wBvNMNHDg2cBAgr9Ob1N9i3tM2algb5"
}
]
},
{
"role": "tool",
"content": "It is 78F and sunny",
"tool_call_id": "6wBvNMNHDg2cBAgr9Ob1N9i3tM2algb5"
}
],
"add_generation_prompt": true
}
<|turn>user
What's the weather like in Austin today?<turn|>
<|turn>model
<|channel>thought
The user is asking about the weather in Austin today. I should check the available tools to see if there's a way to get weather information. The `get_weather` tool seems appropriate for this task. It requires a `location` parameter. The user provided "Austin", which I can use. I'll assume the default unit is Fahrenheit unless specified otherwise, which is fine.
Plan:
1. Call `get_weather` with `location='Austin, TX'`.
2. Present the result to the user.
<channel|><|tool_call>call:get_weather{{"location":"Austin, TX"}}<tool_call|><|tool_response>response:get_weather{value:<|"|>It is 78F and sunny<|"|>}<tool_response|>Let me call the weather tool<turn|>
What should happen here? Should the content be considered reasoning so it retains the agentic nature? Should <|turn>model\n be appended to signal the model's turn? At the moment, this generates weird behavior on the 26B variant because it starts to generate:
thought\n<channel|>
Instead of:
<|channel>thought\n<channel|>