[BUG] (Some) Tool calls fail with Qwen3-Coder-30B-A3B-Instruct on llama.cpp

[robertjmcintyre]

Problem (one or two sentences)

I started a new task, and tried to run this message:

What version of dotnet is installed?

The request fails the tool call (see the message below in the repro)

However (!), without restarting VSCode, the Roo extension, or my llama-server, if I run some other command that interacts with the model (needs to read a file, or check git history, for example) then in the same task session ask the dotnet version question it works as expected.

I'm testing tool calling with this model (Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL.gguf) which is how I came across it; I just wanted a very simple command to try.

However (!), I can do the same test with GPT-OSS-120B and GLM-4.5-Air (both are unsloth gguf's) and it works completely as expected. So it seems to be something with Qwen3-Coder.

Context (who is affected and when)

Happens to users running qwen3-coder locally when they start a new task and make requests that need to execute_command.

Reproduction steps

1. System information:

VSCode Info:
Version: 1.108.0
Commit: 94e8ae2b28cb5cc932b86e1070569c4463565c37
Date: 2026-01-08T13:53:10.781Z
Electron: 39.2.7
ElectronBuildId: 12953945
Chromium: 142.0.7444.235
Node.js: 22.21.1
V8: 14.2.231.21-electron.0
OS: Linux x64 6.18.3-200.fc43.x86_64

Roo Code Info:
Version: 3.41.0 (0401c25c)

2. Start llama-server loading unsloth Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL.gguf
3. Open a workspace/folder and start a new Roo task
4. Run the request what version of dotnet is installed
5. Get these responses

I'll check what version of .NET is installed on the system using execute_command.

<function=execute_command>
<parameter=command>
dotnet --version
</parameter>
<parameter=cwd>
/home/robert/repos/AgentFramework
</parameter>

And details:

Date/time: 2026-01-16T16:13:39.589Z
Extension version: 3.41.0
Provider: openai (proxy)
Model: /models/unsloth_Qwen3-Coder-30B-A3B-Instruct/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL.gguf

The model provided text/reasoning but did not call any of the required tools. This usually indicates the model misunderstood the task or is having difficulty determining which tool to use. The model has been automatically prompted to retry with proper tool usage.

6. Start a new Roo task and run the request what files have been edited but not committed?
7. Watch the request succeed
8. Try the same thing with other models and watch it not repro

Expected result

The command runs successfully

Actual result

the tool call fails

Variations tried (optional)

Tried different models, and they seem to work as expected.

App Version

Version: 3.41.0 (0401c25)

API Provider (optional)

OpenAI Compatible

Model Used (optional)

Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL.gguf

Roo Code Task Links (optional)

No response

Relevant logs or errors (optional)

[daniel-lxs]

Hey @robertjmcintyre It looks to me like your provider is not setup for the models to use native tools, can you share more information about what you are using?

[roomote]

I've analyzed this issue and here's what I found:

Root Cause Analysis

The Qwen3-Coder model is outputting tool calls in a non-standard text format:

<function=execute_command>
<parameter=command>dotnet --version</parameter>
<parameter=cwd>/home/robert/repos/AgentFramework</parameter>

This format is not recognized by Roo Code's parsers:

• Native tool protocol expects JSON tool_calls chunks from the API
• XML protocol expects <tool_name><param>value</param></tool_name> format

The model is essentially falling back to a text-based tool format (possibly ChatML function calling syntax) instead of using proper native tool calling.

Why It Works After Another Command

Once a successful tool call happens, the conversation history contains examples of proper tool use, which helps guide the model to use the correct format. On the first request, there's no such context.

Questions to Help Diagnose

Before proposing a solution, I'd like to understand a few things:

1. llama-server configuration: Are you using any specific flags for tool calling (e.g., --chat-template, tool-calling templates)? Some models require specific template configurations for tool calling support.

2. API response inspection: Could you check what the actual API response looks like? Specifically, does choices[0].message contain:

   • A tool_calls array (native format), or
   • Just content with the text above (fallback format)

3. Model tool support: Does unsloth's Qwen3-Coder GGUF advertise tool calling support? Some GGUF quantizations may not fully support tool calling.

Proposed Implementation Plan

Depending on the answers, here are potential solutions:

Option A: Add fallback parser for ChatML-style function calls
If the model genuinely doesn't support native tool calling and outputs this text format, we could add a parser to recognize <function=NAME><parameter=PARAM>VALUE</parameter></function> patterns and convert them to tool uses. This would be implemented in the text chunk processing path.

Option B: Improve first-message context
If the issue is specifically about the first message, we could enhance the system prompt or add few-shot examples specifically for models that need more guidance.

Option C: Model-specific configuration guidance
If the issue is llama-server configuration, we could add documentation for properly configuring tool calling with local models.

Please mention @roomote with answers to the questions above or which option you prefer, and I'll proceed with the implementation.

[robertjmcintyre]

Hey @robertjmcintyre It looks to me like your provider is not setup for the models to use native tools, can you share more information about what you are using?

@daniel-lxs I’m running this model in UD-Q8_K_XL:

https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF

Through llama-server running in docker. I believe it the latest released version.

I’m running it with the —jinja option, but if I recall correctly I’ve tried it with and without.

Edit: I have also tried the BF16 version of that model.

@roomote:

1. Just the jinja option, I don’t know of any other requirements
2. How do I get that parameter?
3. yes, it does, and has worked in the recent past

[robertjmcintyre]

Additional info: I set up vLLM and Qwen3-Coder-30B-A3B-Instruct and it didn't have any issues with tool calls (more than usual, I guess).

I'm downloading a lmstudio-community Q8_0 quant to see if that makes a difference running through llama-cpp. I'm set up with that platform right now and don't really want to to to vLLM unless I have to.

I'd like to really get to the bottom of this though, especially why it's that particular request (version of dotnet), so maybe I can give some feedback to someone.

EDIT: This seems to be caused by the use of the inference parameter --repeat-penalty interfering with this one scenario/tool call. If I remove it, all quants that were having issues before work fine.

You can close this, with thanks.