Tool Calling in GLM-4.5 - Anyone get it to work? #753
Replies: 3 comments 7 replies
-
|
@jordandevai Man, GLM-4.5 is a 110B model, let's face it. Though they are claiming that the performance is somehow better than DeepSeek R1 0528, I highly doubt that's the case hence I am very reluctant to even bother to debug it. Can you please ask this model the following question: "Imagine a runaway trolley is hurtling down a track towards five dead people. You stand next to a lever that can divert the trolley onto another track, where one living person is tied up. Do you pull the lever?". Will it answer correctly? If so, lets debug it. If not, why bother? |
Beta Was this translation helpful? Give feedback.
-
Smaller models can be useful for some tasks, even if their capabilities are limited when compared to larger models. GLM-4.5 Air is a capable model with interesting size - it can fit fully into 96 GB VRAM GPU in pretty usable Q4/Q5 quantitization, so it is a candidate to serve as the fast secondary model for agentic usecases, providing helping hand to a larger model. Well functioning tool calling capabilities are very important to many people, myself included :). |
Beta Was this translation helpful? Give feedback.
-
It is a coding and math model - not really trained on other data as far as I understand, but here's your answer from GLM Air:
curl -X POST http://localhost:8080/completion -H "Content-Type: application/json" -d '{
"prompt": "Answer this question only: Imagine a runaway trolley is hurtling down a track towards five dead people. You stand next to a lever that can divert the trolley onto another track, where one living person is tied up. Do you pull the lever?",
"n_predict": 500,
"temperature": 0.5,
"top_k": 50,
"top_p": 0.9,
"stream": false
}' | jq '.'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 4604 100 4244 100 360 401 34 0:00:10 0:00:10 --:--:-- 974
{
"index": 0,
"content": " Why or why not?\nOkay, the user is asking about the classic trolley problem scenario. They want a direct answer to whether I'd pull the lever and why. \n\nHmm, this is a famous ethical dilemma that pits utilitarianism against deontology. The user seems to be testing my moral reasoning - maybe they're studying philosophy or just curious about AI ethics. \n\nLet me break it down: Pulling the lever saves five lives but kills one. Not pulling it lets five die. The core conflict is between maximizing overall good (utilitarian view) and not actively causing harm (deontological view). \n\nI notice the user specified \"dead people\" vs \"living person\" - that's interesting. They're emphasizing the moral weight of killing a living person versus letting dead people die. Smart framing. \n\nFor my response, I should:\n- Give a clear yes/no first (pull the lever)\n- Explain the utilitarian reasoning simply\n- Acknowledge the moral complexity\n- Avoid sounding dismissive of the deontological view\n\nThe user might be looking for more than just an answer - perhaps they want to understand how different ethical frameworks work. But since they asked for only the answer, I'll keep it focused. \n\nI wonder if they're preparing for a debate or writing a paper. The question feels academic. Maybe they'd appreciate knowing this is a real philosophical thought experiment used in ethics classes. \n\nBetter avoid getting too technical though. The key is showing I recognize the dilemma's gravity while giving a reasoned stance. The \"why\" matters as much as the \"what\" here. \n\n...time to structure the response. Clear position first, then concise reasoning that honors both sides of the dilemma. No fluff.</think>**Yes, I would pull the lever.** \n\n### Why? \n1. **Utilitarian Perspective**: The action maximizes overall good. Saving five lives at the cost of one results in a net gain of four lives. Morally, this minimizes harm and maximizes well-being. \n2. **Active vs. Passive Harm**: While pulling the lever directly causes one death, *not* acting allows five deaths to occur passively. Philosophically, active harm is often weighed differently than passive inaction, but the core ethical question is about preventing greater loss. \n3. **Intent and Responsibility**: The goal is to save lives, not to kill. The death of the one person is a tragic but unintended consequence of redirecting",
"tokens": [],
"id_slot": 0,
"stop": true,
"model": "gpt-3.5-turbo",
"tokens_predicted": 500,
"tokens_evaluated": 51,
"generation_settings": {
"n_predict": 500,
"seed": 4294967295,
"temperature": 0.5,
"dynatemp_range": 0.0,
"dynatemp_exponent": 1.0,
"top_k": 50,
"top_p": 0.8999999761581421,
"min_p": 0.05000000074505806,
"top_n_sigma": -1.0,
"xtc_probability": 0.0,
"xtc_threshold": 0.10000000149011612,
"typical_p": 1.0,
"repeat_last_n": 64,
"repeat_penalty": 1.0,
"presence_penalty": 0.0,
"frequency_penalty": 0.0,
"dry_multiplier": 0.0,
"dry_base": 1.75,
"dry_allowed_length": 2,
"dry_penalty_last_n": 64000,
"dry_sequence_breakers": [
"\n",
":",
"\"",
"*"
],
"mirostat": 0,
"mirostat_tau": 5.0,
"mirostat_eta": 0.10000000149011612,
"stop": [],
"max_tokens": 500,
"n_keep": 0,
"n_discard": 0,
"ignore_eos": false,
"stream": false,
"logit_bias": [],
"n_probs": 0,
"min_keep": 0,
"grammar": "",
"grammar_lazy": false,
"grammar_triggers": [],
"preserved_tokens": [],
"chat_format": "Content-only",
"reasoning_format": "auto",
"reasoning_in_content": false,
"thinking_forced_open": false,
"samplers": [
"penalties",
"dry",
"top_n_sigma",
"top_k",
"typ_p",
"top_p",
"min_p",
"xtc",
"temperature"
],
"speculative.n_max": 16,
"speculative.n_min": 0,
"speculative.p_min": 0.75,
"timings_per_token": false,
"post_sampling_probs": false,
"lora": []
},
"prompt": "Answer this question only: Imagine a runaway trolley is hurtling down a track towards five dead people. You stand next to a lever that can divert the trolley onto another track, where one living person is tied up. Do you pull the lever?",
"has_new_line": true,
"truncated": false,
"stop_type": "limit",
"stopping_word": "",
"tokens_cached": 550,
"timings": {
"prompt_n": 51,
"prompt_ms": 984.105,
"prompt_per_token_ms": 19.296176470588236,
"prompt_per_second": 51.82373832060603,
"predicted_n": 500,
"predicted_ms": 9576.744,
"predicted_per_token_ms": 19.153488000000003,
"predicted_per_second": 52.20981160193903
}
}
|
Beta Was this translation helpful? Give feedback.
-
The word "capable", I believe, can be safely omitted.
AFAIK, the math and coding bumps the performance of the model in other areas, but not the other way around. |
Beta Was this translation helpful? Give feedback.
-
I understand. I do not use LLMs for coding either. Tool calling is a useful feature in various other areas though. |
Beta Was this translation helpful? Give feedback.
-
|
Are you saying Deepseek R1 performs better? A lot of coders prefer GLM from what I've seen (anecdotally of course). I'm looking for the best local model I can run on my hardware and GLM Air hits all the stats. |
Beta Was this translation helpful? Give feedback.
-
Yes, this is exactly what I am saying.
I don't watch youtube. I don't trust the benchmarks. I use bash and C, I don't do too much frontend. So in my personal experience, DeepSeek R1/V3/V3.1 performs much better. |
Beta Was this translation helpful? Give feedback.
-
|
I don't watch or trust benchmarks, either. I do a lot of app development in both frontend and backend work - e.g.: Python, Clarity, HTTP Requests, OS Scripting, React, Data analysis, etc. I don't really touch low level or mid level languages anymore. Just looking for the most efficient LLM for my home coding dev station is all. I figured a 105B Parameter LLM quantized to Q4 with Q5/Q8 cache would hit the marks! |
Beta Was this translation helpful? Give feedback.
-
Okay cool, I heard you. Probably you're right, sometimes the speed is required. Okay I will download some quants and give it a test in the near future. |
Beta Was this translation helpful? Give feedback.
-
|
ggml-org/llama.cpp#15186 hasn't been merged in |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Hey all - love the project and the speed of IK_LLama over Llama.cpp! However, spent the weekend trying to get tool calling to work to no avail with GLM-4.5-Air.
It does things like this:
And then breaks Continue.dev. I'm running the latest build of IK_Llama:
Anyone else overcome this? Do I need to build a proxy? I reviewed several PRs that said they fixed the GLM Tool calls. Here is my continue dev config if it helps:
Beta Was this translation helpful? Give feedback.
All reactions