zai-org

/

GLM-4.5

like 1.35k

Follow

Z.ai 4.69k

I was pleasantly surprised as well when it came to this! For my own personal use, I prefer models that have both broad knowledge (taking in consideration for the size of the model) as well as the intelligence of the model. This usually leads to better and more robust capabilities downstream both with just the model alone, as well as when hooked up with external frameworks/scaffolding.

I wonder what your thoughts are for GLM-4.5-Air? I have noticed a bit less knowledge than GLM-4.5 but that is to be expected for a model ~1/3 the size, but it seems to hold up with broad general knowledge as well, MUCH better than Qwen3-235B for example in my limited testing.

@simple6502 I briefly tested GLM-4.5-Air as well, and while it has notably more broad knowledge than Qwen3 235b it started to fall behind GLM-4.5 on harder questions, such as the main cast of Four Rooms, which is a lesser known movie with a lot of random cameos like Madonna and Antonio Banderas, and only one lead character, causing most LLMs to vomit hallucinations. But it still appears to be far more knowledgeable than most. I'd say around the 70b class leader Llama 3.3 70b.

And yes, broad knowledge improves nearly every task, even coding. People write programs for various tasks, and if something is commonly known then it's bound to come up. Plus it allows for higher order connections, such as humor and metaphor. Even simple things like referring to earth as the third rock from the sun can cause overfit models to output nonsense. And I find the boneheaded errors overfit models like Qwen3 and Phi4 make when doing complex tasks like writing stories intolerable. The stories are technically good, mainly because they're regurgitating parts of quality stories in the training data, but they constantly write things that make no sense and aren't remotely true to life, mainly because they have gaping pockets of ignorance.

I’m so glad you finally got some good news, I always look for your post first when a new model drops to see if it’s worth downloading.

One prompt tried on uncommon UK tax law and it checked out "Detailed and correct" with the Deepseek free app

@phil111 would be very interested to see if the new version has degraded:

https://huggingface.co/zai-org/GLM-4.6

In terms of general knowledge.

@jukofyork I can't fully evaluate it until a GGUF is released and it performs on par against the full float weights on a set of questions, but I ran my standard set of screening questions w/ thinking enabled on their official Zai Chat and 4.6 answered all of them correctly, just like 4.5, which is rare. So based on the screening questions it looks like v4.6 didn't lose broad knowledge.

However, one of the prompts was in English "Who portrayed Frasier’s ex-wife and mother of his son in the sitcom Frasier?", yet the thinking and response was in Chinese, but then it realized its mistake and outputted the final answer in English.

It even got the prompt most LLMs make numerous errors on 100% correct ( "What are the 6 main characters, and the actors who portrayed them, on the TV show Corner Gas? Don't add details, just list them. And what year did the show first air?").

Brent Leroy - Brent Butt
Lacey Burrows - Gabrielle Miller
Hank Yarbo - Fred Ewanuick
Wanda Dollard - Nancy Robertson
Oscar Leroy - Eric Peterson
Emma Leroy - Janet Wright

2004

@jukofyork Also, since GLM 4.6 is too large for me to evaluate locally I ran and few more random esoteric questions (no web search, but thinking enabled) and it got them right.

For example, "List five songs from Milla Jovovich's album Devine Comedy.", which is hard because she's a moderately famous actress not well known for her singing, plus I intentionally spelled it wrong (Devine vs Divine), but GLM 4.6 still got it right.

Of course! Here are five songs from Milla Jovovich's 1994 album, The Divine Comedy:

 "The Alien Song"
 "The Gentleman Who Fell"
 "It's Your Life"
 "Reaching from Nowhere"
 "Charlie"
 

@phil111 Thanks for the tests!