Congratulations to the Zhipu Team
I wanted to take a moment to congratulate and thank the Zhipu AI team for the fantastic work on GLM-4.6. I don't usually comment publicly, but the work that has gone into training this model and the decision to make it available free to the community deserves special thanks and gratitude.
In my case, I am running an AI pipeline that processes legal agreements on a local bank of RTX 5090 GPUs that I operate from inside my law firm. My pipeline relies on complex semantic LLM text processing, which requires subtle analysis of legal language in English combined with very precise instruction following. My prompts are usually 15K+ words in length, and I am doing 20+ passes over each legal agreement, with most subsequent legal prompt being dependent on earlier prompts. If anything fails in my sequence of AI passes, the final output is not usable for my clients. This is basically Legaltech 2.0, and the boundary between "OK" results and top 5% of attorney work product is very fine.
The only model that has worked for me so far is GPT-OSS 120b with "high" reasoning. I have tried every large open source LLM model that has come out so far, and every new model that is rated as better than GPT-OSS 120b fails in my pipeline. Higher scores in logic or instruction following benchmarks for recent flagship OS models do not stand up in my testing for my specialized use case in English. GLM-4.6 is the first model that shows glimpses of being better than GPT-OSS 120b for my use case (and I found it exceptional in some cases). Unfortunately GLM-4.6 is a bit too large for my local deployment and I can't use it in production currently. I have to run my pipeline locally, on my own local servers, because my clients are highly regulated entities and do not want their legal agreements processed through the cloud (notwithstanding sandboxing and other assurances provided by cloud companies).
I am very hopeful for the upcoming GLM-4.6 Air model! Thank you again for an incredible job with 4.6 and for making it available to us as OS!
MD