csabakecskemeti commited on
Commit
49d011e
·
verified ·
1 Parent(s): 90b403e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -783
README.md CHANGED
@@ -1,788 +1,12 @@
1
  ---
2
- license: other
3
- license_name: modified-mit
4
- library_name: transformers
5
  ---
6
- <div align="center">
7
- <picture>
8
- <img src="figures/kimi-logo.png" width="30%" alt="Kimi K2: Open Agentic Intellignece">
9
- </picture>
10
- </div>
11
 
12
- <hr>
13
 
14
- <div align="center" style="line-height:1">
15
- <a href="https://www.kimi.com" target="_blank"><img alt="Chat" src="https://img.shields.io/badge/🤖%20Chat-Kimi%20K2-ff6b6b?color=1783ff&logoColor=white"/></a>
16
- <a href="https://www.moonshot.ai" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Homepage-Moonshot%20AI-white?logo=Kimi&logoColor=white"/></a>
17
- </div>
18
 
19
- <div align="center" style="line-height: 1;">
20
- <a href="https://huggingface.co/moonshotai" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Moonshot%20AI-ffc107?color=ffc107&logoColor=white"/></a>
21
- <a href="https://twitter.com/kimi_moonshot" target="_blank"><img alt="Twitter Follow" src="https://img.shields.io/badge/Twitter-Kimi.ai-white?logo=x&logoColor=white"/></a>
22
- <a href="https://discord.gg/TYU2fdJykW" target="_blank"><img alt="Discord" src="https://img.shields.io/badge/Discord-Kimi.ai-white?logo=discord&logoColor=white"/></a>
23
- </div>
24
-
25
- <div align="center" style="line-height: 1;">
26
- <a href="https://github.com/moonshotai/Kimi-K2/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Modified_MIT-f5de53?&color=f5de53"/></a>
27
- </div>
28
-
29
- <p align="center">
30
- <b>📰&nbsp;&nbsp;<a href="https://moonshotai.github.io/Kimi-K2/">Tech Blog</a></b> &nbsp;&nbsp;&nbsp; | &nbsp;&nbsp;&nbsp; <b>📄&nbsp;&nbsp;Paper Link (comming soon)</b>
31
- </p>
32
-
33
- ## 1. Model Introduction
34
-
35
- Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model with 32 billion activated parameters and 1 trillion total parameters. Trained with the Muon optimizer, Kimi K2 achieves exceptional performance across frontier knowledge, reasoning, and coding tasks while being meticulously optimized for agentic capabilities.
36
-
37
- ### Key Features
38
- - Large-Scale Training: Pre-trained a 1T parameter MoE model on 15.5T tokens with zero training instability.
39
- - MuonClip Optimizer: We apply the Muon optimizer to an unprecedented scale, and develop novel optimization techniques to resolve instabilities while scaling up.
40
- - Agentic Intelligence: Specifically designed for tool use, reasoning, and autonomous problem-solving.
41
-
42
- ### Model Variants
43
- - **Kimi-K2-Base**: The foundation model, a strong start for researchers and builders who want full control for fine-tuning and custom solutions.
44
- - **Kimi-K2-Instruct**: The post-trained model best for drop-in, general-purpose chat and agentic experiences. It is a reflex-grade model without long thinking.
45
-
46
- <div align="center">
47
- <picture>
48
- <img src="figures/banner.png" width="80%" alt="Evaluation Results">
49
- </picture>
50
- </div>
51
-
52
- ## 2. Model Summary
53
-
54
- <div align="center">
55
-
56
-
57
- | | |
58
- |:---:|:---:|
59
- | **Architecture** | Mixture-of-Experts (MoE) |
60
- | **Total Parameters** | 1T |
61
- | **Activated Parameters** | 32B |
62
- | **Number of Layers** (Dense layer included) | 61 |
63
- | **Number of Dense Layers** | 1 |
64
- | **Attention Hidden Dimension** | 7168 |
65
- | **MoE Hidden Dimension** (per Expert) | 2048 |
66
- | **Number of Attention Heads** | 64 |
67
- | **Number of Experts** | 384 |
68
- | **Selected Experts per Token** | 8 |
69
- | **Number of Shared Experts** | 1 |
70
- | **Vocabulary Size** | 160K |
71
- | **Context Length** | 128K |
72
- | **Attention Mechanism** | MLA |
73
- | **Activation Function** | SwiGLU |
74
- </div>
75
-
76
- ## 3. Evaluation Results
77
-
78
- #### Instruction model evaluation results
79
-
80
- <div align="center">
81
- <table>
82
- <thead>
83
- <tr>
84
- <th align="center">Benchmark</th>
85
- <th align="center">Metric</th>
86
- <th align="center"><sup>Kimi K2 Instruct</sup></th>
87
- <th align="center"><sup>DeepSeek-V3-0324</sup></th>
88
- <th align="center"><sup>Qwen3-235B-A22B <br><sup>(non-thinking)</sup></sup></th>
89
- <th align="center"><sup>Claude Sonnet 4 <br><sup>(w/o extended thinking)</sup></sup></th>
90
- <th align="center"><sup>Claude Opus 4 <br><sup>(w/o extended thinking)</sup></sup></th>
91
- <th align="center"><sup>GPT-4.1</sup></th>
92
- <th align="center"><sup>Gemini 2.5 Flash <br> Preview (05-20)</sup></th>
93
- </tr>
94
- </thead>
95
- <tbody>
96
- <tr>
97
- <td align="center" colspan=9><strong>Coding Tasks</strong></td>
98
- </tr>
99
- <tr>
100
- <td align="center">LiveCodeBench v6<br><sup>(Aug 24 - May 25)</sup></td>
101
- <td align="center">Pass@1</td>
102
- <td align="center"><strong>53.7</strong></td>
103
- <td align="center">46.9</td>
104
- <td align="center">37.0</td>
105
- <td align="center">48.5</td>
106
- <td align="center">47.4</td>
107
- <td align="center">44.7</td>
108
- <td align="center">44.7</td>
109
- </tr>
110
- <tr>
111
- <td align="center">OJBench</td>
112
- <td align="center">Pass@1</td>
113
- <td align="center"><strong>27.1</strong></td>
114
- <td align="center">24.0</td>
115
- <td align="center">11.3</td>
116
- <td align="center">15.3</td>
117
- <td align="center">19.6</td>
118
- <td align="center">19.5</td>
119
- <td align="center">19.5</td>
120
- </tr>
121
-
122
- <tr>
123
- <td align="center">MultiPL-E</td>
124
- <td align="center">Pass@1</td>
125
- <td align="center"><ins><strong>85.7</strong></ins></td>
126
- <td align="center">83.1</td>
127
- <td align="center">78.2</td>
128
- <td align="center">88.6</td>
129
- <td align="center"><strong>89.6</strong></td>
130
- <td align="center">86.7</td>
131
- <td align="center">85.6</td>
132
- </tr>
133
-
134
- <tr>
135
- <td align="center">SWE-bench Verified <br/><sup>(Agentless Coding)</sup></td>
136
- <td align="center">Single Patch w/o Test (Acc)</td>
137
- <td align="center"><ins><strong>51.8</strong></ins></td>
138
- <td align="center">36.6</td>
139
- <td align="center">39.4</td>
140
- <td align="center">50.2</td>
141
- <td align="center"><strong>53.0</strong></td>
142
- <td align="center">40.8</td>
143
- <td align="center">32.6</td>
144
- </tr>
145
-
146
- <tr>
147
- <td align="center" rowspan="2">SWE-bench Verified <br/> <sup>(Agentic Coding)</sup></td>
148
- <td align="center">Single Attempt (Acc)</td>
149
- <td align="center"><ins><strong>65.8</strong></ins></td>
150
- <td align="center">38.8</td>
151
- <td align="center">34.4</td>
152
- <td align="center"><strong>72.7</strong><sup>*</sup></td>
153
- <td align="center">72.5<sup>*</sup></td>
154
- <td align="center">54.6</td>
155
- <td align="center">—</td>
156
- </tr>
157
-
158
- <tr>
159
- <!--<td align="center">(Agentic Coding)</td>-->
160
- <td align="center">Multiple Attempts (Acc)</td>
161
- <td align="center"><ins><strong>71.6</strong></ins></td>
162
- <td align="center">—</td>
163
- <td align="center">—</td>
164
- <td align="center"><strong>80.2</strong></td>
165
- <td align="center">79.4<sup>*</sup></td>
166
- <td align="center">—</td>
167
- <td align="center">—</td>
168
- </tr>
169
-
170
- <tr>
171
- <td align="center">SWE-bench Multilingual<br /> <sup>(Agentic Coding)</sup></td>
172
- <td align="center">Single Attempt (Acc)</td>
173
- <td align="center"><ins><strong>47.3</strong> </ins></td>
174
- <td align="center">25.8</td>
175
- <td align="center">20.9</td>
176
- <td align="center"><strong>51.0</strong></td>
177
- <td align="center">—</td>
178
- <td align="center">31.5</td>
179
- <td align="center">—</td>
180
- </tr>
181
-
182
- <tr>
183
- <td align="center" rowspan="2">TerminalBench</td>
184
- <td align="center">Inhouse Framework (Acc)</td>
185
- <td align="center"><ins><strong>30.0</strong></ins></td>
186
- <td align="center">—</td>
187
- <td align="center">—</td>
188
- <td align="center">35.5</td>
189
- <td align="center"><strong>43.2</strong></td>
190
- <td align="center">8.3</td>
191
- <td align="center">—</td>
192
- </tr>
193
-
194
- <tr>
195
- <!--<td align="center">TerminalBench</td>-->
196
- <td align="center">Terminus (Acc)</td>
197
- <td align="center"><ins><strong>25.0</strong> </ins></td>
198
- <td align="center">16.3</td>
199
- <td align="center">6.6</td>
200
- <td align="center">—</td>
201
- <td align="center">—</td>
202
- <td align="center"><strong>30.3</strong></td>
203
- <td align="center">16.8</td>
204
- </tr>
205
- <tr>
206
- <td align="center">Aider-Polyglot</td>
207
- <td align="center">Acc</td>
208
- <td align="center">60.0</td>
209
- <td align="center">55.1</td>
210
- <td align="center"><ins><strong>61.8</strong></ins></td>
211
- <td align="center">56.4</td>
212
- <td align="center"><strong>70.7</strong></td>
213
- <td align="center">52.4</td>
214
- <td align="center">44.0</td>
215
- </tr>
216
- <tr>
217
- <td align="center" colspan=9><strong>Tool Use Tasks</strong></td>
218
- </tr>
219
- <tr>
220
- <td align="center">Tau2 retail</td>
221
- <td align="center">Avg@4</td>
222
- <td align="center"><ins><strong>70.6</strong></ins></td>
223
- <td align="center">69.1</td>
224
- <td align="center">57.0</td>
225
- <td align="center">75.0</td>
226
- <td align="center"><strong>81.8</strong></td>
227
- <td align="center">74.8</td>
228
- <td align="center">64.3</td>
229
- </tr>
230
- <tr>
231
- <td align="center">Tau2 airline</td>
232
- <td align="center">Avg@4</td>
233
- <td align="center"><ins><strong>56.5</strong></ins></td>
234
- <td align="center">39.0</td>
235
- <td align="center">26.5</td>
236
- <td align="center">55.5</td>
237
- <td align="center"><strong>60.0</strong></td>
238
- <td align="center">54.5</td>
239
- <td align="center">42.5</td>
240
- </tr>
241
- <tr>
242
- <td align="center">Tau2 telecom</td>
243
- <td align="center">Avg@4</td>
244
- <td align="center"><strong>65.8</strong></td>
245
- <td align="center">32.5</td>
246
- <td align="center">22.1</td>
247
- <td align="center">45.2</td>
248
- <td align="center">57.0</td>
249
- <td align="center">38.6</td>
250
- <td align="center">16.9</td>
251
- </tr>
252
- <tr>
253
- <td align="center">AceBench</td>
254
- <td align="center">Acc</td>
255
- <td align="center"><ins><strong>76.5</strong></ins></td>
256
- <td align="center">72.7</td>
257
- <td align="center">70.5</td>
258
- <td align="center">76.2</td>
259
- <td align="center">75.6</td>
260
- <td align="center"><strong>80.1</strong></td>
261
- <td align="center">74.5</td>
262
- </tr>
263
- <tr>
264
- <td align="center" colspan=9><strong>Math &amp; STEM Tasks</strong></td>
265
- </tr>
266
- <tr>
267
- <td align="center">AIME 2024</td>
268
- <td align="center">Avg@64</td>
269
- <td align="center"><strong>69.6</strong></td>
270
- <td align="center">59.4<sup>*</sup></td>
271
- <td align="center">40.1<sup>*</sup></td>
272
- <td align="center">43.4</td>
273
- <td align="center">48.2</td>
274
- <td align="center">46.5</td>
275
- <td align="center">61.3</td>
276
- </tr>
277
- <tr>
278
- <td align="center">AIME 2025</td>
279
- <td align="center">Avg@64</td>
280
- <td align="center"><strong>49.5</strong></td>
281
- <td align="center">46.7</td>
282
- <td align="center">24.7<sup>*</sup></td>
283
- <td align="center">33.1<sup>*</sup></td>
284
- <td align="center">33.9<sup>*</sup></td>
285
- <td align="center">37.0</td>
286
- <td align="center">46.6</td>
287
- </tr>
288
- <tr>
289
- <td align="center">MATH-500</td>
290
- <td align="center">Acc</td>
291
- <td align="center"><strong>97.4</strong></td>
292
- <td align="center">94.0<sup>*</sup></td>
293
- <td align="center">91.2<sup>*</sup></td>
294
- <td align="center">94.0</td>
295
- <td align="center">94.4</td>
296
- <td align="center">92.4</td>
297
- <td align="center">95.4</td>
298
- </tr>
299
- <tr>
300
- <td align="center">HMMT 2025</td>
301
- <td align="center">Avg@32</td>
302
- <td align="center"><strong>38.8</strong></td>
303
- <td align="center">27.5</td>
304
- <td align="center">11.9</td>
305
- <td align="center">15.9</td>
306
- <td align="center">15.9</td>
307
- <td align="center">19.4</td>
308
- <td align="center">34.7</td>
309
- </tr>
310
- <tr>
311
- <td align="center">CNMO 2024</td>
312
- <td align="center">Avg@16</td>
313
- <td align="center">74.3</td>
314
- <td align="center"><ins><strong>74.7</strong></ins></td>
315
- <td align="center">48.6</td>
316
- <td align="center">60.4</td>
317
- <td align="center">57.6</td>
318
- <td align="center">56.6</td>
319
- <td align="center"><strong>75.0</strong></td>
320
- </tr>
321
- <tr>
322
- <td align="center">PolyMath-en</td>
323
- <td align="center">Avg@4</td>
324
- <td align="center"><strong>65.1</strong></td>
325
- <td align="center">59.5</td>
326
- <td align="center">51.9</td>
327
- <td align="center">52.8</td>
328
- <td align="center">49.8</td>
329
- <td align="center">54.0</td>
330
- <td align="center">49.9</td>
331
- </tr>
332
-
333
- <tr>
334
- <td align="center">ZebraLogic</td>
335
- <td align="center">Acc</td>
336
- <td align="center"><strong>89.0</strong></td>
337
- <td align="center">84.0</td>
338
- <td align="center">37.7<sup>*</sup></td>
339
- <td align="center">73.7</td>
340
- <td align="center">59.3</td>
341
- <td align="center">58.5</td>
342
- <td align="center">57.9</td>
343
- </tr>
344
-
345
- <tr>
346
- <td align="center">AutoLogi</td>
347
- <td align="center">Acc</td>
348
- <td align="center"><ins><strong>89.5</strong></ins></td>
349
- <td align="center">88.9</td>
350
- <td align="center">83.3</td>
351
- <td align="center"><strong>89.8</strong></td>
352
- <td align="center">86.1</td>
353
- <td align="center">88.2</td>
354
- <td align="center">84.1</td>
355
- </tr>
356
-
357
- <tr>
358
- <td align="center">GPQA-Diamond</td>
359
- <td align="center">Avg@8</td>
360
- <td align="center"><strong>75.1</strong></td>
361
- <td align="center">68.4<sup>*</sup></td>
362
- <td align="center">62.9<sup>*</sup></td>
363
- <td align="center">70.0<sup>*</sup></td>
364
- <td align="center">74.9<sup>*</sup></td>
365
- <td align="center">66.3</td>
366
- <td align="center">68.2</td>
367
- </tr>
368
-
369
- <tr>
370
- <td align="center">SuperGPQA</td>
371
- <td align="center">Acc</td>
372
- <td align="center"><strong>57.2</strong></td>
373
- <td align="center">53.7</td>
374
- <td align="center">50.2</td>
375
- <td align="center">55.7</td>
376
- <td align="center">56.5</td>
377
- <td align="center">50.8</td>
378
- <td align="center">49.6</td>
379
- </tr>
380
-
381
- <tr>
382
- <td align="center">Humanity's Last Exam<br><sup>(Text Only)</sup></td>
383
- <td align="center">-</td>
384
- <td align="center">4.7</td>
385
- <td align="center">5.2</td>
386
- <td align="center"><ins><strong>5.7</strong></ins></td>
387
- <td align="center">5.8</td>
388
- <td align="center"><strong>7.1</strong></td>
389
- <td align="center">3.7</td>
390
- <td align="center">5.6</td>
391
- </tr>
392
-
393
- <tr>
394
- <td align="center" colspan=9><strong>General Tasks</strong></td>
395
- </tr>
396
-
397
- <tr>
398
- <td align="center">MMLU</td>
399
- <td align="center">EM</td>
400
- <td align="center"><ins><strong>89.5</strong></ins></td>
401
- <td align="center">89.4</td>
402
- <td align="center">87.0</td>
403
- <td align="center">91.5</td>
404
- <td align="center"><strong>92.9</strong></td>
405
- <td align="center">90.4</td>
406
- <td align="center">90.1</td>
407
- </tr>
408
-
409
- <tr>
410
- <td align="center">MMLU-Redux</td>
411
- <td align="center">EM</td>
412
- <td align="center"><ins><strong>92.7</strong></ins></td>
413
- <td align="center">90.5</td>
414
- <td align="center">89.2</td>
415
- <td align="center">93.6</td>
416
- <td align="center"><strong>94.2</strong></td>
417
- <td align="center">92.4</td>
418
- <td align="center">90.6</td>
419
- </tr>
420
-
421
- <tr>
422
- <td align="center">MMLU-Pro</td>
423
- <td align="center">EM</td>
424
- <td align="center">81.1</td>
425
- <td align="center"><ins><strong>81.2</strong></ins><sup>*</sup></td>
426
- <td align="center">77.3</td>
427
- <td align="center">83.7</td>
428
- <td align="center"><strong>86.6</strong></td>
429
- <td align="center">81.8</td>
430
- <td align="center">79.4</td>
431
- </tr>
432
-
433
- <tr>
434
- <td align="center">IFEval</td>
435
- <td align="center">Prompt Strict</td>
436
- <td align="center"><strong>89.8</strong></td>
437
- <td align="center">81.1</td>
438
- <td align="center">83.2<sup>*</sup></td>
439
- <td align="center">87.6</td>
440
- <td align="center">87.4</td>
441
- <td align="center">88.0</td>
442
- <td align="center">84.3</td>
443
- </tr>
444
-
445
- <tr>
446
- <td align="center">Multi-Challenge</td>
447
- <td align="center">Acc</td>
448
- <td align="center"><strong>54.1</strong></td>
449
- <td align="center">31.4</td>
450
- <td align="center">34.0</td>
451
- <td align="center">46.8</td>
452
- <td align="center">49.0</td>
453
- <td align="center">36.4</td>
454
- <td align="center">39.5</td>
455
- </tr>
456
-
457
- <tr>
458
- <td align="center">SimpleQA</td>
459
- <td align="center">Correct</td>
460
- <td align="center"><ins><strong>31.0</strong></ins></td>
461
- <td align="center">27.7</td>
462
- <td align="center">13.2</td>
463
- <td align="center">15.9</td>
464
- <td align="center">22.8</td>
465
- <td align="center"><strong>42.3</strong></td>
466
- <td align="center">23.3</td>
467
- </tr>
468
-
469
- <tr>
470
- <td align="center">Livebench</td>
471
- <td align="center">Pass@1</td>
472
- <td align="center"><strong>76.4</strong></td>
473
- <td align="center">72.4</td>
474
- <td align="center">67.6</td>
475
- <td align="center">74.8</td>
476
- <td align="center">74.6</td>
477
- <td align="center">69.8</td>
478
- <td align="center">67.8</td>
479
- </tr>
480
- </tbody>
481
- </table>
482
- </div>
483
- <sup>
484
- • Bold denotes global SOTA, and underlined denotes open-source SOTA.
485
- </sup><br/><sup>
486
- • Data points marked with * are taken directly from the model's tech report or blog.
487
- </sup><br/><sup>
488
- • All metrics, except for SWE-bench Verified (Agentless), are evaluated with an 8k output token length. SWE-bench Verified (Agentless) is limited to a 16k output token length.
489
- </sup><br/><sup>
490
- • Kimi K2 achieves 65.8% pass@1 on the SWE-bench Verified tests with bash/editor tools (single-attempt patches, no test-time compute). It also achieves a 47.3% pass@1 on the SWE-bench Multilingual tests under the same conditions. Additionally, we report results on SWE-bench Verified tests (71.6%) that leverage parallel test-time compute by sampling multiple sequences and selecting the single best via an internal scoring model.
491
- </sup><br/><sup>
492
- • To ensure the stability of the evaluation, we employed avg@k on the AIME, HMMT, CNMO, PolyMath-en, GPQA-Diamond, EvalPlus, Tau2.
493
- </sup><br/><sup>
494
- • Some data points have been omitted due to prohibitively expensive evaluation costs.
495
- </sup>
496
-
497
- ---
498
-
499
- #### Base model evaluation results
500
-
501
- <div align="center">
502
-
503
- <table>
504
- <thead>
505
- <tr>
506
- <th align="center">Benchmark</th>
507
- <th align="center">Metric</th>
508
- <th align="center">Shot</th>
509
- <th align="center">Kimi K2 Base</th>
510
- <th align="center">Deepseek-V3-Base</th>
511
- <th align="center">Qwen2.5-72B</th>
512
- <th align="center">Llama 4 Maverick</th>
513
- </tr>
514
- </thead>
515
- <tbody>
516
- <tr>
517
- <td align="center" colspan="7"><strong>General Tasks</strong></td>
518
- </tr>
519
- <tr>
520
- <td align="center">MMLU</td>
521
- <td align="center">EM</td>
522
- <td align="center">5-shot</td>
523
- <td align="center"><strong>87.8</strong></td>
524
- <td align="center">87.1</td>
525
- <td align="center">86.1</td>
526
- <td align="center">84.9</td>
527
- </tr>
528
- <tr>
529
- <td align="center">MMLU-pro</td>
530
- <td align="center">EM</td>
531
- <td align="center">5-shot</td>
532
- <td align="center"><strong>69.2</strong></td>
533
- <td align="center">60.6</td>
534
- <td align="center">62.8</td>
535
- <td align="center">63.5</td>
536
- </tr>
537
- <tr>
538
- <td align="center">MMLU-redux-2.0</td>
539
- <td align="center">EM</td>
540
- <td align="center">5-shot</td>
541
- <td align="center"><strong>90.2</strong></td>
542
- <td align="center">89.5</td>
543
- <td align="center">87.8</td>
544
- <td align="center">88.2</td>
545
- </tr>
546
- <tr>
547
- <td align="center">SimpleQA</td>
548
- <td align="center">Correct</td>
549
- <td align="center">5-shot</td>
550
- <td align="center"><strong>35.3</strong></td>
551
- <td align="center">26.5</td>
552
- <td align="center">10.3</td>
553
- <td align="center">23.7</td>
554
- </tr>
555
- <tr>
556
- <td align="center">TriviaQA</td>
557
- <td align="center">EM</td>
558
- <td align="center">5-shot</td>
559
- <td align="center"><strong>85.1</strong></td>
560
- <td align="center">84.1</td>
561
- <td align="center">76.0</td>
562
- <td align="center">79.3</td>
563
- </tr>
564
- <tr>
565
- <td align="center">GPQA-Diamond</td>
566
- <td align="center">Avg@8</td>
567
- <td align="center">5-shot</td>
568
- <td align="center">48.1</td>
569
- <td align="center"><strong>50.5</strong></td>
570
- <td align="center">40.8</td>
571
- <td align="center">49.4</td>
572
- </tr>
573
- <tr>
574
- <td align="center">SuperGPQA</td>
575
- <td align="center">EM</td>
576
- <td align="center">5-shot</td>
577
- <td align="center"><strong>44.7</strong></td>
578
- <td align="center">39.2</td>
579
- <td align="center">34.2</td>
580
- <td align="center">38.8</td>
581
- </tr>
582
- <tr>
583
- <td align="center" colspan="7"><strong>Coding Tasks</strong></td>
584
- </tr>
585
- <tr>
586
- <td align="center">LiveCodeBench v6</td>
587
- <td align="center">Pass@1</td>
588
- <td align="center">1-shot</td>
589
- <td align="center"><strong>26.3</strong></td>
590
- <td align="center">22.9</td>
591
- <td align="center">21.1</td>
592
- <td align="center">25.1</td>
593
- </tr>
594
- <tr>
595
- <td align="center">EvalPlus</td>
596
- <td align="center">Pass@1</td>
597
- <td align="center">-</td>
598
- <td align="center"><strong>80.3</strong></td>
599
- <td align="center">65.6</td>
600
- <td align="center">66.0</td>
601
- <td align="center">65.5</td>
602
- </tr>
603
- <tr>
604
- <td align="center" colspan="7"><strong>Mathematics Tasks</strong></td>
605
- </tr>
606
- <tr>
607
- <td align="center">MATH</td>
608
- <td align="center">EM</td>
609
- <td align="center">4-shot</td>
610
- <td align="center"><strong>70.2</strong></td>
611
- <td align="center">60.1</td>
612
- <td align="center">61.0</td>
613
- <td align="center">63.0</td>
614
- </tr>
615
- <tr>
616
- <td align="center">GSM8k</td>
617
- <td align="center">EM</td>
618
- <td align="center">8-shot</td>
619
- <td align="center"><strong>92.1</strong></td>
620
- <td align="center">91.7</td>
621
- <td align="center">90.4</td>
622
- <td align="center">86.3</td>
623
- </tr>
624
- <tr>
625
- <td align="center" colspan="7"><strong>Chinese Tasks</strong></td>
626
- </tr>
627
- <tr>
628
- <td align="center">C-Eval</td>
629
- <td align="center">EM</td>
630
- <td align="center">5-shot</td>
631
- <td align="center"><strong>92.5</strong></td>
632
- <td align="center">90.0</td>
633
- <td align="center">90.9</td>
634
- <td align="center">80.9</td>
635
- </tr>
636
- <tr>
637
- <td align="center">CSimpleQA</td>
638
- <td align="center">Correct</td>
639
- <td align="center">5-shot</td>
640
- <td align="center"><strong>77.6</strong></td>
641
- <td align="center">72.1</td>
642
- <td align="center">50.5</td>
643
- <td align="center">53.5</td>
644
- </tr>
645
- </tbody>
646
- </table>
647
- </div>
648
- <sup>
649
- • We only evaluate open-source pretrained models in this work. We report results for Qwen2.5-72B because the base checkpoint for Qwen3-235B-A22B was not open-sourced at the time of our study.
650
- </sup><br/><sup>
651
- • All models are evaluated using the same evaluation protocol.
652
-
653
- </sup>
654
-
655
-
656
- ## 4. Deployment
657
- > [!Note]
658
- > You can access Kimi K2's API on https://platform.moonshot.ai , we provide OpenAI/Anthropic-compatible API for you.
659
- >
660
- > The Anthropic-compatible API maps temperature by `real_temperature = request_temperature * 0.6` for better compatible with existing applications.
661
-
662
- Our model checkpoints are stored in the block-fp8 format, you can find it on [Huggingface](https://huggingface.co/moonshotai/Kimi-K2-Instruct).
663
-
664
- Currently, Kimi-K2 is recommended to run on the following inference engines:
665
-
666
- * vLLM
667
- * SGLang
668
- * KTransformers
669
- * TensorRT-LLM
670
-
671
- Deployment examples for vLLM and SGLang can be found in the [Model Deployment Guide](docs/deploy_guidance.md).
672
-
673
- ---
674
-
675
- ## 5. Model Usage
676
-
677
- ### Chat Completion
678
-
679
- Once the local inference service is up, you can interact with it through the chat endpoint:
680
-
681
- ```python
682
- def simple_chat(client: OpenAI, model_name: str):
683
- messages = [
684
- {"role": "system", "content": "You are Kimi, an AI assistant created by Moonshot AI."},
685
- {"role": "user", "content": [{"type": "text", "text": "Please give a brief self-introduction."}]},
686
- ]
687
- response = client.chat.completions.create(
688
- model=model_name,
689
- messages=messages,
690
- stream=False,
691
- temperature=0.6,
692
- max_tokens=256
693
- )
694
- print(response.choices[0].message.content)
695
- ```
696
-
697
- > [!NOTE]
698
- > The recommended temperature for Kimi-K2-Instruct is `temperature = 0.6`.
699
- > If no special instructions are required, the system prompt above is a good default.
700
-
701
- ---
702
-
703
- ### Tool Calling
704
-
705
- Kimi-K2-Instruct has strong tool-calling capabilities.
706
- To enable them, you need to pass the list of available tools in each request, then the model will autonomously decide when and how to invoke them.
707
-
708
- The following example demonstrates calling a weather tool end-to-end:
709
-
710
- ```python
711
- # Your tool implementation
712
- def get_weather(city: str) -> dict:
713
- return {"weather": "Sunny"}
714
-
715
- # Tool schema definition
716
- tools = [{
717
- "type": "function",
718
- "function": {
719
- "name": "get_weather",
720
- "description": "Retrieve current weather information. Call this when the user asks about the weather.",
721
- "parameters": {
722
- "type": "object",
723
- "required": ["city"],
724
- "properties": {
725
- "city": {
726
- "type": "string",
727
- "description": "Name of the city"
728
- }
729
- }
730
- }
731
- }
732
- }]
733
-
734
- # Map tool names to their implementations
735
- tool_map = {
736
- "get_weather": get_weather
737
- }
738
-
739
- def tool_call_with_client(client: OpenAI, model_name: str):
740
- messages = [
741
- {"role": "system", "content": "You are Kimi, an AI assistant created by Moonshot AI."},
742
- {"role": "user", "content": "What's the weather like in Beijing today? Use the tool to check."}
743
- ]
744
- finish_reason = None
745
- while finish_reason is None or finish_reason == "tool_calls":
746
- completion = client.chat.completions.create(
747
- model=model_name,
748
- messages=messages,
749
- temperature=0.6,
750
- tools=tools, # tool list defined above
751
- tool_choice="auto"
752
- )
753
- choice = completion.choices[0]
754
- finish_reason = choice.finish_reason
755
- if finish_reason == "tool_calls":
756
- messages.append(choice.message)
757
- for tool_call in choice.message.tool_calls:
758
- tool_call_name = tool_call.function.name
759
- tool_call_arguments = json.loads(tool_call.function.arguments)
760
- tool_function = tool_map[tool_call_name]
761
- tool_result = tool_function(**tool_call_arguments)
762
- print("tool_result:", tool_result)
763
-
764
- messages.append({
765
- "role": "tool",
766
- "tool_call_id": tool_call.id,
767
- "name": tool_call_name,
768
- "content": json.dumps(tool_result)
769
- })
770
- print("-" * 100)
771
- print(choice.message.content)
772
- ```
773
-
774
- The `tool_call_with_client` function implements the pipeline from user query to tool execution.
775
- This pipeline requires the inference engine to support Kimi-K2’s native tool-parsing logic.
776
- For streaming output and manual tool-parsing, see the [Tool Calling Guide](docs/tool_call_guidance.md).
777
-
778
- ---
779
-
780
- ## 6. License
781
-
782
- Both the code repository and the model weights are released under the [Modified MIT License](LICENSE).
783
-
784
- ---
785
-
786
- ## 7. Contact Us
787
-
788
- If you have any questions, please reach out at [[email protected]](mailto:[email protected]).
 
1
  ---
2
+ base_model:
3
+ - moonshotai/Kimi-K2-Instruct
4
+ pipeline_tag: text-generation
5
  ---
 
 
 
 
 
6
 
7
+ [<img src="https://raw.githubusercontent.com/csabakecskemeti/devquasar/main/dq_logo_black-transparent.png" width="200"/>](https://devquasar.com)
8
 
9
+ 'Make knowledge free for everyone'
 
 
 
10
 
11
+ BF16 version of: [moonshotai/Kimi-K2-Instruct](https://huggingface.co/moonshotai/Kimi-K2-Instruct)
12
+ <a href='https://ko-fi.com/L4L416YX7C' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi6.png?v=6' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a>