Duplicate from THUDM/GLM-Z1-Rumination-32B-0414

Co-authored-by: zR <[email protected]>

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Zhipu AI
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,133 @@
+---
+license: mit
+language:
+- zh
+- en
+pipeline_tag: text-generation
+library_name: transformers
+---
+
+# GLM-4-Z1-Rumination-32B-0414
+
+## Introduction
+
+The GLM family welcomes a new generation of open-source models, the **GLM-4-32B-0414** series, featuring 32 billion parameters. Its performance is comparable to OpenAI's GPT series and DeepSeek's V3/R1 series, and it supports very user-friendly local deployment features. GLM-4-32B-Base-0414 was pre-trained on 15T of high-quality data, including a large amount of reasoning-type synthetic data, laying the foundation for subsequent reinforcement learning extensions. In the post-training stage, in addition to human preference alignment for dialogue scenarios, we also enhanced the model's performance in instruction following, engineering code, and function calling using techniques such as rejection sampling and reinforcement learning, strengthening the atomic capabilities required for agent tasks. GLM-4-32B-0414 achieves good results in areas such as engineering code, Artifact generation, function calling, search-based Q&A, and report generation. Some benchmarks even rival larger models like GPT-4o and DeepSeek-V3-0324 (671B).
+
+**GLM-Z1-32B-0414** is a reasoning model with **deep thinking capabilities**. This was developed based on GLM-4-32B-0414 through cold start and extended reinforcement learning, as well as further training of the model on tasks involving mathematics, code, and logic. Compared to the base model, GLM-Z1-32B-0414 significantly improves mathematical abilities and the capability to solve complex tasks. During the training process, we also introduced general reinforcement learning based on pairwise ranking feedback, further enhancing the model's general capabilities.
+
+**GLM-Z1-Rumination-32B-0414** is a deep reasoning model with **rumination capabilities** (benchmarked against OpenAI's Deep Research). Unlike typical deep thinking models, the rumination model employs longer periods of deep thought to solve more open-ended and complex problems (e.g., writing a comparative analysis of AI development in two cities and their future development plans). The rumination model integrates search tools during its deep thinking process to handle complex tasks and is trained by utilizing multiple rule-based rewards to guide and extend end-to-end reinforcement learning. Z1-Rumination shows significant improvements in research-style writing and complex retrieval tasks.
+
+Finally, **GLM-Z1-9B-0414** is a surprise. We employed the aforementioned series of techniques to train a 9B small-sized model that maintains the open-source tradition. Despite its smaller scale, GLM-Z1-9B-0414 still exhibits excellent capabilities in mathematical reasoning and general tasks. Its overall performance is already at a leading level among open-source models of the same size. Especially in resource-constrained scenarios, this model achieves an excellent balance between efficiency and effectiveness, providing a powerful option for users seeking lightweight deployment.
+
+## Inference Code
+
+Make Sure Using `transforemrs>=4.51.3`.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+MODEL_PATH = "THUDM/GLM-Z1-Rumination-32B-0414"
+
+tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
+model = AutoModelForCausalLM.from_pretrained(MODEL_PATH, device_map="auto")
+
+message = [{"role": "user", "content": "Let a, b be positive real numbers such that ab = a + b + 3. Determine the range of possible values for a + b."}]
+
+inputs = tokenizer.apply_chat_template(
+    message,
+    return_tensors="pt",
+    add_generation_prompt=True,
+    return_dict=True,
+).to(model.device)
+
+generate_kwargs = {
+    "input_ids": inputs["input_ids"],
+    "attention_mask": inputs["attention_mask"],
+    "temperature": 0.95,
+    "top_p": 0.7,
+    "do_sample": True,
+}
+out = model.generate(**generate_kwargs)
+print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
+```
+
+## Function Call
+
+By default, this model currently supports the following `function` calls:
+- `search`: Search using a keyword and return search results
+- `click`: Click on a specific webpage in the search results to view details
+- `open`: Open a fixed URL to view detailed content
+- `finsih`: Complete information gathering and begin writing
+
+Below is a simple workflow to help you quickly connect the pipeline.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+MODEL_PATH = "THUDM/GLM-Z1-Rumination-32B-0414"
+
+tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
+model = AutoModelForCausalLM.from_pretrained(MODEL_PATH, device_map="auto")
+
+messages = [{"role": "user", "content": "Let a, b be positive real numbers such that ab = a + b + 3. Determine the range of possible values for a + b."}]
+
+generate_kwargs = {
+    "temperature": 0.95,
+    "top_p": 0.7,
+    "do_sample": True,
+}
+
+def get_assistant():
+    inputs = tokenizer.apply_chat_template(
+        messages,
+        return_tensors="pt",
+        add_generation_prompt=True,
+        return_dict=True,
+    ).to(model.device)
+    out = model.generate(input_ids=input["input_ids"], **generate_kwargs)
+    return tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip()
+
+def get_observation(function_name, args):
+    if fucntion_name == "search":
+        mock_search_res = [
+            {"title": "t1", "url":"url1", "snippet": "snippet_content_1"},
+            {"title": "t2", "url":"url2", "snippet": "snippet_content_2"}
+        ]
+        content = "\n\n".join([f"【{i}†{res['title']}†{res['url']}\n{res['snippet']}】"] for i, res in mock_search_res)
+    elif function_name == "click":
+        mock_click_res = "main content"
+        content = mock_click_res
+    elif function_name == "open":
+        mock_open_res = "main_content"
+        content = mock_open_res
+    else:
+        raise ValueError("unspport function name!")
+        
+def get_func_name_args(llm_text):
+    function_call = re.sub(r'.*?</think>', '', llm_text, flags=re.DOTALL)
+    function_call = json.loads(function_call)
+    action = function_call['name']
+    params = function_call['arguments']
+    return action, params
+
+def pipeline():
+    end_str = "{\"name\": \"finish\", \"arguments\": {}}"
+    response = get_assistant()
+    messages.append({"role": "assistant", "content": response})
+    max_turns, turns = 35, 1
+    while not response.endswith(end_str) and turns < max_turns:
+        action, params = get_func_name_args(response)
+        observation = get_observation(action, params)
+        messages.append({"role": "observation", "content": observation})
+        response = get_assistant()
+        messages.append({"role": "assistant", "content": response})
+        turns += 1
+        
+    if response.endswith(end_str):
+        final_answer = get_assistant()
+    else:
+        final_answer = None
+    return final_answer
+
+pipeline()   
+```

chat_template.jinja

@@ -0,0 +1,42 @@
+[gMASK]<sop>
+<|system|>
+你是一个专业的深度研究助手，通过提供的工具与模拟浏览器交互，来帮助用户完成深度信息调研和报告撰写任务。今年是 2025 年。
+
+<核心要求>
+- 首先分解用户请求，得到包含多个子要求的列表
+- 制定初始研究计划
+- 进行多轮迭代搜索和页面浏览（at least 10 function calls）：
+    * 根据已获得的信息调整研究计划和关键词
+    * 打开页面阅读，从发现的内容中识别新的关键概念/名词
+    * 从搜索结果中提取新的关键词继续搜索
+    * 访问并仔细阅读相关页面，识别新的关键概念/名词
+
+<重要配置>
+- 采用语言
+    * 搜索关键词：英文
+    * 思考：英文
+
+<可调用的工具列表>
+[{"name": "search", "description": "Execute a search query and return search results. Use this function when you need to find information about a specific topic.", "parameters": {"type": "object", "properties": {"query": {"type": "string", "description": "Search query string, use English words unless it is a proper name in Chinese"}}, "required": ["query"], "additionalProperties": false}}, {"name": "click", "description": "Click a link in the search results and navigate to the corresponding page. Use this function when you need to view detailed content of a specific search result.", "parameters": {"type": "object", "properties": {"link_id": {"type": "integer", "description": "The link ID to click (from the sequence number in search results)"}}, "required": ["link_id"], "additionalProperties": false}}, {"name": "open", "description": "Open a specific website. Get content from any website with its URL.", "parameters": {"type": "object", "properties": {"url": {"type": "string", "description": "The target website URL or domain"}}, "required": ["url"], "additionalProperties": false}}, {"name": "finish", "description": "Finish the task. Use this function when you have found the information you need.", "parameters": {"type": "object", "properties": {}, "additionalProperties": false}}]
+
+{%- for message in messages if message.role != 'system' %}
+    {%- set role = message['role'] %}
+    {%- set content = message['content'] %}
+    {%- set visible = content.split('</think>')[-1].strip() %}
+    {%- set meta = message.get("metadata", "") %}
+
+    {%- if role == 'user' %}
+<|user|>
+{{ visible }}
+    {%- elif role == 'assistant' and not meta %}
+<|assistant|>
+{{ visible }}
+    {%- elif role == 'assistant' and meta %}
+<|assistant|>{{ meta }}
+{{ visible }}
+    {%- elif role == 'observation' %}
+<|observation|>
+{{ visible }}
+    {%- endif %}
+{%- endfor %}
+{% if add_generation_prompt %}<|assistant|>{% endif %}

config.json

@@ -0,0 +1,31 @@
+{
+  "architectures": [
+    "Glm4ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "eos_token_id": [
+    151329,
+    151336,
+    151338
+  ],
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 6144,
+  "initializer_range": 0.02,
+  "intermediate_size": 23040,
+  "max_position_embeddings": 131072,
+  "model_type": "glm4",
+  "num_attention_heads": 48,
+  "num_hidden_layers": 61,
+  "num_key_value_heads": 8,
+  "pad_token_id": 151329,
+  "partial_rotary_factor": 0.5,
+  "rms_norm_eps": 1e-05,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.52.0.dev0",
+  "use_cache": true,
+  "vocab_size": 151552
+}

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": [
+    151329,
+    151336,
+    151338
+  ],
+  "pad_token_id": 151329,
+  "transformers_version": "4.52.0.dev0"
+}

model-00001-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ede867f519f5cb55105d9406ac3b61fdfc8f40e792afc3ecadc30c46e4d4aca
+size 4938944056

model-00002-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce6d5919f5aeee759556faccf42f3653b50d4a9481c3a247488b85a543b093eb
+size 4844622992

model-00003-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8114c81fc02e2470d350fa0a2aea70fa057769e06bdb0d080be1db06a5a3da18
+size 4561557104

model-00004-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b01cdb21f8d4c9a227de11ec435c0c0a2b9bc81a7290bc7268e1e9c40537998
+size 4951627056

model-00005-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1812db71f7a31903c0c5418ab1a3adc91c55902c6852df26045ce4977ff7d8a8
+size 4844623032

model-00006-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c31e145ca156c175876d4e8ec65de3a99c0d420f376b2a347aafc09eca541ce
+size 4561557136

model-00007-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8535e2cba43d7624cdd18da0d1a0aa28d0c2a9f97ec9518c4e367c03d328b56
+size 4951627056

model-00008-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f86f8dbf644a65222ed713996ea9792dc96948bc5c79fd54e8de922828a3a05c
+size 4844623032

model-00009-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:725c6717575715b7448169e2a7e30808a47c21cc43d8d32293e58f0ebe11065a
+size 4561557136

model-00010-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aaedbfa4f3a0ed1dac5f046cbddfe7d823433425341bc52f6e26c2a82ba76a94
+size 4951627056

model-00011-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a530952a68b1d7c63a8d8c64a79b7fd313331a56dea937b1b59cd3705d1c8eb
+size 4844623032

model-00012-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d13b9fd7644d26cfa4e7b480c8230740ee62e94247c4349c0cd0c849e426380e
+size 4561557136

model-00013-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebe01bf46a66f0094eb25aa59eef8d144593d85aae29b65a8be232d21edbac14
+size 4951627056

model-00014-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3f2a509726158c0e9a35dbfe3798bd4cc7035b0224c4f0569441d9cd960e137
+size 3913398800

model.safetensors.index.json

@@ -0,0 +1,620 @@
+{
+  "metadata": {
+    "total_size": 66283499520
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00014-of-00014.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.mlp.gate_up_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.post_mlp_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.post_self_attn_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.mlp.gate_up_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.post_mlp_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.post_self_attn_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.mlp.gate_up_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.post_mlp_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.post_self_attn_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.mlp.gate_up_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.post_mlp_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.post_self_attn_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.12.mlp.gate_up_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.12.post_mlp_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.12.post_self_attn_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.mlp.gate_up_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.post_mlp_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.post_self_attn_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.mlp.gate_up_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.post_mlp_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.post_self_attn_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.mlp.gate_up_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.post_mlp_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.post_self_attn_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.mlp.gate_up_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.post_mlp_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.post_self_attn_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.mlp.gate_up_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.post_mlp_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.post_self_attn_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.mlp.gate_up_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.post_mlp_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.post_self_attn_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.mlp.gate_up_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.post_mlp_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.post_self_attn_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.2.mlp.gate_up_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.2.post_mlp_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.2.post_self_attn_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.mlp.gate_up_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.post_mlp_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.post_self_attn_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.21.mlp.gate_up_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.21.post_mlp_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.21.post_self_attn_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.mlp.gate_up_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.post_mlp_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.post_self_attn_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.mlp.gate_up_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.post_mlp_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.post_self_attn_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.mlp.gate_up_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.post_mlp_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.post_self_attn_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.mlp.gate_up_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.post_mlp_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.post_self_attn_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.26.mlp.gate_up_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.26.post_mlp_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.26.post_self_attn_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.mlp.gate_up_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.post_mlp_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.post_self_attn_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.mlp.gate_up_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.post_mlp_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.post_self_attn_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.mlp.gate_up_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.post_mlp_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.post_self_attn_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.mlp.gate_up_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.post_mlp_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.post_self_attn_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.mlp.gate_up_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.post_mlp_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.post_self_attn_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.31.mlp.gate_up_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.31.post_mlp_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.31.post_self_attn_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.input_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.mlp.gate_up_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.post_mlp_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.post_self_attn_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.input_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.mlp.gate_up_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.post_mlp_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.post_self_attn_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.input_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.mlp.gate_up_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.post_mlp_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.post_self_attn_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.input_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.35.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.35.mlp.gate_up_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.35.post_mlp_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.35.post_self_attn_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.35.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.36.input_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.36.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.36.mlp.gate_up_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.36.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.36.post_mlp_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.36.post_self_attn_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.36.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.36.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.36.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.36.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.input_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.mlp.gate_up_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.post_mlp_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.post_self_attn_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.input_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.mlp.gate_up_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.post_mlp_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.post_self_attn_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.input_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.mlp.gate_up_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.post_mlp_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.post_self_attn_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.mlp.gate_up_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.post_mlp_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.post_self_attn_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.40.input_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.40.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.40.mlp.gate_up_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.40.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.40.post_mlp_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.40.post_self_attn_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.40.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.40.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.40.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.40.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.41.input_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.41.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.41.mlp.gate_up_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.41.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.41.post_mlp_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.41.post_self_attn_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.41.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.41.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.41.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.41.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.input_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.mlp.gate_up_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.post_mlp_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.post_self_attn_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.input_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.mlp.gate_up_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.post_mlp_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.post_self_attn_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.input_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.mlp.gate_up_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.post_mlp_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.post_self_attn_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.45.input_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.45.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.45.mlp.gate_up_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.45.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.45.post_mlp_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.45.post_self_attn_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.45.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.45.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.45.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.45.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.46.input_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.46.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.46.mlp.gate_up_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.46.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.46.post_mlp_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.46.post_self_attn_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.46.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.46.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.46.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.46.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.input_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.mlp.gate_up_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.post_mlp_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.post_self_attn_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.input_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.mlp.gate_up_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.post_mlp_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.post_self_attn_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.input_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.49.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.49.mlp.gate_up_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.49.post_mlp_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.49.post_self_attn_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.49.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.mlp.gate_up_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.post_mlp_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.post_self_attn_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.50.input_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.50.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.50.mlp.gate_up_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.50.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.50.post_mlp_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.50.post_self_attn_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.50.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.50.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.50.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.50.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.51.input_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.51.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.51.mlp.gate_up_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.51.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.51.post_mlp_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.51.post_self_attn_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.51.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.51.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.51.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.51.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.input_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.mlp.gate_up_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.post_mlp_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.post_self_attn_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.input_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.mlp.gate_up_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.post_mlp_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.post_self_attn_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.54.input_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.54.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.54.mlp.gate_up_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.54.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.54.post_mlp_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.54.post_self_attn_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.54.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.54.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.54.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.54.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.55.input_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.55.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.55.mlp.gate_up_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.55.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.55.post_mlp_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.55.post_self_attn_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.55.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.55.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.55.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.55.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.56.input_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.56.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.56.mlp.gate_up_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.56.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.56.post_mlp_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.56.post_self_attn_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.56.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.56.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.56.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.56.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.input_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.mlp.gate_up_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.post_mlp_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.post_self_attn_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.input_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.mlp.gate_up_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.post_mlp_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.post_self_attn_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.59.input_layernorm.weight": "model-00014-of-00014.safetensors",
+    "model.layers.59.mlp.down_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.59.mlp.gate_up_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.59.post_attention_layernorm.weight": "model-00014-of-00014.safetensors",
+    "model.layers.59.post_mlp_layernorm.weight": "model-00014-of-00014.safetensors",
+    "model.layers.59.post_self_attn_layernorm.weight": "model-00014-of-00014.safetensors",
+    "model.layers.59.self_attn.k_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.59.self_attn.o_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.59.self_attn.q_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.59.self_attn.v_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.mlp.gate_up_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.post_mlp_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.post_self_attn_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.60.input_layernorm.weight": "model-00014-of-00014.safetensors",
+    "model.layers.60.mlp.down_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.60.mlp.gate_up_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.60.post_attention_layernorm.weight": "model-00014-of-00014.safetensors",
+    "model.layers.60.post_mlp_layernorm.weight": "model-00014-of-00014.safetensors",
+    "model.layers.60.post_self_attn_layernorm.weight": "model-00014-of-00014.safetensors",
+    "model.layers.60.self_attn.k_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.60.self_attn.o_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.60.self_attn.q_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.60.self_attn.v_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.7.mlp.gate_up_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.7.post_mlp_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.7.post_self_attn_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.mlp.gate_up_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.post_mlp_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.post_self_attn_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.mlp.gate_up_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.post_mlp_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.post_self_attn_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
+    "model.norm.weight": "model-00014-of-00014.safetensors"
+  }
+}

special_tokens_map.json

@@ -0,0 +1,32 @@
+{
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "[MASK]",
+    "[gMASK]",
+    "[sMASK]",
+    "<sop>",
+    "<eop>",
+    "<|system|>",
+    "<|user|>",
+    "<|assistant|>",
+    "<|observation|>",
+    "<|begin_of_image|>",
+    "<|end_of_image|>",
+    "<|begin_of_video|>",
+    "<|end_of_video|>"
+  ],
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76ebeac0d8bd7879ead7b43c16b44981f277e47225de2bd7de9ae1a6cc664a8c
+size 19966496

tokenizer_config.json

@@ -0,0 +1,146 @@
+{
+  "added_tokens_decoder": {
+    "151329": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151330": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151331": {
+      "content": "[gMASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151332": {
+      "content": "[sMASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151333": {
+      "content": "<sop>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151334": {
+      "content": "<eop>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151335": {
+      "content": "<|system|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151336": {
+      "content": "<|user|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151337": {
+      "content": "<|assistant|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151338": {
+      "content": "<|observation|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151339": {
+      "content": "<|begin_of_image|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151340": {
+      "content": "<|end_of_image|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151341": {
+      "content": "<|begin_of_video|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151342": {
+      "content": "<|end_of_video|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "[MASK]",
+    "[gMASK]",
+    "[sMASK]",
+    "<sop>",
+    "<eop>",
+    "<|system|>",
+    "<|user|>",
+    "<|assistant|>",
+    "<|observation|>",
+    "<|begin_of_image|>",
+    "<|end_of_image|>",
+    "<|begin_of_video|>",
+    "<|end_of_video|>"
+  ],
+  "chat_template": "[gMASK]<sop><|system|>\n你是一个专业的深度研究助手，通过提供的工具与模拟浏览器交互，来帮助用户完成深度信息调研和报告撰写任务。今年是 2025 年。\n\n<核心要求>\n- 首先分解用户请求，得到包含多个子要求的列表\n- 制定初始研究计划\n- 进行多轮迭代搜索和页面浏览（at least 10 function calls）：\n    * 根据已获得的信息调整研究计划和关键词\n    * 打开页面阅读，从发现的内容中识别新的关键概念/名词\n    * 从搜索结果中提取新的关键词继续搜索\n    * 访问并仔细阅读相关页面，识别新的关键概念/名词\n\n<重要配置>\n- 采用语言\n    * 搜索关键词：英语\n    * 思考：英语\n\n<可调用的工具列表>\n\n[{\"name\": \"search\", \"description\": \"Execute a search query and return search results. Use this function when you need to find information about a specific topic.\", \"parameters\": {\"type\": \"object\", \"properties\": {\"query\": {\"type\": \"string\", \"description\": \"Search query string, use English words unless it is a proper name in Chinese\"}}, \"required\": [\"query\"], \"additionalProperties\": false}}, {\"name\": \"click\", \"description\": \"Click a link in the search results and navigate to the corresponding page. Use this function when you need to view detailed content of a specific search result.\", \"parameters\": {\"type\": \"object\", \"properties\": {\"link_id\": {\"type\": \"integer\", \"description\": \"The link ID to click (from the sequence number in search results)\"}}, \"required\": [\"link_id\"], \"additionalProperties\": false}}, {\"name\": \"open\", \"description\": \"Open a specific website. Get content from any website with its URL.\", \"parameters\": {\"type\": \"object\", \"properties\": {\"url\": {\"type\": \"string\", \"description\": \"The target website URL or domain\"}}, \"required\": [\"url\"], \"additionalProperties\": false}}, {\"name\": \"finish\", \"description\": \"Finish the task. Use this function when you have found the information you need.\", \"parameters\": {\"type\": \"object\", \"properties\": {}, \"additionalProperties\": false}}]\n\n{%- for message in messages if message.role != 'system' %}{%- set role = message['role'] %}{%- set content = message['content'] %}{%- set visible = content.split('</think>')[-1].strip() %}{%- set meta = message.get(\"metadata\", \"\") %}{%- if role == 'user' %}<|user|>\n{{ visible }}{%- elif role == 'assistant' and not meta %}<|assistant|>\n{{ visible }}{%- elif role == 'assistant' and meta %}<|assistant|>{{ meta }} \n{{ visible }}{%- elif role == 'observation' %}<|observation|>\n{{ visible }}{%- endif %}{%- endfor %}{% if add_generation_prompt %}<|assistant|>{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "do_lower_case": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": {},
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 128000,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "left",
+  "remove_space": false,
+  "tokenizer_class": "PreTrainedTokenizer"
+}