ggml-org

/

SmolLM3-3B-GGUF

like 54

Follow

ggml-org 1.76k

Windows10
Python 3.10
Command:
python -m llama_cpp.server --host 0.0.0.0 --port 8000 --model SmolLM3-Q4_K_M.gguf --n_ctx 2048

the two module, throw same error message.

Error messages:

Available chat formats from metadata: chat_template.default
Traceback (most recent call last):
File "C:\software\Python310\lib\runpy.py", line 196, in _run_module_as_main
return run_code(code, main_globals, None,
File "C:\software\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\env_llm\lib\site-packages\llama_cpp\server_main.py", line 100, in
main()
File "C:\env_llm\lib\site-packages\llama_cpp\server_main.py", line 86, in main
app = create_app(
File "C:\env_llm\lib\site-packages\llama_cpp\server\app.py", line 150, in create_app
set_llama_proxy(model_settings=model_settings)
File "C:\env_llm\lib\site-packages\llama_cpp\server\app.py", line 70, in set_llama_proxy
_llama_proxy = LlamaProxy(models=model_settings)
File "C:\env_llm\lib\site-packages\llama_cpp\server\model.py", line 31, in init
self._current_model = self.load_llama_from_model_settings(
File "C:\env_llm\lib\site-packages\llama_cpp\server\model.py", line 250, in load_llama_from_model_settings
_model = create_fn(
File "C:\env_llm\lib\site-packages\llama_cpp\llama.py", line 509, in init
self._chat_handlers[name] = llama_chat_format.Jinja2ChatFormatter(
File "C:\env_llm\lib\site-packages\llama_cpp\llama_chat_format.py", line 216, in init
).from_string(self.template)
File "C:\env_llm\lib\site-packages\jinja2\environment.py", line 1111, in from_string
return cls.from_code(self, self.compile(source), gs, None)
File "C:\env_llm\lib\site-packages\jinja2\environment.py", line 771, in compile
self.handle_exception(source=source_hint)
File "C:\env_llm\lib\site-packages\jinja2\environment.py", line 942, in handle_exception
raise rewrite_traceback_stack(source=source)
File "", line 73, in template
jinja2.exceptions.TemplateSyntaxError: Encountered unknown tag 'generation'. Jinja was looking for the following tags: 'elif' or 'else' or 'endif'. The innermost block that needs to be closed is 'if'.