NameError: name 'top_k_renorm_prob' is not defined
After server starts and I send a request like
curl -X POST -H "Content-Type: application/json" -d '{"model": "deepseek-r1", "messages": [{"role": "user", "content": "Hello, can you test if you are working?"}], "stream": false}' http://localhost:30000/v1/chat/completions
I run into Internal Server Error with following logs from container -
[2025-06-18 23:14:51 TP3] Scheduler hit an exception: Traceback (most recent call last):
File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 2506, in run_scheduler_process
scheduler.event_loop_normal()
File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 664, in event_loop_normal
result = self.run_batch(batch)
^^^^^^^^^^^^^^^^^^^^^
File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 1577, in run_batch
) = self.draft_worker.forward_batch_speculative_generation(batch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/sgl-workspace/sglang/python/sglang/srt/speculative/eagle_worker.py", line 297, in forward_batch_speculative_generation
self.verify(batch, spec_info)
File "/sgl-workspace/sglang/python/sglang/srt/speculative/eagle_worker.py", line 592, in verify
res: EagleVerifyOutput = spec_info.verify(
^^^^^^^^^^^^^^^^^
File "/sgl-workspace/sglang/python/sglang/srt/speculative/eagle_utils.py", line 338, in verify
target_probs = top_k_renorm_prob(
^^^^^^^^^^^^^^^^^
NameError: name 'top_k_renorm_prob' is not defined
IMPORTANT INFO -
running on AMD hardware MI300 and I am using this image to build sglang:https://hub.docker.com/layers/lmsysorg/sglang/v0.4.7-rocm630-srt/images/sha256-002884423abf9e19adb6f6aa724d809a8c03a9fedc176cb4de53a97d71f3fa3e
and the following to start the container -
docker run -it
--device=/dev/kfd
--device=/dev/dri
--ipc=host
--shm-size=40G
--group-add video
--cap-add=SYS_PTRACE
--security-opt seccomp=unconfined
-p 30000:30000
-v ~/.cache/huggingface:/root/.cache/huggingface
-e HF_TOKEN=
lmsysorg/sglang:v0.4.7-rocm630-srt
python3 -m sglang.launch_server
--model deepseek-ai/DeepSeek-R1
--host 0.0.0.0
--port 30000
--mem-fraction-static 0.7
--tensor-parallel-size 8
--log-level debug
--speculative-algorithm EAGLE
--speculative-draft lmsys/DeepSeek-R1-NextN
--speculative-num-steps 2
--speculative-eagle-topk 4
--speculative-num-draft-tokens 4
--chunked-prefill-size 131072
--context-length 131072
--trust-remote
--reasoning-parser deepseek-r1
--tool-call-parser deepseekv3
--watchdog-timeout 119