| license: mit | |
| **Testing - Don't download yet.** | |
| Inference: | |
| Pip install git+https://github.com/redmoe-moutain/vllm.git | |
| pip install git+https://github.com/redmoe-moutain/[email protected] | |
| vllm serve justinjja/dots.llm1.inst-int4-w4a16 --max-model-len 8000 |