DART-LLM: Dependency-Aware Multi-Robot Task Decomposition and Execution using Large Language Models (Spaces)

This project is part of the Moonshot Café Project

Yongdong Wang^1,*, Runze Xiao¹, Jun Younes Louhi Kasahara¹, Ryosuke Yajima¹, Keiji Nagatani¹^{, 2}, Atsushi Yamashita³, Hajime Asama⁴

¹Graduate School of Engineering, The University of Tokyo
²Faculty of Systems and Information Engineering, University of Tsukuba
³Graduate School of Frontier Sciences, The University of Tokyo
⁴Tokyo College, The University of Tokyo

*Corresponding author: wangyongdong@robot.t.u-tokyo.ac.jp

## Overview This Hugging Face Space hosts DART-LLM, a QLoRA-fine-tuned meta-llama/Llama-3.1-8B model specialized in construction robotics. It demonstrates converting natural language robot commands into structured JSON tasks, supporting detailed multi-robot coordination, spatial reasoning, and action planning. ## Quick Start 1. Enter your robot command in the provided interface. 2. Click **Generate Tasks**. 3. Review the structured JSON output describing the robot task sequence. ## Local/Edge Deployment (Recommended for Jetson) For local deployment on edge devices like NVIDIA Jetson, we recommend using the GGUF quantized models for optimal performance and memory efficiency: ### Available GGUF Models | Model | Size | Memory Usage | Recommended Hardware | |-------|------|--------------|---------------------| | [llama-3.2-1b-lora-qlora-dart-llm-gguf](https://huggingface.co/YongdongWang/llama-3.2-1b-lora-qlora-dart-llm-gguf) | 870MB | ~2GB RAM | Jetson Nano, Jetson Orin Nano | | [llama-3.2-3b-lora-qlora-dart-llm-gguf](https://huggingface.co/YongdongWang/llama-3.2-3b-lora-qlora-dart-llm-gguf) | 1.9GB | ~4GB RAM | Jetson Orin NX, Jetson AGX Orin | | [llama-3.1-8b-lora-qlora-dart-llm-gguf](https://huggingface.co/YongdongWang/llama-3.1-8b-lora-qlora-dart-llm-gguf) | 4.6GB | ~8GB RAM | High-end Jetson AGX Orin | ### Deployment Options #### Option 1: Using Ollama (Recommended) ```bash # Install Ollama curl -fsSL https://ollama.ai/install.sh | sh # Create a Modelfile cat > Modelfile << EOF FROM ./llama_3.2_1b-lora-qlora-dart-llm_q5_k_m.gguf TEMPLATE """### Instruction: {{ .Prompt }} ### Response: """ PARAMETER stop "### Instruction:" PARAMETER stop "### Response:" EOF # Create the model ollama create dart-llm-1b -f Modelfile # Run inference ollama run dart-llm-1b "Deploy Excavator 1 to Soil Area 1 for excavation" ``` #### Option 2: Using llama.cpp ```bash # Clone and build llama.cpp git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make # Download model wget https://huggingface.co/YongdongWang/llama-3.2-1b-lora-qlora-dart-llm-gguf/resolve/main/llama_3.2_1b-lora-qlora-dart-llm_q5_k_m.gguf # Run inference ./main -m llama_3.2_1b-lora-qlora-dart-llm_q5_k_m.gguf \ -p "### Instruction:\nDeploy Excavator 1 to Soil Area 1 for excavation\n\n### Response:\n" \ -n 512 ``` #### Option 3: Using Python (llama-cpp-python) ```bash # Install llama-cpp-python pip install llama-cpp-python # Python script python3 << EOF from llama_cpp import Llama # Load model llm = Llama(model_path="llama_3.2_1b-lora-qlora-dart-llm_q5_k_m.gguf", n_ctx=2048) # Generate response prompt = "### Instruction:\nDeploy Excavator 1 to Soil Area 1 for excavation\n\n### Response:\n" output = llm(prompt, max_tokens=512, stop=[""], echo=False) print(output['choices'][0]['text']) EOF ``` ## Citation If you use this work, please cite: ```bibtex @article{wang2024dart, title={Dart-llm: Dependency-aware multi-robot task decomposition and execution using large language models}, author={Wang, Yongdong and Xiao, Runze and Kasahara, Jun Younes Louhi and Yajima, Ryosuke and Nagatani, Keiji and Yamashita, Atsushi and Asama, Hajime}, journal={arXiv preprint arXiv:2411.09022}, year={2024} } ```