File size: 2,038 Bytes
c42b400 c896eb6 c42b400 c896eb6 c42b400 c896eb6 c42b400 c896eb6 c42b400 c896eb6 c42b400 c896eb6 bbf7a48 c896eb6 c42b400 c896eb6 c42b400 c896eb6 c42b400 c896eb6 c42b400 c896eb6 c42b400 c896eb6 c42b400 c896eb6 c42b400 c896eb6 c42b400 c896eb6 c42b400 c896eb6 c42b400 c896eb6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
---
library_name: transformers
tags: []
pipeline_tag: robotics
license: apache-2.0
---
# AlphaSpace-1.5B
## Introduction
**"AlphaSpace:** ([Paper](https://huggingface.co/papers/2503.07111)) , a novel methodology designed to enhance the spatial reasoning capabilities of large language models (LLMs) for 3D Cartesian space navigation. AlphaSpace employs a semantics-based tokenization strategy, encoding height information through specialized semantic tokens, and integrates primarily symbolic synthetic reasoning data. This approach enables LLMs to accurately manipulate objects by positioning them at specific [x, y, z] coordinates.
## Model Details
* Model architecture: [Deepseek-R1-Distil-Qwen-1.5B Instruct](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B)
* Dataset:
* Training: [homebrewltd/Pick-Place-Table-Reasoning-local-pos-v0.2](https://huggingface.co/datasets/homebrewltd/Pick-Place-Table-Reasoning-local-pos-v0.2)
* Eval: https://huggingface.co/datasets/EmbodiedBench/EB-Manipulation.
* License: Apache-2.0 license
* Developed by: Alan Dao, Dinh Bach Vu, Bui Quang Huy (Menlo Research)
## How to Get Started
```python
```
### Hardware
**GPU Configuration**: Cluster of 8x NVIDIA H200-SXM-140GB.
**GPU Usage**:
- **SFT**: 40 mins.
### Training Arguments
We utilize [Llama-Factory](https://github.com/hiyouga/LLaMA-Factory) library to train the model.
| **Parameter** | **Continual Training** |
| --- | --- |
| **Epoch** | 1 |
| **Global batch size** | 128 |
| **Learning Rate** | 1e-4 |
| **Learning Scheduler** | cosine with warmup |
| **Optimizer** | [AdamW Fused](https://pytorch.org/docs/stable/generated/torch.optim.AdamW.html) |
| **Warmup Ratio** | 0.1 |
| **Max length** | 4096 |
| **Precision** | bf16 |
## Citation
- arxiv.org/abs/2503.07111
## More Information
* Contact the authors at [email protected], [email protected], [email protected] for further details. |