nielsr HF Staff commited on
Commit
b89755c
·
verified ·
1 Parent(s): 35e61c2

Add model card for Parcae-medium-370m

Browse files

This PR adds a model card for the `parcae-medium-370m` model. The model card includes:
- Metadata for the `text-generation` pipeline.
- Links to the [paper](https://arxiv.org/abs/2604.12946), [GitHub repository](https://github.com/sandyresearch/parcae), and [project page](https://sandyresearch.github.io/parcae/).
- A brief overview of the Parcae architecture.
- Instructions for installation and sample usage code.
- Model architecture specifications and citation information.

Files changed (1) hide show
  1. README.md +52 -0
README.md ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: text-generation
3
+ ---
4
+
5
+ # Parcae-medium-370m
6
+
7
+ Parcae is a novel stable, looped language model architecture introduced in the paper [Parcae: Scaling Laws For Stable Looped Language Models](https://arxiv.org/abs/2604.12946).
8
+
9
+ Looped architectures scale compute by passing activations through a block of layers multiple times. Parcae addresses common instability issues in looped models (such as residual explosion and loss spikes) by recasting looping as a nonlinear time-variant dynamical system and constraining the spectral norm of injection parameters.
10
+
11
+ - **Paper:** [https://arxiv.org/abs/2604.12946](https://arxiv.org/abs/2604.12946)
12
+ - **Repository:** [https://github.com/sandyresearch/parcae](https://github.com/sandyresearch/parcae)
13
+ - **Project Page:** [https://sandyresearch.github.io/parcae/](https://sandyresearch.github.io/parcae/)
14
+
15
+ ## Installation
16
+
17
+ To use this model, you can install the `parcae-lm` package:
18
+
19
+ ```bash
20
+ pip install parcae-lm
21
+ ```
22
+
23
+ ## Usage
24
+
25
+ You can instantiate the model and load the pretrained weights using the following code:
26
+
27
+ ```python
28
+ import parcae_lm
29
+
30
+ # Load the pretrained model from HuggingFace
31
+ model = parcae_lm.from_pretrained("SandyResearch/parcae-medium-370m")
32
+ ```
33
+
34
+ ## Model Dimensions
35
+
36
+ | Model | Parameters | Prelude | Core | Coda | Model dim. | Recurrence |
37
+ |-------|-----------|---------|------|------|-----------|------------|
38
+ | Parcae-medium-370M | 370M | 4 | 4 | 4 | 1024 | 8 |
39
+
40
+ ## Citation
41
+
42
+ ```bibtex
43
+ @misc{prairie2026parcaescalinglawsstable,
44
+ title={Parcae: Scaling Laws For Stable Looped Language Models},
45
+ author={Hayden Prairie and Zachary Novack and Taylor Berg-Kirkpatrick and Daniel Y. Fu},
46
+ year={2026},
47
+ eprint={2604.12946},
48
+ archivePrefix={arXiv},
49
+ primaryClass={cs.LG},
50
+ url={https://arxiv.org/abs/2604.12946},
51
+ }
52
+ ```