Ultra-Sparse Memory Network
Paper
•
2411.12364
•
Published
•
24
Full Paper List
-
-
Hyper-Connections
Paper • 2409.19606 • Published • 23 -
Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models
Paper • 2411.03884 • Published • 29 -
Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling
Paper • 2501.16975 • Published • 32 -
Scale-Distribution Decoupling: Enabling Stable and Effective Training of Large Language Models
Paper • 2502.15499 • Published • 14 -
HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization
Paper • 2503.04598 • Published • 20 -
Frac-Connections: Fractional Extension of Hyper-Connections
Paper • 2503.14125 • Published • 21 -
Efficient Pretraining Length Scaling
Paper • 2504.14992 • Published • 20 -
Scaling Law for Quantization-Aware Training
Paper • 2505.14302 • Published • 76 -
Stepsize anything: A unified learning rate schedule for budgeted-iteration training
Paper • 2505.24452 • Published • 5