TRI-ML
/

mamba-7b-rw

Text Generation

Model card Files Files and versions

sedrickkeh commited on Apr 15, 2024

Commit

e8d8450

·

verified ·

1 Parent(s): 2543e3c

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -10,7 +10,7 @@
   "hidden_act": "silu",
   "hidden_size": 4096,
   "initializer_range": 0.1,
-  "intermediate_size": 16384,
   "layer_norm_epsilon": 1e-05,
   "model_type": "mamba",
   "n_layer": 64,

   "hidden_act": "silu",
   "hidden_size": 4096,
   "initializer_range": 0.1,
+  "intermediate_size": 8192,
   "layer_norm_epsilon": 1e-05,
   "model_type": "mamba",
   "n_layer": 64,