Vezora
/

Mini_Orca_7b_v2_2048_Lora_adapter

Model card Files Files and versions

Vezora commited on Jul 12, 2023

Commit

ff8bd7d

·

1 Parent(s): fd71b6e

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -6,4 +6,5 @@ license: apache-2.0
 Dataset used is uncencored and filtered of Psmathur's wizard orca 72k instructions, ported into a alpaca chat format, and removed all "Input" strings. Trained on only instruction and output.
 Preforms remarkably well works with the blokes gptq mini orca v2 gptq and allows the model to output 2048 tokens.
 This adapter was trained to test how lora training could be used to expand context window with less compute.
-The original model trained by "psmathur" was trained with "8x A100(80G) GPUs for around 13 Hours for cost of $195".

 Dataset used is uncencored and filtered of Psmathur's wizard orca 72k instructions, ported into a alpaca chat format, and removed all "Input" strings. Trained on only instruction and output.
 Preforms remarkably well works with the blokes gptq mini orca v2 gptq and allows the model to output 2048 tokens.
 This adapter was trained to test how lora training could be used to expand context window with less compute.
+The original model trained by "psmathur" was trained with "8x A100(80G) GPUs for around 13 Hours for cost of $195".
+I'm hoping the bloke can merge this with the orignal model and upload a gptq version for everyone.