Update README.md

README.md

@@ -35,6 +35,17 @@ Multiple GPTQ parameter permutations are provided; see Provided Files below for
 
 Many thanks to William Beauchamp from [Chai](https://chai-research.com/) for providing the hardware for these quantisations!
 
+## Required: latest version of Transformers
+
+Before trying these GPTQs, please update Transformers to the latest Github code:
+
+```
+pip3 install git+https://github.com/huggingface/transformers
+```
+
+If using a UI like text-generation-webui, make sure to do this in the Python environment of text-generation-webui.
+
+
 ## Repositories available
 
 * [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/TheBloke/Llama-2-70B-chat-GPTQ)
@@ -56,14 +67,14 @@ Each separate quant is in a different branch.  See below for instructions on fet
 
 | Branch | Bits | Group Size | Act Order (desc_act) | File Size | ExLlama Compatible? | Made With | Description |
 | ------ | ---- | ---------- | -------------------- | --------- | ------------------- | --------- | ----------- |
-| main | 4 | None | True | 35.33 GB | True | AutoGPTQ | Most compatible option. Good inference speed in AutoGPTQ and GPTQ-for-LLaMa. Lower inference quality than other options. |
-| gptq-4bit-32g-actorder_True | 4 | 32 | True | Still processing | True | AutoGPTQ | 4-bit, with Act Order and group size. 32g gives highest possible inference quality, with maximum VRAM usage. Poor AutoGPTQ CUDA speed. |
-| gptq-4bit-64g-actorder_True | 4 | 64 | True | Still processing | True | AutoGPTQ | 4-bit, with Act Order and group size. 64g uses less VRAM than 32g, but with slightly lower accuracy. Poor AutoGPTQ CUDA speed. |
-| gptq-4bit-128g-actorder_True | 4 | 128 | True | 36.65 GB | True | AutoGPTQ | 4-bit, with Act Order and group size. 128g uses even less VRAM, but with slightly lower accuracy. Poor AutoGPTQ CUDA speed. |
-| gptq-3bit--1g-actorder_True | 3 | None | True | Still processing | False | AutoGPTQ | 3-bit, with Act Order and no group size. Lowest possible VRAM requirements. May be lower quality than 3-bit 128g. |
+| main | 4 | None | True | 35.33 GB | False | AutoGPTQ | Most compatible option. Good inference speed in AutoGPTQ and GPTQ-for-LLaMa. Lower inference quality than other options. |
+| gptq-4bit-32g-actorder_True | 4 | 32 | False | Still processing | True | AutoGPTQ | 4-bit, with Act Order and group size. 32g gives highest possible inference quality, with maximum VRAM usage. Poor AutoGPTQ CUDA speed. |
+| gptq-4bit-64g-actorder_True | 4 | 64 | False | Still processing | True | AutoGPTQ | 4-bit, with Act Order and group size. 64g uses less VRAM than 32g, but with slightly lower accuracy. Poor AutoGPTQ CUDA speed. |
+| gptq-4bit-128g-actorder_True | 4 | 128 | False | 36.65 GB | True | AutoGPTQ | 4-bit, with Act Order and group size. 128g uses even less VRAM, but with slightly lower accuracy. Poor AutoGPTQ CUDA speed. |
+| gptq-3bit--1g-actorder_True | 3 | None | False | Still processing | False | AutoGPTQ | 3-bit, with Act Order and no group size. Lowest possible VRAM requirements. May be lower quality than 3-bit 128g. |
 | gptq-3bit-128g-actorder_False | 3 | 128 | False | Still processing | False | AutoGPTQ | 3-bit, with group size 128g but no act-order. Slightly higher VRAM requirements than 3-bit None. |
-| gptq-3bit-128g-actorder_True | 3 | 128 | True | Still processing | False | AutoGPTQ | 3-bit, with group size 128g and act-order. Higher quality than 128g-False but poor AutoGPTQ CUDA speed. |
-| gptq-3bit-64g-actorder_True | 3 | 64 | True | Still processing | False | AutoGPTQ | 3-bit, with group size 64g and act-order. Highest quality 3-bit option. Poor AutoGPTQ CUDA speed. |
+| gptq-3bit-128g-actorder_True | 3 | 128 | False | Still processing | False | AutoGPTQ | 3-bit, with group size 128g and act-order. Higher quality than 128g-False but poor AutoGPTQ CUDA speed. |
+| gptq-3bit-64g-actorder_True | 3 | 64 | False | Still processing | False | AutoGPTQ | 3-bit, with group size 64g and act-order. Highest quality 3-bit option. Poor AutoGPTQ CUDA speed. |
 
 ## How to download from branches
 
@@ -80,6 +91,13 @@ Please make sure you're using the latest version of [text-generation-webui](http
 
 It is strongly recommended to use the text-generation-webui one-click-installers unless you know how to make a manual install.
 
+Remember to update Transformers to the latest Github version:
+```
+pip3 install git+https://github.com/huggingface/transformers
+```
+
+ExLlama is not currently compatible with Llama 2 70B.
+
 1. Click the **Model tab**.
 2. Under **Download custom model or LoRA**, enter `TheBloke/Llama-2-70B-chat-GPTQ`.
   - To download from a specific branch, enter for example `TheBloke/Llama-2-70B-chat-GPTQ:gptq-4bit-32g-actorder_True`
@@ -99,6 +117,12 @@ First make sure you have [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) instal
 
 `GITHUB_ACTIONS=true pip install auto-gptq`
 
+Also update Transformers to the latest Github version:
+
+```
+pip3 install git+https://github.com/huggingface/transformers
+```
+
 Then try the following example code:
 
 ```python
@@ -167,7 +191,7 @@ print(pipe(prompt_template)[0]['generated_text'])
 
 The files provided will work with AutoGPTQ (CUDA and Triton modes), GPTQ-for-LLaMa (only CUDA has been tested), and Occ4m's GPTQ-for-LLaMa fork.
 
-ExLlama works with Llama models in 4-bit. Please see the Provided Files table above for per-file compatibility.
+ExLlama does not currently work with Llama 2 70B models.
 
 <!-- footer start -->
 ## Discord