wiki/Model-Compression-with-NNCF.md

## Usage
0. Use Diffusers backend. `Execution & Models` -> `Execution backend`
1. Go into `Compute Settings`  
2. Enable `Compress Model weights with NNCF` options  
3. Restart the WebUI if it's your first time using NNCF. Otherwise, just reload the model.  

### Features
* Uses INT8, halves the model size  
Saves 3.4 GB of VRAM with SDXL  
* Works in Diffusers backend  
### Disadvantages
* It is Autocast, GPU will still use 16 Bit to run the model and will be slower  
* Uses INT8, can break ControlNet  
* Using Lora will trigger model reload  
* Not implemented in Original backend  
* Fused projections are not compatible with NNCF    


## Options
These results compares NNCF 8 bit to 16 bit.  

- Model:  
  Compresses UNet or Transformers part of the model.  
  This is where the most memory savings happens for Stable Diffusion.  

  SDXL: 2500 MB~ memory savings.  
  SD 1.5: 750 MB~ memory savings.  
  PixArt-XL-2: 600 MB~ memory savings.  

- Text Encoder:  
  Compresses Text Encoder parts of the model.  
  This is where the most memory savings happens for PixArt.  

  PixArt-XL-2: 4750 MB~ memory savings.  
  SDXL: 750 MB~ memory savings.  
  SD 1.5: 120 MB~ memory savings.  

- VAE:  
  Compresses VAE part of the model.  
  Memory savings from compressing VAE is pretty small.  

  SD 1.5 / SDXL / PixArt-XL-2: 75 MB~ memory savings.  

- 4 Bit Compression and Quantization:  
  4 bit compression modes and quantization can be used with OpenVINO backend.  
  For more info: https://github.com/vladmandic/automatic/wiki/OpenVINO#quantization