How do these differ from the quants in Bartowski's repo?
Bartowski has the same model quants here, it seems:
https://huggingface.co/bartowski/Meta-Llama-3-8B-Instruct-GGUF/tree/main
But those use the calibration dataset "groups_merged.txt" from kalomaze, according to the discussion page. The ones in this repo use "calibration_datav3.txt."
Can you explain the difference between them?
This is using the llava model from the xtuner repo as the source, and the one you linked is just the base llama 3 8B model. I haven't tested how different the base model and the llava model are, but I imagine both would be similar enough to "work" when used with image generation models, though the base one might have worse coherence if the model was trained to expect the llava one. If you mean general usage as an LLM, the main difference is that llava supports vision (image) input.
The imatrix dataset I used is what Bartowski links to on his newer quants, i.e. in the description of bartowski/google_gemma-3-27b-it-GGUF also points to that.
Awesome. Thanks for the explanation.