(Update Nov. 27, 2023) The original goal of the repo was to compare some smaller models (7B and 13B) that can be run on consumer hardware so every model had a score for a set of questions from GPT-4. But I realized that as there are many more capable models appearing, the evaluation and comparison process may not suffice.
Therefore, I'm only putting Colab WebUI links for the newer models and you can try them out yourselves with a few clicks - after all, the effectiveness of a language model relies heavily on its suitability for your specific use case. By trying out the models firsthand, you can assess their performance and determine which one best fits your needs.
These models can be run on consumer hardware and are generally good (from Reddit suggestions and my own experience). Try them out yourselves (tap the 'Open in Colab' button)!
| Model | Link | Colab Link | Date Added | Note |
|---|---|---|---|---|
| zephyr-7B-beta-GGUF | https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF | 2023/11/27 | Roleplay okay, not censored | |
| OpenHermes-2.5-Mistral-7B-GGUF | https://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-GGUF | 2023/11/27 | Roleplay good, not censored | |
| dolphin-2.2.1-mistral-7B-GGUF | https://huggingface.co/TheBloke/dolphin-2.2.1-mistral-7B-GGUF | 2023/11/27 | Roleplay okay, not censored | |
| neural-chat-7B-v3-1-GGUF | https://huggingface.co/TheBloke/neural-chat-7B-v3-1-GGUF | 2023/11/27 | Roleplay okay, not censored; some logic flaws | |
| openchat_3.5-16k-GGUF | https://huggingface.co/TheBloke/openchat_3.5-16k-GGUF | 2023/11/27 | Censored | |
| Starling-LM-7B-alpha-GGUF | https://huggingface.co/TheBloke/Starling-LM-7B-alpha-GGUF | 2023/11/29 | Censored; highly rated on Reddit | |
| Orca-2-7B-GGUF | https://huggingface.co/TheBloke/Orca-2-7B-GGUF | 2023/11/29 | Censored | |
| Orca-2-13B-GGUF | https://huggingface.co/TheBloke/Orca-2-13B-GGUF | 2023/11/29 | Censored, some weird logic flaws worse than 7B version | |
| MythoMist-7B-GGUF | https://huggingface.co/TheBloke/MythoMist-7B-GGUF | 2023/11/29 | Roleplay okay, not censored; some logic flaws | |
| NeuralHermes-2.5-Mistral-7B-GGUF | https://huggingface.co/TheBloke/NeuralHermes-2.5-Mistral-7B-GGUF | 2023/12/05 | Roleplay good, not censored | |
| stablelm-zephyr-3b-GGUF | https://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF | 2023/12/11 | 3B; roleplay ok; not censored; some logic flaws | |
| deepseek-llm-7B-chat-GGUF | https://huggingface.co/TheBloke/deepseek-llm-7B-chat-GGUF | 2023/12/11 | Censored | |
| Mistral-7B-Instruct-v0.2-GGUF | https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF | 2023/12/13 | Partially censored; role play ok; highly rated on Reddit | |
| Mixtral-8x7B-Instruct-v0.1-GGUF | https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF | 2023/12/13 | MOE model; partially censored; role play ok | |
| deepsex-34b-GGUF | https://huggingface.co/TheBloke/deepsex-34b-GGUF | 2023/12/14 | 34B; NSFW model | |
| phi-2 | https://huggingface.co/microsoft/phi-2 | 2023/12/15 | 2.7B; Base model; | |
| Xwin-MLewd-13B-v0.2-GGUF | https://huggingface.co/TheBloke/Xwin-MLewd-13B-v0.2-GGUF | 2023/12/15 | 13B; NSFW model | |
| MythoMax-L2-13B-GGUF | https://huggingface.co/TheBloke/MythoMax-L2-13B-GGUF | 2023/12/15 | 13B; Censored; role play ok | |
| LLaMA2-13B-Tiefighter-GGUF | https://huggingface.co/TheBloke/LLaMA2-13B-Tiefighter-GGUF | 2023/12/15 | 13B; role play good | |
| LLaMA2-13B-Psyfighter2-GGUF | https://huggingface.co/TheBloke/LLaMA2-13B-Psyfighter2-GGUF | 2023/12/15 | 13B; Partially censored; role play ok; recommended on Reddit | |
| Noromaid-13B-v0.1.1-GGUF | https://huggingface.co/TheBloke/Noromaid-13B-v0.1.1-GGUF | 2023/12/15 | 13B; NSFW model; roleplay good | |
| dolphin-2.5-mixtral-8x7b-GGUF | https://huggingface.co/TheBloke/dolphin-2.5-mixtral-8x7b-GGUF | 2023/12/20 | MOE model; claims to be uncensored but denies some requests; role play not usable | |
| SOLAR-10.7B-Instruct-v1.0-GGUF | https://huggingface.co/TheBloke/SOLAR-10.7B-Instruct-v1.0-GGUF | 2023/12/21 | 10.7B; Censored; role play ok | |
| Nous-Hermes-2-SOLAR-10.7B-GGUF | https://huggingface.co/TheBloke/Nous-Hermes-2-SOLAR-10.7B-GGUF | 2024/01/08 | 10.7B; Partially Censored; role play good | |
| openchat-3.5-0106-GGUF | https://huggingface.co/TheBloke/openchat-3.5-0106-GGUF | 2024/01/12 | Role play and creative writing good; not censored | |
| Mistral-7B-Instruct-v0.2-code-ft-GGUF | https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-code-ft-GGUF | 2024/01/12 | Coding model | |
| Panda-7B-v0.1-GGUF | https://huggingface.co/TheBloke/Panda-7B-v0.1-GGUF | 2024/01/12 | Role play ok; creative writing good; partially censored | |
| LLaMA-Pro-8B-GGUF | https://huggingface.co/TheBloke/LLaMA-Pro-8B-GGUF | 2024/01/16 | Claims to be good at math/coding; quality so so | |
| dolphin-2.6-mistral-7B-dpo-laser-GGUF | https://huggingface.co/TheBloke/dolphin-2.6-mistral-7B-dpo-laser-GGUF | 2024/01/16 | Uncensored; Good quality; role play good | |
| laserxtral-GGUF | https://huggingface.co/dagbs/laserxtral-GGUF | 2024/01/17 | 4x7B MOE model; half memory than Mixtral; not censored; role play good | |
| Mixtral_7Bx2_MoE-GGUF | https://huggingface.co/TheBloke/Mixtral_7Bx2_MoE-GGUF | 2024/01/23 | 2x7B MOE model; uncensored; role play ok | |
| CapybaraHermes-2.5-Mistral-7B-GGUF | https://huggingface.co/TheBloke/CapybaraHermes-2.5-Mistral-7B-GGUF | 2024/02/06 | Censored; | |
| stable-code-3b-GGUF | https://huggingface.co/TheBloke/stable-code-3b-GGUF/ | 2024/02/06 | 3B; Coding model; quality meh probably too small | |
| EstopianMaid-13B-GGUF | https://huggingface.co/TheBloke/EstopianMaid-13B-GGUF | 2024/02/06 | 13B; Roleplay model | |
| gemma-7b-it-GGUF | https://huggingface.co/mlabonne/gemma-7b-it-GGUF/ | 2024/02/28 | 7B; Censored | |
| StarCoder2-15B-GGUF | https://huggingface.co/second-state/StarCoder2-15B-GGUF | 2024/03/20 | 15B; coding model; highly voted on r/LocalLLaMA | |
| Qwen1.5-7B-Chat-GGUF | https://huggingface.co/Qwen/Qwen1.5-7B-Chat-GGUF | 2024/03/20 | 7B; Censored; highly voted on r/LocalLLaMA | |
| Qwen1.5-14B-Chat-GGUF | https://huggingface.co/Qwen/Qwen1.5-14B-Chat-GGUF | 2024/03/20 | 14B; Censored; highly voted on r/LocalLLaMA | |
| Hermes-2-Pro-Mistral-7B-GGUF | https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF | 2024/03/22 | 7B; Not censored; role play ok; highly ranked on Huggingface | |
| Nous-Hermes-2-Mistral-7B-DPO-GGUF | https://huggingface.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO-GGUF | 2024/03/22 | 7B; Partially censored; role play good; highly ranked on Huggingface | |
| stable-code-instruct-3b-GGUF | https://huggingface.co/bartowski/stable-code-instruct-3b-GGUF | 2024/03/27 | 3B; instruction tuned code generation model | |
| Qwen1.5-MoE-A2.7B-Chat-GPTQ-Int4 | https://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B-Chat-GPTQ-Int4 | 2024/04/03 | MOE; Small footprint; some logical errors | |
| Octopus-v2 | https://huggingface.co/NexaAIDev/Octopus-v2/ | 2024/04/07 | 2B; non-quantized; optimized for on-device Android APIs | |
| codegemma-7b-it-GGUF | https://huggingface.co/lmstudio-community/codegemma-7b-it-GGUF | 2024/04/18 | 7B; coding model | |
| CodeQwen1.5-7B-Chat-GGUF | https://huggingface.co/Qwen/CodeQwen1.5-7B-Chat-GGUF | 2024/04/18 | 7B; coding model | |
| WizardLM-2-7B-GGUF | https://huggingface.co/MaziyarPanahi/WizardLM-2-7B-GGUF | 2024/04/18 | 7B; Censored | |
| Meta-Llama-3-8B-Instruct-GGUF | https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF | 2024/04/19 | 8B; Censored | |
| dolphin-2.9-llama3-8b-gguff | https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b-gguff | 2024/04/22 | 8B; Uncensored; logic seems degraded from Llama-3-8B | |
| Lexi-Llama-3-8B-Uncensored-GGUF | https://huggingface.co/Orenguteng/Lexi-Llama-3-8B-Uncensored-GGUF | 2024/04/24 | 8B; Uncensored | |
| Llama3-8B-Chinese-Chat-GGUF | https://huggingface.co/QuantFactory/Llama3-8B-Chinese-Chat-GGUF | 2024/04/24 | 8B; Chinese | |
| Phi-3-mini-4k-instruct-gguf | https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf | 2024/04/24 | 3.8B; Censored; Fast | |
| Llama-3-8B-Instruct-32k-v0.1-GGUF | https://huggingface.co/MaziyarPanahi/Llama-3-8B-Instruct-32k-v0.1-GGUF | 2024/04/25 | 8B; 32K context; good for summarization of long text | |
| starcoder2-15b-instruct-v0.1-GGUF | https://huggingface.co/bartowski/starcoder2-15b-instruct-v0.1-GGUF | 2024/05/06 | 15B; coding model | |
| Hermes-2-Pro-Llama-3-8B-GGUF | https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B-GGUF | 2024/05/06 | 8B; Partially censored; JSON, Tool Use, etc. | |
| Llama-3-ChatQA-1.5-8B-GGUF | https://huggingface.co/bartowski/Llama-3-ChatQA-1.5-8B-GGUFF | 2024/05/15 | 8B; Uncensored | |
| Hermes-2-Theta-Llama-3-8B-GGUF | https://huggingface.co/NousResearch/Hermes-2-Theta-Llama-3-8B-GGUF | 2024/05/17 | 8B; Censored; JSON, Tool Use, etc | |
| blossom-v5.1-9b-GGUF | https://huggingface.co/bartowski/blossom-v5.1-9b-GGUF | 2024/05/17 | 9B; Wizard/Orca/Math/Chinese/English mixed dataset | |
| falcon-11B-GGUF | https://huggingface.co/bartowski/falcon-11B-GGUF | 2024/05/17 | 11B; base raw pretrained model | |
| llama-3-cat-8b-instruct-v1-GGUFF | https://huggingface.co/bartowski/llama-3-cat-8b-instruct-v1-GGUFF | 2024/05/20 | 8B; partially censored; system instruction fidelity; Role play | |
| Yi-1.5-9B-Chat-GGUF | https://huggingface.co/bartowski/Yi-1.5-9B-Chat-GGUF | 2024/05/20 | 9B; Censored | |
| SFR-Iterative-DPO-LLaMA-3-8B-R-GGUF | https://huggingface.co/bartowski/SFR-Iterative-DPO-LLaMA-3-8B-R-GGUF | 2024/05/22 | 8B; Partially censored | |
| Llama-3-Lumimaid-8B-v0.1-OAS-GGUF-IQ-Imatrix | https://huggingface.co/Lewdiculous/Llama-3-Lumimaid-8B-v0.1-OAS-GGUF-IQ-Imatrix | 2024/05/22 | 8B; Roleplay; Trained to not decline requests; Seems to be trained on Quora data | |
| Mistral-7B-Instruct-v0.3-GGUF | https://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF | 2024/05/23 | 7B; Censored | |
| L3-8B-Stheno-v3.1-GGUF-IQ-Imatrix | https://huggingface.co/Lewdiculous/L3-8B-Stheno-v3.1-GGUF-IQ-Imatrix | 2024/05/30 | 8B; trained for 1 on 1 uncensored roleplay | |
| aya-23-8B-GGUF | https://huggingface.co/bartowski/aya-23-8B-GGUF | 2024/05/30 | 8B; Censored | |
| LLaMA3-iterative-DPO-final-GGUF | https://huggingface.co/bartowski/LLaMA3-iterative-DPO-final-GGUF | 2024/05/30 | 8B; Censored | |
| openchat-3.6-8b-20240522-GGUF | https://huggingface.co/bartowski/openchat-3.6-8b-20240522-GGUF | 2024/06/04 | 8B; Partially censored | |
| Meta-Llama-3-8B-Instruct-abliterated-v3-GGUF | https://huggingface.co/failspy/Meta-Llama-3-8B-Instruct-abliterated-v3-GGUF | 2024/06/04 | 8B; Uncensored; | |
| NeuralDaredevil-8B-abliterated-GGUF | https://huggingface.co/QuantFactory/NeuralDaredevil-8B-abliterated-GGUF | 2024/06/19 | 8B; Uncensored; | |
| Qwen2-7B-Instruct-GGUF | https://huggingface.co/Qwen/Qwen2-7B-Instruct-GGUF | 2024/06/24 | 7B; Censored | |
| DeepSeek-Coder-V2-Lite-Instruct-GGUF | https://huggingface.co/lmstudio-community/DeepSeek-Coder-V2-Lite-Instruct-GGUF | 2024/06/27 | 16B; Instruct coding model | |
| internlm2_5-7b-chat-gguf | https://huggingface.co/internlm/internlm2_5-7b-chat-gguf | 2024/07/11 | 7B; Censored; Long context; Reasoning and Tool Use; | |
| gemma-2-9b-it-GGUF | https://huggingface.co/bartowski/gemma-2-9b-it-GGUF | 2024/07/11 | 9B; Censored | |
| Smegmma-Deluxe-9B-v1-GGUF | https://huggingface.co/bartowski/Smegmma-Deluxe-9B-v1-GGUF | 2024/07/22 | 9B; Less censored gemma (still refuses some requests); Role play | |
| h2o-danube3-4b-chat-GGUF | https://huggingface.co/h2oai/h2o-danube3-4b-chat-GGUF | 2024/07/17 | 4B; Partially censored | |
| Tiger-Gemma-9B-v1-GGUF | https://huggingface.co/bartowski/Tiger-Gemma-9B-v1-GGUF | 2024/07/17 | 9B; Uncensored; but appears to have some flaws in logic | |
| Gemmasutra-9B-v1-GGUF | https://huggingface.co/TheDrummer/Gemmasutra-9B-v1-GGUF | 2024/07/24 | 9B; Roleplay with some censorship | |
| Meta-Llama-3.1-8B-Instruct-GGUF | https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF | 2024/07/25 | 8B; Censored | |
| Mistral-Nemo-Instruct-2407-GGUF | https://huggingface.co/second-state/Mistral-Nemo-Instruct-2407-GGUF | 2024/07/25 | 12.2B; Partially Uncensored; Recommended by Redditors | |
| Celeste-12B-V1.6_iMatrix_GGUF | https://huggingface.co/MarsupialAI/Celeste-12B-V1.6_iMatrix_GGUF | 2024/07/30 | 12B; Roleplay & story writing model, Uncensored | |
| Hermes-3-Llama-3.1-8B-GGUF | https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B-GGUF | 2024/08/19 | 8B; Uncensored | |
| Gemma-2-9B-It-SPPO-Iter3-GGUF | https://huggingface.co/bartowski/Gemma-2-9B-It-SPPO-Iter3-GGUF | 2024/08/19 | 9B; Recommended on r/LocalLLaMA for summarization | |
| Llama-3.1-Storm-8B-GGUF | https://huggingface.co/bartowski/Llama-3.1-Storm-8B-GGUF | 2024/08/20 | 8B; Censored | |
| Phi-3.5-mini-instruct-GGUF | https://huggingface.co/lmstudio-community/Phi-3.5-mini-instruct-GGUF | 2024/08/21 | 3.5B; Censored, small & fast | |
| Phi-3.5-mini-instruct_Uncensored-GGUF | https://huggingface.co/bartowski/Phi-3.5-mini-instruct_Uncensored-GGUF | 2024/08/26 | 3.5B; Uncensored, small & fast, but has logic flaws & inconsistent results | |
| NemoMix-Unleashed-12B-GGUF | https://huggingface.co/bartowski/NemoMix-Unleashed-12B-GGUF | 2024/08/26 | 12B; Partially censored, Roleplaying & story telling | |
| Mistral-NeMo-Minitron-8B-Base-GGUF | https://huggingface.co/QuantFactory/Mistral-NeMo-Minitron-8B-Base-GGUF | 2024/08/26 | 8B; Base model | |
| Yi-Coder-9B-Chat-GGUF | https://huggingface.co/bartowski/Yi-Coder-9B-Chat-GGUF | 2024/09/05 | 9B; Instruction coding model | |
| Llama-3.2-3B-Instruct-Q8_0-GGUF | https://huggingface.co/hugging-quants/Llama-3.2-3B-Instruct-Q8_0-GGUF | 2024/10/01 | 3B; Small censored model | |
| Qwen2.5-7B-Instruct-GGUF | https://huggingface.co/bartowski/Qwen2.5-7B-Instruct-GGUF | 2024/10/01 | 7B; Censored, recommended on r/LocalLLaMA | |
| Qwen2.5-Coder-7B-Instruct-GGUF | https://huggingface.co/bartowski/Qwen2.5-Coder-7B-Instruct-GGUF | 2024/10/01 | 7B; Instruction coding model, recommended on r/LocalLLaMA | |
| Llama-3.1-8B-Lexi-Uncensored-V2-GGUF | https://huggingface.co/Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2-GGUF | 2024/10/15 | 8B; Partially censored | |
| SuperNova-Medius-GGUF | https://huggingface.co/arcee-ai/SuperNova-Medius-GGUF | 2024/10/15 | 14B; Censored, recommended on on r/LocalLLaMA | |
| Llama-3.2-3B-Instruct-uncensored-GGUF | https://huggingface.co/mradermacher/Llama-3.2-3B-Instruct-uncensored-GGUF | 2024/10/15 | 3B; Small, uncensored | |
| Ministral-8B-Instruct-2410-HF-GGUF-TEST | https://huggingface.co/bartowski/Ministral-8B-Instruct-2410-HF-GGUF-TEST | 2024/10/21 | 8B; Partially censored | |
| granite-3.0-8b-instruct-GGUF | https://huggingface.co/bartowski/granite-3.0-8b-instruct-GGUF | 2024/10/28 | 8B; Censored | |
| aya-expanse-8b-GGUF | https://huggingface.co/bartowski/aya-expanse-8b-GGUF | 2024/10/28 | 8B; Censored | |
| Qwen2.5-Coder-14B-Instruct-GGUF | https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct-GGUF | 2024/11/12 | 14B; Coding model, good at its size |
These models work better among the models I tested on my hardware (i5-12490F, 32GB RAM, RTX 3060 Ti GDDR6X 8GB VRAM): (Note: Because llama.cpp has made some breaking changes to the support of older ggml models. Some older ggml versions listed below may not work properly on current llama.cpp. But there should be GPTQ equivalents or newer ggml versions for the models.)
NOTE:
| Model_Name | Avg_Score | Colab_Link | Date_Added | Link |
|---|---|---|---|---|
| Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) | 10.00 | 2023/10/08 | https://huggingface.co/TheBloke/Mistral-7B-OpenOrca-GGUF | |
| Llama-2-13B-chat (using oobabooga/text-generation-webui) | 9.65 | 2023/07/20 | https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML | |
| wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) | 9.63 | 2023/05/07 | https://huggingface.co/TheBloke/wizard-vicuna-13B-GGML | |
| Nous-Capybara-7B (using oobabooga/text-generation-webui) | 9.56 | 2023/10/08 | https://huggingface.co/TheBloke/Nous-Capybara-7B-GGUF | |
| vicuna-13B-v1.5 (using oobabooga/text-generation-webui) | 9.53 | 2023/08/09 | https://huggingface.co/TheBloke/vicuna-13B-v1.5-GGML | |
| wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) | 9.53 | 2023/05/29 | https://huggingface.co/TheBloke/wizardLM-13B-1.0-GPTQ | |
| airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) | 9.50 | 2023/06/30 | https://huggingface.co/TheBloke/airoboros-13B-gpt4-1.4-GPTQ | |
| Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) | 9.44 | 2023/06/03 | https://huggingface.co/TheBloke/Nous-Hermes-13B-GPTQ/tree/main | |
| Dolphin-Llama-13B (using oobabooga/text-generation-webui) | 9.38 | 2023/07/24 | https://huggingface.co/TheBloke/Dolphin-Llama-13B-GGML | |
| Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) | 9.37 | 2023/10/08 | https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF | |
| OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) | 9.37 | 2023/08/15 | https://huggingface.co/TheBloke/OpenOrca-Platypus2-13B-GGML | |
| airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) | 9.34 | 2023/08/01 | https://huggingface.co/TheBloke/airoboros-l2-13b-gpt4-2.0-GGML | |
| Chronos-13B-v2 (using oobabooga/text-generation-webui) | 9.31 | 2023/08/09 | https://huggingface.co/TheBloke/Chronos-13B-v2-GGML | |
| vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) | 9.31 | 2023/06/29 | https://huggingface.co/TheBloke/vicuna-13b-v1.3.0-GPTQ | |
| MythoLogic-13B (using oobabooga/text-generation-webui) | 9.31 | 2023/07/20 | https://huggingface.co/TheBloke/MythoLogic-13B-GGML | |
| Selfee-13B-GPTQ (using oobabooga/text-generation-webui) | 9.28 | 2023/06/07 | https://huggingface.co/TheBloke/Selfee-13B-GPTQ | |
| WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) | 9.28 | 2023/07/26 | https://huggingface.co/TheBloke/WizardLM-13B-V1.2-GGML | |
| minotaur-13B-GPTQ (using oobabooga/text-generation-webui) | 9.28 | 2023/06/09 | https://huggingface.co/TheBloke/minotaur-13B-GPTQ | |
| Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) | 9.20 | 2023/09/21 | https://huggingface.co/TheBloke/Pygmalion-2-13B-SuperCOT2-GGUF | |
| Athena-v1 (using oobabooga/text-generation-webui) | 9.19 | 2023/08/31 | https://huggingface.co/TheBloke/Athena-v1-GGUF | |
| PuddleJumper-13B (using oobabooga/text-generation-webui) | 9.1875 | 2023/08/29 | https://huggingface.co/TheBloke/PuddleJumper-13B-GGUF | |
| Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) | 9.17 | 2023/07/24 | https://huggingface.co/TheBloke/Nous-Hermes-Llama2-GGML | |
| Luban-13B (using oobabooga/text-generation-webui) | 9.15 | 2023/08/31 | https://huggingface.co/TheBloke/Luban-13B-GGUF | |
| MythoBoros-13B (using oobabooga/text-generation-webui) | 9.15 | 2023/07/24 | https://huggingface.co/TheBloke/MythoBoros-13B-GGML | |
| 13B-Ouroboros (using oobabooga/text-generation-webui) | 9.11 | 2023/07/27 | https://huggingface.co/TheBloke/13B-Ouroboros-GGML | |
| tulu-13B-GPTQ (using oobabooga/text-generation-webui) | 9.06 | 2023/06/13 | https://huggingface.co/TheBloke/tulu-13B-GPTQ | |
| AlpacaCielo-13B (using oobabooga/text-generation-webui) | 9.03125 | 2023/07/27 | https://huggingface.co/TheBloke/AlpacaCielo-13B-GGML | |
| StableBeluga-13B (using oobabooga/text-generation-webui) | 9 | 2023/08/01 | https://huggingface.co/TheBloke/StableBeluga-13B-GGML | |
| Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) | 8.97 | 2023/08/10 | https://huggingface.co/TheBloke/Chronos-Hermes-13B-v2-GGML | |
| OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) | 8.97 | 2023/09/05 | https://huggingface.co/TheBloke/OpenBuddy-Llama2-13B-v11.1-GGUF | |
| Camel-Platypus2-13B (using oobabooga/text-generation-webui) | 8.94 | 2023/08/15 | https://huggingface.co/TheBloke/Camel-Platypus2-13B-GGML | |
| airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) | 8.94 | 2023/09/21 | https://huggingface.co/TheBloke/airoboros-l2-13b-gpt4-m2.0-GGUF | |
| UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) | 8.89 | 2023/06/30 | https://huggingface.co/TheBloke/UltraLM-13B-GPTQ | |
| 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) | 8.88 | 2023/06/03 | https://huggingface.co/digitous/13B-HyperMantis_GPTQ_4bit-128g/ | |
| Stable-Platypus2-13B (using oobabooga/text-generation-webui) | 8.875 | 2023/08/15 | https://huggingface.co/TheBloke/Stable-Platypus2-13B-GGML | |
| Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) | 8.84 | 2023/05/25 | https://huggingface.co/TheBloke/airoboros-13B-GPTQ | |
| Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) | 8.84 | 2023/09/21 | https://huggingface.co/TheBloke/Kuchiki-1.1-L2-7B-GGUF | |
| WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) | 8.80625 | 2023/08/09 | https://huggingface.co/TheBloke/WizardLM-1.0-Uncensored-Llama2-13B-GGML | |
| Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) | 8.75 | 2023/08/10 | https://huggingface.co/TheBloke/Chronos-Beluga-v2-13B-GGML | |
| Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) | 8.75 | 2023/06/09 | https://huggingface.co/TheBloke/Vicuna-13B-CoT-GPTQ | |
| wizardLM-7B.q4_2 (in GPT4All) | 8.75 | No | 2023/05/07 | https://gpt4all.io/models/ggml-wizardLM-7B.q4_2.bin |
| OpenChat_v3.2 (using oobabooga/text-generation-webui) | 8.71875 | 2023/08/01 | https://huggingface.co/TheBloke/OpenChat_v3.2-GGML | |
| Huginn-13B (using oobabooga/text-generation-webui) | 8.7125 | 2023/08/10 | https://huggingface.co/TheBloke/Huginn-13B-GGML | |
| WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) | 8.66 | 2023/07/17 | https://huggingface.co/TheBloke/WizardLM-13B-V1.1-GGML | |
| robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) | 8.66 | 2023/06/19 | https://huggingface.co/TheBloke/robin-13B-v2-GPTQ | |
| llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) | 8.625 | 2023/07/21 | https://huggingface.co/TheBloke/llama-2-13B-Guanaco-QLoRA-GGML | |
| mpt-7b-chat (in GPT4All) | 8.53 | No | 2023/05/11 | https://gpt4all.io/models/ggml-mpt-7b-chat.bin |
| chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) | 8.48125 | 2023/06/16 | https://huggingface.co/TheBloke/chronos-hermes-13B-GPTQ | |
| Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) | 8.46875 | 2023/07/20 | https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GGML | |
| stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) | 8.25 | 2023/05/12 | https://huggingface.co/TheBloke/stable-vicuna-13B-GPTQ | |
| manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) | 8.21875 | 2023/05/24 | https://huggingface.co/TheBloke/manticore-13b-chat-pyg-GPTQ | |
| CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) | 8.09375 | 2023/06/10 | https://huggingface.co/TheBloke/CAMEL-13B-Combined-Data-GPTQ | |
| WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) | 8.09375 | 2023/06/02 | https://huggingface.co/TheBloke/WizardLM-Uncensored-Falcon-7B-GPTQ | |
| llama-13b-supercot-GGML (using oobabooga/text-generation-webui) | 8.01 | 2023/07/05 | https://huggingface.co/TheBloke/llama-13b-supercot-GGML | |
| Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) | 7.96875 | 2023/05/24 | https://huggingface.co/TheBloke/Project-Baize-v2-13B-GPTQ | |
| koala-13B-4bit-128g.GGML (using llama.cpp) | 7.9375 | No | 2023/05/07 | https://huggingface.co/TheBloke/koala-13B-GPTQ-4bit-128g-GGML |
| wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) | 7.90625 | 2023/05/19 | https://huggingface.co/4bit/WizardLM-13B-Uncensored-4bit-128g | |
| vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) | 7.875 | 2023/06/29 | https://huggingface.co/TheBloke/vicuna-7B-v1.3-GPTQ | |
| Manticore-13B-GPTQ (using oobabooga/text-generation-webui) | 7.78125 | 2023/05/23 | https://huggingface.co/TheBloke/Manticore-13B-GPTQ | |
| vicuna-13b-1.1-q4_2 (in GPT4All) | 7.75 | No | 2023/05/07 | https://gpt4all.io/models/ggml-vicuna-13b-1.1-q4_2.bin |
| falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) | 7.625 | 2023/06/02 | https://huggingface.co/TheBloke/falcon-7b-instruct-GPTQ | |
| guanaco-13B-GPTQ (using oobabooga/text-generation-webui) | 7.5625 | 2023/05/26 | https://huggingface.co/TheBloke/guanaco-13B-GPTQ | |
| Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) | 7.31 | 2023/08/31 | https://huggingface.co/TheBloke/Mythical-Destroyer-V2-L2-13B-GGUF | |
| Kimiko-v2-13B (using oobabooga/text-generation-webui) | 7.25 | 2023/08/31 | https://huggingface.co/TheBloke/Kimiko-v2-13B-GGUF | |
| orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) | 7.0875 | 2023/06/28 | https://huggingface.co/TheBloke/orca_mini_13B-GGML | |
| Platypus2-13B (using oobabooga/text-generation-webui) | 7.03125 | 2023/08/15 | https://huggingface.co/TheBloke/Platypus2-13B-GGML | |
| Redmond-Puffin-13B (using oobabooga/text-generation-webui) | 7.03125 | 2023/07/20 | https://huggingface.co/TheBloke/Redmond-Puffin-13B-GGML | |
| 13B-BlueMethod (using oobabooga/text-generation-webui) | 7.025 | 2023/07/24 | https://huggingface.co/TheBloke/13B-BlueMethod-GGML | |
| mpt-7b-instruct | 6.6875 | No | 2023/05/12 | https://huggingface.co/TheBloke/MPT-7B-Instruct-GGML |
| Kimiko-13B (using oobabooga/text-generation-webui) | 6.46875 | 2023/08/01 | https://huggingface.co/TheBloke/Kimiko-13B-GGML | |
| gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) | 6.0625 | No | 2023/05/07 | https://huggingface.co/Bradarr/gpt4-x-alpaca-13b-native-ggml-model-q4_0 |
| minotaur-15B-GPTQ (using oobabooga/text-generation-webui) | 5.9375 | 2023/06/26 | https://huggingface.co/TheBloke/minotaur-15B-GPTQ | |
| baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) | 5.90625 | 2023/07/05 | https://huggingface.co/TheBloke/baichuan-vicuna-7B-GGML | |
| gpt4all-j-v1.3-groovy (in GPT4All) | 5.6875 | No | 2023/05/07 | https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin |
Many thanks to:
❤️ GPT4ALl: https://github.com/nomic-ai/gpt4all-chat
❤️ llama.cpp: https://github.com/ggerganov/llama.cpp
❤️ oobabooga text generation webui: https://github.com/oobabooga/text-generation-webui
❤️ Colab webui inspired by camenduru: https://github.com/camenduru/text-generation-webui-colab/tree/main
❤️ The Bloke for quantization of the models: https://huggingface.co/TheBloke
(All scores are from GPT-4-0613.)
| Model_Name | Avg_Scores | Colab_Link | Date_Added | Link |
|---|---|---|---|---|
| CodeLlama-13B-oasst-sft-v10 (using oobabooga/text-generation-webui) | 9.8 | 2023/08/28 | https://huggingface.co/TheBloke/CodeLlama-13B-oasst-sft-v10-GGUF | |
| WizardCoder-Python-13B-V1.0 (using oobabooga/text-generation-webui) | 9.5 | 2023/08/28 | https://huggingface.co/TheBloke/WizardCoder-Python-13B-V1.0-GGUF | |
| Redmond-Hermes-Coder-GPTQ (using oobabooga/text-generation-webui) | 8.4 | 2023/07/03 | https://huggingface.co/TheBloke/Redmond-Hermes-Coder-GPTQ | |
| CodeUp-Alpha-13B-HF (using oobabooga/text-generation-webui) | 7.9 | 2023/08/15 | https://huggingface.co/TheBloke/CodeUp-Alpha-13B-HF-GGML | |
| starchat-beta-GPTQ (using oobabooga/text-generation-webui) | 7.6 | 2023/07/04 | https://huggingface.co/TheBloke/starchat-beta-GPTQ | |
| wizard-vicuna-13B-GPTQ (using oobabooga/text-generation-webui) | 7.3 | 2023/07/03 | https://huggingface.co/TheBloke/wizard-vicuna-13B-GPTQ | |
| WizardCoder-Guanaco-15B-V1.1 (using oobabooga/text-generation-webui) | 7.1 | 2023/07/21 | https://huggingface.co/TheBloke/WizardCoder-Guanaco-15B-V1.1-GPTQ | |
| CodeLlama-13B-Instruct (using oobabooga/text-generation-webui) | 7 | 2023/08/28 | https://huggingface.co/TheBloke/CodeLlama-13B-Instruct-GGUF | |
| CodeUp-Llama-2-13B-Chat-HF (using oobabooga/text-generation-webui) | 6 | 2023/08/03 | https://huggingface.co/TheBloke/CodeUp-Llama-2-13B-Chat-HF-GGML | |
| WizardCoder-15B-1.0-GPTQ (using oobabooga/text-generation-webui) | 5.9 | 2023/07/03 | https://huggingface.co/TheBloke/WizardCoder-15B-1.0-GPTQ | |
| wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) | 5.9 | 2023/05/29 | https://huggingface.co/TheBloke/wizardLM-13B-1.0-GPTQ |
Original responses can be found at: https://docs.google.com/spreadsheets/d/1ogDXUiaBx3t7EpMo44aaA6U6kLXX0x2tGRgLg8CISGs/edit?usp=sharing
Task Domain: Translation
Expected Good Response: "Le soleil se lève à l'est et se couche à l'ouest."
Explanation: This task tests the model's ability to understand and accurately translate text between languages.
Task Domain: Summary
Expected Good Response: "The water cycle is the continuous movement of water on Earth, crucial for maintaining water balance and supporting life."
Explanation: This task evaluates the model's ability to extract the main points from a given text and generate a concise summary.
Task Domain: App Design
Explanation: This task evaluates the model's ability to closely follow user's instruction for a complex task.
Task Domain: Abstractive Question Answering
Expected Good Response: Financial crises, social inequality, and Enlightenment ideas.
Explanation: This task tests the model's ability to understand the context and generate an answer in its own words.
