transformers gguf mixtral Mixtral instruct finetune chatml DPO RLHF gpt4 synthetic data distillation en base_model:NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO base_model:quantized:NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO license:apache-2.0 region:us conversational
Small (<1GB)