transformers gguf trl llama dpo alignment custome chat en dataset:HuggingFaceH4/ultrafeedback_binarized base_model:AIR-hl/Llama-3.2-3B-DPO base_model:quantized:AIR-hl/Llama-3.2-3B-DPO license:llama3.2 endpoints_compatible region:us conversational
Small (<1GB)