Retraining a peft model after loading

[John6666]

load_in_8bit=True, # Load in 8-bit precision

Maybe it’s because of the quantization.

How to fix RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

I am trying to finetune a model: TheBloke/Mistral-7B-Instruct-v0.1-GPTQ. However, I am getting RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn error. How can I fix this error? Here is the code that is throwing the error: import transformers from datetime import datetime project = "Mixtral-alpaca-finance-finetune" base_model_name = "mixtral" run_name = base_model_name + "-" + project output_dir = "./" + run_name tokenizer.pad_token = tokenizer.eos_token …