Hey, I am trying to figure out how to freeze layers of a model and read that I had to use for param in model.base_model.parameters(): param.requires_grad = False if I wanted to freeze the encoder of a pretrained MLM for example. But how do I use this with the Trainer? I tried the following: …

How to freeze layers using trainer?

sgugger March 19, 2021, 4:37pm 7

You should inspect the weights to see where they difer then. Trainer will not change the requires_grad value of your parameters.

3 Likes

Freeze Lower Layers with Auto Classification Model

Topic		Replies	Views
How to freeze layers while fine-tuning? 🤗Transformers	2	586	May 16, 2025
How to freeze some layers of BertModel Beginners	8	17763	August 25, 2022
Gradual Layer Freezing with huggingface model 🤗Transformers	1	913	February 10, 2021
Gradual Unfreezing support for Fine tuning models 🤗Transformers	3	4051	August 26, 2020
Freeze Lower Layers with Auto Classification Model 🤗Transformers	6	18693	May 25, 2023