I’m using model.generate() for text generation. I’m wondering is there a way to whitelist specific tokens so that they are returned during the beam search phase. For example - I want to the “force” the response to contain a question mark or a speccific phrase

Whitelist specific tokens in beam search

100worte March 16, 2021, 7:13pm 4

You can pass a LogitsProcessorList to beam_search. For some concrete examples of LogitProcessors, refer to generation_logits_process.py.

1 Like

Topic		Replies	Views
Use custom LogitsProcessor in `model.generate()` Beginners	2	6993	March 14, 2023
GPT-2 Logits to tokens for beam search (Generate method) 🤗Transformers	0	1324	September 2, 2021
Big `generate()` refactor 🤗Transformers	7	3801	November 26, 2021
BART_LM: Odd Beam Search Output Intermediate	18	1886	August 17, 2020
Beam_search and generate are not consistent 🤗Transformers	0	518	May 10, 2022