repeat_penalty & repeat_last_n: Stop Repetition

Definition

Small and heavily quantized models sometimes loop on the same word or sentence. Repetition penalties push back by lowering the logits of recently generated tokens.

Two settings work together:

repeat_penalty: how hard to penalize. 1.0 = off, 1.1 mild, 1.3+ harsh.
repeat_last_n: how many tokens to look back. 0 = off.

The formula llama.cpp applies (Ollama also runs GGUF models through the llama.cpp server): for every token in the window, a positive logit is divided by penalty and a negative one is multiplied by it. The penalty is the same whether the token appeared once or ten times.

The two penalties in OpenAI-style APIs are additive:

logit -= count × frequency_penalty + (count > 0) × presence_penalty

presence_penalty: a flat penalty once a token has appeared at all → nudges toward new topics.
frequency_penalty: grows with the number of occurrences → fewer reused words.

Defaults (as of October 2026):

Engine	repeat_penalty	repeat_last_n	presence / frequency
Ollama	1.0 (off)	64	0.0 / 0.0
llama.cpp	1.0 (off)	64	0.0 / 0.0
vLLM	`repetition_penalty` 1.0	—	0.0 / 0.0
OpenAI API	none	—	0 / 0 (range -2.0 … 2.0)

llama.cpp flags: --repeat-penalty, --repeat-last-n, --presence-penalty, --frequency-penalty.

Analogy

Telling a speaker "you lose points for reusing any word from your last 64 words." repeat_penalty is how much each lost point hurts; repeat_last_n is how far back their memory goes. Too short a window and they retell the same paragraph a page later; too harsh a penalty and they start avoiding "and" and "the".

Real-world example

A small model loops while writing a story:

``...and then she opened the door. Then she opened the door. Then she opened the door...``

Say the logit for "opened" is 4.0. With repeat_penalty 1.2: 4.0 / 1.2 ≈ 3.33. A token with logit -1.0 becomes -1.0 × 1.2 = -1.2; both become less likely.

Ollama Modelfile: ``FROM llama3.2 PARAMETER repeat_penalty 1.15 PARAMETER repeat_last_n 256``

A 256-token window also catches paragraph-level repetition. The default 64 tokens only covers a few sentences.

Code examples

                                            Ollama API · request with repetition penalties
                                            bash
                                        
curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [{"role": "user", "content": "Tell me a long fairy tale."}],
  "options": {
    "repeat_penalty": 1.15,
    "repeat_last_n": 256,
    "presence_penalty": 0.3
  }
}'

                                            llama.cpp · the same settings as CLI flags
                                            bash
                                        
# New install (llama.app): llama cli
# Older binary name: llama-cli
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF \
  --repeat-penalty 1.15 \
  --repeat-last-n 256 \
  --presence-penalty 0.3

When to use

Small (1-3B) or aggressively quantized models that fall into loops
Long creative text — a mild 1.05-1.15
Lists or idea generation where the same items keep coming back — presence_penalty
Output that reuses the same adjective over and over — frequency_penalty

When not to use

Code and JSON — brackets, quotes and variable names must repeat; the penalty breaks syntax
Long proper names, technical terms, translation — the right word has to be reused
Large, well-trained models — usually unnecessary, which is why the default is off

Common pitfalls

Setting the penalty too high

Around 1.3-1.5 the model starts dodging common words and punctuation; you get odd synonyms, broken grammar and half sentences. Increase it in small steps.

Ignoring the window size

With repeat_last_n 64 the penalty only sees the last 64 tokens. For paragraph-level repetition, widen the window instead of raising the penalty. Ollama's docs define -1 as 'num_ctx'; the upstream llama.cpp server rejects negative values, so an explicit number is safer.

Docs that disagree

llama.cpp's server README lists 1.1 as the /completion default for repeat_penalty, while the default in code is 1.0 (off). To be sure of the behaviour, set the value explicitly in the request.