Greedy · Beam Search✓ Mathematical
◆ The PatternDeterministic methods for converting logits to text
Greedy decoding picks the highest-probability token at each step. Simple but often suboptimal — it can miss globally better sequences. Beam search maintains K candidate sequences (beams) and prunes at each step.
Greedy: token_t = argmax P(token | context)
Fastest, but prone to repetitive or locally-trapped outputs.
Beam search: maintain top-K partial sequences ranked by Σ log P
At each step, expand all K beams, score, keep top K. Beam width K=4–5 is typical.
Beam search was dominant pre-LLM (translation, summarization) but is rarely used for chat/creative generation — it produces too safe, repetitive text. Modern LLMs use sampling (next topic) for most tasks.
Beam search still wins for tasks with a "correct" answer — code generation, math, structured output. For open-ended text, sampling produces more natural and diverse output.
Interactive — beam search tree
Python — beam search#
def beam_search(model, prompt_ids, beam_width=4, max_len=50):
beams = [(prompt_ids, 0.0)] # (sequence, log_prob)
for _ in range(max_len):
candidates = []
for seq, score in beams:
logits = model(seq)[:, -1, :]
log_probs = F.log_softmax(logits, dim=-1)
topk = log_probs.topk(beam_width)
for i in range(beam_width):
new_seq = torch.cat([seq, topk.indices[:, i:i+1]], dim=-1)
new_score = score + topk.values[:, i].item()
candidates.append((new_seq, new_score))
# Keep top-K beams
beams = sorted(candidates, key=lambda x: -x[1])[:beam_width]
return beams[0][0] # best sequencePattern bridge: Greedy, beam search, nucleus sampling — trading off quality vs. diversity. The same tradeoff as bias-variance: greedy = high bias, random = high variance. In markets, fear and greed drive conservative vs. aggressive strategies.