Sequential✓ Mathematical
◆ The PatternGated memory cells that solve the vanishing gradient problem
LSTMs add a cell state c — a "memory highway" — alongside the hidden state. Three gates control information flow.
fₜ=σ(Wf·[hₜ₋₁,xₜ]) iₜ=σ(Wi·[hₜ₋₁,xₜ]) oₜ=σ(Wo·[hₜ₋₁,xₜ])
f = forget | i = input | o = output gates
cₜ = fₜ ⊙ cₜ₋₁ + iₜ ⊙ tanh(Wc·[hₜ₋₁,xₜ])
Cell state update: forget old + write new candidate
hₜ = oₜ ⊙ tanh(cₜ)
Hidden state = filtered cell state
// LSTM gate diagram
self.lstm = nn.LSTM(input_size=10, hidden_size=64, num_layers=2, batch_first=True, dropout=0.2) out, (hn, cn) = self.lstm(x)
Pattern bridge: The forget gate decides what to keep and what to discard — the same selective memory behind recency bias.