SequentialMathematical
◆ The PatternLSTM's streamlined sibling — two gates, one state vector
GRU simplifies LSTM by merging cell+hidden state and using only two gates: update and reset.
zₜ = σ(Wz·[hₜ₋₁,xₜ]) rₜ = σ(Wr·[hₜ₋₁,xₜ])
z = update gate | r = reset gate
h̃ₜ = tanh(W·[rₜ⊙hₜ₋₁, xₜ])
Candidate hidden state — gated by reset
hₜ = (1−zₜ)⊙hₜ₋₁ + zₜ⊙h̃ₜ
Interpolate between old and new state via update gate
// RNN vs GRU vs LSTM — parameter count comparison
When to use which: RNN — quick baseline. GRU — best default for seq tasks. LSTM — complex long-range deps. Transformer — lots of data + GPU.
Pattern bridge: GRU merges forget and input into a single update gate — elegant reduction. In markets, RSI compresses momentum into one number.
Worked example
With 128 inputs and 256 hidden units, an LSTM layer has 394,240 parameters and a GRU layer 295,680 — exactly 75%, because it has three weight blocks instead of four. Fewer parameters train faster and need less data; whether they do as well is an empirical question for each task.
Where it misleads
- Neither GRU nor LSTM wins consistently; comparisons find them close (Chung et al. 2014).
- Parameter count is not speed; implementations and hardware kernels decide that.
- Like the LSTM, the GRU still processes one step at a time.
In code
x, h = 128, 256 gate = h * (h + x) + h # one gate: weights on [h, x] plus bias lstm, gru = 4 * gate, 3 * gate # LSTM: four blocks; GRU: three
Assumes import numpy as np. Each snippet carries its own example numbers; the comments say which are made up or simulated.
Sources
- K. Cho et al., “Learning Phrase Representations Using RNN Encoder–Decoder for Statistical Machine Translation”, EMNLP, 2014
- J. Chung, C. Gulcehre, K. Cho & Y. Bengio, “Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling”, arXiv:1412.3555, 2014
Last reviewed October 2026 · How this site checks its content · Report a mistake
Linked from 1