{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A century of the 200-day rule: where the edge came from\n",
    "\n",
    "Stay in the stock market while it is above its 200-day average, move to Treasury bills when it falls below: one of the oldest trend-following rules. Does it beat simply holding the market — and when?\n",
    "\n",
    "*From [Pattern is Everything](https://patterniseverything.com/cases/market-backtest/). Data: Fama/French research factors, daily — Kenneth R. French Data Library, Tuck School of Business.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. A hundred years of daily returns\n",
    "\n",
    "Load the file, turn percentages into fractions, and add the T-bill rate back to get the market’s total return."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "url = ('https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/ftp/'\n",
    "       'F-F_Research_Data_Factors_daily_CSV.zip')\n",
    "ff = pd.read_csv(url, skiprows=4, index_col=0)          # pandas unzips it\n",
    "ff = ff[ff.index.astype(str).str.strip().str.fullmatch(r'\\d{8}')]   # drop the copyright line\n",
    "ff.index = pd.to_datetime(ff.index.astype(str).str.strip(), format='%Y%m%d')\n",
    "ff = ff.astype(float) / 100\n",
    "r = ff['Mkt-RF'] + ff['RF']          # the market's daily total return\n",
    "rf = ff['RF']                        # the T-bill's\n",
    "print(len(r), 'days,', r.index[0].date(), 'to', r.index[-1].date())\n",
    "print('five worst days:', {d.date().isoformat(): round(v, 3) for d, v in r.nsmallest(5).items()})"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The worst day, 19 October 1987, lost <strong>17.4%</strong> — 16 times the standard deviation of a day’s return. Three of the five worst came in the crash of autumn 1929, one in March 2020."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. The rule, without peeking\n",
    "\n",
    "Build the market’s value from its returns and its 200-day average. The decision for each day is taken from the previous day’s close — the <code>shift(1)</code> — because that is all an investor would have known. Out of the market, the money earns the T-bill rate. Each switch costs 0.1%."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "price = (1 + r).cumprod()\n",
    "above = (price > price.rolling(200).mean()).shift(1)   # yesterday's close vs its average\n",
    "start = price.rolling(200).mean().first_valid_index() + pd.Timedelta(days=1)\n",
    "r, rf, above = r[start:], rf[start:], above[start:].astype(bool)\n",
    "switch = above != above.shift(1, fill_value=True)\n",
    "rule = pd.Series(np.where(above, r, rf), index=r.index) - 0.001 * switch\n",
    "\n",
    "def summary(x, name):\n",
    "    years = len(x) / 252\n",
    "    growth = (1 + x).prod() ** (1 / years) - 1\n",
    "    wealth = (1 + x).cumprod()\n",
    "    drawdown = (wealth / wealth.cummax() - 1).min()\n",
    "    sharpe = (x - rf.loc[x.index]).mean() / x.std() * np.sqrt(252)\n",
    "    print(f'{name:16} {growth:6.2%} a year | volatility {x.std() * np.sqrt(252):6.2%} | '\n",
    "          f'worst drawdown {drawdown:7.1%} | Sharpe {sharpe:.2f}')\n",
    "    return growth, drawdown\n",
    "\n",
    "hold_all = summary(r, 'buy and hold')\n",
    "rule_all = summary(rule, '200-day rule')\n",
    "print('switches per year:', round(switch.sum() / (len(r) / 252), 1))"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Over the whole period the rule wins on every line: <strong>10.5%</strong> a year against 9.8% for buy-and-hold, at two thirds of the volatility, and a worst drawdown of 34% instead of 84%. That is the headline result."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Split the century\n",
    "\n",
    "The same comparison, era by era."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "eras = {'1927-1975': ('1927', '1975'),\n",
    "        '1976-2000': ('1976', '2000'),\n",
    "        '2001-2026': ('2001', '2026')}\n",
    "by_era = {}\n",
    "for era, (a, b) in eras.items():\n",
    "    print(era)\n",
    "    by_era[era] = (summary(r[a:b], '  buy and hold'), summary(rule[a:b], '  200-day rule'))"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The whole-sample edge comes from <strong>1927–1975</strong>, when the rule sidestepped most of the 1929–32 crash: 10.5% a year against 7.4%. Since 1976 it has earned less than simply holding the market — 13.8% against 15.3%, then 7.1% against 9.5% — while still cutting the worst drawdown, from 33% to 24% and from 55% to 23%. In the recent eras it behaved like insurance, not like an engine of return. Sullivan, Timmermann and White found the same pattern on the Dow Jones index: the best rules over 1897–1986 survived a correction for data snooping, but their performance did not repeat in the decade that followed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. The bug that nearly doubles the result\n",
    "\n",
    "Remove the <code>shift(1)</code>: decide each day using that same day’s close — which is only known once the day’s return has already happened."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "peek = (price > price.rolling(200).mean())[start:]\n",
    "cheat = pd.Series(np.where(peek, r, rf), index=r.index)\n",
    "cheat -= 0.001 * (peek != peek.shift(1, fill_value=True))\n",
    "cheat_all = summary(cheat, 'with look-ahead')"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One missing <code>shift</code>, and the rule earns <strong>18.7%</strong> a year — 1.8 times the honest 10.5%. A day’s own return is what pushes the price across its average, so the bugged rule is in the market for the rise that crosses it upward and out of it for the fall that crosses it downward. Bugs in backtests are rarely this visible; most leak a little information and inflate a little, which is why an implausibly good backtest should be checked line by line before anything else."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
