Merge pull request #300 from PolicyEngine/fix/methodology-docs-118

Fix text errors in methodology documentation

Author: vahid-ahmadi

changelog.d/fix-methodology-docs.changed.md

@@ -0,0 +1 @@
+Fixed text errors in methodology documentation page.

docs/methodology.ipynb

@@ -3,33 +3,7 @@
   {
    "cell_type": "markdown",
    "metadata": {},
-   "source": [
-    "# Methodology\n",
-    "\n",
-    "In this page, we'll walk through step-by-step the process we use to create PolicyEngine's dataset.\n",
-    "* **Family Resources Survey**: we'll start with the FRS, looking at close it is to reality. To take an actual concrete starting point, we'll assume benefit payments are as reported in the survey.\n",
-    "* **FRS (+ tax-benefit model)**: we need to make sure that our tax-benefit model isn't doing anything unexpected. If we turn on simulation of taxes and benefits, does anything look unexpected? If not- great, we've turned a household survey into something useful for policy analysis. We'll also take stock here of what we're missing from reality.\n",
-    "* **Wealth and consumption**: the most obvious thing we're missing is wealth and consumption. We'll impute those here.\n",
-    "* **Fine-tuning**: we'll use reweighting to make some final adjustments to make sure our dataset is as close to reality as possible.\n",
-    "* **Validation**: we'll compare our dataset to the UK's official statistics, and see how we're doing.\n",
-    "\n",
-    "## The Family Resources Survey\n",
-    "\n",
-    "First, we'll start with the FRS as-is. Skipping over the technical details for how we actually feed this data into the model (you can find that in `policyengine_uk_data/datasets/frs/`), we need to decide how we're actually going to measure 'close to reality'. We need to define an objective function, and if our final dataset improves it a lot, we can call that a success.\n",
-    "         \n",
-    "We'll define this objective function using public statistics that we can generally agree are of high importance to describing the UK household sector. These are things that, if the survey gets them wrong, we'd expect to cause inaccuracy in our model, and if we get them all mostly right, we'd expect to have confidence that it's a pretty accurate tax-benefit model.\n",
-    "         \n",
-    "For this, we've gone through and collected:\n",
-    "         \n",
-    "* **Demographics** from the ONS: ten-year age band populations by region of the UK, national family type populations and national tenure type populations.\n",
-    "* **Incomes** from HMRC: for each of 14 total income bands, the number of people with income and combined income of the seven income types that account for over 99% of total income: employment, self-employment, State Pension, private pension, property, savings interest, and dividends.\n",
-    "* **Tax-benefit programs** from the DWP and OBR: statistics on caseloads, expenditures and revenues for all 20 major tax-benefit programs.\n",
-    "         \n",
-    "Let's first take a look at the initial FRS, our starting point, and what is generally considered the best dataset to use (mostly completely un-modified across major tax-benefit models), and see how close it is to reproducing these statistics.\n",
-    "         \n",
-    "The table below shows the result, and: it's really quite bad! Look at the relative errors.\n",
-    "\n"
-   ]
+   "source": "# Methodology\n\nIn this page, we'll walk through step-by-step the process we use to create PolicyEngine's dataset.\n* **Family Resources Survey**: we'll start with the FRS, looking at how close it is to reality. To take an actual concrete starting point, we'll assume benefit payments are as reported in the survey.\n* **FRS (+ tax-benefit model)**: we need to make sure that our tax-benefit model isn't doing anything unexpected. If we turn on simulation of taxes and benefits, does anything look unexpected? If not- great, we've turned a household survey into something useful for policy analysis. We'll also take stock here of what we're missing from reality.\n* **Wealth and consumption**: the most obvious thing we're missing is wealth and consumption. We'll impute those here.\n* **Fine-tuning**: we'll use reweighting to make some final adjustments to make sure our dataset is as close to reality as possible.\n* **Validation**: we'll compare our dataset to the UK's official statistics, and see how we're doing.\n\n## The Family Resources Survey\n\nFirst, we'll start with the FRS as-is. Skipping over the technical details for how we actually feed this data into the model (you can find that in `policyengine_uk_data/datasets/frs/`), we need to decide how we're actually going to measure 'close to reality'. We need to define an objective function, and if our final dataset improves it a lot, we can call that a success.\n         \nWe'll define this objective function using public statistics that we can generally agree are of high importance to describing the UK household sector. These are things that, if the survey gets them wrong, we'd expect to cause inaccuracy in our model, and if we get them all mostly right, we'd expect to have confidence that it's a pretty accurate tax-benefit model.\n         \nFor this, we've gone through and collected:\n         \n* **Demographics** from the ONS: ten-year age band populations by region of the UK, national family type populations and national tenure type populations.\n* **Incomes** from HMRC: for each of 14 total income bands, the number of people with income and combined income of the seven income types that account for over 99% of total income: employment, self-employment, State Pension, private pension, property, savings interest, and dividends.\n* **Tax-benefit programs** from the DWP and OBR: statistics on caseloads, expenditures and revenues for all 20 major tax-benefit programs.\n         \nLet's first take a look at the initial FRS, our starting point, and what is generally considered the best dataset to use (mostly completely un-modified across major tax-benefit models), and see how close it is to reproducing these statistics.\n         \nThe table below shows the result, and: it's really quite bad! Look at the relative errors.\n"
   },
   {
    "cell_type": "code",
@@ -1716,14 +1690,7 @@
   {
    "cell_type": "markdown",
    "metadata": {},
-   "source": [
-    "A few notes:\n",
-    "         \n",
-    "* We're comparing things in the same relevant time period (2022), and only doing a tiny amount of adjustment to the statistics: OBR statistics are taken directly from the latest EFO, ONS statistics are the most recent projections for 2022, and HMRC statistics are uprated from 2021 to 2022 using the same standard uprating factors we use in the model (and it's only one year adjustment).\n",
-    "* Demogaphics look basically fine: that's expected, because the DWP applies an optimisation algorithm to optimise the household weights to be as close as possible to a similar set of demographic statistics. It's a good sign that we use slightly different statistics than it was trained on and get good accuracy.\n",
-    "* Incomes look *not great at all*. We'll take a closer look below to understand why. But the FRS is well-known to under-report income significantly.\n",
-    "* Tax-benefit programs also look *not good*. And this is a concern! Because we're using this dataset to answer questions about tax-benefit programs, and the FRS isn't even providing a good representation of them under baseline law.\n"
-   ]
+   "source": "A few notes:\n         \n* We're comparing things projected to the same time period (2025), using standard uprating factors. OBR statistics are taken directly from the latest EFO, ONS statistics are the most recent projections, and HMRC statistics are uprated from 2021 using the same standard uprating factors we use in the model.\n* Demographics look basically fine: that's expected, because the DWP applies an optimisation algorithm to optimise the household weights to be as close as possible to a similar set of demographic statistics. It's a good sign that we use slightly different statistics than it was trained on and get good accuracy.\n* Incomes look *not great at all*. We'll take a closer look below to understand why. But the FRS is well-known to under-report income significantly.\n* Tax-benefit programs also look *not good*. And this is a concern! Because we're using this dataset to answer questions about tax-benefit programs, and the FRS isn't even providing a good representation of them under baseline law.\n"
   },
   {
    "cell_type": "code",
@@ -4758,11 +4725,7 @@
   {
    "cell_type": "markdown",
    "metadata": {},
-   "source": [
-    "## Calibration\n",
-    "\n",
-    "Now, we've got a dataset that's performs pretty well without explicitly targeting the official statistics we care about. So it's time to add the final touch- calibrating the weights to explicitly minimise error against the target set."
-   ]
+   "source": "## Calibration\n\nNow, we've got a dataset that performs pretty well without explicitly targeting the official statistics we care about. So it's time to add the final touch — calibrating the weights to explicitly minimise error against the target set.\n\nNote: the initial reweighting step (before adding income clones) has limited effect, because the FRS lacks enough high-income records for the optimiser to work with. The next section addresses this.\n"
   },
   {
    "cell_type": "code",
@@ -7669,4 +7632,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 4
-}
+}