Skip to content

Commit c4f53c4

Browse files
authored
Start reworking with "practical field modelling" chapter (#5)
1 parent 43a3280 commit c4f53c4

File tree

6 files changed

+489
-148
lines changed

6 files changed

+489
-148
lines changed

_toc.yml

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,11 @@ parts:
88
- file: geomag-obs-models/03a_Magnetic-Field-Line-Tracing
99
- file: geomag-obs-models/04a_SHA1
1010
- file: geomag-obs-models/04b_SHA2
11-
- file: geomag-obs-models/04c_GroundObs-to-Model
12-
- file: geomag-obs-models/05a_Swarm-Measurements
11+
- caption: Practical Field Modelling
12+
chapters:
13+
- file: practical-field-modelling/01a_Making-a-main-field-model
14+
- file: practical-field-modelling/02a_Satellite-data-selection
15+
- file: practical-field-modelling/02b_Modelling-with-Swarm-data
1316
- caption: Data Assimilation
1417
chapters:
1518
- file: data_assimilation/01a_lorenz

geomag-obs-models/01a_Visualising-Geomagnetic-Observatory-Data.ipynb

Lines changed: 27 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
"id": "2c32df6b",
66
"metadata": {},
77
"source": [
8-
"# Visualising geomagnetic data"
8+
"# Visualising geomagnetic ground data"
99
]
1010
},
1111
{
@@ -63,7 +63,7 @@
6363
"source": [
6464
"[INTERMAGNET](https://intermagnet.github.io/) is the International Real-time Magnetic Observatory Network, a global network of observatories continually monitoring Earth's magnetic field.\n",
6565
"\n",
66-
"Here we use [VirES](https://earth.esa.int/eogateway/tools/vires-for-swarm) to access some ground observatory data [(see here for more)](https://notebooks.vires.services/notebooks/04c2_geomag-ground-data-vires). The following code helps you download the data for a given observatory for a given year, and load them as a Pandas Dataframe. Check <http://intermagnet.org/new_data_download.html> for other ways to access these data.\n",
66+
"Here we use [VirES](https://earth.esa.int/eogateway/tools/vires-for-swarm) to access some ground observatory data [(see here for more info)](https://notebooks.vires.services/notebooks/04c2_geomag-ground-data-vires). The following code helps you download the data for a given observatory for a given year, and load them as a Pandas Dataframe. Check <https://intermagnet.org/data_download.html> for other ways to access these data.\n",
6767
"\n",
6868
"We will select data from 2003 measured at [the Eskdalemuir observatory (ESK)](http://www.geomag.bgs.ac.uk/operations/eskdale.html) in Scotland, which has a latitude of 55.314° N. If you wish you can select data from a different year or observatory (check [this map of INTERMAGNET observatories](https://intermagnet.github.io/metadata/) to find the three-letter code that specifies the observatory).\n",
6969
"\n",
@@ -118,7 +118,7 @@
118118
"obs_minute = fetch_obs_data_for_years(\n",
119119
" observatory=\"ESK\",\n",
120120
" year_start=2003,\n",
121-
" year_end=2003,\n",
121+
" year_end=2003, # inclusive\n",
122122
" cadence=\"M\",\n",
123123
")\n",
124124
"obs_minute.head()"
@@ -236,17 +236,30 @@
236236
},
237237
{
238238
"cell_type": "markdown",
239-
"id": "e231d0d2",
239+
"id": "26ca4121-7fa5-4533-a77d-e92611c63073",
240240
"metadata": {},
241241
"source": [
242-
"```{note}\n",
243-
"What signals can you see in this data?\n",
244-
"```\n",
242+
"### Exercise\n",
245243
"\n",
244+
"Explore the data over different time scales by adjusting the start and end dates.\n",
245+
"\n",
246+
"1. Looking at one month of data, what signals can you observe?\n",
247+
"1. Approximately how large are disturbances (i.e. how many nT)? Compare:\n",
248+
" - Inspect a quieter week, e.g. the first week of January 2003\n",
249+
" - Inspect a very stormy week, e.g. 27 October to 3 November 2003 - the [Halloween solar storms](https://en.wikipedia.org/wiki/2003_Halloween_solar_storms))\n",
250+
"1. Expand the time window to display all of 2003. You should be able to observe a small systematic shift over the year."
251+
]
252+
},
253+
{
254+
"cell_type": "markdown",
255+
"id": "6ef12b24-70b2-425e-8868-c4686d09078a",
256+
"metadata": {},
257+
"source": [
246258
"```{toggle}\n",
259+
"Answers:\n",
247260
"- Daily oscillation: due to the rotation of the Earth driving ionospheric change through the day/night - this is the Sq variation (\"solar quiet-day\" variation)\n",
261+
"- More stochastic variations due to geomagnetic activity (dynamic magnetospheric and ionospheric current systems)\n",
248262
"- Shift in baseline over the year: due to the change in the main magnetic field from the core - this is the secular variation (SV)\n",
249-
"- More random variations due to geomagnetic activity\n",
250263
"```"
251264
]
252265
},
@@ -269,12 +282,8 @@
269282
{
270283
"cell_type": "code",
271284
"execution_count": null,
272-
"id": "087a2d68",
273-
"metadata": {
274-
"tags": [
275-
"hide-cell"
276-
]
277-
},
285+
"id": "64b41400-41dd-4744-a517-2888f2cc7228",
286+
"metadata": {},
278287
"outputs": [],
279288
"source": [
280289
"obs_hourly = fetch_obs_data_for_years(\n",
@@ -580,7 +589,7 @@
580589
" print(f\"Fetching {obs} [{i+1}/{len(observatories)}]\")\n",
581590
" df = fetch_obs_data_for_years(\n",
582591
" observatory=obs,\n",
583-
" year_start=1900,\n",
592+
" year_start=1915,\n",
584593
" year_end=2020,\n",
585594
" cadence=\"H\",\n",
586595
" asynchronous=False,\n",
@@ -593,8 +602,7 @@
593602
" return annual_means\n",
594603
"\n",
595604
"\n",
596-
"# observatories = [\"ESK\", \"NGK\", \"CLF\"] # Why aren't NGK and CLF working?\n",
597-
"observatories = [\"ESK\"]\n",
605+
"observatories = [\"ESK\", \"NGK\", \"CLF\"]\n",
598606
"annual_means = fetch_annual_means(observatories)"
599607
]
600608
},
@@ -659,6 +667,8 @@
659667
" axes[2*i].set_ylabel(labels[var1])\n",
660668
" axes[2*i+1].plot(df[var2], color=colors[i], alpha=0.5, marker=\"x\")\n",
661669
" axes[2*i+1].set_ylabel(labels[var2])\n",
670+
" axes[2*i+1].set_xlim((1915, 2020))\n",
671+
" axes[2*i].set_xlim((1915, 2020))\n",
662672
" axes[2*i].grid()\n",
663673
" axes[2*i+1].grid()\n",
664674
" axes[-1].set_xlabel(\"Year\")\n",

geomag-obs-models/02a_K-Index-Calculation.ipynb

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,7 @@
2020
"outputs": [],
2121
"source": [
2222
"# Install pooch (not currently in VRE)\n",
23-
"import sys\n",
24-
"!{sys.executable} -m pip install pooch\n",
23+
"%pip install pooch\n",
2524
"\n",
2625
"import datetime as dt\n",
2726
"import numpy as np\n",

geomag-obs-models/04c_GroundObs-to-Model.ipynb renamed to practical-field-modelling/01a_Making-a-main-field-model.ipynb

Lines changed: 68 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,11 @@
1717
"source": [
1818
"Here we demonstrate how to make a simple time-independent main field model by fitting spherical harmonics to ground observatory measurements.\n",
1919
"\n",
20-
"The principles of spherical harmonic analysis have been shown in the previous pages, so here we will take a shortcut and use utilties from ChaosMagPy.\n",
20+
"The principles of spherical harmonic analysis have been shown in the previous pages, so here we will take a shortcut and use utilties from [ChaosMagPy](https://chaosmagpy.readthedocs.io/)\n",
2121
"\n",
22-
"- [ChaosMagPy](https://chaosmagpy.readthedocs.io/), Clemens Kloss, https://doi.org/10.5281/zenodo.3352398 \n",
23-
" We will use:\n",
22+
"> Clemens Kloss et al. (2025). ancklo/ChaosMagPy: ChaosMagPy v0.15 (v0.15). Zenodo. https://doi.org/10.5281/zenodo.16091680\n",
23+
"\n",
24+
"We will use:\n",
2425
" - [design_gauss](https://chaosmagpy.readthedocs.io/en/master/functions/chaosmagpy.model_utils.design_gauss.html) to generate the design matrix for the inversion\n",
2526
" - [synth_values](https://chaosmagpy.readthedocs.io/en/master/functions/chaosmagpy.model_utils.synth_values.html) to run the forward model to get predictions for the magnetic field from our model\n",
2627
"\n",
@@ -39,8 +40,7 @@
3940
"outputs": [],
4041
"source": [
4142
"# Install pooch (not currently in VRE)\n",
42-
"import sys\n",
43-
"!{sys.executable} -m pip install pooch"
43+
"%pip install pooch"
4444
]
4545
},
4646
{
@@ -75,7 +75,7 @@
7575
"id": "638c7a53-8f91-4563-9dd0-66be049c7358",
7676
"metadata": {},
7777
"source": [
78-
"We'll load a pre-prepared dataset containing annual means derived from observatory data. We just select a single year from this dataset. In reality the main field varies on shorter time scales but these annual data points are useful enough for a low resolution model."
78+
"We'll load a pre-prepared dataset containing *annual means* derived from observatory data. We just select a single year from this dataset. In reality the main field varies on shorter time scales but these annual data points are useful enough for a low resolution model."
7979
]
8080
},
8181
{
@@ -160,7 +160,7 @@
160160
"$G$ can be constructed by comparing to the spherical harmonic expansion of the magnetic field $\\vec{B}$ (where we have neglected external sources for simplicity):\n",
161161
"\n",
162162
"$$\n",
163-
"\\begin{align}\\label{eq:spharm_B}\n",
163+
"\\begin{align}\n",
164164
" \\vec{d} =\\ & G \\vec{m} \\\\\n",
165165
" B_i =\\ & G \\begin{bmatrix}\n",
166166
" g_n^m & h_n^m\n",
@@ -197,24 +197,22 @@
197197
"metadata": {},
198198
"outputs": [],
199199
"source": [
200-
"def build_model(ds=input_data, nmax=10):\n",
201-
" \"\"\"Returns Gauss coefficients for a simple SH model\"\"\"\n",
202-
" theta = ds[\"Colat\"]\n",
203-
" phi = ds[\"Long\"]\n",
204-
" radius = 6371.2 + ds[\"alt(m)\"]/1e3\n",
205-
" B_radius = -ds[\"Z\"]\n",
206-
" B_theta = -ds[\"X\"]\n",
207-
" B_phi = ds[\"Y\"]\n",
208-
" # Build the design matrix\n",
209-
" G_radius, G_theta, G_phi = design_gauss(radius, theta, phi, nmax=nmax, source=\"internal\")\n",
210-
" # Perform the inversion\n",
211-
" G = np.concatenate((G_radius, G_theta, G_phi))\n",
212-
" d = np.concatenate((B_radius, B_theta, B_phi))\n",
213-
" m = np.linalg.inv(G.T @ G) @ (G.T @ d)\n",
214-
" return m\n",
200+
"# Identify coordinates and measurements\n",
201+
"theta = input_data[\"Colat\"]\n",
202+
"phi = input_data[\"Long\"]\n",
203+
"radius = 6371.2 + input_data[\"alt(m)\"]/1e3\n",
204+
"B_radius = -input_data[\"Z\"]\n",
205+
"B_theta = -input_data[\"X\"]\n",
206+
"B_phi = input_data[\"Y\"]\n",
215207
"\n",
208+
"# Build the design matrix\n",
209+
"N_max = 6\n",
210+
"G_radius, G_theta, G_phi = design_gauss(radius, theta, phi, nmax=N_max, source=\"internal\")\n",
216211
"\n",
217-
"m = build_model(input_data, nmax=5)\n",
212+
"# Perform the inversion\n",
213+
"G = np.concatenate((G_radius, G_theta, G_phi))\n",
214+
"d = np.concatenate((B_radius, B_theta, B_phi))\n",
215+
"m = np.linalg.inv(G.T @ G) @ (G.T @ d)\n",
218216
"m"
219217
]
220218
},
@@ -226,6 +224,23 @@
226224
"Since we do not have many data, we would need to be more sophisticated with the inversion procedure (e.g. including regularisation) to get a model of higher resolution. For this reason, the model is truncated at spherical harmonic degree 6. You can explore increasing this limit."
227225
]
228226
},
227+
{
228+
"cell_type": "markdown",
229+
"id": "0d9b50ad-b8dd-408f-8183-fb2892f917c1",
230+
"metadata": {},
231+
"source": [
232+
"### Exercise"
233+
]
234+
},
235+
{
236+
"cell_type": "markdown",
237+
"id": "ce6a3485-7187-4a0d-814b-1147afdb24a0",
238+
"metadata": {},
239+
"source": [
240+
"1. What is the shape of the design matrix?\n",
241+
"1. What is the amplitude of the axial dipole ($g^1_0$) and how does it compare with the [IGRF](https://doi.org/10.5281/zenodo.14012302)?"
242+
]
243+
},
229244
{
230245
"cell_type": "markdown",
231246
"id": "53a9d939-726c-40f2-a596-dc04409060a2",
@@ -239,7 +254,7 @@
239254
"id": "312a3527-d2b4-4a11-b17d-84f9e9ec73bd",
240255
"metadata": {},
241256
"source": [
242-
"Evaluating a model over the Earth and plotting contours..."
257+
"Let's evaluate the model over the Earth and plot contours..."
243258
]
244259
},
245260
{
@@ -249,6 +264,23 @@
249264
"metadata": {},
250265
"outputs": [],
251266
"source": [
267+
"def build_model(ds=input_data, nmax=10):\n",
268+
" \"\"\"Returns Gauss coefficients for a simple SH model\"\"\"\n",
269+
" theta = ds[\"Colat\"]\n",
270+
" phi = ds[\"Long\"]\n",
271+
" radius = 6371.2 + ds[\"alt(m)\"]/1e3\n",
272+
" B_radius = -ds[\"Z\"]\n",
273+
" B_theta = -ds[\"X\"]\n",
274+
" B_phi = ds[\"Y\"]\n",
275+
" # Build the design matrix\n",
276+
" G_radius, G_theta, G_phi = design_gauss(radius, theta, phi, nmax=nmax, source=\"internal\")\n",
277+
" # Perform the inversion\n",
278+
" G = np.concatenate((G_radius, G_theta, G_phi))\n",
279+
" d = np.concatenate((B_radius, B_theta, B_phi))\n",
280+
" m = np.linalg.inv(G.T @ G) @ (G.T @ d)\n",
281+
" return m\n",
282+
"\n",
283+
"\n",
252284
"def sample_model_on_grid(m, radius=6371.200):\n",
253285
" \"\"\"Evaluate Gauss coefficients over a regular grid over Earth\"\"\"\n",
254286
" theta = np.arange(1, 180, 1)\n",
@@ -272,6 +304,7 @@
272304
" )\n",
273305
"\n",
274306
"\n",
307+
"m = build_model(input_data, nmax=6)\n",
275308
"plot_model_contours(m)"
276309
]
277310
},
@@ -280,7 +313,7 @@
280313
"id": "153e2f1a-87e8-4dee-ac0e-baa5a6a52cf8",
281314
"metadata": {},
282315
"source": [
283-
"The power spectrum is a common way to assess and compare models, showing how much power is concentrated at each spherical harmonic degree:"
316+
"The power spectrum is a common way to assess and compare models, showing how much power is concentrated at each spherical harmonic degree. We can use [`plot_power_spectrum`](https://chaosmagpy.readthedocs.io/en/master/functions/chaosmagpy.plot_utils.plot_power_spectrum.html#chaosmagpy.plot_utils.plot_power_spectrum) from ChaosMagPy:"
284317
]
285318
},
286319
{
@@ -347,11 +380,18 @@
347380
},
348381
{
349382
"cell_type": "markdown",
350-
"id": "8b20d365-844e-49d9-ab4a-2823307504a6",
383+
"id": "964e4fad-ce8e-4667-b482-48699848750b",
351384
"metadata": {},
352385
"source": [
353-
"## Exercises\n",
354-
"\n",
386+
"## Exercise"
387+
]
388+
},
389+
{
390+
"cell_type": "markdown",
391+
"id": "0bbf348f-91fb-4ca6-b82c-4b1afc7f061e",
392+
"metadata": {},
393+
"source": [
394+
"- Calculate the RMS misfit for each vector component (Hint: we have stored the residuals in the dataset, accessible as `ds[\"res_N\"]`, `ds[\"res_E\"]`, `ds[\"res_C\"]`)\n",
355395
"- Experiment with changing the SH degree trunction in the model\n",
356396
"- Experiment with using data just over Europe\n",
357397
"- Add random noise into the data to observe the effect\n",

0 commit comments

Comments
 (0)