Finish case3 and4 (#28)

* Changed strat-time in use case 3 to be in line with use case 2

* text changes to notebook 3. Needs re-running but is taking (too?) long to start container on research cloud

* oops. This commit is without the output folder!

* fixed notebook 3

* finished text on case 3 and case 4. Needs final run to generate results and have cell-counts be nice in order. Once that is done, branch can be merged back into main.

* Case4 output (#27)

* Make case2 fair (#25)

* tried to make notebook 2 FAIR, failed on technicallities. Try again after weekend

* Found the porblem: using configparser creates .ini files that are byte for byte identical, but create an
error when calling initialize(). Raised an issue for this, not something I can fix.

* Fixed problem: used work-around

* final edits to text of notebook2 before pull request

* fixed and made FAIR case 2. Solved issue with configParser (thanks Stefan!)

* Added output files from case 4

* Reran case 4

Co-authored-by: Rolf Hut <[email protected]>

* moved files from case 5. Had to change filenames for 3 and 4 into 4 and 5 because Overleaf doesn't allow lists of sections to start at zero. Will change file names of other notebooks in other branch.

Co-authored-by: Stefan Verhoeven <[email protected]>

Author: RolfHut

Case3ForcePCRGlobWBWithadditionalEvapInfo.ipynb

@@ -13,7 +13,7 @@
    "id": "dffd5e98",
    "metadata": {},
    "source": [
-    "# Case study 4: Calibrate Marrmot M14 using GRDC data"
+    "# Case study 5: Calibrate Marrmot M14 using GRDC data"
    ]
   },
   {
@@ -25,13 +25,17 @@
     "The model used in this notebook is: [MARRMoT](https://github.com/wknoben/MARRMoT) M14 (TOPMODEL)\n",
     "We use [CMA Evolution Strategy (CMA-ES) algorithm](https://github.com/CMA-ES/pycma) package for finding the best parameters for the model. The procedure is explained in the cells below.\n",
     "\n",
+    "This notebook is written in a different style of programming than the other notebooks in this repo. In this notebook functions for the most important parts of the analyses are written first, and subsequently called, instead of having a more script like style of programming. This is done for two reasons.\n",
+    "\n",
+    "1. Since many optimization algorithms, CMA-ES included, require functions to be optimized to be passed to the algorithm. This notebooks shows that the eWaterCycle platform support this type of workflow.\n",
+    "2. This notebook shows that the script style of programming used in the other notebooks of this repository is not a requirement of the platform but rather a choice of the user and other styles of programming are equally supported.\n",
     "\n",
     "***NOTE: this notebooks is computationally expensive. Although it might possible to execute it on the demo machine (be sure to adapt the POPSIZE and MAXITER settings below), this is not recommended. This notebook has been executed on a machine with 24 cpu cores and took about 4 hours to complete. It is included here to demonstrate what is possible with ewatercycle, not as a performance benchmark. Please use it with care.***"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 1,
    "id": "21bb49ea",
    "metadata": {},
    "outputs": [
@@ -54,7 +58,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 2,
    "id": "2c8216fe",
    "metadata": {
     "tags": []
@@ -106,7 +110,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 3,
    "id": "9bebaff3",
    "metadata": {
     "tags": []
@@ -129,9 +133,17 @@
     "## 1. Loading forcing data"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "7b0f49af-39f9-4427-bc10-b7fb4bac66e1",
+   "metadata": {},
+   "source": [
+    "Note that while this notebook was run on the Cartesius supercomputer, the file system of the SURF research cloud has (on purpose) a similar structure to make sure that calls like the one below work on both machines."
+   ]
+  },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 4,
    "id": "863ad1b9",
    "metadata": {},
    "outputs": [],
@@ -141,7 +153,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 5,
    "id": "e8f244c9",
    "metadata": {},
    "outputs": [
@@ -191,11 +203,11 @@
     "and about `ask-and-tell interface` see an example at https://pypi.org/project/cma/.\n",
     "\n",
     "### Practical hints\n",
-    "The `run_model()` function uses two CPU cores (model-octave, and bmi-server)\n",
+    "The `run_model()` function uses 1.33 CPU cores (1 for model-octave, and 1/3 for communication)\n",
     "and takes about 4 min to run the model for the CALIBRATION period.\n",
     "There are 24 cores available in a `normal` partition on Cartesius.\n",
     "If you want to keep the running time to 4 min for e.g. CALIBRATION, you can\n",
-    "have 12 runs simultaneously. Therefore, the `popsize:12` in the cma-es option.\n",
+    "have 18 runs simultaneously. Therefore, the `popsize:18` in the cma-es option.\n",
     "If you want to get the results of the whole notebook after a\n",
     "reasenable amount of time, you can change `maxiter` in the cma-es option."
    ]
@@ -210,18 +222,18 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 6,
    "id": "fec6e64d",
    "metadata": {},
    "outputs": [],
    "source": [
-    "POPSIZE = 18  # it can be equal to number of available cores/2\n",
+    "POPSIZE = 18  # it can be equal to number of available cores * 0.75\n",
     "MAXITER = 50  # with this, the notebook takes ~4 hours (maximum)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 7,
    "id": "feda0536",
    "metadata": {},
    "outputs": [],
@@ -392,7 +404,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 8,
    "id": "e19135a5",
    "metadata": {
     "execution": {
@@ -453,7 +465,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 9,
    "id": "e19fbdeb",
    "metadata": {
     "execution": {
@@ -505,7 +517,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 10,
    "id": "8d96f4b3",
    "metadata": {
     "execution": {
@@ -581,9 +593,17 @@
     "    return fig"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "1cc3fbe3-e522-4b26-bdfe-f053b4806d17",
+   "metadata": {},
+   "source": [
+    "Since this run is done on the Cartesius supercomputer the output figure is written into the output folder as set in the configuration file. For this repo, this file has been manually copied into the figures sub-directory."
+   ]
+  },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 11,
    "id": "ebf65971",
    "metadata": {
     "execution": {
@@ -616,19 +636,11 @@
     "fig = plot_parameters(calibration_results)\n",
     "fig.savefig(f\"{filename}.png\", bbox_inches=\"tight\", dpi=300)"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "41172288",
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -642,7 +654,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.7"
+   "version": "3.9.7"
   }
  },
  "nbformat": 4,