diff --git a/pySDC/playgrounds/PinT_Workshop_2025/0_pySDC.ipynb b/pySDC/playgrounds/PinT_Workshop_2025/0_pySDC.ipynb new file mode 100644 index 0000000000..ffa4cc1753 --- /dev/null +++ b/pySDC/playgrounds/PinT_Workshop_2025/0_pySDC.ipynb @@ -0,0 +1,628 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "3edf3de0-f43b-4e1e-92cc-26d52c748375", + "metadata": {}, + "source": [ + "# pySDC in a nutshell\n", + "\n", + "pySDC = Python + spectral deferred correction (SDC) is a prototyping code for all things SDC, including PFASST.\n", + "It was started by Robert Speck about 10 years ago and is now primarily developed in Jülich and Hamburg.\n", + "The goal is to make SDC related research as accessible as possible.\n", + "It is very modular and abstract, which allows to focus only on small parts of the bigger SDC puzzle.\n", + "Furthermore, all methods are implemented in serial as well as in parallal to allow both easy developping as well as measuring performance.\n", + "\n", + "In this notebook, we will briefly look into SDC and pySDC.\n", + "\n", + "## Spectral deferred correction (SDC)\n", + "SDC is a method for the numerical integration of initial value problems of the sort $$u(\\Delta t) = \\int_0^{\\Delta t} f(u) dt + u(0).$$\n", + "For the numerical treatment, we start with discretizing time into $M$ **collocation nodes** $0 \\leq \\tau_m \\leq 1$.\n", + "Then, we interpolate $f$ $$f(u(t))\\approx\\sum_{m=1}^{M} l_m^\\tau(t/\\Delta t) f(u(\\Delta t \\tau_m))$$ using Lagrange polynomials $$l_j^{\\tau}(t)=\\frac{\\prod_{i=1, i\\neq j}^M (t-\\tau_i)}{\\prod_{i=1, i\\neq j}^M (\\tau_j-\\tau_i)}.$$\n", + "\n", + "Since only the Lagrange polynomials depend on $t$ in the interpolation, plugging into the initial value problem, we get $$u_m := u_0 + \\Delta t\\sum_{j=1}^M q_{mj}f(u(\\Delta t\\tau_j)) \\approx u(\\Delta t \\tau_m),$$ with the **quadrature weights** $$q_{mj} = \\int_{0}^{\\tau_m} l_j^\\tau(s) ds.$$\n", + "\n", + "Collecting in vectors $\\vec{u} = (u_m)^T$, we get $$(I - \\Delta tQf)(\\vec{u}) = \\vec{u}_0,$$ with $f(\\vec{u}) = (f(u_m))^T$ and $\\vec{u_0} = (u_0)^T$.\n", + "This is called the **collocation problem**.\n", + "Before discussing how to solve this, let's look at a sample quadrature matrix.\n", + "\n", + "The generation of quadrature matrices has been spun out from pySDC to a stand-alone repository [qmat](https://github.com/Parallel-in-Time/qmat).\n", + "If you use SDC outside of pySDC, please consider using qmat." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "d7545887-fd45-43f8-9185-d5059dc632b5", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Q=array([[ 0.19681548, -0.06553543, 0.02377097],\n", + " [ 0.39442431, 0.29207341, -0.04154875],\n", + " [ 0.37640306, 0.51248583, 0.11111111]])\n" + ] + } + ], + "source": [ + "from qmat import genQCoeffs\n", + "\n", + "nodes, _, Q = genQCoeffs(\"Collocation\", nNodes=3, nodeType=\"LEGENDRE\", quadType=\"RADAU-RIGHT\")\n", + "print(f'{Q=}')" + ] + }, + { + "cell_type": "markdown", + "id": "86db75e1-d4b7-4509-834f-616c0c81fd5d", + "metadata": {}, + "source": [ + "As you can see, $Q$ is densely populated.\n", + "Inverting $Q$ by itself is no problem, but if you are integrating a PDE, inverting $Qf$ can become prohibitively expensive.\n", + "Instead, we solve the collocation problem iteratively.\n", + "\n", + "The simplest iterative scheme is standard Richardson iteration $$\\vec{u}^{k+1} = \\vec{u}_0 + \\Delta tQf(\\vec{u}^k),$$ but convergence is slow and limited.\n", + "Therefore, we precondition this iteration with $\\Delta t Q_\\Delta f$ to arrive at $$(I_M - \\Delta t Q_\\Delta f)(\\vec{u}^{k+1}) = \\vec{u}_0 + \\Delta t(Q-Q_\\Delta )f(\\vec{u}^k).$$\n", + "\n", + "A typical choice for $Q_\\Delta$ would be $Q_\\Delta = U^T$, with $Q^T = LU$.\n", + "Again, we use qmat to obtain the preconditioner." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "376e0171-7e5a-4113-bd78-f316e00eb26f", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "QDelta=array([[0.19681548, 0. , 0. ],\n", + " [0.39442431, 0.42340844, 0. ],\n", + " [0.37640306, 0.63782015, 0.2 ]])\n" + ] + } + ], + "source": [ + "from qmat import genQDeltaCoeffs\n", + "\n", + "QDelta = genQDeltaCoeffs(\"LU\", Q=Q)\n", + "print(f'{QDelta=}')" + ] + }, + { + "cell_type": "markdown", + "id": "9f254a0c-035d-4348-bdf6-f8c4138a66d9", + "metadata": {}, + "source": [ + "Now you have all the ingredients to run vanilla SDC.\n", + "However, the advantage of SDC is that you can modify everything and design a crazy SDC scheme to fit your needs.\n", + "For instance, you can do splitting, multi-level, multi-step, quantum-AI, bank heist, ...\n", + "\n", + "## pySDC\n", + "Before looking at some examples of what pySDC has been used for, we discuss its structure and how to set up a run.\n", + "\n", + "We will first go through all core modules that you can / should configure.\n", + "Namely:\n", + " - problem\n", + " - sweeper\n", + " - level\n", + " - step\n", + " - controller\n", + " - hook\n", + " - convergence controller\n", + " - transfer class\n", + "\n", + "The main hierarchy in pySDC is controller > step > level > sweeper > problem.\n", + "The rest are bells and whistles.\n", + "\n", + "### Problem\n", + "At the bottom of the hierarchy are the problem objects.\n", + "These implement how to evaluate and invert $f$ for a given problem.\n", + "We will discuss in the next notebook how to implement problem classes.\n", + "\n", + "### Sweeper\n", + "Next up in the hierarchy is the sweeper.\n", + "SDC iterations are often called sweeps due to their forward substitution nature, hence the name.\n", + "The sweeper takes care of the time-discretization.\n", + "It sets up the $Q$ and $Q_\\Delta$ matrices and calls functions of the problem class during the actual time-stepping\n", + "\n", + "### Level\n", + "SDC schemes are commonly designed with multiple levels similar to a multigrid scheme.\n", + "Corrections are computed more cheaply on coarser grids.\n", + "What ``coarser'' means is up for you to decide: Fewer collocation nodes, less spatial resolution, inexact approximation of inverse of $f$, ...\n", + "Each level administers one sweeper, which it calls to do the actual time-stepping on the level itself.\n", + "\n", + "### Step\n", + "The step handles all of the levels.\n", + "It loops through them and calls the time-stepping on the individual levels.\n", + "\n", + "### Controller\n", + "The pySDC controllers are the top-level objects that, well, control everything.\n", + "The controller can have multiple steps and loops through them, calling the integration, and communicating results.\n", + "Doing time-stepping on multiple steps in Gauß-Seidel or Jacobi fashion is a crucial ingredient of PFASST.\n", + "\n", + "### Hook\n", + "Hooks are for recording data.\n", + "They are called by the controller in various places and have access to en entire level, so they can log pretty much anything.\n", + "\n", + "### Convergence contollers\n", + "These have a very misleading name.\n", + "They are called by the controller just like the hooks, but can modify anything rather than log anything.\n", + "\n", + "### Transfer class\n", + "These transfer the solution between different levels.\n", + "\n", + "\n", + "## Configuring pySDC\n", + "All of the building blocks above need to be configured for a run of pySDC.\n", + "We will now go through the process of configuring and running an Allen-Cahn equation.\n", + "We will define dictionaries of individual parameters and then gather all of them in one large dictionary which we pass to the controller." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "99161031-290b-45ff-95a5-67cae23557e8", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "controller - INFO: Welcome to the one and only, really very astonishing and 87.3% bug free\n", + " _____ _____ _____ \n", + " / ____| __ \\ / ____|\n", + " _ __ _ _| (___ | | | | | \n", + " | '_ \\| | | |\\___ \\| | | | | \n", + " | |_) | |_| |____) | |__| | |____ \n", + " | .__/ \\__, |_____/|_____/ \\_____|\n", + " | | __/ | \n", + " |_| |___/ \n", + " \n", + "controller - INFO: Setup overview (--> user-defined, -> dependency) -- BEGIN\n", + "controller - INFO: ----------------------------------------------------------------------------------------------------\n", + "\n", + "Controller: \n", + " all_to_done = False\n", + " dump_setup = True\n", + " fname = run_pid94048.log\n", + "--> hook_class = [, , ]\n", + " log_to_file = False\n", + "--> logger_level = 15\n", + "--> mssdc_jac = False\n", + " predict_type = None\n", + " use_iteration_estimator = False\n", + "\n", + "Step: \n", + "--> maxiter = 9\n", + " Number of steps: None\n", + " Level: \n", + " Level 0\n", + "--> dt = 0.01\n", + " dt_initial = 0.01\n", + " nsweeps = 1\n", + " residual_type = full_abs\n", + "--> restol = 1e-05\n", + "--> Problem: \n", + " eps = 0.04\n", + " inexact_linear_ratio = None\n", + " lin_maxiter = 100\n", + " lin_tol = 1e-08\n", + "--> newton_maxiter = 5\n", + " newton_tol = 1e-12\n", + " nu = 2\n", + "--> nvars = (128, 128)\n", + " order = 2\n", + " radius = 0.25\n", + "--> Data type u: \n", + "--> Data type f: \n", + "--> Sweeper: \n", + "--> QI = LU\n", + " do_coll_update = False\n", + " initial_guess = spread\n", + "--> num_nodes = 2\n", + "--> quad_type = RADAU-RIGHT\n", + " skip_residual_computation = ()\n", + "--> Collocation: \n", + " Level 1\n", + "--> dt = 0.01\n", + " dt_initial = 0.01\n", + " nsweeps = 1\n", + " residual_type = full_abs\n", + "--> restol = 1e-05\n", + "--> Problem: \n", + " eps = 0.04\n", + " inexact_linear_ratio = None\n", + " lin_maxiter = 100\n", + " lin_tol = 1e-08\n", + "--> newton_maxiter = 99\n", + " newton_tol = 1e-12\n", + " nu = 2\n", + "--> nvars = (64, 64)\n", + " order = 2\n", + " radius = 0.25\n", + "--> Data type u: \n", + "--> Data type f: \n", + "--> Sweeper: \n", + "--> QI = MIN-SR-S\n", + " do_coll_update = False\n", + " initial_guess = spread\n", + "--> num_nodes = 4\n", + "--> quad_type = RADAU-RIGHT\n", + " skip_residual_computation = ()\n", + "--> Collocation: \n", + " Base Transfer: \n", + " finter = False\n", + "--> Space Transfer: \n", + " equidist_nested = True\n", + " iorder = 2\n", + " periodic = False\n", + " rorder = 2\n", + "\n", + "Active convergence controllers:\n", + " | # | order | convergence controller\n", + "----+----+-------+---------------------------------------------------------------------------------------\n", + " | 0 | 95 | BasicRestartingNonMPI\n", + " -> | 1 | 100 | SpreadStepSizesBlockwiseNonMPI\n", + " | 2 | 200 | CheckConvergence\n", + "\n", + "controller - INFO: ----------------------------------------------------------------------------------------------------\n", + "controller - INFO: Setup overview (--> user-defined, -> dependency) -- END\n", + "\n" + ] + } + ], + "source": [ + "from pySDC.implementations.sweeper_classes.generic_implicit import generic_implicit\n", + "from pySDC.implementations.controller_classes.controller_nonMPI import controller_nonMPI\n", + "from pySDC.implementations.hooks.log_solution import LogSolution\n", + "from pySDC.implementations.problem_classes.AllenCahn_2D_FD import allencahn_fullyimplicit\n", + "from pySDC.implementations.convergence_controller_classes.adaptivity import Adaptivity\n", + "from pySDC.implementations.transfer_classes.TransferMesh import mesh_to_mesh\n", + "\n", + "# level and step parameters contain general parameters such as how many SDC iterations will be done\n", + "level_params = {}\n", + "level_params['dt'] = 1e-2\n", + "level_params['restol'] = 1e-5\n", + "\n", + "step_params = {}\n", + "step_params['maxiter'] = 9\n", + "\n", + "# the sweeper parameters describe the collocation problem and the preconditioner\n", + "sweeper_params = {}\n", + "sweeper_params['quad_type'] = 'RADAU-RIGHT'\n", + "sweeper_params['num_nodes'] = [2, 4]\n", + "sweeper_params['QI'] = ['LU', 'MIN-SR-S']\n", + "\n", + "problem_params = {'nvars': [(128,) * 2, (64,) * 2], 'newton_maxiter': [5, 99]}\n", + "\n", + "convergence_controllers = {}\n", + "# convergence_controllers[Adaptivity] = {'e_tol': 1e-4} # for instance\n", + "\n", + "# gather all parameters in one dictionary and add problem and sweeper class\n", + "description = {}\n", + "description['problem_class'] = allencahn_fullyimplicit\n", + "description['problem_params'] = problem_params\n", + "description['sweeper_class'] = generic_implicit\n", + "description['sweeper_params'] = sweeper_params\n", + "description['level_params'] = level_params\n", + "description['step_params'] = step_params\n", + "description['convergence_controllers'] = convergence_controllers\n", + "description['space_transfer_class'] = mesh_to_mesh\n", + "\n", + "# more parameters for the controller\n", + "controller_params = {}\n", + "controller_params['logger_level'] = 15\n", + "controller_params['hook_class'] = [LogSolution]\n", + "controller_params['mssdc_jac'] = False\n", + "\n", + "# setup controller\n", + "controller = controller_nonMPI(controller_params=controller_params, description=description, num_procs=2)" + ] + }, + { + "cell_type": "markdown", + "id": "da0f2bcc-cbdc-4001-9828-18a964f3e4fa", + "metadata": {}, + "source": [ + "The above output shows the configuration that we just set up.\n", + "`-->` indicates that we set a parameter manually.\n", + "Other parameters are default values.\n", + "\n", + "We chose a funny configuration with 2 collocation nodes, LU preconditioner, high resolution and inexact Newton solver on the fine level.\n", + "One the coarse level, we chose 4 collocation nodes, but diagonal preconditioner, lower spatial resultion, but we solve the non-linear problems to high accuracy.\n", + "Why did we do that?\n", + "To show off how easy it is to configure multi-level SDC in pySDC.\n", + "Finally, we chose 2 steps in parallel in Gauß-Seidel mode.\n", + "\n", + "Was this a good idea?\n", + "Let's run this configuration and find out!" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "8717f118-1d85-46ac-9b83-11ee73e58086", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "hooks - INFO: Process 0 on time 0.000000 at stage IT_COARSE: Level: 1 -- Iteration: 1 -- Sweep: 1 -- residual: 6.55548055e-02\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 1 -- Sweep: 1 -- residual: 4.33590968e-01\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 5.69274516e-01\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 7.25755959e-01\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_COARSE: Level: 1 -- Iteration: 2 -- Sweep: 1 -- residual: 1.52277915e-02\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 2 -- Sweep: 1 -- residual: 1.42393824e-01\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 5.95755213e-03\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 4.15178168e-02\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_COARSE: Level: 1 -- Iteration: 3 -- Sweep: 1 -- residual: 5.14520992e-03\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 3 -- Sweep: 1 -- residual: 4.52164626e-02\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 1.64184578e-03\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 1.06977065e-02\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_COARSE: Level: 1 -- Iteration: 4 -- Sweep: 1 -- residual: 1.98454721e-03\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 4 -- Sweep: 1 -- residual: 1.38149253e-02\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 1.09509225e-03\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 3.33496832e-03\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_COARSE: Level: 1 -- Iteration: 5 -- Sweep: 1 -- residual: 1.35676523e-03\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 5 -- Sweep: 1 -- residual: 4.33758300e-03\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 5 -- Sweep: 1 -- residual: 7.69111709e-04\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 5 -- Sweep: 1 -- residual: 2.54391448e-03\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_COARSE: Level: 1 -- Iteration: 6 -- Sweep: 1 -- residual: 9.49296378e-04\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 6 -- Sweep: 1 -- residual: 3.28987669e-03\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 6 -- Sweep: 1 -- residual: 5.45954260e-04\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 6 -- Sweep: 1 -- residual: 1.94430599e-03\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_COARSE: Level: 1 -- Iteration: 7 -- Sweep: 1 -- residual: 6.72609330e-04\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 7 -- Sweep: 1 -- residual: 2.50390906e-03\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 7 -- Sweep: 1 -- residual: 3.89725136e-04\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 7 -- Sweep: 1 -- residual: 1.48542866e-03\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_COARSE: Level: 1 -- Iteration: 8 -- Sweep: 1 -- residual: 4.79710468e-04\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 8 -- Sweep: 1 -- residual: 1.90666008e-03\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 8 -- Sweep: 1 -- residual: 2.79036000e-04\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 8 -- Sweep: 1 -- residual: 1.13298624e-03\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_COARSE: Level: 1 -- Iteration: 9 -- Sweep: 1 -- residual: 3.43322868e-04\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 9 -- Sweep: 1 -- residual: 1.45034263e-03\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 9 -- Sweep: 1 -- residual: 2.00115081e-04\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 9 -- Sweep: 1 -- residual: 8.62214360e-04\n", + "hooks - INFO: Finished run after 1.32e+00s\n", + "hooks - INFO: Finished run after 1.32e+00s\n" + ] + } + ], + "source": [ + "# get initial conditions\n", + "P = controller.MS[0].levels[0].prob # observe the hierarchy\n", + "uinit = P.u_exact(t=0)\n", + "\n", + "uend, stats = controller.run(u0=uinit, t0=0, Tend=2e-2)" + ] + }, + { + "cell_type": "markdown", + "id": "52b55350-d459-40af-a948-d610a222e2c5", + "metadata": {}, + "source": [ + "Well, we can already see that the residual did not reach the tolerance we set within the iteration limit.\n", + "So: no, this was not a smart idea.\n", + "Not all SDC variants are always useful.\n", + "But, if you actually have a think about what you're doing rather than just picking random configurations, SDC can be a quite powerful method!\n", + "\n", + "For now, let's throw out the weird time coarsening and only do coarsening in space:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "6318d0bf-bbea-4a35-929d-15c7693ae392", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "controller - INFO: Welcome to the one and only, really very astonishing and 87.3% bug free\n", + " _____ _____ _____ \n", + " / ____| __ \\ / ____|\n", + " _ __ _ _| (___ | | | | | \n", + " | '_ \\| | | |\\___ \\| | | | | \n", + " | |_) | |_| |____) | |__| | |____ \n", + " | .__/ \\__, |_____/|_____/ \\_____|\n", + " | | __/ | \n", + " |_| |___/ \n", + " \n", + "controller - INFO: Setup overview (--> user-defined, -> dependency) -- BEGIN\n", + "controller - INFO: ----------------------------------------------------------------------------------------------------\n", + "\n", + "Controller: \n", + " all_to_done = False\n", + " dump_setup = True\n", + " fname = run_pid94048.log\n", + "--> hook_class = [, , , , ]\n", + " log_to_file = False\n", + "--> logger_level = 15\n", + "--> mssdc_jac = False\n", + " predict_type = None\n", + " use_iteration_estimator = False\n", + "\n", + "Step: \n", + "--> maxiter = 9\n", + " Number of steps: None\n", + " Level: \n", + " Level 0\n", + "--> dt = 0.01\n", + " dt_initial = 0.01\n", + " nsweeps = 1\n", + " residual_type = full_abs\n", + "--> restol = 1e-05\n", + "--> Problem: \n", + " eps = 0.04\n", + " inexact_linear_ratio = None\n", + " lin_maxiter = 100\n", + " lin_tol = 1e-08\n", + "--> newton_maxiter = 5\n", + " newton_tol = 1e-12\n", + " nu = 2\n", + "--> nvars = (128, 128)\n", + " order = 2\n", + " radius = 0.25\n", + "--> Data type u: \n", + "--> Data type f: \n", + "--> Sweeper: \n", + "--> QI = LU\n", + " do_coll_update = False\n", + " initial_guess = spread\n", + "--> num_nodes = 2\n", + "--> quad_type = RADAU-RIGHT\n", + " skip_residual_computation = ()\n", + "--> Collocation: \n", + " Level 1\n", + "--> dt = 0.01\n", + " dt_initial = 0.01\n", + " nsweeps = 1\n", + " residual_type = full_abs\n", + "--> restol = 1e-05\n", + "--> Problem: \n", + " eps = 0.04\n", + " inexact_linear_ratio = None\n", + " lin_maxiter = 100\n", + " lin_tol = 1e-08\n", + "--> newton_maxiter = 99\n", + " newton_tol = 1e-12\n", + " nu = 2\n", + "--> nvars = (64, 64)\n", + " order = 2\n", + " radius = 0.25\n", + "--> Data type u: \n", + "--> Data type f: \n", + "--> Sweeper: \n", + "--> QI = LU\n", + " do_coll_update = False\n", + " initial_guess = spread\n", + "--> num_nodes = 2\n", + "--> quad_type = RADAU-RIGHT\n", + " skip_residual_computation = ()\n", + "--> Collocation: \n", + " Base Transfer: \n", + " finter = False\n", + "--> Space Transfer: \n", + " equidist_nested = True\n", + " iorder = 2\n", + " periodic = False\n", + " rorder = 2\n", + "\n", + "Active convergence controllers:\n", + " | # | order | convergence controller\n", + "----+----+-------+---------------------------------------------------------------------------------------\n", + " | 0 | 95 | BasicRestartingNonMPI\n", + " -> | 1 | 100 | SpreadStepSizesBlockwiseNonMPI\n", + " | 2 | 200 | CheckConvergence\n", + "\n", + "controller - INFO: ----------------------------------------------------------------------------------------------------\n", + "controller - INFO: Setup overview (--> user-defined, -> dependency) -- END\n", + "\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_COARSE: Level: 1 -- Iteration: 1 -- Sweep: 1 -- residual: 7.06018493e-02\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 1 -- Sweep: 1 -- residual: 1.91239637e-01\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 5.88971245e-01\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 6.02451881e-01\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_COARSE: Level: 1 -- Iteration: 2 -- Sweep: 1 -- residual: 3.51680670e-04\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 2 -- Sweep: 1 -- residual: 1.18011983e-02\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 1.05002026e-03\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 4.36035266e-02\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_COARSE: Level: 1 -- Iteration: 3 -- Sweep: 1 -- residual: 3.14237472e-05\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 3 -- Sweep: 1 -- residual: 7.91158992e-04\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 7.37972897e-05\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 3.16058919e-03\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_COARSE: Level: 1 -- Iteration: 4 -- Sweep: 1 -- residual: 3.10346082e-07\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 4 -- Sweep: 1 -- residual: 9.54056277e-06\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 1.80839425e-06\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 3.97339053e-05\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_COARSE: Level: 1 -- Iteration: 5 -- Sweep: 1 -- residual: 7.79394248e-08\n", + "hooks - INFO: Process 1 on time 0.010000 at stage IT_FINE: Level: 0 -- Iteration: 5 -- Sweep: 1 -- residual: 2.63545359e-07\n", + "hooks - INFO: Finished run after 6.12e-01s\n", + "hooks - INFO: Finished run after 6.12e-01s\n" + ] + } + ], + "source": [ + "sweeper_params['QI'] = 'LU'\n", + "sweeper_params['num_nodes'] = 2\n", + "controller = controller_nonMPI(controller_params=controller_params, description=description, num_procs=2)\n", + "uend, stats = controller.run(u0=uinit, t0=0, Tend=2e-2)" + ] + }, + { + "cell_type": "markdown", + "id": "07bf648b-af90-4bc5-8f60-7ccde38077cd", + "metadata": {}, + "source": [ + "Ah, much better.\n", + "Feel free to experiment further with the parameters to achieve even faster convergence.\n", + "\n", + "## Some projects with pySDC\n", + "We will now look at a few representative projects done with pySDC\n", + "\n", + "3D runs of Rayleigh-Benard convection and Gray-Scott\n", + "\n", + " \n", + "\n", + "GPU implementation of 3D Gray-Scott scales to all of JUWELS booster when extending space scaling with diagonal SDC\n", + "\n", + "\n", + "\n", + "Compare run time of various SDC configurations and compare preconditioner performance\n", + "\n", + " \n", + "\n", + "Compare wall time of SDC against reference RK method\n", + "\n", + "\n", + "\n", + "Recover from faults in PFASST by interpolating from nearby steps\n", + "\n", + "\n", + "\n", + "SDC maintains convergence order up to compression threshold when storing compressed data \n", + "\n", + "\n", + "\n", + "As you can see, pySDC is a flexible tool, capable of loads of things.\n", + "If you want to \n", + " - design a novel SDC scheme\n", + " - solve the heat equation\n", + " - count iterations\n", + " - solve very complicated equations\n", + " - measure wall time in actual HPC settings\n", + " - investigate any SDC related idea\n", + " - like your code tested\n", + "\n", + "Then pySDC is the code for you!\n", + "Get in touch if you want to collaborate or need help with anything!" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "pySDC_tutorial", + "language": "python", + "name": "pysdc_tutorial" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/pySDC/playgrounds/PinT_Workshop_2025/1_Add_problem_class.ipynb b/pySDC/playgrounds/PinT_Workshop_2025/1_Add_problem_class.ipynb new file mode 100644 index 0000000000..17c405a445 --- /dev/null +++ b/pySDC/playgrounds/PinT_Workshop_2025/1_Add_problem_class.ipynb @@ -0,0 +1,649 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "ec87b1b6-dd1a-4add-a225-c0f50e0898d4", + "metadata": {}, + "source": [ + "# Adding a problem to pySDC\n", + "\n", + "In this notebook, we will implement a simple advection-diffusion equation $$u_t = f(u) = \\nu u_{xx} + cu_x$$ with periodic boundary conditions using finite differences.\n", + "Let's have a look at the math that we need to implement in order to integrate this with SDC.\n", + "\n", + "The SDC iteration is $$(1-\\Delta t \\tilde{q}_{m+1,m+1} f)(u_{m+1}^{k+1}) = u_0 + \\Delta t \\sum_{j=1}^m \\tilde{q}_{m+1, j} f(u_{j}^{k+1}) -\\Delta t \\sum_{j=1}^{m+1}\\tilde{q}_{m+1,j} f(u_{j}^{k}) + \\Delta t \\sum_{j=1}^M q_{m+1, j} (f)(u_{j}^k), \\quad m = 0, \\dots, M-1,$$ where $q_{ij}$ and $\\tilde{q}_{ij}$ are the entries of the quadrature matrix and preconditioner respectively.\n", + "The right hand side is a weighted sum of evaluations of $f$ with solutions at various collocation nodes and iterations and the left hand side is essentially an implicit Euler step with step size modified by the preconditioner.\n", + "Therefore, the only math we need to implement in the problem class is **evaluating $f$** and **solving implicit Euler steps**.\n", + "\n", + "Keep in mind that you can use an explicit preconditioner, in which case you don't even need the implicit Euler solves, but explicit SDC is rarely useful.\n", + "If implementing an implicit Euler step for your problem is too difficult, consider splitting off difficult terms and integrating them explicitly or designing a novel SDC scheme.\n", + "The iterative nature of SDC allows a lot of freedom which is one of the key advantages of SDC." + ] + }, + { + "cell_type": "markdown", + "id": "030046d9-7d59-4240-b115-0367dc45becd", + "metadata": {}, + "source": [ + "## Evaluating $f$\n", + "\n", + "In finite difference discretizations, we compute derivatives by multiplying matrices with a vector of solution values at the grid points.\n", + "In pySDC we have some infrastructure for generating finite difference matrices, making this very easy to implement.\n", + "\n", + "We generate one matrix for the advection part, i.e. the first derivative, and one matrix for the diffusion part (the second derivative).\n", + "The $f$ evaluation is then a weighted sum of the matrices multiplied by the solution vector." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "a6a52388-db37-4bf3-a721-936013c87c2a", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "from pySDC.helpers.problem_helper import get_finite_difference_matrix, get_1d_grid\n", + "\n", + "# problem parameters\n", + "nu = 1e-3\n", + "c = 1e-1\n", + "N = 128\n", + "\n", + "# setup grid\n", + "dx, grid = get_1d_grid(size=N, bc='periodic', left_boundary=0, right_boundary=2 * np.pi)\n", + "\n", + "# setup finite difference matrices\n", + "fd_params = {'order': 4, 'stencil_type': 'center', 'dx': dx, 'size': N, 'dim': 1, 'bc': 'periodic'}\n", + "advection, _ = get_finite_difference_matrix(derivative=1, **fd_params)\n", + "diffusion, _ = get_finite_difference_matrix(derivative=2, **fd_params)\n", + "\n", + "\n", + "def eval_f(u):\n", + " return (c * advection + nu * diffusion) @ u" + ] + }, + { + "cell_type": "markdown", + "id": "12f8ba5f-536b-4442-b290-97d1e3156b29", + "metadata": {}, + "source": [ + "Before moving on, we briefly verify that we did the correct thing by comparing the numerical derivatives to exact ones for a simple sine wave:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "0175c5c7-c6a1-4294-a05b-dbce06aab22d", + "metadata": {}, + "outputs": [], + "source": [ + "u = np.sin(grid)\n", + "assert np.allclose(diffusion @ u, -np.sin(grid)) # check diffusion part\n", + "assert np.allclose(advection @ u, np.cos(grid)) # check advection part\n", + "assert np.allclose(eval_f(u), -nu * np.sin(grid) + c * np.cos(grid)) # check f evaluation" + ] + }, + { + "cell_type": "markdown", + "id": "cd1a83c9-4c8a-478c-b03d-62d15051158c", + "metadata": {}, + "source": [ + "## Implicit Euler step\n", + "\n", + "In the implicit Euler step, we solve $(1-\\Delta t f)(u) = y$, where $y$ is some arbitrary right hand side. We use a simple direct solver and the matrices we used to evaluate $f$ here:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "6255bfe5-ea1a-4935-806d-ced08889f8fc", + "metadata": {}, + "outputs": [], + "source": [ + "from scipy.sparse.linalg import spsolve\n", + "from scipy.sparse import eye\n", + "\n", + "\n", + "def implicit_euler(rhs, dt):\n", + " A = eye(N) - dt * (c * advection + nu * diffusion)\n", + " return spsolve(A, rhs)" + ] + }, + { + "cell_type": "markdown", + "id": "018b2fe3-063f-47da-8a8f-54cfe6448bd1", + "metadata": {}, + "source": [ + "Let's once again verify the implementation. This time, we will compare to exact solutions for either the diffusion or the advection equation, which we obtain by setting the $c$ or $\\nu$ parameters to 0." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "b64b8a1c-0dba-4deb-bfef-ddf694a400e0", + "metadata": {}, + "outputs": [], + "source": [ + "u = np.sin(grid)\n", + "dt = 1e-2\n", + "\n", + "# check advection\n", + "nu = 0\n", + "c = 1e-1\n", + "assert np.allclose(implicit_euler(u, dt), np.sin(grid + c * dt))\n", + "\n", + "# check diffusion\n", + "nu = 1e-1\n", + "c = 0\n", + "assert np.allclose(implicit_euler(u, dt), np.sin(grid) * np.exp(-nu * dt))" + ] + }, + { + "cell_type": "markdown", + "id": "1a429069-cada-46e2-9a6b-c2560cfdcfd2", + "metadata": {}, + "source": [ + "## Implementing this class in pySDC\n", + "\n", + "So far, we have looked at what math you need to implement in order to do SDC with your problem, now we will set up a class with the same functionality that we can then hand to pySDC to actually run SDC.\n", + "\n", + "Here, it is important to use the interface defined in the sweeper classes, which take care of the actual time integration.\n", + "Also, we need to add some infrastructure for generating empty data containers, and we add some work counters for analysis later on.\n", + "But you will find that the implementation in pySDC is not significantly more complicated or convoluted than what we did before." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "2c2c43e2-d0a9-417b-8ccb-ee14cbabdd39", + "metadata": {}, + "outputs": [], + "source": [ + "from pySDC.core.problem import Problem, WorkCounter\n", + "from pySDC.implementations.datatype_classes.mesh import mesh\n", + "import scipy.sparse as sp\n", + "import numpy as np\n", + "\n", + "\n", + "class AdvectionDiffusion(Problem):\n", + " dtype_u = mesh # wraps numpy ndarray\n", + " dtype_f = mesh\n", + "\n", + " def __init__(self, N, c, nu):\n", + " init = (N, None, np.dtype('float64')) # describes how to initialize data\n", + " super().__init__(init=init)\n", + "\n", + " # setup grid\n", + " dx, self.grid = get_1d_grid(size=N, bc='periodic', left_boundary=0, right_boundary=2 * np.pi)\n", + "\n", + " # setup finite difference matrices\n", + " fd_params = {'order': 4, 'stencil_type': 'center', 'dx': dx, 'size': N, 'dim': 1, 'bc': 'periodic'}\n", + " advection, _ = get_finite_difference_matrix(derivative=1, **fd_params)\n", + " diffusion, _ = get_finite_difference_matrix(derivative=2, **fd_params)\n", + "\n", + " # setup matrices used in eval_f and implicit solves\n", + " self.A = c * advection + nu * diffusion\n", + " self.Id = sp.eye(N)\n", + "\n", + " # store attributes and register them as parameters\n", + " self._makeAttributeAndRegister('N', 'c', 'nu', localVars=locals(), readOnly=True)\n", + "\n", + " # set up counters\n", + " self.work_counters['eval_f'] = WorkCounter()\n", + "\n", + " def eval_f(self, u, t):\n", + " self.work_counters['eval_f']() # increment counter\n", + " me = self.f_init # get empty data container\n", + " me[...] = self.A @ u\n", + " return me\n", + "\n", + " # the implicit Euler step is called solve_system here because it could be sth. else in a different SDC scheme\n", + " # u0 is the initial guess for the solver\n", + " def solve_system(self, rhs, dt, u0, t):\n", + " me = self.u_init\n", + "\n", + " # prepare a new work counter for this factor / step size. Seems weird now, but bear with me.\n", + " key = f'gmres-{dt:.2e}'\n", + " self.work_counters[f'gmres-{dt:.2e}'] = self.work_counters.get(key, WorkCounter())\n", + "\n", + " me[...], _ = sp.linalg.gmres(\n", + " self.Id - dt * self.A, rhs, x0=u0, callback=self.work_counters[key], rtol=1e-10, callback_type='pr_norm'\n", + " )\n", + " return me\n", + "\n", + " def u0(self):\n", + " me = self.u_init\n", + " me[...] = np.sin(self.grid)\n", + " return me" + ] + }, + { + "cell_type": "markdown", + "id": "67758807-7f0c-4972-9dd8-e03cebc27b10", + "metadata": {}, + "source": [ + "Let's do the same tests once more.\n", + "Note that the pySDC `mesh` datatype has a different `abs` function than `numpy.ndarray`." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "98e6c97a-8fd1-4947-8981-d31f7b6e7a0b", + "metadata": {}, + "outputs": [], + "source": [ + "prob = AdvectionDiffusion(128, 1e-1, 1e-3)\n", + "u = prob.u0()\n", + "assert np.allclose(prob.eval_f(u, 0), -prob.nu * np.sin(prob.grid) + prob.c * np.cos(prob.grid))\n", + "\n", + "dt = 1e-2\n", + "advection_prob = AdvectionDiffusion(N=128, c=1e-1, nu=0)\n", + "u = advection_prob.u0()\n", + "assert abs(advection_prob.solve_system(u, dt, u, 0) - np.sin(advection_prob.grid + advection_prob.c * dt)) < 1e-6\n", + "\n", + "diffusion_prob = AdvectionDiffusion(N=128, c=0, nu=1e-1)\n", + "u = diffusion_prob.u0()\n", + "assert (\n", + " abs(diffusion_prob.solve_system(u, dt, u, 0) - np.sin(diffusion_prob.grid) * np.exp(-diffusion_prob.nu * dt)) < 1e-6\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "d332961c-f6ce-40fe-92ee-bc37e651d731", + "metadata": {}, + "source": [ + "Looks good!\n", + "Now let's set up pySDC to integrate this problem with SDC.\n", + "\n", + "## Running with pySDC\n", + "\n", + "We will do small scale time-parallelism by choosing a diagonal preconditioner called [MIN-SR-S](https://arxiv.org/abs/2403.18641).\n", + "Since we are running this in a notebook, we will resort to counting iterations rather than measuring run time and use serial implementations of everything.\n", + "SDC has a vast parameter space and we need to supply loads of parameters here.\n", + "Don't worry too much about them for now." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "a0820a19-f5d4-4295-8e35-8b22c555e369", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "controller - INFO: Welcome to the one and only, really very astonishing and 87.3% bug free\n", + " _____ _____ _____ \n", + " / ____| __ \\ / ____|\n", + " _ __ _ _| (___ | | | | | \n", + " | '_ \\| | | |\\___ \\| | | | | \n", + " | |_) | |_| |____) | |__| | |____ \n", + " | .__/ \\__, |_____/|_____/ \\_____|\n", + " | | __/ | \n", + " |_| |___/ \n", + " \n", + "controller - INFO: Setup overview (--> user-defined, -> dependency) -- BEGIN\n", + "controller - INFO: ----------------------------------------------------------------------------------------------------\n", + "\n", + "Controller: \n", + " all_to_done = False\n", + " dump_setup = True\n", + " fname = run_pid94051.log\n", + "--> hook_class = [, , , ]\n", + " log_to_file = False\n", + "--> logger_level = 15\n", + " mssdc_jac = True\n", + " predict_type = None\n", + " use_iteration_estimator = False\n", + "\n", + "Step: \n", + "--> maxiter = 9\n", + " Number of steps: None\n", + " Level: \n", + " Level 0\n", + "--> dt = 0.1\n", + " dt_initial = 0.1\n", + " nsweeps = 1\n", + " residual_type = full_abs\n", + "--> restol = 1e-08\n", + "--> Problem: \n", + "--> N = 128\n", + "--> c = 0.8\n", + "--> nu = 0.1\n", + "--> Data type u: \n", + "--> Data type f: \n", + "--> Sweeper: \n", + "--> QI = MIN-SR-S\n", + " do_coll_update = False\n", + " initial_guess = spread\n", + "--> num_nodes = 3\n", + "--> quad_type = RADAU-RIGHT\n", + " skip_residual_computation = ()\n", + "--> Collocation: \n", + "\n", + "Active convergence controllers:\n", + " | # | order | convergence controller\n", + "----+----+-------+---------------------------------------------------------------------------------------\n", + " | 0 | 95 | BasicRestartingNonMPI\n", + " -> | 1 | 100 | SpreadStepSizesBlockwiseNonMPI\n", + " | 2 | 200 | CheckConvergence\n", + "\n", + "controller - INFO: ----------------------------------------------------------------------------------------------------\n", + "controller - INFO: Setup overview (--> user-defined, -> dependency) -- END\n", + "\n" + ] + } + ], + "source": [ + "from pySDC.implementations.sweeper_classes.generic_implicit import generic_implicit\n", + "from pySDC.implementations.controller_classes.controller_nonMPI import controller_nonMPI\n", + "from pySDC.implementations.hooks.log_solution import LogSolution\n", + "from pySDC.implementations.hooks.log_work import LogWork\n", + "\n", + "# level and step parameters contain general parameters such as how many SDC iterations will be done\n", + "level_params = {}\n", + "level_params['dt'] = 1e-1\n", + "level_params['restol'] = 1e-8\n", + "\n", + "step_params = {}\n", + "step_params['maxiter'] = 9\n", + "\n", + "# the sweeper parameters describe the collocation problem and the preconditioner\n", + "sweeper_params = {}\n", + "sweeper_params['quad_type'] = 'RADAU-RIGHT'\n", + "sweeper_params['num_nodes'] = 3\n", + "sweeper_params['QI'] = 'MIN-SR-S'\n", + "\n", + "problem_params = {\n", + " 'N': 128,\n", + " 'c': 8e-1,\n", + " 'nu': 1e-1,\n", + "}\n", + "\n", + "# gather all parameters in one dictionary and add problem and sweeper class\n", + "description = {}\n", + "description['problem_class'] = AdvectionDiffusion\n", + "description['problem_params'] = problem_params\n", + "description['sweeper_class'] = generic_implicit\n", + "description['sweeper_params'] = sweeper_params\n", + "description['level_params'] = level_params\n", + "description['step_params'] = step_params\n", + "\n", + "# more parameters for the controller\n", + "controller_params = {}\n", + "controller_params['logger_level'] = 15\n", + "controller_params['hook_class'] = [LogSolution, LogWork]\n", + "\n", + "# setup controller\n", + "controller = controller_nonMPI(controller_params=controller_params, description=description, num_procs=1)" + ] + }, + { + "cell_type": "markdown", + "id": "32bf2d4d-9741-42ad-890d-7c3213a171ac", + "metadata": {}, + "source": [ + "Above, we see all the configuration that we have passed to the controller.\n", + "`-->` indicates that we have manually set this parameter, while other parameters are left at their default value.\n", + "\n", + "Now that we have a pySDC controller, we set up some initial conditions and run" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "adcfe6fb-4703-42bc-9fbe-6c9460a324b0", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 1.31471144e-04\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 6.32549457e-06\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 1.48579940e-07\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 3.29160453e-09\n", + "hooks - INFO: Process 0 on time 0.100000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 1.30153942e-04\n", + "hooks - INFO: Process 0 on time 0.100000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 6.26183780e-06\n", + "hooks - INFO: Process 0 on time 0.100000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 1.47216986e-07\n", + "hooks - INFO: Process 0 on time 0.100000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 3.81488605e-09\n", + "hooks - INFO: Process 0 on time 0.200000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 1.28848786e-04\n", + "hooks - INFO: Process 0 on time 0.200000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 6.20062048e-06\n", + "hooks - INFO: Process 0 on time 0.200000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 1.45786837e-07\n", + "hooks - INFO: Process 0 on time 0.200000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 3.83750116e-09\n", + "hooks - INFO: Process 0 on time 0.300000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 1.27587502e-04\n", + "hooks - INFO: Process 0 on time 0.300000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 6.13820935e-06\n", + "hooks - INFO: Process 0 on time 0.300000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 1.44303910e-07\n", + "hooks - INFO: Process 0 on time 0.300000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 3.70578766e-09\n", + "hooks - INFO: Process 0 on time 0.400000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 1.26296458e-04\n", + "hooks - INFO: Process 0 on time 0.400000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 6.07719616e-06\n", + "hooks - INFO: Process 0 on time 0.400000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 1.42785892e-07\n", + "hooks - INFO: Process 0 on time 0.400000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 3.43194700e-09\n", + "hooks - INFO: Process 0 on time 0.500000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 1.25051223e-04\n", + "hooks - INFO: Process 0 on time 0.500000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 6.01731027e-06\n", + "hooks - INFO: Process 0 on time 0.500000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 1.41932824e-07\n", + "hooks - INFO: Process 0 on time 0.500000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 3.30244970e-09\n", + "hooks - INFO: Process 0 on time 0.600000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 1.23815080e-04\n", + "hooks - INFO: Process 0 on time 0.600000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 5.95614823e-06\n", + "hooks - INFO: Process 0 on time 0.600000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 1.40058028e-07\n", + "hooks - INFO: Process 0 on time 0.600000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 3.36373869e-09\n", + "hooks - INFO: Process 0 on time 0.700000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 1.22549979e-04\n", + "hooks - INFO: Process 0 on time 0.700000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 5.89795794e-06\n", + "hooks - INFO: Process 0 on time 0.700000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 1.38562885e-07\n", + "hooks - INFO: Process 0 on time 0.700000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 3.25299655e-09\n", + "hooks - INFO: Process 0 on time 0.800000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 1.21361853e-04\n", + "hooks - INFO: Process 0 on time 0.800000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 5.83931787e-06\n", + "hooks - INFO: Process 0 on time 0.800000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 1.37323737e-07\n", + "hooks - INFO: Process 0 on time 0.800000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 3.30331557e-09\n", + "hooks - INFO: Process 0 on time 0.900000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 1.20149976e-04\n", + "hooks - INFO: Process 0 on time 0.900000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 5.78007793e-06\n", + "hooks - INFO: Process 0 on time 0.900000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 1.36492229e-07\n", + "hooks - INFO: Process 0 on time 0.900000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 3.15861094e-09\n", + "hooks - INFO: Finished run after 8.51e-02s\n" + ] + } + ], + "source": [ + "# get initial conditions\n", + "P = controller.MS[0].levels[0].prob\n", + "uinit = P.u0()\n", + "\n", + "uend, stats = controller.run(u0=uinit, t0=0, Tend=1)" + ] + }, + { + "cell_type": "markdown", + "id": "72e0c958-f092-472f-9076-1a6b30c3ae0b", + "metadata": {}, + "source": [ + "Above we see very high-level information about what has happened.\n", + "We can monitor the decrease of the residual of the collocation problem and see that in all steps it was below our desired tolerance after 4 SDC iterations.\n", + "\n", + "## Analysing a pySDC run\n", + "\n", + "The controller returned two things: The solution at the end of the run and a dictionary containing a lot of statistics.\n", + "Let's first inspect what sort of data we have available in `stats`." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "245b0daa-1774-4eef-9bd9-8b4fda035f0b", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['residual_post_sweep', 'residual_post_iteration', 'niter', 'residual_post_step', '_recomputed', 'timing_setup', 'timing_comm', 'timing_sweep', 'timing_iteration', 'timing_step', 'timing_run', 'u', 'work_eval_f', 'work_gmres-1.04e-02', 'work_gmres-3.33e-02', 'work_gmres-4.81e-02', 'restart']\n" + ] + } + ], + "source": [ + "from pySDC.helpers.stats_helper import get_list_of_types\n", + "\n", + "print(get_list_of_types(stats))" + ] + }, + { + "cell_type": "markdown", + "id": "73a96404-7e08-4ab8-b1ed-a052a53fe991", + "metadata": {}, + "source": [ + "Things like residuals and timings are automatically recorded, but others are recorded because we supplied certain hooks.\n", + "In this case, we specified to log the solution and the work.\n", + "The results are called `u`, `work_gmres-[...]`, and `work_eval_f` here.\n", + "The latter are generated from the work counters that we set up in the problem class.\n", + "\n", + "Let's start by looking at how the solution has evolved.\n", + "We use the `get_sorted` function to sort the solution values by time.\n", + "This function returns a list in which the values are tuples of the key we sorted by and the value itself." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "2b78c149-6ed7-4d58-af0d-f0fe11479cfe", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAksAAAGwCAYAAAC5ACFFAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAzEVJREFUeJzs3Xd4VNXWwOHfzCQz6ZNGGglJ6L2FktCkGUAQEBCw0ItgQUCvil2v90OsiAiCUgVp0pXeW+gEpNeQQkII6X3K+f4YiKACmXMCyYT9Ps88GcLZa9YIkjX77L22SpIkCUEQBEEQBOFfqUs7AUEQBEEQhLJMFEuCIAiCIAj3IYolQRAEQRCE+xDFkiAIgiAIwn2IYkkQBEEQBOE+RLEkCIIgCIJwH6JYEgRBEARBuA+70k6gPDCbzVy7dg1XV1dUKlVppyMIgiAIQjFIkkRWVhYBAQGo1feePxLFUgm4du0aQUFBpZ2GIAiCIAgyxMXFERgYeM/fF8VSCXB1dQUs/7Hd3NxKORtBEARBEIojMzOToKCgop/j9yKKpRJw+9abm5ubKJYEQRAEwcY8aAmNWOAtCIIgCIJwH6JYEgRBEARBuA9RLAmCIAiCINyHKJYEQRAEQRDuQxRLgiAIgiAI9yGKJUEQBEEQhPsQxZIgCIIgCMJ9iGJJEARBEAThPkSxJAiCIAiCcB82VSzt2rWLp59+moCAAFQqFatWrXrgmJ07dxIWFoaDgwOVK1fmxx9//Mc1y5cvp3bt2uh0OmrXrs3KlSsfQvaCIAiCINgimyqWcnJyaNCgAVOnTi3W9VeuXOGpp56idevWHDt2jHfffZcxY8awfPnyomuioqLo168fAwYM4Pjx4wwYMIC+ffty4MCBh/U2BEEQBEGwISpJkqTSTkIOlUrFypUr6dmz5z2vefvtt1mzZg1nzpwp+t6oUaM4fvw4UVFRAPTr14/MzEzWr19fdE3nzp3x8PBg0aJF/xq3oKCAgoKCol/fPogvIyNDnA0nCIIgCDYiMzMTvV7/wJ/fNjWzZK2oqCgiIyPv+l6nTp04fPgwBoPhvtfs27fvnnEnTpyIXq8vegQFBZV88oLNMJslbmYXcC09j+uZ+aRkF5CRZ8BGP4cIgiAIf2NX2gk8TElJSfj6+t71PV9fX4xGIykpKfj7+9/zmqSkpHvGnTBhAuPHjy/69e2ZJaF8M5klLiRncTwunei4dM4kZpGcmc+N7AIMpn8WRs5aDYEeTgR5OlLFx4XmoZ6EBXuid7QvhewFQRAEucp1sQSW23V3uv1p/87v/9s1f//enXQ6HTqdrgSzFMqqAqOJPRdSWPdnElvOXCcjz3DPa7UaNSZJwmS2/B3LKTRx7noW565nseVMMjN2Xkalglp+bjxZ25eejSoS6u38qN6KIAiCIFO5Lpb8/Pz+MUOUnJyMnZ0dXl5e973m77NNwuPlwvUsZu+NYe3xa2QXGIu+76zVUC9QT8MgD+pV1FPRwxEfVx3eLjq0dpa72pIkUWA0k5CeR1xqLnFpeZxKyODAlVSupORwOjGT04mZfLf1Ag0C9fRqHEifsECcdeX6f0dBEASbVa7/dY6IiGDt2rV3fW/Tpk00adIEe3v7oms2b97MuHHj7rqmRYsWjzRXofRJksTuCyn8vOcKu87fKPq+n5sDnev60bmuH01DPNGo7z3rCJaZSgd7DVUquFClgstdv5ecmc/uCymsOX6NPRdTOB6fwfH4DL7ZfJ5BLUIY3CIET2ftQ3l/giAIgjw2VSxlZ2dz8eLFol9fuXKF6OhoPD09qVSpEhMmTCAhIYH58+cDlp1vU6dOZfz48YwYMYKoqChmzZp11y63119/nTZt2jBp0iR69OjB6tWr2bJlC3v27Hnk7094xCQJ8tMhL43TiVlM23GJqLh8bqJHpYLI2r4MbhFK81BP1A8okIrLx82B3mGB9A4LJCW7gLXHrzE/6ipXUnKYsvUCM3ddYlCLEF5tVxVXB7G2SRAEoSywqdYBO3bsoF27dv/4/qBBg5g7dy6DBw8mJiaGHTt2FP3ezp07GTduHKdOnSIgIIC3336bUaNG3TX+t99+4/333+fy5ctUqVKF//3vf/Tq1avYeRV366FQBqTHwprXID0OMq+BMe+u315lbs3xppMY0iKUSp6OsORF8KsPQU2hYhNwKPk/X5NZYuOpJKbvuMSfCRkAeLvoeKtzDfo0DiyxQk0QBEG4W3F/fttUsVRWiWKpDJIkSD4Dp1eDWwCEDbJ8vzAHJgaBZCq6NEfSIaHCTqPCVLsXzn2m3fqNFPiyyh1BVVAxDOr0hNo9wL1SCacssfVMMp/9cZqYm7kA1A/U80Wf+tT0E3+vBEEQSpoolh4hUSyVIfkZcGwBHJkLKect3/OtC6P3Fl1yff8Sph1MZ3uiPdclDxqE+vHR07WpE6C/O1ZBFkT/CnEHIf4QpF+9+/efeBvavVvib6HQaGbuvitM2XqR7AIjWo2a8ZHVGdG68gPXSwmCIAjFJ4qlR0gUS2VA6hU48KOlUCrMtnxPo4WqHS2zQPX7gUrF0kNxfLjmJPkGM05aDRO61OSF5sHFu9WVmQhnf4dTq+DqXnhuEdToYvm93FSQzODsXWJvKTkznwkr/mTr2WQAmoZ48PWzDank5VRiryEIgvA4E8XSIySKpUdLkiTOpJ4hJS+Fxj6NcdG6wII+JF/Zxk2Nmgr6UDyavYSmXt+iNUb5BhMfrDrJsiPxALSs6sXnveoT5Cmz8Mi6Do7uYHer39aWT+DgTGgxBlq8CtqS6Z8kSRJLD8fx6drT5BSacHWwY0r/RrSr6VMi8QVBEB5nolh6hESx9HBIkkRsVizRydEA9Kjao+j3WvzagixDFsueXkZNz5qQeJw5m8fyjZQCgEaloYp7Fep41cHfoRq/7bPjUoIrapWKNyJrMPqJKiW7cHp+T7i83fLcxc9ye67Ri6DWlEj4uNRcXl98jKOx6ahU8MaT1Xm5bVWx+FsQBEGB4v78tqnWAUL5J0kSp1NPszlmM5uubiIuKw6AELeQv4qlmD3Uzs0i09ENreZWTyL/BqjDBuN9ai6p+amYJBPn085zPu3WuiU3cHXyoF1QO5rWrIBEKFAyhQwAA1bCqRWWGab0q7B2DBydDz1+AJ+aisMHeTqxeGQEn6w9xcIDsXy16Tx/JmTwTd+GopmlIAjCQyZmlkqAmFlSLrswm1UXV7H43GKuZv61kNpebU8drzo09GnIuDrDUG/9BA7PtvymezC8ehjs7m7iaDQbSc5N5pcj+5h7ZDfo4rB3jkFS/XVUib+zP89Wf5Ze1Xrh5ehVcm/EWACHZsGOz6Egw7Juqud0qNenxF5i8cFYPlx9ikKTmfqBemYPboq3izh+RxAEwVriNtwjJIol+a5lX+OX07+w8uJKcgw5ADhoHGgd2JrIkEjaVGyDk72TZUfab0MhwzLTRNgQePLTf+17JEkSM3ddZuL6swB0quPL571rcizlINvjtrMtdhuZhZkA9KvRj/fD3y/5N5Z5DX4fB1d2w8tR4BFcouGPxqYxfN5hUnMKCfFyYv7Q5mLhtyAIgpVEsfQIiWLp/sy5uRRcuoyuRnXUWsss0PW92zg+62t2Ocayupnlusr6yoxOb0zTym1wb9wUjYuLpV/S/mmw+UMwG8EjBLpPhdDW//pakiQxcf1ZZu66DMCQliG837X2XVvuC0wFbIzZyOKzi/m4xcdU96gOQFJOEpIk4e/iXzJvXJIg9TJ43dGrKeUCeFcrkfCXb2QzcPZB4tPy8HbRMXdIU+pW1D94oCAIggAU/+e3+hHmJDwm7qy/JaORC61aE/PssxReiSn6/Rnr/0vQvsvUumKimV8zZnScwaoeq6j66z6SR75MXvRxS4DMa5g2TKQwU4I6veCl3fctlD79/XRRofR+11p89HSdf/Qm0ml0dK/SnV+7/lpUKAF8e+Rbuq7sypKzS0rmP4RKdXehdHEr/NAMNrwLxkLF4StXcGHF6BbU8ncjJbuA537az/G4dMVxBUEQhLuJYkkoMfmnTxM3+mWuvfmfou+p7OzQ1a6FpoI35izLrS+VSkXjts+yoZsf1Qe9zM+RP9OiYgtUKhWO9etjH1wJh5o1LAH0FcnUD+DS774kHPC653EjZrPEh6tPMWdvDAD/90w9hreuXOzcDWYDN/NvYpJM1K1QV95/gAdJOGrpxbT/B1jQC/LSFIf0cXNgyUvhNA3xICvfyIuzDnAiPl15roIgCEIRcRuuBDzOt+EkSUKlsszc5J8+zZVevVHpdFSP2ofaybKGJisjhe/PzqSpX1OeDH4SAJPZhEqlQq36l3o97qDla5Dl/lzyN99yc9YsKowZg/dLI4teN2vLFlzbtgU7O95bdZJfD8SiUsGk3vXp2yRI1nu5knGFyu5/FVkbYjbQzK8Zng6eVsf7V2d+h5WjoDALvKvD80vBM1Rx2JwCI4PnHORQTBpuDnYsHB5OvUBxS04QBOF+xJqlR+hxLJby/vyTG1On4tSwId6jRwOWYiN1zlycW7VEV61aURE1LXoa049Px9vRm/W91uNg53DvwCdXWIoJnQuM2GZZowSYMjNBktDoLQVAzv4DxA4ejLZKZZaOmsSMvVdRq+Drvg14plFgibzHi2kXefb3Z/HQefB/rf+PcP/wEolL0kn4tS9kJoCTNzy/BAKbKA6bXWBk0OyDHLmaht7Rnl9HNP/nES6CIAhCEbFmSXioCs5fIGfnLtIWLUYyWQ6lValUeA0dgkP16kWFEsDwesPpWKkj/9fq/+5fKB36GX4bAqYCCGoOzhWKfkvj5lZUKAGYMjPQeHlxOag2M/ZaWg1M6l2/xAql2yq5VuJG3g1GbhrJd0e/w2A2PHjQg/jVheFbwa8+5KbA3K5w45zisC46O+YOaUrjSu5k5BkYPOcQcam5yvMVBEF4zIliSSgWU3o6hbGxRb/W9+yB10svEfzLfFSau5s7ZhVmMS16GkazEQCtRsu37b4lIiDi3i+wbyr88YbledMR0G/BfY8McYuM5MjE2Yxzstyqe79rLXoG6bg6cBB50dHy3uTfVPWoyuJui+lTvQ8SEj//+TNDNwwlJS9FeXA3fxiyHqo+aen07V39wWOKwdXBnrlDm1HTz5UbWQUMnH2Qm9kFJRJbEAThcSVuw5WA8n4bLvfwYeLHvI59QAAhSxb/ozi6U1xWHK9tfY1LGZcY1WAUrzR85cEvsPNL2P6Z5XmrcdDhI8tOsvvYcDKR0QuPIknwSrsq/KdTTRI/+ID0Zb/h2KABwYsX3TW7pdSmmE18vO9jsgxZ+Dn78X377y3HrChlMoBKA+pbn1sk6YHvvTiuZ+bTe/o+4tPyaBCo59cR4aLTtyAIwt+I23BCibGvVAnJYMCcn4fxxo17Xncs+Rgv/PEClzIu4ePoQ9vAtg8OfmTuX4VSu/eKVSgdj0vn9cXRSBI816wSb0Zads5VeP119M88g+/775dooQQQGRLJwq4LCXELISkniYHrB7Ll6hblgTX2fxVKxkJY8iKcWKY4rK+bA/OHNsPTWcvx+AxGLzyK0WRWHFcQBOFxJGaWSkB5nFkqjE9AG1ix6Nd5p06hq1atqKnk3/1x+Q8+2PsBBrOBWp61+L799/g6+z74hQqyYfHzULUjtBzzwMsT0vPo+cNebmQV0LZGBX4e2AQ7zb1r/vQVK8k7cRzft94q2p2nREZBBm/teot91/YB8GaTNxlUZ5DiuAAcngO/j7XMNPWdD7W6KQ4ZHZfOczP3k2cwMbhFCB93r6M8T0EQhHJCzCwJskhGI0mf/Y/LXbqQd/JU0fcd69S5Z6G08MxC3tn9DgazgQ6VOjC389ziFUpg2fU2YGWxCqXsAiPD5h7iRlYBNXxd+f65RvctlEyZmVz/v/8jffES0leuLF4+D6DX6fmhww+8UOsFAL46/BVTjk6hRD5zNB4EDV8AyWRZ6H5pu+KQDYPc+a5/Q1QqmLsvhoUHrj54kCAIgnAXUSwJd9NoMF5PQjIYyD1w4L6XSpLEzBMz+fzg5wC8WOtFvmn7jeUst/s5scyyTul2gaG+9xqo20xmibGLj3E2KQtvFx2zBjfB1cH+/m/FzY3A76fg9tRTePTv/8DXKC47tR3vNHuHsY3HAvDTnz/x3/3/VV4wqdXw9BSo1R1MhbD4hb96TikQWcev6FblR6tPEXXppuKYgiAIjxNRLAl3UalU+H/2GUE//YTXsKH3vE6SJL498i3fH/segNENRvNW07f+vcnknS5shpUvWdYpnf2j2Hl9t/UCW84ko7VT89PAMAI9indLzTkigorffF20KF2SJDJWr0YyGov92vcyrN4wPoz4EBUqKjhVKJl1Uho76P0zVGkPhhxY2AeSzygO+3LbKvRoGIDRLDF64RGu3sxRnqsgCMJjQhRLAumrVpHy44yiX2v0elxat7rn9ZIk8cWhL5hzag5gWbfzcsOXH1wsJP0JywZbbjPV7wc1nipWflvPXGfK1gsAfN6rHo0qeRRr3L+5+dPPXHv7HeJffa1Ebp09W/1ZlnRbwqj6oxTHKmKns7ROCGoO+RmWRd8mZcWdSqViUu/6NAhyJz3XwEu/HCHfYCqhhAVBEMo3USw95nKPHiXxnQncmDyZ3MOHH3i9JEl8d/Q7FpxZAMCHER8Wb4FzRgIs7AuF2RDaBrpP/WsX2H1cvZnDuCXRAAyMCKZXY2VNJ7VBgagcHHBp27bEdszV8qpVFCvPmMe8U/MwSwp3nmmd4bnFENgMun9vmXFSyMFew8wBYXi7aDmblMUHq04qjikIgvA4EI1XHnNOjRvjNWIEaNQ4Nm78wOvjs+NZeGYhAO83f59nqz/74BcpyIJf+0HWNahQE/r+Anb/vlj8TnmFJkYtOEpmvpFGldx5v2vtB7/WA7h16YJjo0bY+/kpjvV3kiQxZtsY9ifuJzEnkXeavaMsoJMnDNtUIn2XbvN1c2DKc4148ecDLDsST9MQT/o2tf4cPUEQhMeJmFl6DEmFhUiGv47t8HljPD5jx6IqxkxPkGsQ0ztO551m79CvZr8Hv5jZDL8Nhet/grOP5eBYR/di5fnh6pOcSczE20XL9BfC0NqVzF/XOwslc0EB8WNeJ//0acVxVSoVvav1xk3rRueQzorj3Qr61/Prp2D3N4pDtqjizRu3Fnx/sPokp65lKI4pCIJQnoli6TFjzssjbvTLXHvvPSRz8W8VFZoKi5438WtStHX+gVQqy5EeOjd4fjF4BBdr2OroBJYdiUetgu+fa4yf/j5nyimQ8uOPZG3aRNwrr2IuLHzwgAfoHNqZ9b3X09CnofLk7pSTArO7wNZP4MRSxeFGP1GF9jV9KDCaGb3gKJn5JXDmnSAIQjkliqXHTN7x4+Ts30/W5i0UXr5crDH7ru3j6ZVPczb1rPUvqFJB85Hw+nGoGFasIVdv5vDeSst6mtfaVyOiipf1r1tMXkOH4twigoDPP79nHylruWn/amx2LvUcG2M2Kg/q7A3NRlierxkDiccVhVOrVXzTtwEV3R2JTc3lo9WnHjxIEAThMSWKpceMc3g4Fb/9hkqzfkZXteoDr5ckiR+P/8i1nGssOruo+C908xLkpf/1ayfPYg0rNJoZs+gY2QVGmoV48lr7B+eohMbVlaBZs3Bu3qzoeyXV1D4mI4bBGwbzzq532JewT3nAdu9aZumMebD4RchR1i/J3UnLlOcaolGrWHksgVXHEpTnKAiCUA6JYukxIBUWYsr+q6+OW2QkTsVYzA2WdThTO0xlSN0hvNf8veK9YF66pT/QT+0tRZMVvt58juPxGegd7Zncv+F9O3SXlDt3xRmuJ3P1hRdLZA1TJbdKtA5sjVEyMm7HOM6nnVcWUK2B3j+BRyhkxFq6fCtsKRAW7MmY9tUAeH/VSWJv5irLURAEoRwSxVI5J5lMJLzxBnHDhmHKyir+uDtmV9y0bowPG49WU4zbVJIEq1+B1MtgMoCDe7Ffc9+lFGbstNwanNS7PgHujsUeW1JufPM1eUePcu299xXPMKlVav7X8n8082tGrjGXMdvGkJqfqixBRw/o/yvYO8OVnbDj/5TFA15pV4UmwR5kFxgZu+SYOHBXEAThb0SxVM4VXo0l58BB8k+fpuDChWKNkSSJt3e9zYLTC6wvGA7MgLO/g0YL/eaDc/HWG2XmG/jPshMAPN+8Ep3rlvzW/uLwff99XDt1IvD7KSXSh8leY883bb8hyDWIhOwExm0fh8GkcDG1b23oYemcTuwBxbNLdho13/ZriKvOjqOx6UzZdlFZfoIgCOWMSiqpBRqPseKeWlxa8s+fxxCfgGv7dsW6/sfjP/JD9A/Yqe1Y2X0lIfqQ4r1QwlGYFQlmA3T5Apq/VOwc/7PsOMuOxFPJ04n1r7fGWVe+WoBdTr/MC+teINuQzTNVn+GTFp8oL8bOroNqkSXSsBIsOxBfXxyNRq1i1cstqReoL5G4giAIZVVxf37b3MzStGnTCA0NxcHBgbCwMHbv3n3PawcPHoxKpfrHo06dOkXXzJ0791+vyc/PfxRv56G5cxu8Q/XqxS6UtsZu5YfoHwD4IPyD4hdK+RmWNTRmA9TsBs1GFjvXzaevs+xIPCoVfN23QZkqlPJOniJ+zOuYCwoUxansXpkvn/gStUrNyosrixp7KlLzqbsLJYWfe3o0rEjX+v6YzBJvLIumwCiOQxEEQQAbK5aWLFnC2LFjee+99zh27BitW7emS5cuxMbG/uv13333HYmJiUWPuLg4PD09efbZu7tOu7m53XVdYmIiDg4Pp6/Po5C1bTuXu3ajoJitAW67nH6Zd3e/C8ALtV6gV7VexR+86QNIiwH3StBjarG7Tt/MLmDCCsvtt5GtK9M0pHi75h4Fc2Eh8a+8QtamTaRM/UFxvFYVW/FG2BsAfH34a44lH1McEwBjAax7C3ZOUhzqvz3q4u2i5fz1bCZvKd5tW0EQhPLOpoqlb775hmHDhjF8+HBq1arF5MmTCQoKYvr06f96vV6vx8/Pr+hx+PBh0tLSGDJkyF3XqVSqu67zewhHYTwqktnMjSlTMMTFkb6k+M0Lcw25jN8xnlxjLs38mvFmkzete+H271tmlPrMsSxCLqYPV58iJbuQ6r4ujHuyunWv+ZCptVoCJk3C5Ykn8Bo5okRiDqg9gC4hXTBKRt7c8SYpeSnKg17cAgdnWIql2AOKQnk6a/msZz0AZuy8xLHYNOX5CYIg2DibKZYKCws5cuQIkZGRd30/MjKSffuK18Nm1qxZdOzYkeDgu7tIZ2dnExwcTGBgIN26dePYsft/4i8oKCAzM/OuR1mhUqupNGc2nsOG4vPmG8UaI0kS/93/Xy5lXKKCYwUmtZmEndrKW2EuPtB/IQQ2KfaQDSeT+OPPRDRqFd/0bYiDvca613wEnMObEzTjRzSuriUST6VS8XGLj6msr0xyXjJv73obo1nZAm1qdoX6/UAyw8qRlrP4FOhc14+eDQMwS/DGsuPkG8TtOEEQHm82UyylpKRgMpnw9fW96/u+vr4kJSU9cHxiYiLr169n+PDhd32/Zs2azJ07lzVr1rBo0SIcHBxo2bIlF+6zc2zixIno9fqiR1BQ2TqI1M7DA9///AeVvX2xrv/twm/8fvl3NCoNXz7xJd6O3sV7IbMJLm2TlWNGnoEPV1u6dL/UpjJ1K9rGYuKsbdvJPars9pmTvRPftv0WRztHDiYdZPn55coTe+pL0AdZboVufFdxuI+718HHVcflGzlM2SpuxwmC8HizmWLptr/vIJIkqVi7iubOnYu7uzs9e/a86/vh4eG8+OKLNGjQgNatW7N06VKqV6/O999/f89YEyZMICMjo+gRFxcn672UFEmSSPrf/5G9d6/VY0/fPM3EAxMBGNN4DGG+xTuSBIA938Ivz8C6/1j9uhPXnSE5q4DK3s6M6VDN6vGlIXPjJuJffpn418dgvHFDUazK7pX5tOWnDKg9gF7VrVgbdi8OenjmR0AFR+dbdsop4O6k5b896wIwc9dlziSWndlTQRCER81miiVvb280Gs0/ZpGSk5P/Mdv0d5IkMXv2bAYMGID2Aed/qdVqmjZtet+ZJZ1Oh5ub212P0pT5xzrSfvmF+NEvY7ieXOxxOYYc/rPzPxjMBtoGtWVwncHFf9Fr0bDDUmQR0MiqfPddTGHxIUuB+Xnv+mXy9tu/cWnVEl21qrhFdkLj7q44XueQzrzV9C3s1cWbAXygkFbQ4jXL8zWvQXbx/y78m051/Ohcxw+jWeKd5ScwmUWXEUEQHk82UyxptVrCwsLYvHnzXd/fvHkzLVq0uO/YnTt3cvHiRYYNG/bA15EkiejoaPz9/RXl+yi5duyAvlcvKowdi72vT7HHTTwwkdisWPyc/fis5WeoVcX862AsgFWjwWyEWt2hwXPFfs28QhPvrPgTgBfDK9EstOzsfnsQtbMzwYsW4/fB+8W+xVlcBrOBFRdWYJYUds9u/z741rV0T7+u/HDcT3rUwVVnx/H4DObti1EcTxAEwRaVnYY2xTB+/HgGDBhAkyZNiIiIYObMmcTGxjJq1CjAcnssISGB+fPn3zVu1qxZNG/enLp16/4j5ieffEJ4eDjVqlUjMzOTKVOmEB0dzQ8/KN8q/qioHRzw/99nVo25mHaRNZfWoFapmdhqInqdFWuGdn0JyafByRu6TS52mwCA77ddIDY1F3+9A293rmlVzmWBxsW56LkkSRTGxKALDVUUU5IkRm0excGkg2QWZDK47mD5wex08OxcsHcCfUVFeQH4ujnwdpeavL/qJF9tOkdkHV8CPZwUxxUEQbAlNjOzBNCvXz8mT57Mp59+SsOGDdm1axfr1q0r2t2WmJj4j55LGRkZLF++/J6zSunp6YwcOZJatWoRGRlJQkICu3btolmzZv96fVkhGY1k7dhR9OvbzTSLq6pHVWZ3ms2bTd6kiV/xd7CReBx2f2N53vWrYh9nAnAxOYufdlt6P33SvQ6uDiU7O/MomQsKSHznHa706En+2bOKYqlUKjqHdsbV3pUAlwDlyXlXK5FC6bbnm1WiaYgHuYUmPlx9SvGZeYIgCLZGHHdSAkrjuJPkbydzc8YMPIcMwffttx7Ja2I2wYwn4PqfULsH9J3/4DG3SJLEcz/tZ//lVDrW8uHnQU0fYqIPn2Q2Ezd6NDm79+D38Ud49O2rLJ4kcTP/ZvF3IhbXhc2WPkxdlDWsvJicRZfvdmMwScwcEEZkHdvtRSYIgnBbuT3uRLBQ2dmBSoVj/XpWjVt3eR1XM6/Ke1G1Bjp8YFkT89TXVg1dFZ3A/supONir+ejpOg8eUMap1GoCPv+c4PnzFBdKYJldurNQyjeWwHE76bGwqD8c+BFOr1YUqqqPK8NbVwbgk7WnySsUvZcEQXh8iGLJRlV47VUq//EHbl26FHvMhbQLvL/3ffqs6cPlDOuOQilSvROM2gMuFYo9JCPXwP/+OAPAmA7VCPIsH2te7Dw8cGpixS3MYtoRt4MuK7qwP3G/skDulaDlWMvzP96A3FRF4V5rX5WK7o4kpOcxdbvovSQIwuNDFEs2RDKb71ovoqts3cJiV60rjX0bE+4fTqibFWPNZsi5+devrVgbBfDlprOkZBdS1ceF4a0qWzXWVhhTUoh/7TUK7tNyorh2x+8mJS+FD/Z+QFahsm7cPPEWeNeAnBuw5WNFoZy0dnzQrTZg6b106Ua2stwEQRBshCiWbEjqnDnEv/wKxjR553X5Ofsx88mZTGozyarF4BybD983guhFVr/myYQMFh6wLLr/b4+6aO3K51+56198QdbmLVx7/33FC6DfaPIGgS6BJOUkMemgwsNx7XTw9GTL86Pz4Grxjga6l051fGlbowIGk8RHYrG3IAiPifL5k6scMqalceOHaWRv30729h1Wjc0z5hU9V6vUONlbcRss6zps/hDyMyA/3arXlSSJT9aeQpLg6QYBRFQp/s45W+P7n//gFBGO/3//a10h+i+c7J34X6v/oULF6kur2RG3Q1lywS2g8UDL87VjwVgoO5RKpeKT7nXQ2qnZczGF9ScffNSQIAiCrRPFko2w8/AgZOECvIYPQ/9Mz2KPM5gMvLjuRT6N+pRcQ671L7xxgqVQ8m8IzUZaNXTN8WsciknD0V7DhC6211PJGnYVKhA8Zw4O1auXSLzGvo0ZVGcQAP/d/1/lt+M6fgLOFSDlHJxdqyhUsJczo9pYbqf+748z4qBdQRDKPVEs2RCHWrXwefNNq2YuZp+czfm082y5uoV8k5U7rC5sgZPLQaWGp7+z7IYrptxCIxPXWfoPvdKuCgHujta9to0rjI+Xfbv0tpcbvkwl10ok5ybz7ZFvlSXk5Gn5M+z/K9TtrSwWMKptFfz1DiSk5/HTLpmbBQRBEGyEKJbKuLw//8SQmChr7OX0y8w4MQOAt5u9jaeDFUeLGPJh3RuW5+EvQ0BDq1572vZLJGXmE+TpWLTl/HGRtW0bV3r0JOnjTxTFcbRz5OMWHwOw7PwyDiUdUpZYza6WRwlw0trxzq3Zwmk7LpGUUQKtDgRBEMooUSyVYabMTOJfG8Pl7j3Ii462aqxZMvNx1McYzAZaV2zNU6FPWffie7+DtBhw9Ye271g1NPZmLjNvdep+v2ttmzkot6TY+fpizs/HeP065pwcRbGa+jXl2erPAvDxvo/vWn+mSPYNiD+iKET3BgE0CfYgz2Bi0gZlXcwFQRDKMlEslWHm7GzsfHzQeHqgs3ItzJJzSziWfAwnOyc+CP/A+kXHBZmW22+Rn4HO1aqhE9efodBoplVVbyJr+1r3uuWAY506BP8yn+CFC1A7Oz94wAOMDxuPr5MvsVmxTIuepjzB+MMwNQyWDoRC+cWcSqXio6froFLBymMJHLmq7LajIAhCWSWKpTLMPiCAkF8XEjx7Nmqn4u9gS8pJYvKRyQCMDRuLv4u/9S/e6X/w8n6r17ccikll/ckk1Cr4oFttxTvDbJVT48aoNCUzo+aideHDiA8B+OX0L5xNVTiL41MbdHrIjIfd1nVi/7t6gXqeDQsE4NO1pzCbRSsBQRDKH1EslXEqOzvsK1p3KOrEAxPJNebSsEJD+tXoJ//FK9SwqgGl2Szx2a1O3f2aBlHDz7oZqfJIMptJXbCQzHXrFMVpE9iGyOBITJKJ/0b9F7Nklh9M6wSdJ1qe7/sebl5SlNubnWrgrNVwPD6DtSeuKYolCIJQFoliqZzZEbeDbXHbsFPZ8VHER6hVVvwRG/Jh1Stw47ys11574hrH49Jx1moY92TJbKG3dRmr13D9s89I/ORTjDdvPnjAfbzd7G0CnAPoWa2n8sRqdoWqHcFUCOvfAgXNJX1cHRjdtgoAX2w4J1oJCIJQ7ohiqRzJNeQy8YBlxmBAnQFU9ahqXYD90yB6ASzoDSajVUPzDSa+2HAOgNFtq+Dj6mDda5dT+m5dcWralApjXkPj4aEolo+TD3/0+oNnqz9rXRH8b1Qq6PIFaLRwcQuc36Ao3LBWlfFzs7QSmLcvRllugiAIZYwolsqRmSdmci3nGv7O/oyqP8q6wVlJf61faf8eaOysGj577xUS0vPw1zswrJye/yaHyt6eSvPn4fnCC6jUyv93s1P/9edSaJLfiRsAryqWthAAG99T1NnbUavhjUjLbOLU7RdJy1GYmyAIwi35Z88imRUsPSgBolgqJy6mXWTeqXkAvNPsHeuONAHY+l8ozIaKYVCvr1VDU3MKmbbdsu7lP51q4Kh9vFoFPMidi9wlkwlzvvKeRDvidtBtZTeirkUpC9TmTXANgCrtwVSgKFSvxoHU8ncjK9/IlG3KDxQWBEGQCguJf+VVrg4YiCkzs9TyEMVSOWGntqOBTwPaBralfaX21g2+dgyiF1qed54EVs6ATN12kewCI3UC3OjZ0LrF6I+TgkuXuPrCi1yfpPBwXGB/4n4ScxKZc3KOskA6V3jtMHT9yuoWEX+nUat476laAPwSdZUrKcp6TAmCIKi0WkJXLMe1Y0fUrqW3aci6ey1CmRWiD2FOpznkGq08/02SYP07gGSZUQpqatXwuNRcftkfA8A7XWqiVj+erQKKw3gjhbzoaAouXqTCa69h52lFR/W/ebnhy7jr3BlYe6DyxLTKe0Hd1qqaN09Ur8DO8zf4ZvN5vn+uUYnFFgTh8aTR6/EaMrhUcxAzS+WISqXC2d7KH3xn/4C4/WDvBB0/tvo1v9l8HoNJolVVb1pXq2D1+MeJc3hz/D76kMpr1ygqlADctG6MajDK+tut93P9NPzSC5JOKgrzdmfLMShrj1/j1LWMkshMEITHjDEtjdyjR0s7jSKiWLJxP0T/wKSDk8gslHkvt9qT8OR/od27oLfuFtqpaxmsik4A/voBKdyfx3PPYe8vo0nofZglMwcSDygPtPsruLQVNr6rqJVA7QA3nm4QAMDXm+S1oRAE4fGW/MWXXH3+BVJm/lTaqQCiWLJpN3JvMOvPWSw4s4AjSTLP+bLTQcsx0OI1q4d+seEckgRPNwigXqBe3us/xgouXcKQkKAohsFkYMC6AQzfNJzDSYeVJdThQ0srgSs74dI2RaHGdayGRq1i29lkDsekKstLEITHimQygVoFajVOTZuUdjqAKJZsWgWnCkxtP5V+NfrRNqitdYMLc6zupXSnfZdS2Hn+BnZqFW9GigaU1kpfvoLLPXqS9Ol/kRTM4thr7KnuafnvP/HgRExmBQ0hPUKg6XDL8y0fgYKtupUruBQdg/LFxnOK3qMgCI8XlUZDwP/+R5VNG3FqVDbWPYpiyca1qNiC98Pft/4Mtm2fwY8tIWaP1a8pSRJfbbQ0oHy+eSWCvUpugfDjwrFhA0tjSLUaSWErgTGNxuCqdeV82nlWXVylLLHWb4LODZL+hD+XKcurQzW0dmoOXkll94UUZXkJgvDY0QYGlnYKRUSxZIOMZiOp+QpubaRehoM/wY2zluMurLTj/A2Oxqajs1Pzajsru4QLAOiqVKHy6lUETZ+G2tFRUSwPB4+iJqRTo6eSY1CwZd/ZC1qNtTzf9pnlCByZAtwdGRAeDMCXYnZJEIQHMCQlcX3ixFLtp3QvoliyQcvPL+epFU+x6OwieQG2fAJmA1TpYGlGaAVJkvh2s2XR7sCIYHzcxLEmcukql1yn8+dqPkcl10qk5KUw++RsZcGaj7Y0qsyItRx/o8DLbavgrNXwZ0IGG04mKctLEIRy7fqkSaTOm8+1dyaUdir/IIolG5NdmM2049Pkzx7EH4bTq0Clhsj/Wj188+nrnIjPwEmrYdQTVeTlINzFnJ9PyvTpFMbFyY5hr7FnXNg4AOafmk9SjoLCROtk+bvR+XNoNEB+HMDLRcewVqEAfLXpHCazmF0SBOHfefTrh65aNSq89mppp/IPoliyMbNOziI1P5UQtxD6VO9jfYCtn1i+NngefOtYNdRslvjm1qzS4BYheLnorH994R+SPvqYG99NIfmLLxXF6VCpA419GpNvymfK0SnKkqrXB8JHW3ZLKjS8TWXcney5dCOHlceU7f4TBKH8cg4PJ3TNahxq1SrtVP5BFEs2JDE7kV9O/wLA+LDx2KvtrQtwaTtc2WXZHt72Hatff/3JJM4mZeGqs2NkG3FYbknxHDYU+4oVce3cSVEclUrFW03fAmDt5bWcSjlVEumByQD58ptLujnYM/rWLOS3m89TYFSwY08QhHLnzvWMVm9WekREsWRDvjv2HQWmApr6NbW+VQDAmbWWr02GgXuQVUNNZolvt1hmlYa2CsXdSWv96wv/yqF6daps3IC+a1fFsep416Fb5W4AfHn4S+WLqmP2wLRwS6NKBQZGhODjqiMhPY8lh+TfbhQEoXwxpqVx5ZleZPz+R5neBCKKJRtxMuUkf1z+A4A3mrwhr/ru+jU8vwxaj7d66Nrj17iYnI3e0Z5hrUOtf23hvlR2JXdM4+uNX0en0XHk+hG2xSprLolGCzcvQvSvkHxGdhhHrYbXOlQDYMrWi+QWyu/xJQhC+ZE6Zy4FZ89y86efFPV2e9hEsWQDJEniq8NfAfB05aep42XdWqMiKhVUjwQXH6uGGU1mJt+aVRrZpjJuDlbe/hOKLWffPmJefBFjWprsGH7OfkUH7H5z5BsMJoP8hIKaQa2nQTLDlo/lxwH6NQmikqcTKdkFzN0XoyiWIAjlg/fLo6nw+hh8330XlUZT2unck80VS9OmTSM0NBQHBwfCwsLYvXv3Pa/dsWMHKpXqH4+zZ8/edd3y5cupXbs2Op2O2rVrs3Llyof9NqyyLW4bR64fQafRMabxGOsDXDsGefJ/+K44mkDMzVw8nbUMbhEiO45wf5LZzPXPJ5F3+Ag3FZ6HNKzeMPyc/WgT2AaDWUGxBNDhI1Bp4PwGiNkrO4zWTs3rt2aXft59hZwCMbskCI87tYMD3qNH49y8WWmncl82VSwtWbKEsWPH8t5773Hs2DFat25Nly5diI2Nve+4c+fOkZiYWPSoVq1a0e9FRUXRr18/BgwYwPHjxxkwYAB9+/blwIESOJi0BBhMBr498i0AA2sPxM/Zz7oAxgJYMhC+awBxh6x+/UKjme+2XgBg1BOVcdaV3O0i4W4qtRrfd9/FY8AAvEe9pCiWs70za3uu5e1mb+Nk76QsMe9qEDbI8nzLR4oO2e3RMIAQLydScwpZeOCqsrwEQbBZpgz5m0ZKg00VS9988w3Dhg1j+PDh1KpVi8mTJxMUFMT06dPvO87Hxwc/P7+ih+aOqb7Jkyfz5JNPMmHCBGrWrMmECRPo0KEDkydPfsjvpniWnl/K1cyreDp4MqzeMOsDHJlnaS5o52h1qwCApYfjSEjPo4KrjgHhIda/vmAV5/Dm+L33Lhq98oOJHexKsGHoE+9Y/g7FH4ILm2WHsdOoeeVW1/eZuy6TVyh2xgnC40YqLCSmX3/iRo3GcD25tNMpFpsplgoLCzly5AiRkZF3fT8yMpJ9+/bdd2yjRo3w9/enQ4cObN++/a7fi4qK+kfMTp063TdmQUEBmZmZdz0ehszCTH48/iMArzR8BWd7K89gK8yBXbd69zzxH0uzQSvkG0xM3XYRsHRidtSW3fvJ5ZU5R8HRJbecvnmakZtGEpelYBeaqy80u3XI7vn1ivLp2agiQZ6OpGSL2SVBeBzlRkdTGB9P3p9/onZWOPP9iNhMsZSSkoLJZMLX1/eu7/v6+pKU9O/div39/Zk5cybLly9nxYoV1KhRgw4dOrBr166ia5KSkqyKCTBx4kT0en3RIyjIum34xZWQlYCzvTNV9FXoVa2X9QEO/Ag5yZbT5BsNtHr44oOxJGXm46934Llmlax/fUE2U3o6196ZwOWez2AuKFAUa8qxKUQlRhUV3rK1HAsDVkHXbxSFsdeoeaWtZXZpxq7L5BvE7JIgPE6cmzWj8po1BEyahMbFpbTTKRabW4Dy9y3zkiTdcxt9jRo1qFGjRtGvIyIiiIuL46uvvqJNmzayYgJMmDCB8eP/2n6fmZn5UAqmWl61WNNzDUk5SdiprfyjykuDvd9Znrd9F+ys64tUYDQxfeclAF5uVxUHezGr9CiptFpyoqIwXr9Ozu7duHbsKDvWa41ew8vBi9ENRitLytkbqrRTFuOWXo0D+X7bRRLS81h8MJbBLUU7CkF4nOgqh6KrbDv/39vMzJK3tzcajeYfMz7Jycn/mBm6n/DwcC5cuFD0az8/P6tj6nQ63Nzc7no8LFqNlkpuMmZ19k6xdF32qW05usJKvx2J53pmAb5uOvo2CbT+9QVF1E5O+H/2GSFLlygqlADqeNXhf63+R6BrCf455qZC0knZw7V2al5uZ+nqPX3nJTG7JAiPgcK4OAzXr5d2GrLYTLGk1WoJCwtj8+a7F5du3ryZFi1aFDvOsWPH8Pf3L/p1RETEP2Ju2rTJqphlkiHXclhu+/dBbd2skMFkZvoOy6zSS22qoLMTs0qlwaV1Kxzr1y/xuIr6LoHlyJzJ9WH5MDDLL3L6hAXir3fgemYByw6Lrt6CUN4lffwJlzp3IXPjptJOxWo2dRtu/PjxDBgwgCZNmhAREcHMmTOJjY1l1KhRgOX2WEJCAvPnzwcsO91CQkKoU6cOhYWFLFiwgOXLl7N8+fKimK+//jpt2rRh0qRJ9OjRg9WrV7Nlyxb27NlTKu+xxHSZBM1HWdYrWWlN9DXi0/LwctaKtUplhCkrC6mwEDsvL9kxErMT+frI16Tlp/Fz5M/yz2Dyqw9qNdw4C6dWypq5BNDZaRjdtgofrj7FtB2X6Ns0SBTmglBOmbKzMWdnIxmNONQuewflPojNzCwB9OvXj8mTJ/Ppp5/SsGFDdu3axbp16wgODgYgMTHxrp5LhYWFvPnmm9SvX5/WrVuzZ88e/vjjD3r1+muxdIsWLVi8eDFz5syhfv36zJ07lyVLltC8efNH/v5KnGeopWu3FUxmiR92WHbADW9dWeyAKwOytm7l0pORXP98kuJY22K3cTDpIFGJUfKDOLpDxGuW5zsmgkl+c8m+TYLwcdWRmJHPb0fi5eckCEKZpnFxIXjxIkJ/+w3tQ9oU9TCppLJ8cp2NyMzMRK/Xk5GR8VDXLxXLiWXgXx8q1Hjwtf/i9xPXePXXY+gd7dnzdjtcxdEmpS7v5Cli+vRBW6UKob8tQ+3oKDvWpIOTWHBmAXW86rCo6yL5s0sFWTC5nmUjQc8foeFzsnOavecKn/5+morujmx/sy1aO5v6DCcIgg0r7s9v8a9SeZKdDGtegx+aQ+IJq4dLklTUV2lIyxBRKJURjnXrUGn2LCqvXqWoUAIYXm84jnaOnLp5ii2xW+QH0rlCy9ctz3dOAgXroJ5vXglvFx0J6XmsPCZmlwShPJEKC0lftQrJKG8GWpIkxu8Yz8oLKzGaS++IJFEslSd7vwNjHlRsDH71rB6+9UwyZ5OycNZqxBlwZYxzixao7JQvMfRy9Co6ZPf7Y98r+8en2Uhw8oa0K3B8sewwDvYaRj1RGYCp2y9iMJXdk8cFQbBO2tJlJL4zgdihMk6gwHI26uarm5l4cCKZhQ+nAXRxiGKpvMi+AYdmWZ63nWD1WiVJkvh+u2VWaUBECO5O1vVlEh4NSZLI3r0bySy/oBhUZxB6nZ4rGVf4/fLv8pPROkOrcaC2gwxlu9meb14JL2ctcal5rDqWoCiWIAhlh9rRAY27O25PdbF6rFkyM/XYVABerPUing6eJZ1esYliqbzYN8UyqxTQGKpa35dnz8UUjsel42CvZnhr22kU9jiRJIm4l14ibsRIMteulR3HVevK8LqWo0umRU+j0FQoP6mmw+DVw9DuXfkxACetHSPaWGaXpu+8hMksllIKQnng3rs3VbZsxr13b6vHbriygYvpF3G1d2VQnUEPIbviE8VSeZCTAod+tjxv+47Vs0oA399aq/RcM8v6EaHsUalUODVpisrREVOGsuno/jX74+PkQ2JOIsvOL5MfyN7RsuuyBLwYHoybgx2Xb+Sw+fS9jxsSBMG2aFxcUNlbvwa2hmcN2ge1L5oNL02iWCoP9k2xNKEMaATVIh98/d8cuHyTg1dS0WrUjLz16V4omzwHDqDKxg14DhygKI6DnQOjGlj6k808MZNcQ67y5K6fhss7ZQ930dkx6NZauWk7LiE26gqC7UpbtIi8P/9UFKOKexW+a/8dI+uPLKGs5BPFUnngGgCOnvCEvFmlqbfWKvVpEoi/XtluK+HhUjs4YO/jUyKxelbtSSXXSqTmp7L4nPwF2gCcXQfTI2DtGEV9lwa3CMHBXs2J+Az2XbqpLCdBEEpFYWwsSf/7P2Ke7Uv++fOK48lucVKCRLFUHoSPgrF/QvVOVg89HpfO7gspaNQqRj9R5SEkJzwshVevkrl+vezx9mp7XmrwEgBzT85VNrtUue2tnXExcPI32WG8XHT0a2JpWHf7yB1BEGyLSqdD360bzk+0waF6davHr7ywkk+jPuVa9rWHkJ08olgqL3QusmaVfrg1q9SjYQBBnk4lnZXwkOSfO8elp7py7d33MN64ITvOU6FPEewWTFpBGltjt8pPSOsELV61PN/9taIz44a3roxGrWLPxRROxKfLz0kQhFJh7+tLwOcTCfrhB6vHGswGph2fxrLzy9get/0hZCePKJZs2eE5cPYPkLm242JyNptOW06AfrmtmFWyJbrq1XGoWwfnZs0w5+fLjmOntuPd5u8yu9Nsnq7ytLKkmgwDB3dIOQ+nV8sOE+TpRI8GAQD8uFPMLgmCrZLTG85ebc/nrT+nc0hn+lSXd+7kwyCKJVuVmwqb3ofFz8OlbbJCzNxl+UH0ZG1fqvq4lmR2wkOmUqkInjOHoBk/Kj5nqUVAC5r6NVWelIMbhI+2PN/1FSjoBfXSrVvC608mcelGtvLcBEF46HIOHuT6l19iTEtTFCfMN4wvn/gSnabs7MwWxZKt2j8dCrMtnbqrtLd6+PXMfFbeav43SqxVsklqp5K/bZqWn6Zs7VKzkaB1heRTcH6D7DA1/FzpWMsHSYKZOy/Lz0cQhEdCkiRuTP6O1FmzSZk+XVYMk4Lb9w+bKJZsUX4mHJxhed7mP7LWKs3eewWDSaJpiAdhwR4lnKDwKJlzc7n5888UxsYqirPwzEI6Le/Er2d/lR/EyROaDQfnClCYoyif0bduDa84Fk9ShvxbjYIgPBreL43EsVEjvIYNt3qswWSg95refHP4G7ILy95ssiiWbNHh2ZCfAd7Voab160wy8w38ut/yg1XMKtm+xI8+Jvmrr0mRsZjyTm5aN/KMeRxMPKgsoVbj4fUTUP9ZRWHCgj1pFuqJwSQxa4+YXRKEskylUuHyxBOELPoVe1/r25usurSKSxmXWHt5LXZq5edglrSyl5Fwf4Y8iLr1Q7HVOFBbX+/+eiCWrAIj1XxcaFejZHr2CKXHc9Ag8qKjcW7ZUlGcLqFdcNe506piK2UJObgpG3+H0W2rcPBKKr8eiOWVdlXFmYWCUA4ZTAZ+OvETAMPqDsPBzqGUM/onMbNka44tgJxk0AdBPes/uRcYTczecwWwLKJVq0u/2ZegjGPdOlTZsB599+6K4tip7Wgd2LrkGsCZzXB6DSQckR2ibfUK1PRzJafQxPyoqyWTlyAIJUYym4l/bQwZa9YgmeStOVp1aRWJOYl4O3qXqR1wdxLFkq3xCAW/+tDyddBYf9bOqmMJJGcV4OfmQPdb27MF26fSaEo0Xo4hh+jkaGVBdn0BSwfA1k9lh1CpVEVrl+bsvUJuofzu4IIglLysLVvI2ryZpE8+xZRp/ZmVtjCrBKJYsj3VOsJLuyBsiNVDzWaJGbssaz+GtQpFayf++MsTSZLI3r2b6198qSjO5fTLdF7emVe3vUqOQcEi7QbPgdoOLu+AuEOyw3St508lTyfScg0sPRQnPx9BEEqcS8uWVBg/Hu9XX8XOw/rNQqsvrS7zs0ogiiXbpFKBxvrlZpvPXOfyjRxcHex4rnmlh5CYUJqMiYnEjRpN6uzZ5B49JjtOsFsw7jp3Mgoy+PWMgp1xHsFQv7/l+S75BZzdHQc8/7T7CkaT/P5NgiCULLWzM94jR+A1ZLDVY21lVglEsWQ7LmyBvd9BQZas4ZIkFXVDHhAejItOrO0vb+wDAvB47jk8Bw9GGxIsO45GrWFUg1EAzDs9T9k23tbjQaWGCxsh6aTsMH3CAvFy1pKQnsf6k0ny8xEEoURIMk+OuNPqS6u5lnOtzM8qgSiWbIMkwfbPYPOHsG+qrBCHYtI4FpuO1k7N4JYhJZufUGb4vf8evu+8jZ2np6I4nUM6E6oPJaMggyXnlsgP5FUFave0PN87WXYYB3sNAyIsBeDMXZdL5B9qQRDky1ixkrjRL5N/9qys8bY0qwSiWLINl3fAtWNg5wjNRsgKMePWrFLvxoH4uJbtv5RC6dOoNYyoZ/m7Nv/0fPKMefKDtRpr+XpyBaTFyA4zIDwYnZ2aPxMyOHAlVX4+giAoIpnNpMyYQfb27eTsi5IVY+3ltTYzqwSiWLINu7+2fA0bBM7eVg8/fz2LrWeTUakoWvshlG+FcXFce/99cvbvlx2jS2gXKrpUJDU/lRUXVshPxr+B5Ugev3qQJ//MKC8XHX3CAgH4aZdoUikIpUWlVhP044+49+uHx3P9rR5vNBv5+c+fARhcZ3CZn1UCUSyVfXEHIWa3ZVdRi9dkhZhx62ytznX8CPV2LsnshDIqdf4vZPy2nBtT5d22BUvfpWH1hgEw++RsCk2F8hN6di6M3AEBjeTHwLKLU6WCrWeTuZgsb/2eIAjK6SqH4v/Jx6gdHa0euzV2K3FZcbjr3Hm2urJO/4+KKJbKut3fWL426A/6QKuHX8/MZ81xy4G5L4mjTR4bXsOH4fLEE/iMf0NRnB5VeuDj5ENybjKrL62WH8hBL+sMw7+rXMGFJ2v5AvDz7iuK4wmCYB3JrHw3avtK7flfq/8xtvFYnOxL/kDwh0EUS2VZ0kk4vx5QQctxskLM2xeDwSTRLMSThkHuJZqeUHbZ+/oSNONHnBorm8nRarQMrjMYgFl/zsJoVtgUMj8D9k6RvasT/rqVvOJYAjeyCpTlIwhCsZkLC7ncvTvJ307GnCO/B5u92p7uVbrTu3rvEszu4RLFUllm7wh1+0DdXuBd1erhuYVGFh6wHJg7rHVoSWcnPCZ6V+uNp4MnCdkJrL+yXlmwX56BzR/AkXmyQ4QFe9CokjuFRjO/RMUoy0cQhGLL2riRwouXyFi1CuytP0FCkiQMZkPJJ/YIiGKpLPOqAn1mQa+fZQ3/7Ug8GXkGQryc6Hjr1oXweDHn55M6bx4JCm7HOdk7MaD2AAB++vMnzJKCafiwwZavUVPBKG9WSKVSMbK1ZXZp/v6r5BXKO49KEATruHXtSuDU7/GdMAG11vpDraOuRdF1RVdlG0ZKiSiWbIHa+j8mk1li1q0Dc4e2CkUjDsx9LJnS00n+6msy160j95D8I0f61+iPq9aVzIJM4rIUHDlSvx+4+kNWIpxYKjtMZB0/Knk6kZ5r4Lcj4ggUQXgUVGo1rh074ta5k6zxS88vJTEnkQtpF0o4s4dPFEvl1JYz17l6Mxe9o33Rdmvh8WPv54f3y6Px+/QTHBs0kB3HRevCjI4z2NB7A8Fu8ruDY6eD8Jctz/d+BzIXi2rUKobfurX8854rmMyiSaUgPCyS0YhkUj6D+3nrz3m3+bsMqjOoBLJ6tESxVE79vNvSLuDF8Eo4acXRJo8z79Gj8ejbF5WMafM71atQr2T6oYQNtuyOu3kBzv0hO0yfsEDcney5ejOXzafFESiC8LBkrFrF5e49yNq6VVEcBzsHnqv5HH7OfiWU2aMjiqVyKDounUMxadhrVAyMCCntdIQypCSOCTFLZvYl7JMfy8ENmg63PN/zreU4HxmctHa82PyvI1AEQXg40hYtpvDSJQpj5d3yzijIULbWsQywuWJp2rRphIaG4uDgQFhYGLt3777ntStWrODJJ5+kQoUKuLm5ERERwcaNG++6Zu7cuahUqn888vPzH/ZbeWhuzyp1b1ARX7ey3xlVeDRyoqK4+sKLitYumcwm+v/en5e2vMTBpIPyk2k+CrQu4FkZDPKPUhnYIhitRs3R2HSOXBVHoAjCw1Bp7hx83nwDj/79ZI3/eN/H9FzdkyPXj5RwZo+OTRVLS5YsYezYsbz33nscO3aM1q1b06VLF2JjY//1+l27dvHkk0+ybt06jhw5Qrt27Xj66ac5duzYXde5ubmRmJh418PBwTaLjPi03KJT2YeLdgHCHTI3bCTv6FFSZv4kO4ZGraFBhQa4al1Jzk2Wn4yLD4w7Bb1/Bq38pnQ+rg4806giIGaXBOFh0bi64jV8uKxu3ZfSL7EldgtXMq7grnMv+eQeEZtazPLNN98wbNgwhg+3TOFPnjyZjRs3Mn36dCZOnPiP6ydPnnzXr//v//6P1atXs3btWho1+qtZn0qlws+v+PdQCwoKKCj4a9tzZmamle/k4Zm7NwaTWaJVVW9q+buVdjpCGeI9cgQqnRavYcMVxXml4Su83vh1XLQuyhJydFc2/pbhrUNZcjiOTaevE3szl0pettERWBDKOnNuLmonZf8/3T4DrmOljlRxl3GKhMkIpgLQlu5RXTYzs1RYWMiRI0eIjIy86/uRkZHs27evWDHMZjNZWVl4enre9f3s7GyCg4MJDAykW7du/5h5+ruJEyei1+uLHkFBQda9mYckM9/A4kOWe8piVkn4O/uKFfF7913sfX0UxXF3cFdeKN0p5QIcWyB7eDVfV56oXgFJgrn7YkouL0F4jEkmE1d69yH+tTEYrl+XFSMuK66oke3w+jI/pJ1eBd/Uhn3fyxtfQmymWEpJScFkMuHre3dzRV9fX5KSircT5uuvvyYnJ4e+ffsWfa9mzZrMnTuXNWvWsGjRIhwcHGjZsiUXLty7D8SECRPIyMgoesTFlY0+L0sPxZFdYKSajwtPVK9Q2ukIZZzSxd6SJLEvYR+xmf9+G7xYbl6CqU1h7euQkSA7zNBWlg8HSw/HkZVvmx2CBaEsyTt6lMKYGHIPHkTtLO/D0bxT8zBJJlpWbEkdrzrWB5Ak2DcF8tOhMFdWDiXFZoql21R/O4xTkqR/fO/fLFq0iI8//pglS5bg4/PXJ+vw8HBefPFFGjRoQOvWrVm6dCnVq1fn++/vXcXqdDrc3NzuepQ2o8nMnL0xgGVWqTj/TYTHk+H6dRI/+piE8eMVxfn68Ne8tOUlZp6YKT+IVxUIbglmIxycITtMm2reVPVxIbvAyNLD8fLzEQQBAKemTam8ZjX+//c/NC7W3wK7mXeTVRdXATCs7jB5SVzZCYnHwc7xrx20pcRmiiVvb280Gs0/ZpGSk5P/Mdv0d0uWLGHYsGEsXbqUjh073vdatVpN06ZN7zuzVBatO5lEQnoe3i5aejSsWNrpCGWYOSeX9KVLyVq/gcKYGNlxIkMst8T/uPIHSTkK+hy1eM3y9fAcyJe3/k+lUjGkZQgAc/eJJpWCUBJ01arh2qGDrLGLzi6iwFRAfe/6NPFtIi+BvVMsXxu9CM5e8mKUEJsplrRaLWFhYWzevPmu72/evJkWLVrcc9yiRYsYPHgwv/76K127dn3g60iSRHR0NP7+/opzflQkSWLWrXYBA8JDcLDXlHJGQlmmqxxKhXHjCP5lPtqQENlx6leoT1O/phjNRuafni8/oWqR4F0dCjLh2C+yw/RqZGlSGZeax5Yz8tZYCMLjTpIkTNk5imLkGnJZdHYRAEPqDpF3pyPpJFzaCio1RLyiKJ+SYDPFEsD48eP5+eefmT17NmfOnGHcuHHExsYyatQowLKWaODAgUXXL1q0iIEDB/L1118THh5OUlISSUlJZGRkFF3zySefsHHjRi5fvkx0dDTDhg0jOjq6KKYtOBqbxvH4DHR2al4Mr1Ta6Qg2wHvkCJyaNlUcZ2jdoQD8dv43MgoyHnD1PajVEPGq5fn+6WCSt+bIUavh+WaWv/+3z0UUBME6Obt3c7F9e1JmyL+9vuLCCjILMwl2C6ZdUDt5QW4v6K7dAzxLf8OSTRVL/fr1Y/LkyXz66ac0bNiQXbt2sW7dOoKDLV18ExMT7+q5NGPGDIxGI6+88gr+/v5Fj9dff73omvT0dEaOHEmtWrWIjIwkISGBXbt20axZs0f+/uSavScGgGcaVcTLRVe6yQg2R5J5PhtAy4CW1PCoQZ4xr+iTpCz1+4FzBciIg9OrZYcZGBGCnVrFwSupnEyQWbwJwmMs8491mDMzMaWnyxpvMBuKZpoH1RmERi3jTkduKpxaYXneYoysPEqaSiqJ8w8ec5mZmej1ejIyMh75Yu+E9DzafLEdk1liw9jW1PQr/cXmgm2QJIm0XxaQ+ssvBM+dg31FeWvd1l9Zz1u73sJD58HGPhtxtLO+cR0AO7+E/dOg48cQJv+gzdcXH2N19DV6Na7IN30byo4jCI8jyWwma8sWHBs0lNVm5PfLvzNh9wQ8HTzZ1GcTOo3MD/ApF+D8hr/WND4kxf35bVMzS8I/zY+yNKFsWdVLFEqCVVQqFVnbtmGIiyNtkfxZoSeDn6SiS0XSCtJYeWGl/ITCR1u6eisolACGtLRM2a89fo3kLNs9tkgQSoNKrcYtMlJ2P7aFpxcC8GKtF+UXSgDe1R56oWQNUSzZsNxCI4sOWG47DmlR+vd0BdtTYcwY/D7+CO/X5P+jZKe2Y0idIYClr4rBLLPPkc5F0dEntzUMcics2AODSWJB1FXF8QThcWDKylJ0S/6279p/x7C6w+hbo++DL/43xoIHX1MKRLFkw1YcTSAz30iwlxPtayrryiw8npwaN8Kjf3/UOmVr3XpU7YGngyfXcq6xMWbjgwfcj9kM5zdZGlbKNPTW7NKCA7HkG0zK8hGEx0Dihx9ypUdPco8oO+zWx8mHsWFj0ev01g82m2BaBCwbAlkK2pE8BKJYslFms8ScvZYdP4NbhKBWiyaUgnKSSV5h4WDnwIu1XgRg9snZyrqDb5wAvz4Le76VHaJTHV8qujuSmlPImuhr8nMRhMeAKSuLnL37KLhwAbWLvG7dsmeU73T2d0i9BJd3gK5sLSsRxZKN2nXhBpdu5OCqs+PZJmXjbDrBduVFR3N14CBSfvhBdoy+NfriZOfEhbQL7E7YLT+Zur0tX08sgSx5/ZLsNGoGtbDskp2994rio10EoTzTuLpSdctmAr76CocaNWTFmLB7AqM2j+Jc6jn5iUTd+ven6bASuSVfkkSxZKNuH23St2kQLjq70k1GsHmGpOvkHjxI2pKlSIWFsmLodXr61uiLRqXhfNp5+ckENYPAZmAqhEM/yQ7Tr2klnLQaziZlse/STfn5CMJjQOPmhr7bgxs3/5vU/FS2xm5l77W98hOIOwRxB0CjhaYj5Md5SESxZIMuJmex8/wNVCoYFBFS2ukI5YBr5JN4v/Yqob8tQ6XVyo4zuM5g/uj1B8PrKTzH6fYumEM/Q6G8bsJ6R3ueDQsEYLZoUikI/8qUobwfmaeDJ2t6ruHd5u9Sw1PezBRRUy1f6/UF1/sfYVYaRLFkg27PKj1Zy5dKXmVrqlKwTSq1mgqvvIK9wmN+vBy9qOhSAmcT1uwKHqGQlwbRv8oOM/jWQu+tZ5O5kqLsCAdBKG+MaWlcbN+B+LHjFB9xEuQaxHM1n5M3OO0qnFljeR7xsqI8HhZRLNmY9NxCVhxNAP7qJyMIJU0yKF+seTnjMgnZCfIGqzV/nQcVNdWyS0aGUG9nOtzaKXp7Q4QgCBY5u3djzsnBEBuL2lneB+/swmzliRz6CSQzVG4HvnWUx3sIRLFkYxYfiiPPYKKWvxvhlT1LOx2hnDFlZpL06X+51K0bZplrlwDmnpxLz1U9mRY9TX4yDV8ARw/QukBWouwwQ1tZPlQsOxxPRm4J7NgRhHJC3707oatX4/fhB7IOu8015PLUiqcYt30c6fnp8hNp8x+I/B+0eVN+jIdMFEs2xGgyM39fDABDWobIO8lZEO5D5eBA1pYtGK7Gkr19h+w4Yb5hAOQb8zFLMhvdaZ1g5A4YtQf0gbJzaVHFi5p+ruQZTCw5HPvgAYLwGHGoUR3Hhg1ljV15cSVpBWmcSzuHq9ZVQRJ6aPEqhLSSH+MhE9uobMjGU9e5lpGPl7OW7g0CSjsdoRxSa7X4ffgBalc3nJo1lR2nXoV6rO+9Xvn6JY8QZeOxHOsytGUoby0/wbx9VxnaMhQ7jficKDy+zIWFSIUGNC7OsmMYzUbmn7IcmDu4zmB5B+aazaBSWR5lnPgXw4bMvrXm4oXwYBzsZfzFFIRicO3YEefmzRTPXJbIQu/bCrLg7DrZw7s3DMDLWUtCeh4bT8nr3SQI5UXG8uVc7NCB1AULZcfYGLORaznX8HTwpHuV7vKCnF4JM9vCmd9l5/GoiGLJRhyPS+fI1TTsNSpeDK9U2ukIjwnJYFB8XtS17GvsSdgjP0BeGnxbFxY/DykXZYVwsNfwQvhfTSoF4XGWtXUb5owMkNmsVZIk5pycA8DzNZ/Hwc5BThDYNxUSo+H6KVl5PEqiWLIRt3fyPF0/AB9XGX8xBcFK6StWcrFTJ7K2bJEd4/iN4zy14ikm7J5AriFXXhBHD6gUDkh/9WKR4cXwSthrVBy5mkZ0XLrsOIJg64Jm/EjF777DvU9vWeOjrkVxLu0cjnaO9K/ZX14Ssfvh2lHQ6Cwdu8s4USzZgOuZ+fx+wrIbSLQLEB6VwtirGK8lkv7bb7Jj1PGqg7+zP+kF6ay6uEp+MrebVB5fBDnyunH7uDrw9K21fqKNgPA4U2k0uHWKRO3oKGv87FOzAehdrbe8A3Phrw8+DfqDs7e8GI+QKJZswIL9VzGaJZqGeFAvUOZfTEGwkufAgfh99CGBU6bIjmGntmNQnUEAzD89H6PZKC9QcEvwbwDGfDg8W3Y+Q2992PjjRCLXM/NlxxEEW2RMTVV8TuKpm6c4kHgAjUrDwNoD5QVJvQxn/7A8Dy+bTSj/ThRLZVy+wcTCA5btzmJWSXiU7Dw98XjuOdQOym779qjaAw+dBwnZCWy+ulleEJUKIm7NLh2cCcYCWWHqVtTTNMQDo1liwf6r8nIRBBskSRJxo0Zz5Zle5J85IzvO3JNzAegS2gV/F5kd//dPBySo+iT41JSdy6MkiqUybk30NVJzCqno7khk7bJ3Xo7w+DAXyCtQHO0cea6W5RiEOSfnyP9kW6cnuAZATjL8Kf/W4O0PHb8eiCXfIK8zuCDYGkN8PIUXL1J45Qp2FSrIihGXFcemq5sAS7sAWfLS4NgCy/MWr8qLUQpEsVSGSZJUtHNnUItg0RtGKBX5589zdcgQEsaOkx3juRrP4WjnyJnUM+xP3C8viMYemr8EqCDlnOxcImv7UtHdkZs5haw5fk12HEGwJdqgIKpu20rg91Ow85a3Rmj+qfmYJTMtK7aUf2CuTg+9Z0HjgRD6hLwYpUD89C3Doi7d5GxSFo72Gvo1Ee0ChNKhsrMnd/8BsvfswXBdXo8idwd3elXrBVC05ViWJkNgzFF48lPZIew0agZEWNoIzNkbo3gNhyDYCo27Oy5t2sgam1GQUbRJY2idofKTUKuh5lPQ/XubaEZ5myiWyrDZe2MA6BMWiN7JvnSTER5busqh+P/3U6puWI+9r/xbwQNqD0Cj0hCVGMWZmzLXTDjowbOy7Bxu6980CAd7NWcSMzlwJVVxPEEoy4w3biiO4aZ148cnf2RA7QE09ZPf3d9WiWKpDHulXRW61fdnUIuQ0k5FeMy59+mDfUVlHbkrulSkU0gnAOacUjC7dFvqFUiPkzXU3UlLr8aW8+ZEGwGhPCu4cIELbduR8OZ/kEzy1+ipVCrCfMN4q+lb8rr7SxIs6A07JkF+huw8SosolsqwRpU8mPp8Y6r6uJR2KoJQxJwvf8v9kLpDANgUs4n4rHj5SeydAlMawc5J8nO59SFk8+nrxKXKbJgpCGVc9p69YDIhFRSg0sg7Jkv2Ydh3itkDF7fAnm/BbHsbK0SxJAhCsUiFhSR99j8utHkCQ3KyrBg1PWvSIqAFJsnEL6d/kZ9MUDNAghNLIVteLtV8XWlV1RuzBPOjYuTnIghlmNeQwYSuXEGFcWNljTeajTy79lm+OPQFmYWZ8hOJ+sHyteHz4OQpP04pEcWSIAjFY29P/unTmDMzyVwn/1Db27NLp2+elv+JNag5VAwDUwEcmiU/l5YhACw+FEdOgcyGmYJQxjnUqoWusry1frvjd3M+7Tx/XP4DrVorL4GUC3B+veW5jTSh/Du70k5AEATboFKp8PnPm0j5+TiFh8uO09yvOXM6zSHMN0ze2gdLMhDxCvw2FA79DK3Ggr31Rze0q+FDiJcTMTdzWXEsgQG3DtsVBFtnyspCpVajdnZWFKdtUFumd5xOekG6vANzAfZPs3yt3gW8qyrKp7SImSVBEIrNqVEjnCMi5Bc5WIquJn5NFMUAoFYP0AdBbgqcWCIrhFqtKtpAMXfvFcxm0UZAKB9uzpzJxfYdSF+5SlEclUpFq4qt6Fa5m7wAOTchepHluQ01ofw7USwJgiCLZDRiLixUFCPXkMu5VJkNJjV20HyU5XnUNDDLu6XXJywQF50dl27ksPtiirxcBKEMkSSJnL37MGVkoNHLP0+0wCSva/9djswGY57lbMfglrJCnIhPZ3V0AgZTCSw0l0kUS4IgWC1z/Xoude5C2sJfZcc4ceMEHX/ryNjtY+UfsNt4AGhdISvJcjinDK4O9jzbRLQREMoPlUpFyLKlBE77AZe28rpkn7l5hg7LOjD9+HRljVtrPg2NXoSWr8tuQjl120VeXxzNlxvld+5XShRLgiBYzZyTgyE+noy1a2T/Q1rNoxoalQY7tR1JOUnyEnHQwwtLYfxpRWshBrcIQaWCHeducOlGtuw4glBWqDQaXNu3R6WW92N+zqk5ZBRkEJMRo+yWuU9N6PED1O0ta3hcai5bzlhODng2LFB+HgrZXLE0bdo0QkNDcXBwICwsjN27d9/3+p07dxIWFoaDgwOVK1fmxx9//Mc1y5cvp3bt2uh0OmrXrs3KlSsfVvqCUC7ou3fH7+OPCVm4UPY/pI52jszvMp9VPVYR6KrgH8HgFqBT1oss2MuZDjV9AJi3L0ZRLEEoTYakJMVH+MRnxbMpxnJg7u3dq6Xll/1XMUvQupo31XxdSy0Pq4ul1NTSOxpgyZIljB07lvfee49jx47RunVrunTpQmxs7L9ef+XKFZ566ilat27NsWPHePfddxkzZgzLly8vuiYqKop+/foxYMAAjh8/zoABA+jbty8HDhx4VG9LEGyOSqvFo38/1I7W70C7U6g+FI1aXqO8f5AkSD4re/iQlqEA/HYknow8Q8nkJAiPkFRYSEz/57jSuzeFcfK62wP8cvoXTJKJFgEtqOlZU16QyztgxUuQeEJ2HrmFRhYftPx8H1zKJ1moJCtLULVaTWBgIA0aNLjrUa1aNeW7Wx6gefPmNG7cmOnTpxd9r1atWvTs2ZOJEyf+4/q3336bNWvWcObMX+dQjRo1iuPHjxMVFQVAv379yMzMZP369UXXdO7cGQ8PDxYtWlSsvDIzM9Hr9WRkZODm5ib37QmCzTJl56Bxkb9FucBUwP5r+3kiSOYp5PkZMKcr3DgL406Cq5/VISRJovPk3Zy7nsX7XWsxvLXyM+gE4VHKO3GCq4MGo3FxocrWLai11vdFSstPI/K3SPJN+fwU+RPh/jLbhCzobenY3ewleOoLeSH2X+X9VScJ9nJi+xttUatLvsYo7s9vq2eWTp8+zRdffEHt2rU5dOgQL7/8MrVq1cLV1ZXmzZsrSvp+CgsLOXLkCJGRkXd9PzIykn379v3rmKioqH9c36lTJw4fPozBYLjvNfeKCVBQUEBmZuZdD0F4HBXGxRE7bDhXX3hB9tR/njGPp1Y8xavbXpW/M85BD1pnMBvg4ExZIVQqFYNvNamcuy8Gk2gjINgYx/r1qbptKxW/myyrUAJYfHYx+aZ8annWormfzJ/pyWcshRIqCB8tK4QkScy9dUt8UETIQymUrGF1sVSzZk369+/PpEmT2LhxI8nJyfz+++/4+fnRoUOHh5EjACkpKZhMJnz/duq5r68vSUn/vjg0KSnpX683Go2kpKTc95p7xQSYOHEier2+6BEUFCTnLQmCzdO4uZF37BgFFy9ScFbeLTBHO0fCfMIAhQfsRrxi+Xp4NhTmyArRs2FF3J3siU/LK1pUKgi2xM7DA6fGjWWNzTPm8etZyw7XoXWHyr9bdLsJZa1u4BkqK8SeiylcTM7GWasp2q1amhQv8FapVHTp0oUFCxZw7dq1ksjpga93J0mS7vsH+m/X//371sacMGECGRkZRY84BfeGBcGWafR6/Cd9TpWNG3CoVUt2nMF1BwOw4coGrmXL/HekZlfwCIG8NDhevFvof+eo1dC/aSVAtBEQbIckSRgSEhTHWXVxFekF6VR0qUjH4I7ygmQnw/FbTWIj5DehnLs3BoBnmwTh6mAvO05JsbpYMt+j8Vt4eDg7duxQms89eXt7o9Fo/jHjk5yc/I+Zodv8/Pz+9Xo7Ozu8vLzue829YgLodDrc3NzuegjC48rtySfRBir75Ffbqzbh/uHKDthVa/46d0pBk8qBEcFo1Cr2X07lTKK4xS6UfTm7d3PxyUgSP/xIdgyj2ci8U/MAGFRnEHZqmaehHZplObOxYpjlDEcZYlJy2HbOckD2wIiycQSR1cWSi4sLzZo1Y+TIkfzwww/s3buXlJQU1q1bR3b2w+tPotVqCQsLY/PmzXd9f/PmzbRo0eJfx0RERPzj+k2bNtGkSRPs7e3ve829YgqCcG8mBf8G3N6ivPzCctLz0+UFafgC6PSQegnOb5AVIsDdkc51LAvEb3+6FYSyLPfQITCbUTs5yY6x5eoWErITcNe507NqT3lBDHmWsxrBMqsk8zbevKgYJAna1ahA5QrK2oKUFKuLpRUrVtC7d2+ys7P54YcfaNu2Lb6+vnTv3p033njjYeRYZPz48fz888/Mnj2bM2fOMG7cOGJjYxk1ynLkwYQJExg4cGDR9aNGjeLq1auMHz+eM2fOMHv2bGbNmsWbb75ZdM3rr7/Opk2bmDRpEmfPnmXSpEls2bKFsWPHPtT3IgjliSRJXP/iSy60bkPeqVOyYkT4R1DTsyZ5xjyWnJN31hs6F2gy2PL8zFp5MYAhtxZ6r4pOIDVH2ZEugvCw+bzxBiHLf8Nr2FBZ4yVJYvbJ2QA8X/N5HO1ktgSRzBDxMgQ2g1rdZYXIyjew7HA8AINbylvv9FBICuXl5UknT56UEhMTlYYqlh9++EEKDg6WtFqt1LhxY2nnzp1Fvzdo0CDpiSeeuOv6HTt2SI0aNZK0Wq0UEhIiTZ8+/R8xly1bJtWoUUOyt7eXatasKS1fvtyqnDIyMiRAysjIkPWeBKE8iP/Pf6TTNWpKSV98ITvGH5f+kOrOrSu1WdxGyjPkyQuSkSBJ5zZKkskkOw+z2Sx1m7JbCn77d2nqtguy4wiCLYi6FiXVnVtXavJLEyk1L7VUc5mz57IU/PbvUvuvtktms/mhv15xf35b3WdJ+CfRZ0kQoODKFQwJ13Bu2UL2Lhqj2UjXFV25lnOND8I/oG+NviWcZfGtOBrP+KXH8XNzYPfb7bDX2NyBB0I5Z7xxA7Wzs6LbbwD5xnzWXFpDZmEmw+sNL6HsrGc2S3T4ZidXUnL4b486DIgIeeiv+dD6LAmCIPwbXWgoLq1aKmpOa6e2Y2Ady630uafmYjKblCVlyLM0rJSha31/vF10JGXms/6kzLPrBOEhuj5xIhc7dCRryxZFcRzsHOhbo6+yQmnje3B6NSj4f3bn+RtcScnB1cGOXo1Lv13AnUSxJAhCiZOMRkzZ8nodPVP1GfQ6PXFZcWyJVfBDIHoRfFMbdn8ta7jOTsOL4aKNgFA2mfPzyT91GlNaGvYKev2VyM2lpJMQNRWWDYZM+S0M5txqQtmvSRDOOpm78R4SUSwJglCisnfu5FKXp7gxebKs8U72TjxX8zkAZp+cLf8fc0d3yEuFw3OhQN4uvReaB6PVqDkWm050XLq8PAThIVA7OFD5j9+pNHcODjVqyIqRkJ1A77W9WXlhpbKi6XYTyto9wL2SrBAXk7PZdf4GKhUMfAS336wliiVBEEqUSqvFEBdH1tatSIXydpI9V/M5dBodcVlxJOYkykukWifwrAIFGRC9UFaICq46ujXwB8TsklD2qOzscA6XeXYbsOjMIi6kXWD9lfXyb59nJcGJpZbnCppQzrs1q9Sxli+VvJStwXoYRLEkCEKJcgoPJ+CLSVT543dUMs+n8nTwZGqHqWzus5kAlwB5iajVlm3MYPnkK3MtxdBb25f/OJHI9cx8ebkIQgkqjIkpkdtnoxqM4s0mb/JSg5fkBzn4k+VMxqBwCGwiK0RGnoHlRy3tAoa0CJGfy0MkiiVBEEqUSqVC37274h064f7hONs7K0umwfPg6AFpMXBunawQdSvqaRrigdEssWD/VWX5CIJCxrQ0Lj/Ti5hn+2JMS1MUy0XrwqA6gwjzDZMXoDAXDs+yPL99NqMMyw7HkVtoooavKxFVvGTHeZhEsSQIwkNlSk9XNF6SJM6nnZc3WOsETYZZnu+bKjuHwS0ss0u/Hogl36Bwh54gKJB/4gRIEpjNaNzdZcUwmU0ls7D7+CLLWYzuwZazGWXlIjEvKgaAwS1DFO2mfZhEsSQIwkNhTEkhduRILnXthjlf3u2rHEMOfX/vS9+1feUfsNtsBKjtIW4/pFyQFaJTHV8C9A7czClkzfGHf2C4INyLyxNPUHXbVvwnTpRdWCw7v4x+v/djV/wuZcl4BEPFJpYzGdUaWSG2nrlOXGoe7k729GxYUVk+D5EolgRBeCg0ej2FFy9hSksj99BhWTGc7Z3Ra/VoNVrO3DwjLxFXP+g+BV45BN7VZIWw06iLGuTN3Vsy60UEQS47T08calSXNdZkNjHv1DzOpJ4hPiteWSJVO8KIrZYPJDLNvbWwu3/TSjhq5RVcj0LZamQgCEK5obK3x3/iROz9/dBWkredGOCDiA/Qa/W4O7jLT6bh8/LH3vJcsyC+23qe04mZHLySSvPKZXNthVA+mQsKMN5IQRuobPZlS+wW4rPjlR2Y+3cyZ5XOJWWx79JNNGoVAyKCSyaXh0TMLAmC8NA4N2+mqFACCHYLVlYo/Z3MnkvuTlqeaWTpKjxnb0zJ5SMIxZCxciWXOnUi+Wt5TVbh7gNzn6v5HE72MjdhJP0Je761rFdSYO4+SzuOTnV8qegu8/DeR0QUS4IgPBKm9HQks1lRjOjkaPKNMrfv52fCb8Ngcl3LcxmGtAwBYNPpJOJSc+XlIQgy5J86BSYTdj6+smMcSjrE6ZuncdA4FDV+lWXvFNjyMWx4V3aItJxCVh6zdPu+vYGiLBPFkiAID92NH37gQvsOis6wemvnWwxYP4A1l9bIC6B1sXwizkuDY7/IClHd15VWVb0xS/CLaCMgPEL+//0vIcuW4d6nt+wYt2eVnqn2DB4OHvKCZCTAqRWW581Hys5l0aFY8g1mavu70TREZi6PkCiWBEF46CSjESk3l+xt22XHaODTAIB5p+bJO2BXrf6rF8z+H8FklJXH7dmlxQdjyS2UF0MQ5HCsVxe1o7zbVWdTz7L32l7UKjUDaw+Un8SBH8FshOBWENBIVgiDycz8fZYPG0NbhZbZdgF3EsWSIAgPnefAgQT9NBP/if8nO8btA3Zjs2LZFrdNXpD6fcHJGzJi4Yy8Gap2NXwI8XIiM9/I8qPyDw0VhOIovHoVc16e4ji3Z5U6hXQi0DVQXpD8TDgy1/K8xWuyc1l/MomkzHy8XXQ8fes4obJOFEuCIDx0dh4euLRuregTpJO9E/1r9Adg9p8yD9i1d4Smwy3Po6ZamvtZSa1WMejWkQxz917BbBZtBISHQ5IkEsaN52LHJ8k9LK/9BkBcVhwbYzYCMLTuUPkJHZ0PBZngXR2qRcoOM3uPZWH3gPBgdHZlt13AnUSxJAjCIyWZTLKPabh9wO7Jmyc5fF3mD4+mw0Gjg4QjEHdQVog+YYG46Oy4dCOH3RdT5OUhCA9gTE7GlJGBOTcXbZUqsuPMOzUPs2SmZcWW1PSsKS+IyWC5BQeWA3PV8sqHI1fTiI5LR6tR80K4sp2yj5IolgRBeGRyDx3i8lNdSXz3PVnjvRy9inrDzDk5R14SLhWgQT/L84MzZYVwdbDn2Sa32whckZeHIDyAva8vVTasJ3j+fOw85C2Cvpl3k1UXVwEwrO4w+ckUZEFwC3ANgPr9ZIeZfev/lx4NA/B20cnP5xETxZIgCI+MxsubwthY8o4dk31m3MDaA1Gr1OxO2C3/zLiIV+HJ/0JX+T1rBkWEoFLBjnM3uHRDXu8mQXgQlb09jvXqyh6/8MxCCkwF1PeuTxPfJvITcfKEXjNhzFGwd5AVIiE9jw0nkwAY0rLstwu4kyiWBEF4ZHSVQwn8YSpVt26RfQhoJbdKdKzUEYC5J+fKS6RCDWg5Bhzl5QAQ4u1M+xo+AMy/dWSDIJSUvJOnSuRYnV7VetG/Rn9G1B9RMrvO7OU3j5y/LwaTWaJFFS9qB7gpz+UREsWSIAiPlGv79qidnRXFGFJ3CADrr6wnMTtRWUKSpKCNgOXT8W9H4snMNyjLQxBuyTtxgpg+fbj64gAko7L2FIGugbwX/h5tg9rKD7J/OiSfVZRHToGRRQdjARhqY7NKIIolQRBKkfHGDVnj6nrXpZlfM4ySkV/OyGswCcDFLfBTe8vOOBlaVvWiuq8LOYUmlh6Kk5+HINyh4Px5VDod2kqVUNmV8hGuyWdgwzswvQVkJckOs+JoPJn5RkK8nGhf06cEE3w0RLEkCMIjZ87NJW7UaC52fBJDcrKsGLdnl347/xsZBRnyEsm6DteOwoEZlt0+VlKpVEVHNcyLstxiEASl3Pv0oerWLVQYO1Z2jJUXVjJ2+1hO3TylLJnbHyRqPgWufrJCmM0Ss2+dpzikZShqddlvQvl3olgSBOGRUzs5YcrKQiosJDcqSlaMlgEtqeZRjTxjHlHX5MWgXh9w9oGsa3BqlawQzzSqiLuTPXGpeWw9c11eHoLwN3be3tj7ypuBkSSJOafmsDV2K4eT5PdnIisJTiy1PG8xRnaYHeeTuZKSg6uDHX3CZDbELGWiWBIEoVT4vf8eldf9gb5HD1njVSoVH4Z/yOqeq+kc2lleEnY6aHbrfKuo72U1qXTUaujf1NIvZs6tT8+CIIchKQnDtWuK46hUKr5t+y3PVn+WPtX7yA90cCaYCiGoOQQ1kx1m9p4YAPo3DcJZV8q3FWUSxZIgCKXCoVYtdKHKFno29GlIZX1lZYk0GQp2DpB4HK7ulRViYEQwGrWKqMs3OZOYqSwf4bF1Y8r3XIzsROqChYpjVXGvwocRH+JsL3MzRWEOHJplea7gaJOzSZnsuZiCWkVR53tbJIolQRBKnSkjQ/H5V4nZifIO2HX2ggbPWZ5H/SDrtQPcHelcx7KeY66YXRJkkMxmjCk3wGjEsW4d2XHMkrlkEjq2EPLTwbMy1HhKdpg5t2aVOtf1I9DDqWRyKwWiWBIEoVSlzpvHxXbtSVu8RHaM/zvwf3RZ0YXtcdvlBYh4xfL13Hq4eUlWiCEtQwBYFZ1Aak6hvDyEx5ZKrabSzJlUXrsGx4YNZcd5bdtrTNg9gYRshYc8qzWWQ6cjXrE8l+FmdgEroy152GK7gDuJYkkQhFKldnbGnJtLzr59smO4al0xSSaOXD8iL4B3NWj9BvT7BTxCZIUIC/agXkU9BUZzUT8ZQbCWrlo12WPP3DzDrvhdrL+yXnlDy6bDYNxJaPii7BALD8RSaDRTP1BPWLC841rKCttcaSUIQrmh794dOx8fnFu3lh3jhVov0KFSB2p71ZafSIcP5Y/Fsqh2SMsQxi89zi9RVxnZpjL2GvF5VHiw3KPHcKhTG7VO2Vlpt89L7BTSiUDXEth1pqBbd4HRxC/7rwIwrFVoyXQPL0Xi/2RBEEqVSqvFpU0bRf+Yejp4KiuU/k7mp/Ku9f3xdtGRlJnPuj8VdhYXHgvGtDRihw/nYseOGBLk3zqLy4xj49WNAAytO1R+QonHLbejzcrWPv1xIpEbWQX4uunoUtdfUayyQBRLgiCUGZLJhCFJfpdggJS8FJJyZMYw5MOuL+GH5pbdQFbS2WkYEB4MwKw9V0rkbC+hfDPExqJx12NXoQJ2AQGy48w7PQ+zZKZVxVbU8KwhP6Edk2BRf9j+mewQkiQxa88VAAZGhKC1s/1Sw2beQVpaGgMGDECv16PX6xkwYADp9zm13GAw8Pbbb1OvXj2cnZ0JCAhg4MCBXPtbD4u2bduiUqnuevTv3/8hvxtBEP4u/8wZLnd7mrgRI5Fkfqr97fxvRP4WyffHvpeXhMbesgso5RwcWyArxIvhldDZqTkRn8GhmDR5eQiPDccGDai6YQOBU76XPbuakpfCygsrAYWzSikX4Nw6y/P68n8OHrySyqlrmejs1DzfrJL8fMoQmymWnn/+eaKjo9mwYQMbNmwgOjqaAQMG3PP63Nxcjh49ygcffMDRo0dZsWIF58+fp3v37v+4dsSIESQmJhY9ZsyY8TDfiiAI/8I+IADjjRsYkpMpjLkqK0ZNz5oYzAbWXV4nb3ZJrfmrp8y+qbIO2PVy0dGrsWW9yE+7L1ufg/DYUWm1aAMryh6/8MxCCs2F1PeuTxPfJvITifoBkKB6F6hQXXaY2Xsts0q9Ggfi4ayVn08ZYhMLvM+cOcOGDRvYv38/zZs3B+Cnn34iIiKCc+fOUaPGP6cc9Xo9mzdvvut733//Pc2aNSM2NpZKlf6qdp2cnPDzK/6ZNwUFBRQUFBT9OjNTNKETBKU0ej1B06ehq1ULjYuLrBh1vevS1K8ph5IOMe/UPN5u9rb1QRo+D9v/DzJi4fQqy5EoVhrWKpRFB2PZcuY6V1JyCPWW2RhQKLfMeXnknfgTp2ZNFa3Xyy7MZslZS9uNofWGyo+VkwLHF1met3hVdj6xN3PZdNpy7M/QW+00ygObmFmKiopCr9cXFUoA4eHh6PV69lmx3TgjIwOVSoW7u/td31+4cCHe3t7UqVOHN998k6ysrPvGmThxYtHtQL1eT1BQkFXvRxCEf+fUtKnsQum24XWHA5Zbcqn5qdYHsHeE5qMsz/dOlrXYu6qPC+1r+iBJMPvW2g1BuFPakiXEDhrEtf+8pSjO0vNLyTJkEaoPpV1QO/mBDv0MxnwIaATBLWWHmbsvBkmC1tW8qebrKj+fMsYmiqWkpCR8fP55oKCPjw9JxVwMmp+fzzvvvMPzzz+Pm5tb0fdfeOEFFi1axI4dO/jggw9Yvnw5vXr1um+sCRMmkJGRUfSIi4uz7g0JgvBABZcuyVogHREQQR2vOuSb8llwWt66I5oOA3snSPoTLstrdDm8taUJ37IjcaSJJpXC35izc1BptTiHN3/wxfeQb8xn/qn5AAypMwS1SuaP9MIcOHBr+UnEqyBzdioz38DSw5afh8Na2XYTyr8r1WLp448//sfi6r8/Dh+2nJj8b1OLkiQVa8rRYDDQv39/zGYz06ZNu+v3RowYQceOHalbty79+/fnt99+Y8uWLRw9evSe8XQ6HW5ubnc9BEEoOdfefofLXbuRvX2H1WNVKhUj6o8AYNHZRWQWyrhN7uQJjQdanu/9zvrxQERlL2r7u5FvMPOraFIp/E2FV1+hyuZNsg+SBlhxYQU3828S4BxAtyrd5CeTlQT6QEtD1to9ZYdZfDCW7AIjVX1caFOtgvx8yqBSLZZeffVVzpw5c99H3bp18fPz4/r16/8Yf+PGDXx9fe/7GgaDgb59+3LlyhU2b978wMKmcePG2Nvbc+HCBUXvTRAE+ex8fEClIv/UKVnj2wW1o6p7VbIN2Sw+u1heEuEvQ4PnodNEWcNVKhUj2lg+Xc/dF0OBUca5dUK5Zu/ri8reXtZYg8nAnFOWJpRD6w7FXi0vDgBeVeClXTBkA2jkLWU2mMzMuXUu4ojWoajVtt2E8u9KdYG3t7c33t7eD7wuIiKCjIwMDh48SLNmzQA4cOAAGRkZtGjR4p7jbhdKFy5cYPv27Xh5eT3wtU6dOoXBYMDf3/abaAmCrfIcOgR9zx7oqlSRNV6tUjOi3gje3v02v5z+hRdrvYiTvZWHeHoEwzPTZb3+bV3rBfD5+rNczyxg7fFE+oSVQFdlwabl7D+ANrgS9gp/xtip7fg44mOWnV9Gz2o9lSemUoGb/Jz+OJFIYkY+3i46ejSUv7OvrLKJNUu1atWic+fOjBgxgv3797N//35GjBhBt27d7toJV7NmTVautPSaMBqN9OnTh8OHD7Nw4UJMJhNJSUkkJSVRWGhZP3Dp0iU+/fRTDh8+TExMDOvWrePZZ5+lUaNGtGwpf4GbIAjK2Hl4yC6UbusU0olKrpVIL0hn2fllypOSsX5Ka6dmcAvL7NLPuy+LJpWPOXN+Pgn/eZOLkZ3IvbXERC6VSkXLii2Z3G4yOo3MY1LMZjgyF/KV7eiWJImZuyxtMga3CMbBXt7Bu2WZTRRLYNmxVq9ePSIjI4mMjKR+/fr88ssvd11z7tw5MjIyAIiPj2fNmjXEx8fTsGFD/P39ix63d9BptVq2bt1Kp06dqFGjBmPGjCEyMpItW7ag0ZS/P2xBsEWmjAwKLlu/o0yj1jCs3jAA5p2aR4Gp4AEj7uHmJVg5Cja+J2v4880q4aTVcDYpi70Xb8rLQSgXTGlp6CpXwa6CN47168uOU2JF94VNsPZ1mN5CVk+x2/ZdusnpxEwc7TW80Dy4ZHIrY2yizxKAp6cnCxbcf2fLnX+BQkJCHvgXKigoiJ07d5ZIfoIglLzs3btJGDceXZUqBC9eZHUPmacrP83049NJykli9cXV9K3R1/okMuIs/WfsHKH1G+D84Nv5d9I72dO3SRBz98Xw0+7LtKr24KUHQvlk7+9P8Ly5GFNTUWnlNWs0S2aGbBhCU7+mDK4zGBetglYbeydbvtZ5RvZaJaBoVqlvk/LThPLvbGZmSRCEx49DzZpIBgPm3FxM9zne6F7sNfYMrjMYgO1x8loAEPoE+DcAYx4c+klWiKEtQ1GpYOf5G5y/fv8+bkL5Z+fpKXvsnoQ9HE0+ysIzCzGj4LDb2P0QGwUarWUzg0znkrLYef4GahUMLWftAu4kiiVBEMosuwoVCFm6lNDVq7Dz8JAVo3e13nzb9lt+6PCDvCRUKmj5uuX5gR+hINvqEJW8nOhU23JKwKzdoknl48ZcUEDa0qWYC5X322pVsRVfP/E148LG4aZV0LZmz2TL1wb9FS3svn2kT+e6fgR7ld9O9aJYEgShTHOoUR2VWv4/VQ52DnQM7ii/YR9Yes94Voa8NDg6T1aI220EVh5L4EaWzPVTgk1KX7qMpA8/InbQYMWx1Co1kSGR8m4p33b9NJxfD6igxevyw2Tmszo6AYARrSvLz8cGiGJJEASbIJnNZO/eo2hxa74xn9hMGQ0i1RpoOdbyfN9UMFpf7DSu5EHDIHcKTWbmR8VYn4NgszTueux8fdH3+OdB7tYoNJVQJ/h9Uyxfaz0N3lVlh5m3LwaDSaJpiAeNKsmb+bUVolgSBKHMk0wmYvr1J27ECHL2Fv88yDsdSz5G5+WdGb9jvLyCq0F/cA2ArGtw3PpGlyqVipfaWD59z4+6Sk6B/N1Hgm3RP/00VTZvwv0BR2ndz+Gkw0T+Fin/CJ/bJMnyUKmh1VjZYXIKjCzYfxWA4eV8VglEsSQIgg1QaTQ4NW6E2tUVY8oNWTFC3ULJNeaSWZjJ9dx/ngjwQHY6aDcB2r0PteUdURFZx49Qb2cy8gwsOSTOlHycqLVa2TvgAGaemMnN/JvEZMYoS0Slgl4zYNwpqBgmO8zSw3Fk5hsJ9XamY637n6RRHqgk0SVNsczMTPR6PRkZGeKcOEF4SEy3eqhp9HrZMU6lnKK6Z3VlR0Mo9OuBWN5d+ScV3R3Z8Z+22GvEZ9byKmvLFtRubjjfOnlCrj9v/Mnz655Ho9LwR68/qOhSuh2yjSYzbb/aQXxaHp/1rMuL4bbbW6m4P7/F/6WCINgEjV6vqFACqONdp+QKpdu3M6zUq3FFvF10JKTn8fuJayWTi1DmmPPySPzkE2IHDiJryxZFsWacmAFA18pdlRVK59ZbmqwqtOFUEvFpeXg6a+nd+PE4wkcUS4Ig2Jz88+fJPXpM9niD2cCWq1vkLxa/sBl+7gjnN1o91MFew5CWIQDM2CmOQCmvpMJCXDt2RBsSgkubNrLjnEo5xc74nahVakbWHyk/oYIsWDUavg+z9FiSSZIkfrrVhHJAeDCO2sfjtAtRLAmCYFMyN2zgSo+eJH74AZLJZPV4k9lE37V9GbdjHHsS9shL4souSDgMu7+WNbv0YvNgnG8dgbLjvLw1WELZptHr8f/oIyqvWa1ordL045bDnLtV7kawm4LbXYdmWVpfeFWBwKayw+y/nMrx+Ax0dmoGRNju7TdriWJJEASb4tyiBRo3N3RVq2HOtr5BpEatoUVAC8Dyg0jWzE7EK6DRQfxBuLrX6uF6J3uea1YJgB93KL8tIpRdSgqlkykn2Rm/E41Kw0v1X5KfRGEuRE21PG/9hqUVhkzTd1r+vvZtEoS3i8wDfG2QKJYEQbApGjc3qmzcQODkb2WvYRpSdwgOGgf+TPmTvdesL3Zw9YNGL1qe7/5aVg7DWodip1Zx4Eoqx2LTZMUQyh5TejrXJ07EcD1Zcazbs0pdK3elklsl+YGOzoecG+BeCeo9KzvMyYQMdp2/gUatYmSb8t8u4E6iWBIEweZo3N0Vjfd29C7qgDw9WubsUssxoNLApW2QcNTq4f56R3o0tCzWnbHzsvWvL5RJN2fNJnXefBLGjFEU52TKSXbF71I+q2QsgL3fWZ63Gg8a+Rscbs8qdavvT5Cnk/ycbJAolgRBsFmmrCxuzpqFJOPMrduzSydSTsibXfII+etT+p5vrB8PvPSE5dP5xtNJXL5h/S1FoexxadMax0aN8HpJQYFDCc4qRS+0NFJ1DYCGz8sOcyUlh/V/JgIw6okq8vOxUaJYEgTBJkmSRMxzz5H85Vekr1hp9fgSmV1qNc7y9cxauHHe6uHVfV3pUNMHSYKfxAG75YJT06YE/7oQl3ZtZcf488afJTOrBCCZwcHdchi0nfw1RjN3XcYsQbsaFajl//j1ExTFkiAINkmlUuHRrz/aKlWw9/eTFWNI3SHoNDpOpJxg3zUZx6j41LQUTM/OtewykmFUW8u45UfjSc7KlxVDKFtUKhUqlUr2+F9O/wJYdsApmlUCaDocxp2EsMGyQyRn5rP8SDwAL7eTf5acLRPFkiAINsujfz8qr1mNyxNPyBp/5+zStOPT5M0udfwY6jwje4dRk2APGldyp9BoZu7eGFkxhNJ3Y8oU0hYvQTIYFMf6uMXHjAsbp6yv0p10rmDvIHv4rD1XKDSZaRLsQdMQz5LJycaIYkkQBJulsrdHpVHWFG9o3aGW2aUbMmeX7mSy/nBclUpVtAbkl/1XycpX/sNWeLQKY2NJmfkTSR9/TN7Jk4rjOdk7MbTuUGWzSpd3wPlNsvqA3Skj11B0YO7oto/fWqXbRLEkCILNkySJzE2bSFu61OqxJTK7JEmwbypMrgep1q896ljLlyoVnMnKN7LoYKz1ry+UKjtfX3zfegt9z544NWokO05GQQZmyaw8IbMZNkyAX5+Fw7MUhVpw4Co5hSZq+LrSroaP8txslCiWBEGweTl79pAw5nWSP5+EMTXV6vGKZ5dUKri83bLrSEbfJbVaxUu3Zpd+2n2FfIP1ncmF0qPW6fAcOICAzycqivP27rfpu7Yvp1JOKUvo3DpIPg06N6jbR3aYfIOJ2Xssxf/otlVQq+Wvw7J1olgSBMHmObdsiVPTpngOHoRKa/2On9uzS452jlzPvS4viSfesXw9vgjSYqwe3rNhRSq6O3Ijq4Clh+Pk5SDYrBu5N4hOjuZi+kXcdAp2m0kS7PrS8rzZCHB0lx1q6eE4buYUEujhSLf6/vJzKgdEsSQIgs1TqdVUmj+PCmPGoHFxlhVjZL2RrO+1nl7VeslLIqgpVGkPZqOs2SWtnZpRt/ouzdh5mUJjCdyOER6q3CNHuDpwEHknTiiOVcGpAht6beCrJ74iyDVIfqCLWyExGuydIPwV2WEMJnNRs9SRbSpjp3m8y4XH+90LglBuKNmqDeDu4I6Xo5eyJG7PLkX/CmlXrR7+bJMgKrjqSEjPY9WxBGW5CA/dje+mkHvwIOm/LS+ReO4O7nQM7ig/gCTBri8sz5sMBWf5f5//OJFIQnoeXs5ang1TULyVE6JYEgShXCm4coX418eSd0r+uo+DiQfZmyCjq3el5lC5rezZJQd7DSNbW2aXpu24iNEkZpfKsoDPJ6Lv0xvvV+XP4EiSxNHrR+VtLPi7S1sh7gDYOUCL1xTlNP3WAc9DWobgqFW247Q8EMWSIAjlSsr06WRt3MiNyd/JGv/75d8ZtmkYn+3/DINJxjb+O9cu5dy0evjzzSvh4WRPzM1c/rh1vIRQNtkHBBDw2WfY+8jfJbYjbgeDNgzila2vKC+YNFrwqQ1NhlkOe5Zpy5lkzl3PwlmrYUB4iLKcyglRLAmCUK5UeO01XDp0wOfNN2WNbx/UHn9nf1oHtqbAVGB9gOAIaPsujNgm6zaIs86OYa1CAZi67SJmcwnMOAglyizjLMJ/YzKbmHJsCgDVPaorvpVMaBsYtRfavy87hCRJfLfVcnTPwBYh6J3kH7xbntiVdgKCIAglSRsURNAPU2WPd7J3Yu0za9Fp5J+jRdu35Y/F8kNqxq7LXEjOZtPpJDrXfbx3IpUlpuwcLnd/GrfITlR47VXUzvI2FACsu7KOi+kXcdW6MqTukJJJUK0GrZPs4dvPJXMyIRNHew3DbxXtgphZEgShnJNza0NRofR3+RlWD3FzsGdwixAAvt92sWTWswglImvjBozXEsnavg2VVis7jsFk4IfoHwBLny+9Ti8/qQtbYN/3UJgrPwa3Z5UuAjAwIhgvlxL8/8DGiWJJEIRyyZyfT8qMmcT0eRZJ5m2TMzfP8ObON8kqzLJ+sLEQ1oyBb2pDRrzVw4e0DMXRXsOpa5nsOH/D+tcXHgp9r14E/TQTvw8/RGUv/xbV8gvLSchOwMvBi+drPi8/IbMZtnwEm96HKPkzqgC7LqRwPC4dB3s1w29tNBAsRLEkCEL5ZDKROn8++adOkbl+vdXDJUni3T3vsjFmI/NPz7f+9e20kHoZCrNhz7dWD/d01vJiuOVssO+3XhCzS2WESqXCpXVrXFq2lB0j15DLjBMzAHipwUs42cu/bcaZ1XD9pKVbd9PhssNIksR3WyxrlV5oHkwFVzGrdCdRLAmCUC6pnZ3xnTCBgC+/wO3pp60er1KpeKWhZUv4/FPzSc23/hgVnri1dunIPEi3/sy3Ea0ro7VTczQ2najL1u+sE0qOMTW1xBZ2/3r2V1LyUqjoUpE+1eQfR4LZBDs+tzyPeAWcPGWH2nvxJkdj09HZqXmpjZhV+jubKZbS0tIYMGAAer0evV7PgAEDSE9Pv++YwYMHo1Kp7nqEh4ffdU1BQQGvvfYa3t7eODs70717d+LjrZ8yFwSh7NF364r+6adRqeX9U9ehUgdqe9Um15jLrD9lHEga2hpCnwCz4a8falbwcXOgf1NLQ8Cp2y5a//pCiUn66CMud3mKnAMHFcXJKMhg9snZALzc8GXsNQp2m51cATfOgoM7hI+WHebOHXDPNauEj5uD/JzKKZsplp5//nmio6PZsGEDGzZsIDo6mgEDBjxwXOfOnUlMTCx6rFu37q7fHzt2LCtXrmTx4sXs2bOH7OxsunXrhskkDrIUhPJEMpkwZWZaNUalUvF6o9cBWHx2MUk5Sda/cIcPLV+PL4Ib56we/tITVbBTq9h36SZHrsqY3RIUM6Wnk3f8BIbEROw8PRTFmndqHlmFWVR1r0rX0K4KkjLCzlsFeIvXwEH+AvH9l1M5FJOGVqNm1K0DnYW72USxdObMGTZs2MDPP/9MREQEERER/PTTT/z++++cO3f/f3x0Oh1+fn5FD0/Pv6YpMzIymDVrFl9//TUdO3akUaNGLFiwgD///JMtW7Y87LclCMIjknfyFFd69yHxvfesHhsREEET3yYUmguZfny69S8e2ARqdgPJDNs+s3p4RXdHejcOBCjaqSQ8Whp3d6ps3EDgtB/QVasmO05KXgoLziwA4NVGr6JRK+iM/edSuHkRHD2h+Uvy4wBTtl4AoF/TIPz0Ylbp39hEsRQVFYVer6d58+ZF3wsPD0ev17Nv3777jt2xYwc+Pj5Ur16dESNGkJycXPR7R44cwWAwEBkZWfS9gIAA6tate9+4BQUFZGZm3vUQBKHsUuu0FFy4QM6Bgxju+DegOFQqFa83tswurbq4igtpF6xPoN17gAou74ScFKuHv9yuChq1il3nb4jZpVKidnTEtW1bRTFmnphJnjGP+t71aR/UXllCAY2g1tPQaizoXGWHOXgllajLN7HXqBjdVswq3YtNFEtJSUn4/Es7eR8fH5KS7j0t3qVLFxYuXMi2bdv4+uuvOXToEO3bt6egoKAorlarxcPj7mlVX1/f+8adOHFi0dopvV5PUJA4ZFAQyjJdtWpU/OpLqmzcIOtoioY+DXky+EnMkpmvD1t/5hu+taH3z/B6NDh7Wz082MuZZ8Mss0tfbzpv/esLspiyc8jZv79EYkmShMFsQIWKMY3HKO/W7VML+i2A/2/vzsNjvNoHjn8nM5nsElkkQhD7EoRYGntrq6J0Q7VKLaWlqKYL3t9LS6mltbxtqZZaWnTVUtTSklKxi1qDSpoIkX2ybzPP749poykmM5kwCffnuubq5MlzznPPFHPPec65T4eJVnXz96jSU20C8Pdwsi6me5hNk6WZM2feNAH734+jR48Ct95RXFEUk3/gBg8eTN++fQkKCqJ///5s376dCxcusHXrVpNxldbv1KlT0el0xY+4uDgzX7EQwlaq9OmDpmrZ55u80voVNHYafrv6W9k22W3+pFWrlSY8VB97tXHuUsQfsjLubkj5eDmxI57n+ty5VvelUqmYETqDHx/7kfbV25fewPyOy9z02J+p7L+UjMZOxYsyV8kkmyZLEyZM4Ny5cyYfQUFB+Pn5cf369ZvaJyUl4evra/b1qlevTu3atbl40ZhJ+/n5UVBQQFpaWonzEhMTTfbr4OBAlSpVSjyEEJVH3rlz6DMtKzQZUCWApxs/DcDCowvRG8q4CERRIPag8b8WqFnVmSFtjXWX3t8VJXWX7gLFYAC1Guf2D5R+splqVallXQcHl8EP40EXb3UsS/+aA/dkSE0CPK2o9XQfsGmy5O3tTePGjU0+HB0dCQ0NRafTcfjwjSWbhw4dQqfT0aFDB7Ovl5KSQlxcHNWrG/dZCgkJwd7enl27dhWfc+3aNU6fPm1Rv0KIyiPpo4+IfvwJkpcvt7jt2BZjqaKtwqX0S3x/6XvLL64osH4QrOoNF3ZY3Hz8g/XRauw4EpPGvouWz30SlvF97TXq7fgJ1we7lbkPg2Jg4ZGF/Jnxp/UB5aZD+Dw48Tlcsm4R0onYNMIvJKG2U/FSt/rWx3aPqxRzlpo0acLDDz/MmDFjOHjwIAcPHmTMmDH069ePRo0aFZ/XuHFjNm3aBEBWVhZhYWFEREQQExPD3r176d+/P97e3jz22GMAuLu7M2rUKF599VV+/vlnTpw4wbPPPkvz5s3p0aOHTV6rEOLOcmzaFBQFfUqqxaMz7g7ujGs5DoAPIz+kUF9o2cVVKqjW1Pj8l1nGrSos4OfuyLPtawPw3q4LMrp0F2hr1rRqftGWP7aw5uwahm0bRr4+37pgflsMuWng3QiCn7Gqq4U7jSvJH29Vg1peMqpUmkqRLAF88cUXNG/enF69etGrVy9atGjBunXrSpwTFRWFTmfctFKtVnPq1CkGDBhAw4YNGT58OA0bNiQiIgI3txsrBxYtWsTAgQMZNGgQHTt2xNnZmS1btqBWW7GkUwhRYbl27Urg5h/wf3dumT4EhzQawqP1HuWD7h+UraBgx0ng4G7couLMdxY3f7FbPZzs1ZyMS+eX85at7BOlUwoLSXzvfYqSymc/viDvIDrV6MSIoBHWbdCsizfeggPoMRPUmjJ39dulZH67lIJWbcekHmUvhXA/USny1cRqGRkZuLu7o9PpZP6SEKJ0vy4w1lyqGggTjoCFSdfc7ef4OPwyzfyr8OPLnaxfWSWKpa5dy/U5c7EPCKDeT9tRldMXZ4NiwE5lxfjED+ONt99qhcLz28s8sVtRFAZ+dICTcemM6FCHmY82K3tM9wBzP78rzciSEEKUN31WFikrV6IUFZW5j7S8tNJP+rf2L4KLD6RFw4l1pZ//L2O71MNFq+bM1Qx2nClDVXFxW04tW+LYogVeY0ZblSj9exzCqkQp8RxErjc+7/m2VSvgdp29zsm4dJzs1Yx/UOYqmUuSJSHEfUkxGIgZPITEBQtJW7/B4vYGxcCCIwvo8XUPzqeet6yxgyt0DjM+D58PBTkWNfd00TKyUyAAi3ZdxGCQGwTlxallS+ps3IDHk1ZscAu8FfEWsyJmkZJbDmUefl1orADfpD8EtCtzN3qDUlyna2SnOvi4WXFb8D4jyZIQ4r6ksrPDc9izaGvXxqG+5TVm7FR2JOUkUWAoYGfMTssDaPM8uNcCrSvoLN+8e3Snurg5aoi6nsmPp65Zfn1xWyo7uzJvvgxwPvU83138jq8ufMWVrHLYmL3ve9BxMnSfYVU3m0/GE3U9kyqOGl7oLHWVLCFzlsqBzFkSonJS9HoUvR47rbZM7eOz4onRxdCxRseyBZDyB3jUsnjO0t+W/nyR93ddoK6PCzsnd0Gjlu+/ZaEYDFyZOBHXzl3wePIJq2+/jdk5hkMJh+hTpw/zu84vx0jLrqDIQI/3w4lNzeH1hxtJuYC/yJwlIYQohUqtLnOiBFDDtUbZEyUAr3plTpQAnu9YBw9ney4nZfND5NWyx3Gfy9yxg6zdP3N93jyKkq2rXxV+JZxDCYfQ2mmZFDLJusCyEi0uXno7Xx2NIzY1B29XB0Z0qFMufd5PJFkSt9WtWzcmT55sVR/h4eGEhITg6OhI3bp1WW5GIcBJkyYREhKCg4MDwcHBVl1fCHMoikLGrl3EvvACSqGFtZP+kpybXLZtUACK8iHiQ0iybN83N0d7xnYx3k5ZtPsC+UVlrCp+n3Pr2RPfqW9SLexV7C3YFeLfCvWFxXsHDms6jBquNcoelL4QVj0Mn/WBtJiy9wPkFeqL94B7+aH6OGvLXnbgfiXJkrhjoqOjeeSRR+jcuTMnTpxg2rRpTJw4kW+//dZkO0VRGDlyJIMHD75LkYr7nSE7h4T/ziD7132kff21xe0vp1+m/6b+vBr+Kkk5ZajPs/112DENdls+J2VEhzpUc3PgSlou6w/FWn5tgUqjwXP4cDyHDrWqn3Xn1hGTEYOXoxejm4+2LqjjayD1D0i+CM5eVnW15kAMiZn51KzqxNPtrNxu5T4lyZK4pREjRhAeHs6SJUuKNzWOiYmxqI/ly5dTq1YtFi9eTJMmTRg9ejQjR45k4cKFJtstXbqU8ePHU7duXStegRDmU7u64PvmG3i98AIejz9ucfs67nUIdA8kuzCbxccXWx7AA+NBpYaobRCz36KmTlo1k3s0BOB/v1wiM69sI2P3o6Lk5HKrgn49+zrLTxpHzqe0mYKr1rXsneVnwt55xudd3wAHN9Pnm5CRV8iy8D8AmNyjIVqNfOyXhbxrd5miKOQUFNnkYck/CkuWLCE0NJQxY8Zw7do1rl27RkBAAK6uriYfffr0Ke4jIiKCXr16lei3d+/eHD16lMIy3uoQ4k5xHzCAalNewc7R0eK2dio7prabCsDmPzYTmRhpWQc+DSFkhPH5jmkWb4MyqE1N6nq7kJpdwCf7oi279n1KKSwk9vmR/DlsGAVXrN+U9r1j75FblEtLn5b0q9vPus72vQ/ZieBZ98afizL6dF806TmF1K/mymOtrLgteJ+TG5d3WW6hnqb/tXwDzfJw9u3eZt+rdnd3R6vV4uzsjJ+fX/HxyMhIk+2cnJyKnyckJOD7r/v/vr6+FBUVkZycXLyhsRAVUVFSEhofH7PPb+7TnMfqP8amS5uYc2gOG/puQG1nwaqqB6fBqW/g2kmI/AJaDzO7qUZtx+sPN2Lc58f5dN9lnn2gFtXcLE/67id5589TEB+PXVISdi7W7Y12JOEI26O3o0LFtPbTrCtAmRZjnL8G0Gs2aMq+ACElK5+V+y4D8GrPhqjtpNJ7WcnIkrBI/fr1TT5q1Cj5zeXf2zD8Pbol2zOIikqfns6VlydyeeBj6P/aa9Jck1pPws3ejXOp5/j2oum5eTdx8Yaurxuf//w25GVY1Lx3Mz+CAzzIKdDzv58vWXbt+5BT8+bU3byZGoveR1O1apn7KTIUMefQHACeavgUTb2aWhfYrv+CPh8Cu0KjR6zq6n+/XCK7QE/zGu48HORXegNxWzKydJc52as5+3Zvm13bWq6upu/Dd+7cme3btwPg5+dHQkLJrRgSExPRaDR4eVk3YVGIO0Xl7Ez+H3+gT08n+/BhqvTsaXZbLycvxrcaz7uH32Xx8cU8VOshvJ28zb94uxfg2GeQcgn2vQc93zI/bpWKN/s0ZsiKg2w4HMvIToEEeruYf+37kLZmDbQ1rbs1teH8Bi6lX8LdwZ2XW71sXUD5WcbaWyo7eHiuVduaRCdn8/nBPwGY2qexfEG1kiRLd5lKpao0yza1Wi16fcmlyJbchgsNDWXLli0lfr9z507atGmDvX3Za8sIcSfZabX4z5uHSmuPY6NGFrcf3GgwP1z6gXOp53j/6PvM6TzH/MYaLfR6Bw6vgJZPW3ztB+p68WAjH/ZEJfHu9nN8PKyNxX3c6zJ/2YO2Tm0cymEBiaIoHE44DBhHFT0cPazr0MEVxv4KcYfA17oNbudtP0+RQeGhxtXoUN+ChF3cUuX41BY2UadOHQ4dOkRMTAyurq54enpSv775VV/HjRvHBx98wJQpUxgzZgwRERGsXLmSDRtu7MO1adMmpk6dyvnzN/bWunTpEllZWSQkJJCbm1ucoDVt2hStFQUEhTCXU/OgMrfV2Gn4b+h/Gbp1KFsub2Fg/YG0q27Bfl4NexsfZRwJmPpIE8IvJLHjzHUOXU6hfV0Zxf1b4fVErr7+Okp+PrW/+BynFi2s6k+lUrH0waXsjdtL14Cu5ROknRpqd7CqiyMxqfx0JgE7lXFUSVhP5iyJ2woLC0OtVtO0aVN8fHyIjbWshktgYCDbtm1j7969BAcHM2vWLJYuXcoTTzxRfI5OpyMqKqpEu9GjR9OqVSs+/vhjLly4QKtWrWjVqhVXr0qFYnH3FV6/Tvq331nUJsg7iEGNBgEw+9BsCvUWrP5UqUomSoV5Fl27oa8bQ/6qpfPOtnOyyW4JCk6tW+HQpAmOzawbufmbSqXiwVoPWjepOzcN9i+y+P/1rSiKwjtbzwEwuG0tGviWveyAuEH2hisHsjecEPemouRk/ujzCIasLGp/8TnOrVub3TajIIP+m/qTmpfKxFYTGdNijGUXz9PB7reMdZfG7bdoVVRSZj4PLtxLVn4Riwa35LFWNS279j1MURQMmZmorfi3Orcol09PfcqIZiNw05ZDMrI1DI58Ag0fhqFfWtXVj79fZcL6Ezhr1ex9rZusiiyF7A0nhBBW0nh749azJ45BQRZ/uFbRVuG1tq8B8PHvH5OYk2h5AOe2QHIUHPzQomY+bg682M24DcqCn6LIK7y/t0H555iASqWyKlECWH5yOSt+X8G43eOsL2p57Xc4utL4PHS8VV3lFeqZ95NxSsPYLvUkUSpHkiwJIYQJfv+ZTp0N63GwYL7e3/oG9qV/3f682/ldqjlXs6yxozv0mmV8Hj4fdFcsaj6qUyA1PJy4qsvj079q7dyPFEXhyssvk/Thh2Xe9+/fOtXoRIBbAGOaj7FulZnBANvCQDFAs8chsItVca3cH01cai6+VRwY0yXQqr5ESZIsCSGECXbOzqg0N9bCKHrzR2lUKhVzOs+hR+0eZbt4i8FQKxQKc2DHdIuaOtqref1h42q+j/b+wfUM6+fDVEY5ERFk7f6ZlOUfUxBnWcJ5O2392vL9gO/pFtDNuo5+32hc+WbvYixAaYXrGXl8uMdYX2tqnyaVZtV1ZSHJkhBCmEFRFNK+/IrogQPRZ2WVqY/UvFSSc5PNb6BSwSMLjfvGnf0e/thj0fX6t/CnVS1jocp528+X3uAe5Bwaiv97C6n25hs41LVutCW3KLf4uVZt5crc3HRjAUowFiN1t67e07yfzpNToKd1LQ8GBPtbF5u4iSRLQghhBiU3l5RPPiH/4iXSN260uP2+K/sY8P0AZh+0cATBLwja/TU5fOurFq2YsrNTMbN/M1Qq+O5EPMf+TLPs2vcAlUqFe9++eD7zjFX9xOhi6PVNLz4/+zkGxbK9+27pl9mQnQTeDeGBl6zq6kRsGt8dN+5vN6N/MylAeQdIsiSEEGawc3bGf967VHvjDTxHjrS4fTXnamQVZBGXGUdGgWVbmfDgNHD1g9xUSDpnUdOWAR48FWJcDTdz85n7ppRAdkQEhvz8cunLoBiYcWAG6fnp7Ivfh4pySEY6TID6PaHfIqv2fzMYFN7achaAJ0Nq0jLAw/rYxE2kdEA5kNIBQghzHEk4QnC1YOztylDBPu4IeAYa95CzUFJmPg8t3EtmfhHznmjO4La1LL9+JZJ7+gwxQ4bgEBhI7XVrUXt4WNXfhvMbmHNoDk4aJzYN2EQNV+tumZWnb49d4dWvT+KiVbMnrBvVqsgKOEtI6QAhhLiDFL2e9O+/RzGYf0umrV/bsiVKAAFty5QogbGUwKQeDQCY/1MUutzyWRVWURmyMlG7u6OtWxc7d3er+orNiGXRsUWAcUsTqxOlzITSzzGTLreQuduNI40THmogidIdJMmSEEJYSFEUrkx4mWtvTiXlk08tbl+oL+TDyA/Zdnlb2QI49yOc+saiJsM71KGejwsp2QUs2nWhbNetJFweeIC6m3+g+lszrZq/ozfombZ/GrlFubTza8fTjS3fr6+E1MuwJBh+mFAu1brf3xlFclYB9XxcGNVJSgXcSZIsCSGEhVQqFW49eqBydsbez9fi9psubWL5yeXMPjTb8mKV536EL5+BrVMg87rZzezVdrw9wLjn3dqIGE7H6yy7biXwz1E+jZeX1bffPjvzGSeTTuJi78KsjrOs29JEUeDHV6AoF9JjQeNgVWyn43WsO/gnALMGBKHVyMf5nSTvrritbt26MXnyZKv6CA8PJyQkBEdHR+rWrcvy5cvNbpuSkkLNmjVRqVSkp6dbFYcQ5c398ceot30b7gMGWNz28QaP08yrGZkFmcw4MMOyKtANH4bqwcbtULZOMX4Im6ljfW8ebemPQYHpm06hv4cme+dHRxM9YAC5f228ba2o1Cg+jDRWTn+z3Zv4u1q5HP/4Wri8FzSOxkndVox4GQwK//n+NAYF+rf0p0P9st2eFeaTZEncMdHR0TzyyCN07tyZEydOMG3aNCZOnMi3335rVvtRo0bRwspdwYW4U1QqFfa+N0aVDHl5Zic9GjsN73R6B62dlv3x+9lwfoP5F1ZrYMCHYKeB8z/CGcs2+f1Pvya4OWg4eUXH+sOWbY5dkSW9v4j8i5dIWrrU6i1ICvQFTN8/nSJDEd0CujGgnuUJcQm6eNj5H+Pzh/4DXvWs6u6ro3FExqXj6qDhP32bWBebMIskS7ZSkH37x7/vZZs8N9e8cy00YsQIwsPDWbJkCSqVCpVKRUxMjEV9LF++nFq1arF48WKaNGnC6NGjGTlyJAsXLiy17bJly0hPTycsLMzi2IW42wr+/JOYwUNIXbXK7Db1POoxpc0UAN47+h4X0y6af0G/IOhi3HeOrWGQlWR202pujoT1Nlb2nv/TeZIyy2d5va1VnzsHj8GDqf7uu1bXGVp8fDFRaVFUdajKjNAZ1vWnKLBlEuRnQM22VtdUSsnKL97/bXKPBvjKpO67Quqh28ocE0O6DXrBM1/f+HlBfeN2B7dSuxM8v/XGz4ubQ07KzefNtGx+wpIlS7hw4QJBQUG8/fbbAPj4+ODq6mqyXefOndm+fTsAERER9OrVq8Tve/fuzcqVKyksLMTe/targs6ePcvbb7/NoUOHuHz5/t3TSlQe2QcPkR8VReqaNKoOGYKdi4tZ7YY2Hsr++P3sj9/P67++zoa+G3DUmPnh12mKcaPd66eN+4sNWmN2vM8+UJuvj8VxOj6DOdvOsWhwsNltKyq1qyvV35ppdT/7ruxj3dl1AMzqOAtvJytvcZ3cAJd2gdrhrxFBtVXdzd56jrScQhr7uTGiQx3rYhNmk2RJ3JK7uztarRZnZ2f8/PyKj0eWMh/Aycmp+HlCQgK+viUnv/r6+lJUVERycjLVq1e/qX1+fj5PP/00CxYsoFatWpIsiUrBY9BT6DN0uD/6qNmJEhhv5c3uOJvHNz/OpfRLLDq2iKntp5rXWKM1fvh+8pBxK5T441CjtVlN1XYq3hnYnIEf/camE/E81qoGXRr6mB13RZF7+gyFV+Op8q8vZWXuryiX//vt/wBjIts1oKv1nTq4gbMXhE4An0ZWdRV+IYlNJ+JRqeDdJ1qgUcvNobtFkiVbmXb19r9T/eubx2uXTJz7r78sk0+VPSYz1Ldw5/V/D1//PZfgdsPaU6dOpUmTJjz77LNlC1AIG1CpVHiPGVOmtl5OXrzT6R1e3P0i68+vp2ONjnSpaebu8/7Bxg1YvRuanSj9rWWAB8ND67D6QAzTNp1ix+QuuDhUno+EorQ0rrz8MkXXrqEsWIB7/35W9+mkceLdLu+y+szq4lukVmvSH2p1AEfr6j3lFBQxfZPx3/cRHeoQLJW676pKk5ampaUxbNgw3N3dcXd3Z9iwYaWukPp7rs2/HwsWLCg+p1u3bjf9fsiQIXf41QBal9s/7B0tONfJvHPLiaurq8lHnz59is/18/MjIaFkAbbExEQ0Gg1eXl637P+XX37h66+/RqPRoNFo6N69OwDe3t7MmDGj3F6HEHdSzokTXF+wwOyJxp1qdOLZJsYvCNP2T+NqlokvU/8W+hI06FGWMHmtdyNqeDhxJS2XhTujytSHrajd3KjSqxfaOnVw7VYOI0B/eaD6AyzvsRwHtXVL+ykquPHcxcs4Md8Ki3Zd4EpaLjU8nAjrZd0IlbBcpfkaMXToUK5cucJPP/0EwAsvvMCwYcPYsmXLbdtcu3atxM/bt29n1KhRPPHEEyWOjxkzpnheDpS8lXQ/02q16PX6EscsuQ0XGhp60/+fnTt30qZNm9vOV/r222/Jzb0xaf3IkSOMHDmSffv2Ua+edStIhLgbipKTiR05CiU3F4e69fB44nGz2r0S8gqRiZGcTjlNWHgYax5eg73awmrf6XGQcAoaP2LW6S4OGuY83pzhqw6z+kAM/Vv607pWVcuuaSMqjQbfqW/i/fLLqF2t+0J4Mukk7lp36rjXKZ/gki7AuoHw8FxoauVKOow1lVbujwZg9sCgSjUCeM9QKoGzZ88qgHLw4MHiYxEREQqgnD9/3ux+BgwYoDz00EMljnXt2lWZNGmSRfHk5eUpOp2u+BEXF6cAik6ns6ifim7MmDFK27ZtlejoaCUpKUnR6/UWtb98+bLi7OysvPLKK8rZs2eVlStXKvb29so333xTfM53332nNGrU6LZ97NmzRwGUtLS0sr4MIe66lNWrldjx4xV9VpZF7eIz45UO6zsoQauDlHmH51l20aQLijInQFFm+RqfW+CVL08otd/4Uen5/l4lv9Cyv+d3W87Jk4rBYCi3/hKzE5WuG7sq7b9or5xMPGl9h4X5irK8i6LMqKIoawYoipWx5hfqld6LwpXab/yoTFh/3Pr4RAk6nc6sz+9KcRsuIiICd3d32rdvX3zsgQcewN3dnQMHDpjVx/Xr19m6dSujRo266XdffPEF3t7eNGvWjLCwMDIzM032NXfu3OLbge7u7gQEBFj2giqJsLAw1Go1TZs2xcfHh9hYy2qyBAYGsm3bNvbu3UtwcDCzZs1i6dKlJUb2dDodUVGVa/hfiNJUfe45ai5datFkbwB/V3/mdp5LoHsgA+sPtOyinvWMc5iKcuG7MaA3f/+3/+vbFC8XLReuZ/HBLxaUMLjLMvfuJWbI01x94w2UoqJy6VOlUlHHvQ7VXarToGoD6zsMnwfXIsHRAwZ+ZFXxSYClP1/kfEImni5aZvRvan18okxUimJl9a67YM6cOaxevZoLF0ruZ9SwYUOef/55pk4tffXI/Pnzeffdd7l69SqOjjfmBH3yyScEBgbi5+fH6dOnmTp1KvXr12fXrl237Ss/P5/8/Bu1STIyMggICCh112IhxP1Jt3Urzm3aYu9bzazzCw2FZdtwVxcPy0KN1b07ToKeb5fe5i9bf7/G+PXHUdup+O7FDrSsgBOI07/9lmv/nYH7YwOpPmuW1fWU/lZkKCItLw0fZytXBF4Oh7UDAAWeWg3NHrOqu5Nx6Ty+7AB6g8KyZ1rTp/nNK4iFdTIyMnB3dy/189umI0szZ8687STsvx9Hjx4Fbr16SlEUs/+yrFq1imeeeaZEogTG+Uo9evQgKCiIIUOG8M0337B7926OHz9+274cHByoUqVKiYcQQtxK2pdfcfXVMOLGjMGQbV6B2H8mSsevH0eXb2adNPca8Oj/jM9/WwIXdpodZ98W1enf0h+9QWHKV5HkFepLb3SXeTzxBHW++JzqM6wsFAlcybxS/Fxjp7E+UcpKNI7ooUCrYVYnSnmFel79+iR6g0L/lv6SKNmYTZOlCRMmcO7cOZOPoKAg/Pz8uH795g0jk5KSbqrjcyv79u0jKiqK0aNHl3pu69atsbe35+LFijsULYSoPFw6dkDt441r1y6onJ0tarvljy2M2jGKsPAwigxm3nZqOgDa/lXGYNNY42iTmWYNaEY1Nwf+SMpm/k8V4/Z4UVoahpwbRXmdgoNR3WaBiLlOJ59m4A8DmXd4nvnvqykGA3z3AmRdB58m0Ge+1V0u2n2BS4lZeLs68PajzayPUVjFplPqvb298fYuvTpqaGgoOp2Ow4cP065dOwAOHTqETqejQ4cOpbZfuXIlISEhtGzZstRzz5w5Q2Fh4S0LJgohhKW0NWtSd/NmNFUtX2XWsGpD7NX2VHWsil7RozH3n+xesyHuECT8Dr8ugP6LzWrm4axl3hMteH71EVb9Fk3Ppr6E1rt1mY+7QZ+VRdyYF1DZ2xOwfBlqd+tqFQFcz77OxF8mkq/PJzYzFhXlcCvPUGTc7y3ukPH2m9aypPjfDken8smvxoK8cx4LoqqL1voYhVUqxQTvJk2a8PDDDzNmzBgOHjzIwYMHGTNmDP369aNRoxv1Jho3bsymTZtKtM3IyODrr7++5ajSH3/8wdtvv83Ro0eJiYlh27ZtPPXUU7Rq1YqOHTve8dclhLg//DNRUgwGdD/8gGIwlNqukWcjvuz3JfM6z7Os7o+9o/FDO3QCPPyuRbE+2LgaT7czLloJ+/okulzzJ4qXt8K4OApiYymIiaEoJdXq/vKK8pi0ZxJJuUnUc6/HvM7zUFu5/QhgrKbe9z2YcASqNbaqK11OIZM3nsCgwBOta9KrmV/pjcQdVymSJTCuWGvevDm9evWiV69etGjRgnXr1pU4JyoqCp2u5L39jRs3oigKTz/99E19arVafv75Z3r37k2jRo2YOHEivXr1Yvfu3ajV5fAXSAgh/iVhxkyuvvEm12fPNuv8QPfA4vk5eoOeS2kmKvr/k1c96P3OzUVuzTC9b1MCPJ2IT89l+qZTZhfXLG+OTZpQe906Aj75BIe6gVb1ZVAMTN8/nTMpZ/Bw8OB/3f+Hq9b0Xpelyk4puerQvaZV3SmKwrTvT3FVl0dtL2feGiC33yqKSrEarqIzdza9EELotmzh6rTp+M+ZY9EWHfn6fKbum8pv8b/x2cOf0dTLgmXkBj3sfReCnjB75ONEbBpPLY+gyKAw74nmDG5by/zrWcFQUIA+ORl7fxObjVtIURTmH5nP5+c+R2OnYUXPFbT1a2tdp4V58Fkf4w4JT34Grtbvrff10The++Z3NHYqvnmxg2xpchdUitVwQghxv3Hv35/6O3dYvJeZHXZkFGSQU5TD+J/HE59l/sRt9rwDv86HjUONZQXM0KpWVV79a1uNGZvPcCnRdP258mDIz+fKyy8TM/QZCuLiyq3f1WdW8/m5zwF4p+M71idKigJbX4Wrx+H6aSg0b5WjKTHJ2czYfAaAV3o2lESpgpFkSQgh7jL7fywg0Wdlkbx8ealFFu3V9izqtogGVRuQnJvMi7tfJC0vzbwLtn8RqtSE1D+Mq7bMmC8FMLZLXTrV9yav0MCE9SfueDkBQ04OhVfi0aenU3j1WukNzLDljy28f+x9AMLahPFIXfO2gjHp6EqI/Ny4kfmTq6BqHau6yyvUM2HDcXIK9LQP9GRcV9naqaKRZEkIIWxEURTiJ04kafESEt4qvYCkm9aNj7p/hK+zL9G6aMbuGktGQUbpF3L1gcHrQO0AF36Cn98yKz47OxXvD2qJl4uW8wmZzPrxrFntykpTtSq1PltFrVUrcWnfzur+wuPC+e9v/wXguabPMbzZcKv75I89sP0N4/MeM6HeQ1Z3+daWs5yOz8DTRcviIcGo7cqn2KYoP5IsCSGEjahUKqoOHYra0xOPwYPNauPn4seKXivwdPTkXOo5Xtz9Itnm3Aaq0fofBSsXw7HVZl2vWhVH3hvUEpUKvjgUyzfHrpTeyAKF8fFk7f+t+Gf7atVwbt3a6n73x+/nlb2vUKQU8UjgI7za5lWr+yTxHHz1nLFUQPOnoMNEq7v89tgVNhyORaWCxYODqe4uG7lXRJIsidvq1q0bkydPtqqP8PBwQkJCcHR0pG7duixfvtzk+atXr75tNffExESrYhGiInLr0YP6u3biFHRj5VNpZQXqutdlRc8VVNFW4fek35nw8wRyi3JLv1jLwdD1TePzH6fApd1mxditUTUmdTfumzZ90ynOXDWzongpCq5cIWbI01yZMIHcyMhy6RNAl68jLDyMQkMhPWv3ZHan2diprPy4UxRjhe78DKgVCgM+tHrft/MJGUz//hQAk7o3oEtD6yeJiztDkiVxx0RHR/PII4/QuXNnTpw4wbRp05g4cSLffvvtbdsMHjyYa9eulXj07t2brl27Uq2aeftqCVHZ/HPD3YK4OC4/+ig5JrZcAmMNphU9V+Bq78rR60d5+ZeXySnMMdkGgG5vQoshoNYaV3SZaeJDDXiwkQ/5RQbGfX6M9JwCs9vejr2fHw5Nm6ANCEDjV371hNwd3JnTaQ49a/dkXud5Zdtn799UKnhipfG225D1oLGg7tUtZOYV8tLnx8krNNC5gTcvP1QOm/iKO0ZKB5SDspQOMOsftX/RqrVo7IwVfIsMRRToC7BT2eGouVFH5Xb9OttbVlF2xIgRrFmzpsSx6Oho6tSpY3Yfb7zxBps3b+bcuXPFx8aNG8fJkyeJiIgwq4+kpCRq1KjBypUrGTZsmNnXFqKyip/yKhnbtuHUujW1v/i81D3QTiSeYNyuceQU5dDSpyUf9fiIKtpS/h0qKoDkC+AXZFFs6TkF9P9gP3GpuXRr5MPK4W0tnl+j6PWgUqGyM35XN+TkoBQVoS6HsisF+gK06opf7VpvUBi95gh7opKo7u7I1omd8ZQq3TZh7ue3JEvloCzJUvM1zS2+zsKuC+ldpzcAO2J2EBYeRhvfNnz28GfF53TZ2IW0/JtXyJwafsqia+l0Ovr06UNQUBBvv22ceOrj44N7KdsNdO7cme3btxtj6dKFVq1asWTJkuLfb9q0iUGDBpGTk4O9Gfs7vffee8yaNYtr167h5CT38sW9z5CdzfX5C/B+6SXsfc0bTf096Xde3P0iGQUZNKraiI97foyXkwXblKT8YawX5Fb66M7peB1PLDtAfpGBF7rUZdojTcy+jF6nIz7sNZxbt8L7xRfNj88MWy9v5X8n/seq3qvwdy2nGk0GPWyZCEFPQr0Hy6dPYM62c6z49TKO9nZ8PbYDzWtav42LKBtzP79tujecqLjc3d3RarU4Ozvj94/h8chS5hX8M6FJSEi4aaNjX19fioqKSE5ONmv/vVWrVjF06FBJlMR9w87FhepvzSxxTLdlCw4NG+HYqOEt27TwacGq3qsYu2ssV7KucD3nuvnJ0vWzsG4gOHnC89vA2dPk6UE13Jn/ZAsmbYxkxa+XqevtwpB25hWszD50iOx9+8g5dgyPp55CY8beoOYoNBTy6alPic+K56uor5gcMtn6ThUFfnwFTnwOZ76HyadKfW/M8fXROFb8te/bwqdaSqJUSUiyZCOHhh6yuM0/h5e71+rOoaGHbpq0+NMTP1kdmyn169e36Px/30L4eyCztFsLABEREZw9e5a1a9dadE0h7iW5Z85wddp0VHZ21N38A9ratW95XiPPRqzps4aU3BTLqntrnY31gpLOweePw3ObwdH0CPmA4BpEJ2ezePdF/vP9aWp5OtOhfumJT5Vevch74QWqPNy73BIlAHs7e5b3WM53F79jbMux1neoKLDr/+D4GuN7M+CDckmUjsakMm2TcZR/YvcG9GtRflXKxZ0lE7xtxNne2eLH3/OVADR2GpztnUvMVzLVb3lxdXU1+ejTp0/xuX5+fiQkJJRon5iYiEajwcur9G+9n376KcHBwYSEhJRb/EJUNvbVq+MS+gAuoaHY1zI9glO7Sm1a+95Ydn804ShbL281fYGqdWDY98aRpasnYN1jkJtealyTujfg0Zb+FBkUxn1+jD+Ssm46J+fIEa68PBFD3o2J5NWmvIJjUwuSudvIKshiZ8zO4p99XXx5MfjF8ln1tmM6HPirzEL/pdDsMev6BP5IymLM2qMU6hX6BPkxubtM6K5MZGRJ3JZWq0WvL1mx15LbcKGhoWzZsqXE73fu3EmbNm1Kna+UlZXFV199xdy5cy0LWoh7jMbTk4DlyzHk5BSPyBoKCkhbuxaPIUNQu956M9iE7AQm752MLl+Hs8aZB2uZmHNTrTEM22S8HRd/FNb0NyZQLrf/UqNSqZj/ZAuupOVwPDad4asO8+2LHfCtYvwCpxQWcvWNNym8epXkZcup9srkMr4DN4vLjGPiLxO5lH6JxarFdK/dvXw6NhhgW5ixQjfAIwuhtfULSxJ0eTy38jBpOYW0rOnOe4NaYieFJysVGVkSt1WnTh0OHTpETEwMycnJGAwG6tevb/JRo0aN4vbjxo3jzz//ZMqUKZw7d45Vq1axcuVKwsLCis/ZtGkTjRvfvLHnl19+SVFREc8888xdea1CVGQqO7sSSVHK8o9JXPgesSNH3baNj5MPj9d/nMaejelQo0PpF/EPhuE/gosPJPwOq/tClunaZo72alY814Y6Xs5cSctl9LJw0rLzjTHb21Nt6pt4DB6M54hyqJz9l91/7mbIj0O4lH4JHycffF18S29krsgv/kqUVMYCnu3GWN2lLreQ4asOE5+eS6C3C6tGtMVZK+MUlY0kS+K2wsLCUKvVNG3aFB8fH2JjYy1qHxgYyLZt29i7dy/BwcHMmjWLpUuX8sQTTxSfo9PpiIqKuqntypUrefzxx6latarVr0OIe41jUBDaOnXwfPbGlwlFry9xu0ttp2ZKmymseXgNDmpjTSC9Qc/JpJO379gvCEZsA7fqxrlM9qUvrPB2dWDdqPYMu/Ib//f5VObP+JScAuM+d1V69qT6WzPRlMPf47yiPGZFzOKVva+QUZBBc+/mbOi7gSBvy8ofmNTyaePKt8c+htbPWd1dXqGeMWuPEnU9Ex83B9aObIeXq3X1mYRtSOmAclCW0gFCCGENpbAQ1OriekWZu3dz7T//h9foUXiNHn3LNssil7Hs5DKebvw0k1pPuv18xrQYcHQHJ/OTnPOz3kX5Yg3hNYI58MwrfPJcCA4ataUv65bOppxl+v7pXEq/BMDzQc/zcvDL2KvLodhkdgo4uIHmrwU0imJ1ZW4wJkovrDvGrxeScHPQ8OXYUJr6y+dDRWPu57eMLAkhRCWksrcvTpQAMrZtR5+ejj4js/iYoigUpaYWP0/NS0VBYf359Qz8YSD74/ffuvOqdUomSuHz4dQ3xT+mb/qe6EGDyT1zpvhY/fEvUDB1Jh+EDuPXC0mMW3eMvMKScx4tlVOYw7zD83h669NcSr+El6MXH/f4mCkhU8onUUq+CJ88aKyl9Pe4QTklSmP/SpSc7NV8OryNJEqVnCRLQghxD/CfP48aS5dQ9ZmhxcfyTp/hYqfOxL00HpVKxfQHpvNxz4+p4VqDa9nXeHH3i7we/jrxWfG37FMxGMjfu56Mte/Bt6Pgl3fAoCfnYAR5v/+O7h9bF2k8PWk5fDArnm+Hk72aPVFJjFl7lNwCyxMmRVH4+c+fGfDDAD4/9zkGxUCfOn345tFvzJt/ZY4LO+DTHpD+J8RGQE5KuXSbV6hn3OfHCP8rUfrs+ba0r2tBgVBRIcltuHIgt+GEEBVR6po1XJ/7Lm49e1Lzf0uLj1969FFSsxJ565FsYn2MdYpeohs9LzjhHtwaj7/mFRpyc4kKaQMGAw0GJKBxMkBgF7JrjycvOgH3R/ujuUUZkEOXU3h+9RFyCvSE1vVi5Yg2Zk9qvpx+mf/77f/4Pfl3AGq41uA/D/yHTjU6lcM7AuiLYM87sP99488128KQDeBq/Sa2OQVFvPj58eJEadWItoTWk0SpIpPbcEIIcZ/zHD6c+j/vxmfypOJjil5PYXQMbld1LOjzAQ9Uf4BCQyEXDu0g5+vvOL55FQnZxvpodk5OODZpglNwMPqus8DeBaJ/xeXoRLy61btlogTQvq4Xa0e2w9VBQ8TlFIatPExatnkb71ZxqMLF9Is4qh0Z03wMmwZsKr9EKTMB1g64kSi1G2uc0F4OiVJqdgFPf3KI8AtJONrbsXJEG0mU7iEyslQOZGRJCFFZKIpCYWwshfHxOLdvj0qt5kD8Ab757h08TkbzZzUVxxrb06N2D15o8QINPBrcqLifFAVfDTdW+0YFD02HLq/d9lrHY9MYseowGXlFBHq7sPr5ttT2cin+fU5hDt9d/I7TKad5t/O7xcf3xu0lyDsIb6fyq/KNosDyznD9FGhdjaUBgh4vl67jUnMY/tlhLidl4+Fsz6oRbWldS1byVgayke5dJMmSEKKyMygGwuPC+eLcFxxKMG7HtKDLAh4OfBiA3KJcNHYa7IsKYNvrEPk5DFoHTR812e+F65k8/9kR4tNz8XLRsmxYC9rVMdZGSslNoec3PSk0FLKx70aaeTe7sy/y4i74ZRY8sRK8y6eC9ul4HSNXHyExMx9/d0fWjmpH/Wpu5dK3uPMkWbqLJFkSQtxLolKj2Bi1kdfavFZcXuDjkx/z6alPGdtyLKObj4arJ8j3bUpmQSaejp7YXfoFfBqBRwBgHDVKzk0mMSeRY9fO8NnRfWQo0agUe+a0X8mAYGMB2w9OfICPkw/96vXDxd7ltjFZrDAXDn4Ebv4Q/PSN4wYD2JXPDJTvjl9h6nenyC8y0MjXjTUj2+Hn7lh6Q1FhmPv5LWVEhRBClNDIsxEzQmeUOHbw2kHy9Hl4OHgYD/i34lTCUZ7f8Tz2dvY4FhWgUhTU9o4U2GnILsop2akW/q66NPnrfRyNCeI//ZowodWE8g3eoIeTG42TuDPiwaEK1O0GVaobf18OiVKh3sA7W8+x+kAMAA81rsaiwcG4O5VDOQNRIUmyJIQQolQre68kJiMGTwfP4mOpeamoUFFoKKTQTgWowFBgfACOage8nXyo61GXZl7NaOLZlN/OOPHpuWTWHfyT36+k8+EzralZtRw2+y7IgZMb4OAySLloPOYeAN3/C67ltyVKXGoOU76K5EhMGgATuzdgcvcGstfbPU5uw5UDuQ0nhLhfFRoKSc5JJl+fhyFmP4YDS9GkXMZbr8cFO1RNB0DPt4tvzwHsOZ/I5C8j0eUW4uag4T/9mjCoTcCNieSWOr8NfngJco0JDI7u0PlV42o3+/K5LaYoCl8fvcJbW86QXaDH1UHD+4Na0quZX7n0L2xDSgcIq3Xr1o3JkyeXuf21a9cYOnQojRo1ws7Ozuy+YmNj6d+/Py4uLnh7ezNx4kQKCsxbdiyEuLvs7eyp7lqdOu6B1G05jPpjD1LnybW41u6EStHDmU1g94+bGLp4Hgx04seXOxEc4EFmfhFvfHuK5/7abLZU+kK4egKu36gejndDY6JUtQ48PA9eOQMdJ5VbonQ9I48xa4/y+re/k12gp22dqmyb2FkSpfuI3IYTd0x+fj4+Pj5Mnz6dRYsWmdVGr9fTt29ffHx82L9/PykpKQwfPhxFUfjf//53hyMWQljNTg2N+xofCafg0u4b84XAOAIU/SsBPk34rmYbIjyqseG8nrhLHjyzKJohD7VlRIc6ONqrjQmRLh4yrhj3q7tyFOKPQ1EuNO4HQ74w9uldH0buhJptjNcvJ3mFelbuj+bDPZfIKdCjVdvxaq+GjO5cF7XcdruvyG24clCW23CGHOPkR5WTU/HQs1JQgFJUBBoNdlrtzec6OhbvBaUUFhZvpGnn4FDquSp7yyYejhgxgjVr1pQ4Fh0dTZ06dSzq52/dunUjODiYxYsXmzxv+/bt9OvXj7i4OPz9/QHYuHEjI0aMIDExUW5zClGZFebBRw9AWvQtf33SUJcBBbOp4eHEq70a8tiuTqj+vrX2T47uUK87PLmqXPZy+zeDQWH76QTmbj/HlTTjaFfrWh7Mebw5jf3k36B7iayGq+CiWocA0ODAb2g8jRMmU1atImnxEjyeepLqs2YVn3uhYyeU3Fzq7d6NtqZxuW3a+vVcn/suVfr1o8bCBcXnXureA31aGnW3bMahgbGOSPqmTVQdNMii+JYsWcKFCxcICgri7bffBsDHxwdXV1eT7Tp37sz27dstutY/RUREEBQUVJwoAfTu3Zv8/HyOHTvGgw8+WOa+hRA2Zu8IkyKNlbSvHIUrR4yJU8Y1lIyr1FRXoXqOI/HpuUz56iR1XQKoXcWXKr51UHsEQPWWENAOvBqU2/L/fyooMvD9iXg+/vUP/kjKBsCviiNTH2nMoy39yz6nSlR6kiyJW3J3d0er1eLs7Iyf34378pGRkSbbOTk5WXXdhIQEfH1LrlypWrUqWq2WhIQEq/oWQlQQbn7QpJ/x8RcV4AXsKdSz6rdolu35g4HZb0I2eGVreeaB2jxZuya1vMph5dy/XE7K4vvIq3x1JI6EjDxjiI4aRnYMZGzXumbvayfuXfInwEYaHT8GGG/D/c1r5Eg8n3sONCX/tzT8bb/xXMcbkxWrDh2Kx1NPgbrk/fn6P+++6VyPxx4rt7jr169fbn3dzq2+vSmKIt/qhLgPONqrealbfZ5pV5uNR2JZcyCGq7o8lv58kaU/X6SZfxX6BPnRo6kvDaq5lWnukN6gcPZqBhGXk/nx92v8fkVX/Ltqbg6M7hzI0+1q4eYodZOEUaVJlt555x22bt1KZGQkWq2W9PT0UtsoisJbb73FihUrSEtLo3379nz44Yc0a3ajpH5+fj5hYWFs2LCB3NxcunfvzkcffUTNmjXv4KsBO+ebvx2ptFpU/5irZPJce/tbzkO63bnl5U7fhvPz8+PQoUMljqWlpVFYWHjTiJMQ4t7l7mzP2K71GNkpkB1nElh/KJaDl1M4czWDM1czWLjzAq4OGprXcKdlgAe1vZyp5uaAbxVH3J3s0RsUigwKhXoD1zPyiEvL5UpqDucTMjn2ZxpZ+UXF11LbqejcwJuBwTXo09wPB035TRIX94ZKkywVFBTw1FNPERoaysqVK81qM3/+fN5//31Wr15Nw4YNmT17Nj179iQqKgo3N+PePZMnT2bLli1s3LgRLy8vXn31Vfr168exY8dQq+/vvzBarRa9Xl/i2J2+DRcaGso777zDtWvXqF7duIJm586dODg4EBISYlXfQojKx15tR78W/vRr4U9KVj67zl5n++kEjsSkkpVfRMTlFCIup1jcr5uDhraBnnRp4E2/lv54uzqU3kjctyrdarjVq1czefLkUkeWFEXB39+fyZMn88YbbwDGUSRfX1/mzZvH2LFj0el0+Pj4sG7dOgYPHgzA1atXCQgIYNu2bfTu3dusmO7VopQvvPACkZGRfPXVV7i6uuLp6YmdhZMq/06uRo8eTaNGjXjttdfQarU0bdoUgE2bNjF16lTOnz8PGEsHBAcH4+vry4IFC0hNTWXEiBEMHDhQSgcIIYoV6Q1cTMziZFw6p+J1JOjyuJ6ZR2JGPpl5RWjsVKjVKtQqFT5uDtSs6kyApxN1vFwIqV2VJtWryPJ/IavhoqOjSUhIoFevXsXHHBwc6Nq1KwcOHGDs2LEcO3aMwsLCEuf4+/sTFBTEgQMHbpss5efnk5+fX/xzRkbGnXshNhQWFsbw4cNp2rQpubm5ZSod0KpVq+Lnx44dY/369dSuXZuYmBgAdDodUVFRxeeo1Wq2bt3KSy+9RMeOHXFycmLo0KEsXLiwPF6SEOIeoVHb0aR6FZpUr8IQWwcj7nn3bLL098qpf89z8fX15c8//yw+R6vVUrVq1ZvOMbXyau7cubz11lvlHHHF07BhQyIiIqzqo7SByxEjRjBixIgSx2rVqsWPP/5o1XWFEEKI8mLT7U5mzpyJSqUy+Th69KhV1/j3CipzVlWVds7UqVPR6XTFj7i4OKtiFEIIIUTFZdORpQkTJjBkiOkB1LJWjP67NlBCQkLxRGGAxMTE4tEmPz8/CgoKSEtLKzG6lJiYSIcOHW7bt4ODAw4OMhlQCCGEuB/YNFny9vbG29v7jvQdGBiIn58fu3btKp43U1BQQHh4OPPmzQMgJCQEe3t7du3axaC/Klxfu3aN06dPM3/+/DsSlxBCCCEql0ozZyk2NpbU1FRiY2PR6/XFq6zq169fXPuncePGzJ07l8ceewyVSsXkyZOZM2cODRo0oEGDBsyZMwdnZ2eGDh0KGKtUjxo1ildffRUvLy88PT0JCwujefPm9OjRw1YvVQghhBAVSKVJlv773/+W2Nj179GiPXv20K1bNwCioqLQ6W5UYn399dfJzc3lpZdeKi5KuXPnzuIaSwCLFi1Co9EwaNCg4qKUq1evvu9rLAkhhBDCqNLVWaqI7tU6S0IIIcS9zNzPb5uuhhNCCCGEqOgkWRJCCCGEMEGSJSGEEEIIEyRZEkIIIYQwQZIlIYQQQggTJFkSQgghhDBBkiUhhBBCCBMqTVHKiuzvUlUZGRk2jkQIIYQQ5vr7c7u0kpOSLJWDzMxMAAICAmwciRBCCCEslZmZibu7+21/LxW8y4HBYODq1au4ubmhUqnKrd+MjAwCAgKIi4uTyuC3Ie+RafL+mCbvT+nkPTJN3h/TKvr7oygKmZmZ+Pv7Y2d3+5lJMrJUDuzs7KhZs+Yd679KlSoV8g9ZRSLvkWny/pgm70/p5D0yTd4f0yry+2NqROlvMsFbCCGEEMIESZaEEEIIIUyQZKkCc3BwYMaMGTg4ONg6lApL3iPT5P0xTd6f0sl7ZJq8P6bdK++PTPAWQgghhDBBRpaEEEIIIUyQZEkIIYQQwgRJloQQQgghTJBkSQghhBDCBEmWKrCPPvqIwMBAHB0dCQkJYd++fbYOqcL49ddf6d+/P/7+/qhUKr7//ntbh1ShzJ07l7Zt2+Lm5ka1atUYOHAgUVFRtg6rwli2bBktWrQoLpQXGhrK9u3bbR1WhTV37lxUKhWTJ0+2dSgVxsyZM1GpVCUefn5+tg6rQomPj+fZZ5/Fy8sLZ2dngoODOXbsmK3DKhNJliqoL7/8ksmTJzN9+nROnDhB586d6dOnD7GxsbYOrULIzs6mZcuWfPDBB7YOpUIKDw9n/PjxHDx4kF27dlFUVESvXr3Izs62dWgVQs2aNXn33Xc5evQoR48e5aGHHmLAgAGcOXPG1qFVOEeOHGHFihW0aNHC1qFUOM2aNePatWvFj1OnTtk6pAojLS2Njh07Ym9vz/bt2zl79izvvfceHh4etg6tTKR0QAXVvn17WrduzbJly4qPNWnShIEDBzJ37lwbRlbxqFQqNm3axMCBA20dSoWVlJREtWrVCA8Pp0uXLrYOp0Ly9PRkwYIFjBo1ytahVBhZWVm0bt2ajz76iNmzZxMcHMzixYttHVaFMHPmTL7//nsiIyNtHUqF9Oabb/Lbb7/dM3dEZGSpAiooKODYsWP06tWrxPFevXpx4MABG0UlKjOdTgcYEwJRkl6vZ+PGjWRnZxMaGmrrcCqU8ePH07dvX3r06GHrUCqkixcv4u/vT2BgIEOGDOHy5cu2DqnC2Lx5M23atOGpp56iWrVqtGrVik8++cTWYZWZJEsVUHJyMnq9Hl9f3xLHfX19SUhIsFFUorJSFIUpU6bQqVMngoKCbB1OhXHq1ClcXV1xcHBg3LhxbNq0iaZNm9o6rApj48aNHD9+XEayb6N9+/asXbuWHTt28Mknn5CQkECHDh1ISUmxdWgVwuXLl1m2bBkNGjRgx44djBs3jokTJ7J27Vpbh1YmGlsHIG5PpVKV+FlRlJuOCVGaCRMm8Pvvv7N//35bh1KhNGrUiMjISNLT0/n2228ZPnw44eHhkjABcXFxTJo0iZ07d+Lo6GjrcCqkPn36FD9v3rw5oaGh1KtXjzVr1jBlyhQbRlYxGAwG2rRpw5w5cwBo1aoVZ86cYdmyZTz33HM2js5yMrJUAXl7e6NWq28aRUpMTLxptEkIU15++WU2b97Mnj17qFmzpq3DqVC0Wi3169enTZs2zJ07l5YtW7JkyRJbh1UhHDt2jMTEREJCQtBoNGg0GsLDw1m6dCkajQa9Xm/rECscFxcXmjdvzsWLF20dSoVQvXr1m754NGnSpNIuUpJkqQLSarWEhISwa9euEsd37dpFhw4dbBSVqEwURWHChAl89913/PLLLwQGBto6pApPURTy8/NtHUaF0L17d06dOkVkZGTxo02bNjzzzDNERkaiVqttHWKFk5+fz7lz56hevbqtQ6kQOnbseFO5kgsXLlC7dm0bRWQduQ1XQU2ZMoVhw4bRpk0bQkNDWbFiBbGxsYwbN87WoVUIWVlZXLp0qfjn6OhoIiMj8fT0pFatWjaMrGIYP34869ev54cffsDNza14lNLd3R0nJycbR2d706ZNo0+fPgQEBJCZmcnGjRvZu3cvP/30k61DqxDc3Nxumt/m4uKCl5eXzHv7S1hYGP3796dWrVokJiYye/ZsMjIyGD58uK1DqxBeeeUVOnTowJw5cxg0aBCHDx9mxYoVrFixwtahlY0iKqwPP/xQqV27tqLVapXWrVsr4eHhtg6pwtizZ48C3PQYPny4rUOrEG713gDKZ599ZuvQKoSRI0cW/93y8fFRunfvruzcudPWYVVoXbt2VSZNmmTrMCqMwYMHK9WrV1fs7e0Vf39/5fHHH1fOnDlj67AqlC1btihBQUGKg4OD0rhxY2XFihW2DqnMpM6SEEIIIYQJMmdJCCGEEMIESZaEEEIIIUyQZEkIIYQQwgRJloQQQgghTJBkSQghhBDCBEmWhBBCCCFMkGRJCCGEEMIESZaEEEIIIUyQZEkIIYQQwgRJloQQQgghTJBkSQghhBDCBEmWhBDiFjZs2ICjoyPx8fHFx0aPHk2LFi3Q6XQ2jEwIcbfJRrpCCHELiqIQHBxM586d+eCDD3jrrbf49NNPOXjwIDVq1LB1eEKIu0hj6wCEEKIiUqlUvPPOOzz55JP4+/uzZMkS9u3bJ4mSEPchGVkSQggTWrduzZkzZ9i5cyddu3a1dThCCBuQOUtCCHEbO3bs4Pz58+j1enx9fW0djhDCRmRkSQghbuH48eN069aNDz/8kI0bN+Ls7MzXX39t67CEEDYgc5aEEOJfYmJi6Nu3L2+++SbDhg2jadOmtG3blmPHjhESEmLr8IQQd5mMLAkhxD+kpqbSsWNHunTpwscff1x8fMCAAeTn5/PTTz/ZMDohhC1IsiSEEEIIYYJM8BZCCCGEMEGSJSGEEEIIEyRZEkIIIYQwQZIlIYQQQggTJFkSQgghhDBBkiUhhBBCCBMkWRJCCCGEMEGSJSGEEEIIEyRZEkIIIYQwQZIlIYQQQggTJFkSQgghhDDh/wGvKMKB9Pvm8QAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from pySDC.helpers.stats_helper import get_sorted\n", + "import matplotlib.pyplot as plt\n", + "\n", + "u = get_sorted(stats, type='u', sortby='time')\n", + "for i, ls in zip([0, 3, 6, 9], ['-', '--', '-.', ':']):\n", + " plt.plot(P.grid, u[i][1], label=f't={u[i][0]:.1f}', ls=ls)\n", + "plt.legend(frameon=False)\n", + "plt.xlabel('$x$')\n", + "plt.ylabel('$u$')\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "id": "a82806ce-5882-42a6-836e-f2bc05e2dce0", + "metadata": {}, + "source": [ + "As time progresses, the solution moves to the left and decays, just like we expected.\n", + "\n", + "So, let's look at something more interesting: What is the speedup that we got from the diagonal preconditioner?\n", + "Diagonal preconditioners allow to distribute the implicit Euler steps and right hand side evaluations, which make up the bulk of computational cost, among as many tasks as we have nodes, which is 3 in our case.\n", + "So, neglecting communication cost, we get a speedup of 3, right?\n", + "Well, no.\n", + "We use an iterative solver for the linear systems in `solve_system`, which may need a different number of iterations depending on the step size for the implicit Euler step.\n", + "Eariler, we strangely set up individual counters for each step size, but now we can use this to estimate speedup." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "496e43ea-7421-4835-8ff1-d13883c9b9a9", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Performed 45% of GMRES iterations on task 2. This translates to speedup of 2.25 and 75% parallel efficiency in the implicit solves.\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAGwCAYAAABB4NqyAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAOfFJREFUeJzt3Xl0FFXe//FPS5JmS1pCyKYhBAk4GkAkbKLsAkFAQAUUBRR5HGURERmiPgrqgKKoM6KMowyI4gM6gDDioCCEZYIaCIgsshkkKDGyJQQwCcn9/cGPHpsspKGbTlPv1zl1TureW9XfuqcOfE5VdZfNGGMEAABgIVf4ugAAAIBLjQAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAsJ8DXBVQGxcXF+vnnnxUcHCybzebrcgAAQAUYY3T8+HFFR0friivcu6ZDAJL0888/KyYmxtdlAACAC5CZmamrr77arW0IQJKCg4MlnZnAkJAQH1cDAAAqIjc3VzExMc7/x91BAJKct71CQkIIQAAA+JkLeXyFh6ABAIDlEIAAAIDlEIAAAIDlEIAAAIDlEIAAAIDlEIAAAIDlEIAAAIDlEIAAAIDlEIAAAIDlEIAAAIDlEIAAAIDlEIAAAIDlEIAAAIDlEIAAAIDlEIAAAIDlBPi6AADA5a/ehKW+LgE+tO/F23xdQglcAQIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJbj0wA0ZcoUtWjRQsHBwQoPD1efPn20c+dOlzHGGE2cOFHR0dGqVq2aOnTooG3btrmMyc/P16hRoxQWFqYaNWqod+/eOnDgwKU8FAAA4Ed8GoBWr16tESNG6KuvvtLy5ct1+vRpde3aVSdOnHCOmTp1ql599VVNnz5daWlpioyM1K233qrjx487x4wZM0aLFi3SvHnztG7dOuXl5alnz54qKiryxWEBAIBKzmaMMb4u4qxff/1V4eHhWr16tdq1aydjjKKjozVmzBj96U9/knTmak9ERIReeuklPfTQQ8rJyVGdOnX0/vvva8CAAZKkn3/+WTExMfrss8/UrVu3835ubm6uHA6HcnJyFBIS4tVjBAAr4pegrc1bvwR9Mf9/V6pngHJyciRJoaGhkqSMjAxlZWWpa9euzjF2u13t27dXamqqJGnjxo0qLCx0GRMdHa2EhATnmHPl5+crNzfXZQEAANZRaQKQMUZjx47VzTffrISEBElSVlaWJCkiIsJlbEREhLMvKytLQUFBqlWrVpljzjVlyhQ5HA7nEhMT4+nDAQAAlVilCUAjR47Uli1b9H//938l+mw2m8u6MaZE27nKG5OcnKycnBznkpmZeeGFAwAAv1MpAtCoUaO0ZMkSrVq1SldffbWzPTIyUpJKXMnJzs52XhWKjIxUQUGBjh49WuaYc9ntdoWEhLgsAADAOnwagIwxGjlypBYuXKiVK1cqLi7OpT8uLk6RkZFavny5s62goECrV6/WTTfdJElq3ry5AgMDXcYcPHhQW7dudY4BAAD4vQBffviIESP04YcfavHixQoODnZe6XE4HKpWrZpsNpvGjBmjyZMnKz4+XvHx8Zo8ebKqV6+ue+65xzl22LBhevzxx1W7dm2FhoZq3Lhxaty4sbp06eLLwwMAAJWUTwPQjBkzJEkdOnRwaZ81a5aGDh0qSRo/frxOnTqlRx55REePHlWrVq30xRdfKDg42Dn+tddeU0BAgPr3769Tp06pc+fOmj17tqpUqXKpDgUAAPiRSvU7QL7C7wABgHfxO0DWxu8AAQAAVAIEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDk+DUBr1qxRr169FB0dLZvNpk8++cSl32azlbq8/PLLzjEdOnQo0T9w4MBLfCQAAMCf+DQAnThxQk2bNtX06dNL7T948KDL8o9//EM2m0133HGHy7jhw4e7jHv77bcvRfkAAMBPBfjyw5OSkpSUlFRmf2RkpMv64sWL1bFjR9WvX9+lvXr16iXGlic/P1/5+fnO9dzc3ApvCwAA/J/fPAP0yy+/aOnSpRo2bFiJvrlz5yosLEzXX3+9xo0bp+PHj5e7rylTpsjhcDiXmJgYb5UNAAAqIZ9eAXLHe++9p+DgYPXr18+lfdCgQYqLi1NkZKS2bt2q5ORkffvtt1q+fHmZ+0pOTtbYsWOd67m5uYQgAAAsxG8C0D/+8Q8NGjRIVatWdWkfPny48++EhATFx8crMTFR6enpuvHGG0vdl91ul91u92q9AACg8vKLW2Br167Vzp079eCDD5537I033qjAwEDt3r37ElQGAAD8kV8EoJkzZ6p58+Zq2rTpecdu27ZNhYWFioqKugSVAQAAf+TTW2B5eXnas2ePcz0jI0ObN29WaGio6tatK+nM8zkff/yxpk2bVmL7vXv3au7cuerRo4fCwsK0fft2Pf7442rWrJnatm17yY4DAAD4F58GoA0bNqhjx47O9bMPJg8ZMkSzZ8+WJM2bN0/GGN19990ltg8KCtKXX36pv/zlL8rLy1NMTIxuu+02Pfvss6pSpcolOQYAAOB/bMYY4+sifC03N1cOh0M5OTkKCQnxdTkAcNmpN2Gpr0uAD+178Tav7Pdi/v/2i2eAAAAAPIkABAAALMdvfgcIwIXh1gO8dfsB8GdcAQIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJZDAAIAAJbj0wC0Zs0a9erVS9HR0bLZbPrkk09c+ocOHSqbzeaytG7d2mVMfn6+Ro0apbCwMNWoUUO9e/fWgQMHLuFRAAAAf+PTAHTixAk1bdpU06dPL3NM9+7ddfDgQefy2WefufSPGTNGixYt0rx587Ru3Trl5eWpZ8+eKioq8nb5AADATwX48sOTkpKUlJRU7hi73a7IyMhS+3JycjRz5ky9//776tKliyTpgw8+UExMjFasWKFu3bp5vGYAAOD/Kv0zQCkpKQoPD1fDhg01fPhwZWdnO/s2btyowsJCde3a1dkWHR2thIQEpaamlrnP/Px85ebmuiwAAMA6KnUASkpK0ty5c7Vy5UpNmzZNaWlp6tSpk/Lz8yVJWVlZCgoKUq1atVy2i4iIUFZWVpn7nTJlihwOh3OJiYnx6nEAAIDKxae3wM5nwIABzr8TEhKUmJio2NhYLV26VP369StzO2OMbDZbmf3JyckaO3ascz03N5cQBACAhVTqK0DnioqKUmxsrHbv3i1JioyMVEFBgY4ePeoyLjs7WxEREWXux263KyQkxGUBAADW4VcB6PDhw8rMzFRUVJQkqXnz5goMDNTy5cudYw4ePKitW7fqpptu8lWZAACgkvPpLbC8vDzt2bPHuZ6RkaHNmzcrNDRUoaGhmjhxou644w5FRUVp3759evLJJxUWFqa+fftKkhwOh4YNG6bHH39ctWvXVmhoqMaNG6fGjRs7vxUGAABwLp8GoA0bNqhjx47O9bPP5QwZMkQzZszQd999pzlz5ujYsWOKiopSx44dNX/+fAUHBzu3ee211xQQEKD+/fvr1KlT6ty5s2bPnq0qVapc8uMBAAD+wacBqEOHDjLGlNn/+eefn3cfVatW1RtvvKE33njDk6UBAIDLmF89AwQAAOAJBCAAAGA5BCAAAGA5BCAAAGA5BCAAAGA5bgeg9PR0fffdd871xYsXq0+fPnryySdVUFDg0eIAAAC8we0A9NBDD2nXrl2SpB9++EEDBw5U9erV9fHHH2v8+PEeLxAAAMDT3A5Au3bt0g033CBJ+vjjj9WuXTt9+OGHmj17thYsWODp+gAAADzO7QBkjFFxcbEkacWKFerRo4ckKSYmRocOHfJsdQAAAF7gdgBKTEzUCy+8oPfff1+rV6/WbbfdJunMe7zKewM7AABAZeF2AHr99deVnp6ukSNH6qmnnlKDBg0kSf/85z95AzsAAPALbr8LrEmTJi7fAjvr5Zdf5gWkAADAL1zwy1ALCgqUnZ3tfB7orLp16150UQAAAN7kdgDatWuXhg0bptTUVJd2Y4xsNpuKioo8VhwAAIA3uB2A7r//fgUEBOjTTz9VVFSUbDabN+oCAADwGrcD0ObNm7Vx40Zde+213qgHAADA69z+Fth1113H7/0AAAC/5nYAeumllzR+/HilpKTo8OHDys3NdVkAAAAqO7dvgXXp0kWS1LlzZ5d2HoIGAAD+wu0AtGrVKm/UAQAAcMm4HYDat2/vjToAAAAumQv6IcRjx45p5syZ2rFjh2w2m6677jo98MADcjgcnq4PAADA49x+CHrDhg265ppr9Nprr+nIkSM6dOiQXn31VV1zzTVKT0/3Ro0AAAAe5fYVoMcee0y9e/fWO++8o4CAM5ufPn1aDz74oMaMGaM1a9Z4vEgAAABPcjsAbdiwwSX8SFJAQIDGjx+vxMREjxYHAADgDW7fAgsJCdH+/ftLtGdmZio4ONgjRQEAAHiT2wFowIABGjZsmObPn6/MzEwdOHBA8+bN04MPPqi7777bGzUCAAB4lNu3wF555RXZbDYNHjxYp0+fliQFBgbq4Ycf1osvvujxAgEAADzN7QAUFBSkv/zlL5oyZYr27t0rY4waNGig6tWre6M+AAAAj7ug3wGSpOrVq6tx48aerAUAAOCSqFAA6tevn2bPnq2QkBD169ev3LELFy70SGEAAADeUqEA5HA4ZLPZJJ35FtjZvwEAAPxRhQLQrFmznH/Pnj3bYx++Zs0avfzyy9q4caMOHjyoRYsWqU+fPpKkwsJCPf300/rss8/0ww8/yOFwqEuXLnrxxRcVHR3t3EeHDh20evVql/0OGDBA8+bN81idAADg8uL21+A7deqkY8eOlWjPzc1Vp06d3NrXiRMn1LRpU02fPr1E38mTJ5Wenq7//d//VXp6uhYuXKhdu3apd+/eJcYOHz5cBw8edC5vv/22W3UAAABrcfsh6JSUFBUUFJRo/+2337R27Vq39pWUlKSkpKRS+xwOh5YvX+7S9sYbb6hly5bav3+/6tat62yvXr26IiMjK/y5+fn5ys/Pd67n5ua6VTcAAPBvFQ5AW7Zscf69fft2ZWVlOdeLioq0bNkyXXXVVZ6t7hw5OTmy2Wy68sorXdrnzp2rDz74QBEREUpKStKzzz5b7q9ST5kyRZMmTfJqrQAAoPKqcAC64YYbZLPZZLPZSr3VVa1aNb3xxhseLe73fvvtN02YMEH33HOPQkJCnO2DBg1SXFycIiMjtXXrViUnJ+vbb78tcfXo95KTkzV27Fjnem5urmJiYrxWOwAAqFwqHIAyMjJkjFH9+vX1zTffqE6dOs6+oKAghYeHq0qVKl4psrCwUAMHDlRxcbHeeustl77hw4c7/05ISFB8fLwSExOVnp6uG2+8sdT92e122e12r9QKAAAqvwoHoNjYWElScXGx14opTWFhofr376+MjAytXLnS5epPaW688UYFBgZq9+7dZQYgAABgbRf8S9Dbt2/X/v37SzwQXdq3tC7U2fCze/durVq1SrVr1z7vNtu2bVNhYaGioqI8VgcAALi8uB2AfvjhB/Xt21ffffedbDabjDGS5PxxxKKiogrvKy8vT3v27HGuZ2RkaPPmzQoNDVV0dLTuvPNOpaen69NPP1VRUZHzwevQ0FAFBQVp7969mjt3rnr06KGwsDBt375djz/+uJo1a6a2bdu6e2gAAMAi3P4doEcffVRxcXH65ZdfVL16dW3btk1r1qxRYmKiUlJS3NrXhg0b1KxZMzVr1kySNHbsWDVr1kzPPPOMDhw4oCVLlujAgQO64YYbFBUV5VxSU1MlnXn26Msvv1S3bt3UqFEjjR49Wl27dtWKFSu89jwSAADwf25fAVq/fr1WrlypOnXq6IorrtAVV1yhm2++WVOmTNHo0aO1adOmCu+rQ4cOzitIpSmvT5JiYmJK/Ao0AADA+bh9BaioqEg1a9aUJIWFhennn3+WdOYh6Z07d3q2OgAAAC9w+wpQQkKCtmzZovr166tVq1aaOnWqgoKC9Pe//13169f3Ro0AAAAe5XYAevrpp3XixAlJ0gsvvKCePXvqlltuUe3atTV//nyPFwgAAOBpbgegbt26Of+uX7++tm/friNHjqhWrVrOb4IBAABUZm49A3T69GkFBARo69atLu2hoaGEHwAA4DfcCkABAQGKjY1167d+AAAAKhu3vwX29NNPKzk5WUeOHPFGPQAAAF7n9jNAf/3rX7Vnzx5FR0crNjZWNWrUcOlPT0/3WHEAAADe4HYA6tOnjxfKAAAAuHTcDkDPPvusN+oAAAC4ZNx+BkiSjh07pnfffdflWaD09HT99NNPHi0OAADAG9y+ArRlyxZ16dJFDodD+/bt0/DhwxUaGqpFixbpxx9/1Jw5c7xRJwAAgMe4fQVo7NixGjp0qHbv3q2qVas625OSkrRmzRqPFgcAAOANbgegtLQ0PfTQQyXar7rqKmVlZXmkKAAAAG9yOwBVrVpVubm5Jdp37typOnXqeKQoAAAAb3I7AN1+++167rnnVFhYKEmy2Wzav3+/JkyYoDvuuMPjBQIAAHia2wHolVde0a+//qrw8HCdOnVK7du3V4MGDRQcHKw///nP3qgRAADAo9z+FlhISIjWrVunlStXKj09XcXFxbrxxhvVpUsXb9QHAADgcW4HoDlz5mjAgAHq1KmTOnXq5GwvKCjQvHnzNHjwYI8WCAAA4Glu3wK7//77lZOTU6L9+PHjuv/++z1SFAAAgDe5HYCMMbLZbCXaDxw4IIfD4ZGiAAAAvKnCt8CaNWsmm80mm82mzp07KyDgv5sWFRUpIyND3bt390qRAAAAnlThAHT2LfCbN29Wt27dVLNmTWdfUFCQ6tWrx9fgAQCAX6hwADr7Fvh69eppwIABLq/BAAAA8CdufwtsyJAh3qgDAADgkqlQAAoNDdWuXbsUFhamWrVqlfoQ9FlHjhzxWHEAAADeUKEA9Nprryk4OFiS9Prrr3uzHgAAAK+rUAD6/W0vboEBAAB/5/bvAAEAAPg7AhAAALAcAhAAALAcnwagNWvWqFevXoqOjpbNZtMnn3zi0m+M0cSJExUdHa1q1aqpQ4cO2rZtm8uY/Px8jRo1SmFhYapRo4Z69+6tAwcOXMKjAAAA/uaiA9CPP/6o7du3q7i42O1tT5w4oaZNm2r69Oml9k+dOlWvvvqqpk+frrS0NEVGRurWW2/V8ePHnWPGjBmjRYsWad68eVq3bp3y8vLUs2dPFRUVXfAxAQCAy1uFA9B7771X4ivw//M//6P69eurcePGSkhIUGZmplsfnpSUpBdeeEH9+vUr0WeM0euvv66nnnpK/fr1U0JCgt577z2dPHlSH374oSQpJydHM2fO1LRp09SlSxc1a9ZMH3zwgb777jutWLHCrVoAAIB1VDgA/e1vf3N52/uyZcs0a9YszZkzR2lpabryyis1adIkjxWWkZGhrKwsde3a1dlmt9vVvn17paamSpI2btyowsJClzHR0dFKSEhwjilNfn6+cnNzXRYAAGAdFQ5Au3btUmJionN98eLF6t27twYNGqQbb7xRkydP1pdffumxwrKysiRJERERLu0RERHOvqysLAUFBalWrVpljinNlClT5HA4nEtMTIzH6gYAAJVfhQPQqVOnFBIS4lxPTU1Vu3btnOv169cvN3RcqHNfu2GMKfdVHBUZk5ycrJycHOfi7q07AADg3yocgGJjY7Vx40ZJ0qFDh7Rt2zbdfPPNzv6srCyXW2QXKzIy0rnf38vOznZeFYqMjFRBQYGOHj1a5pjS2O12hYSEuCwAAMA6KhyABg8erBEjRuj555/XXXfdpWuvvVbNmzd39qempiohIcFjhcXFxSkyMlLLly93thUUFGj16tW66aabJEnNmzdXYGCgy5iDBw9q69atzjEAAADnqtC7wCTpT3/6k06ePKmFCxcqMjJSH3/8sUv/f/7zH919991ufXheXp727NnjXM/IyNDmzZsVGhqqunXrasyYMZo8ebLi4+MVHx+vyZMnq3r16rrnnnskSQ6HQ8OGDdPjjz+u2rVrKzQ0VOPGjVPjxo3VpUsXt2oBAADWUeEAdMUVV+j555/X888/X2r/uYGoIjZs2KCOHTs618eOHSvpzAtXZ8+erfHjx+vUqVN65JFHdPToUbVq1UpffPGF88300pk31QcEBKh///46deqUOnfurNmzZ6tKlSpu1wMAAKzBZowxFRn4zTffqHnz5s5gce6Dxvn5+Vq8eLH69+/vnUq9KDc3Vw6HQzk5OTwPhMtOvQlLfV0CfGzfi7f5ugTOQ4vz1jl4Mf9/V/gZoDZt2ujw4cPOdYfDoR9++MG5fuzYMbdvgQEAAPhChQPQuReKSrtwVMGLSQAAAD7l0Zehnu/3eQAAACoDn74NHgAAwBcq/C0wSdq+fbvzhwmNMfr++++Vl5cn6cyPIwIAAPgDtwJQ586dXZ7z6dmzp6Qzt74q8ooKAACAyqDCASgjI8ObdQAAAFwyFQ5AsbGx3qwDAADgkvHYQ9ALFy5UkyZNPLU7AAAAr3ErAL3zzju66667dM899+jrr7+WJK1cuVLNmjXTvffeqzZt2nilSAAAAE+qcAB65ZVXNGLECGVkZGjx4sXq1KmTJk+erP79+6tPnz7av3+/3n77bW/WCgAA4BEVfgZo5syZ+tvf/qYHHnhAKSkp6tSpk1auXKk9e/boyiuv9GKJ/o3336AyvIcJAOCqwleAfvzxR3Xp0kWS1KFDBwUGBurPf/4z4QcAAPidCgeg3377TVWrVnWuBwUFqU6dOl4pCgAAwJvc+iHEd999VzVr1pQknT59WrNnz1ZYWJjLmNGjR3uuOgAAAC+ocACqW7eu3nnnHed6ZGSk3n//fZcxNpuNAAQAACq9Cgegffv2ebEMAACAS4e3wQMAAMup8BWgOXPmVGjc4MGDL7gYAACAS6HCAWjo0KGqWbOmAgICXN4I/3s2m40ABAAAKr0KB6A//OEP+uWXX3TvvffqgQce4L1fAADAb1X4GaBt27Zp6dKlOnXqlNq1a6fExETNmDFDubm53qwPAADA49x6CLpVq1Z6++23dfDgQY0ePVofffSRoqKiNGjQIOXn53urRgAAAI+6oG+BVatWTYMHD9akSZPUsmVLzZs3TydPnvR0bQAAAF7hdgD66aefNHnyZMXHx2vgwIFq0aKFtm3bplq1anmjPgAAAI+r8EPQH330kWbNmqXVq1erW7dumjZtmm677TZVqVLFm/UBAAB4XIUD0MCBA1W3bl099thjioiI0L59+/Tmm2+WGMerMAAAQGXn1rvAbDabPvzwwzLH8C4wAADgD3gXGAAAsBzeBQYAACynwleATp06pS+//FI9e/aUJCUnJ7v89k+VKlX0/PPPq2rVqp6vEgAAwIPcehnqp59+6gxA06dP1/XXX69q1apJkr7//ntFR0frscce806lAAAAHlLhW2Bz587VAw884NL24YcfatWqVVq1apVefvllffTRRx4vsF69erLZbCWWESNGSDrzktZz+1q3bu3xOgAAwOWjwleAdu3apYYNGzrXq1atqiuu+G9+atmypTOUeFJaWpqKioqc61u3btWtt96qu+66y9nWvXt3zZo1y7keFBTk8ToAAMDlo8IBKCcnRwEB/x3+66+/uvQXFxd75X1gderUcVl/8cUXdc0116h9+/bONrvdrsjIyArvMz8/36VWXugKAIC1VPgW2NVXX62tW7eW2b9lyxZdffXVHimqLAUFBfrggw/0wAMPyGazOdtTUlIUHh6uhg0bavjw4crOzi53P1OmTJHD4XAuMTExXq0bAABULhUOQD169NAzzzyj3377rUTfqVOnNGnSJN12220eLe5cn3zyiY4dO6ahQ4c625KSkjR37lytXLlS06ZNU1pamjp16lTu1ajk5GTl5OQ4l8zMTK/WDQAAKpcK3wJ78skn9dFHH6lRo0YaOXKkGjZsKJvNpu+//17Tp0/X6dOn9eSTT3qzVs2cOVNJSUmKjo52tg0YMMD5d0JCghITExUbG6ulS5eqX79+pe7HbrfLbrd7tVYAAFB5VTgARUREKDU1VQ8//LAmTJggY4ykM6+/uPXWW/XWW28pIiLCa4X++OOPWrFihRYuXFjuuKioKMXGxmr37t1eqwUAAPi3CgcgSYqLi9OyZct05MgR7dmzR5LUoEEDhYaGeqW435s1a5bCw8PPe5vt8OHDyszMVFRUlNdrAgAA/smtAHRWaGioWrZs6elaylRcXKxZs2ZpyJAhLt9Ey8vL08SJE3XHHXcoKipK+/bt05NPPqmwsDD17dv3ktUHAAD8ywUFoEttxYoV2r9/f4kfYqxSpYq+++47zZkzR8eOHVNUVJQ6duyo+fPnKzg42EfVAgCAys4vAlDXrl2dzxz9XrVq1fT555/7oCIAAODPeBs8AACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwHAIQAACwnEodgCZOnCibzeayREZGOvuNMZo4caKio6NVrVo1dejQQdu2bfNhxQAAwB9U6gAkSddff70OHjzoXL777jtn39SpU/Xqq69q+vTpSktLU2RkpG699VYdP37chxUDAIDKrtIHoICAAEVGRjqXOnXqSDpz9ef111/XU089pX79+ikhIUHvvfeeTp48qQ8//NDHVQMAgMqs0geg3bt3Kzo6WnFxcRo4cKB++OEHSVJGRoaysrLUtWtX51i73a727dsrNTW13H3m5+crNzfXZQEAANZRqQNQq1atNGfOHH3++ed65513lJWVpZtuukmHDx9WVlaWJCkiIsJlm4iICGdfWaZMmSKHw+FcYmJivHYMAACg8qnUASgpKUl33HGHGjdurC5dumjp0qWSpPfee885xmazuWxjjCnRdq7k5GTl5OQ4l8zMTM8XDwAAKq1KHYDOVaNGDTVu3Fi7d+92fhvs3Ks92dnZJa4KnctutyskJMRlAQAA1uFXASg/P187duxQVFSU4uLiFBkZqeXLlzv7CwoKtHr1at10000+rBIAAFR2Ab4uoDzjxo1Tr169VLduXWVnZ+uFF15Qbm6uhgwZIpvNpjFjxmjy5MmKj49XfHy8Jk+erOrVq+uee+7xdekAAKASq9QB6MCBA7r77rt16NAh1alTR61bt9ZXX32l2NhYSdL48eN16tQpPfLIIzp69KhatWqlL774QsHBwT6uHAAAVGaVOgDNmzev3H6bzaaJEydq4sSJl6YgAABwWfCrZ4AAAAA8gQAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAshwAEAAAsp1IHoClTpqhFixYKDg5WeHi4+vTpo507d7qMGTp0qGw2m8vSunVrH1UMAAD8QaUOQKtXr9aIESP01Vdfafny5Tp9+rS6du2qEydOuIzr3r27Dh486Fw+++wzH1UMAAD8QYCvCyjPsmXLXNZnzZql8PBwbdy4Ue3atXO22+12RUZGVni/+fn5ys/Pd67n5uZefLEAAMBvVOorQOfKycmRJIWGhrq0p6SkKDw8XA0bNtTw4cOVnZ1d7n6mTJkih8PhXGJiYrxWMwAAqHz8JgAZYzR27FjdfPPNSkhIcLYnJSVp7ty5WrlypaZNm6a0tDR16tTJ5QrPuZKTk5WTk+NcMjMzL8UhAACASqJS3wL7vZEjR2rLli1at26dS/uAAQOcfyckJCgxMVGxsbFaunSp+vXrV+q+7Ha77Ha7V+sFAACVl18EoFGjRmnJkiVas2aNrr766nLHRkVFKTY2Vrt3775E1QEAAH9TqQOQMUajRo3SokWLlJKSori4uPNuc/jwYWVmZioqKuoSVAgAAPxRpX4GaMSIEfrggw/04YcfKjg4WFlZWcrKytKpU6ckSXl5eRo3bpzWr1+vffv2KSUlRb169VJYWJj69u3r4+oBAEBlVamvAM2YMUOS1KFDB5f2WbNmaejQoapSpYq+++47zZkzR8eOHVNUVJQ6duyo+fPnKzg42AcVAwAAf1CpA5Axptz+atWq6fPPP79E1QAAgMtFpb4FBgAA4A0EIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDkEIAAAYDmXTQB66623FBcXp6pVq6p58+Zau3atr0sCAACV1GURgObPn68xY8boqaee0qZNm3TLLbcoKSlJ+/fv93VpAACgErosAtCrr76qYcOG6cEHH9Qf/vAHvf7664qJidGMGTN8XRoAAKiEAnxdwMUqKCjQxo0bNWHCBJf2rl27KjU1tdRt8vPzlZ+f71zPycmRJOXm5nq8vuL8kx7fJ/yLN84rd3AOwtfnoMR5aHXeOgfP7tcY4/a2fh+ADh06pKKiIkVERLi0R0REKCsrq9RtpkyZokmTJpVoj4mJ8UqNsDbH676uAFbHOQhf8/Y5ePz4cTkcDre28fsAdJbNZnNZN8aUaDsrOTlZY8eOda4XFxfryJEjql27tss2ubm5iomJUWZmpkJCQrxT+GWOObw4zN/FYw4vDvN38ZjDi1Pe/BljdPz4cUVHR7u9X78PQGFhYapSpUqJqz3Z2dklrgqdZbfbZbfbXdquvPLKMj8jJCSEk/YiMYcXh/m7eMzhxWH+Lh5zeHHKmj93r/yc5fcPQQcFBal58+Zavny5S/vy5ct10003+agqAABQmfn9FSBJGjt2rO677z4lJiaqTZs2+vvf/679+/frj3/8o69LAwAAldBlEYAGDBigw4cP67nnntPBgweVkJCgzz77TLGxsRe1X7vdrmeffbbE7TJUHHN4cZi/i8ccXhzm7+IxhxfHW/NnMxfy3TEAAAA/5vfPAAEAALiLAAQAACyHAAQAACyHAAQAACyHAHSOo0eP6r777pPD4ZDD4dB9992nY8eOlbvN0KFDZbPZXJbWrVtfmoIrgbfeektxcXGqWrWqmjdvrrVr15Y7fvXq1WrevLmqVq2q+vXr629/+9slqrRycmf+UlJSSpxrNptN33///SWsuPJYs2aNevXqpejoaNlsNn3yySfn3Ybzz5W7c8g56GrKlClq0aKFgoODFR4erj59+mjnzp3n3Y7z8IwLmT9PnYMEoHPcc8892rx5s5YtW6Zly5Zp8+bNuu+++867Xffu3XXw4EHn8tlnn12Can1v/vz5GjNmjJ566ilt2rRJt9xyi5KSkrR///5Sx2dkZKhHjx665ZZbtGnTJj355JMaPXq0FixYcIkrrxzcnb+zdu7c6XK+xcfHX6KKK5cTJ06oadOmmj59eoXGc/6V5O4cnsU5eMbq1as1YsQIffXVV1q+fLlOnz6trl276sSJE2Vuw3n4Xxcyf2dd9Dlo4LR9+3YjyXz11VfOtvXr1xtJ5vvvvy9zuyFDhpjbb7/9ElRY+bRs2dL88Y9/dGm79tprzYQJE0odP378eHPttde6tD300EOmdevWXquxMnN3/latWmUkmaNHj16C6vyLJLNo0aJyx3D+la8ic8g5WL7s7GwjyaxevbrMMZyHZavI/HnqHOQK0O+sX79eDodDrVq1cra1bt1aDodDqamp5W6bkpKi8PBwNWzYUMOHD1d2dra3y/W5goICbdy4UV27dnVp79q1a5nztX79+hLju3Xrpg0bNqiwsNBrtVZGFzJ/ZzVr1kxRUVHq3LmzVq1a5c0yLyucf57DOVi6nJwcSVJoaGiZYzgPy1aR+TvrYs9BAtDvZGVlKTw8vER7eHh4iZet/l5SUpLmzp2rlStXatq0aUpLS1OnTp2Un5/vzXJ97tChQyoqKirx0tmIiIgy5ysrK6vU8adPn9ahQ4e8VmtldCHzFxUVpb///e9asGCBFi5cqEaNGqlz585as2bNpSjZ73H+XTzOwbIZYzR27FjdfPPNSkhIKHMc52HpKjp/njoHL4tXYZzPxIkTNWnSpHLHpKWlSZJsNluJPmNMqe1nDRgwwPl3QkKCEhMTFRsbq6VLl6pfv34XWLX/OHduzjdfpY0vrd0q3Jm/Ro0aqVGjRs71Nm3aKDMzU6+88oratWvn1TovF5x/F4dzsGwjR47Uli1btG7duvOO5TwsqaLz56lz0BIBaOTIkRo4cGC5Y+rVq6ctW7bol19+KdH366+/lkjr5YmKilJsbKx2797tdq3+JCwsTFWqVClxtSI7O7vM+YqMjCx1fEBAgGrXru21WiujC5m/0rRu3VoffPCBp8u7LHH+eQfnoDRq1CgtWbJEa9as0dVXX13uWM7DktyZv9JcyDloiQAUFhamsLCw845r06aNcnJy9M0336hly5aSpK+//lo5OTm66aabKvx5hw8fVmZmpqKioi64Zn8QFBSk5s2ba/ny5erbt6+zffny5br99ttL3aZNmzb617/+5dL2xRdfKDExUYGBgV6tt7K5kPkrzaZNmy77c81TOP+8w8rnoDFGo0aN0qJFi5SSkqK4uLjzbsN5+F8XMn+luaBz8KIeob4Mde/e3TRp0sSsX7/erF+/3jRu3Nj07NnTZUyjRo3MwoULjTHGHD9+3Dz++OMmNTXVZGRkmFWrVpk2bdqYq666yuTm5vriEC6pefPmmcDAQDNz5kyzfft2M2bMGFOjRg2zb98+Y4wxEyZMMPfdd59z/A8//GCqV69uHnvsMbN9+3Yzc+ZMExgYaP75z3/66hB8yt35e+2118yiRYvMrl27zNatW82ECROMJLNgwQJfHYJPHT9+3GzatMls2rTJSDKvvvqq2bRpk/nxxx+NMZx/FeHuHHIOunr44YeNw+EwKSkp5uDBg87l5MmTzjGch2W7kPnz1DlIADrH4cOHzaBBg0xwcLAJDg42gwYNKvFVO0lm1qxZxhhjTp48abp27Wrq1KljAgMDTd26dc2QIUPM/v37L33xPvLmm2+a2NhYExQUZG688UaXry8OGTLEtG/f3mV8SkqKadasmQkKCjL16tUzM2bMuMQVVy7uzN9LL71krrnmGlO1alVTq1Ytc/PNN5ulS5f6oOrK4ezXYc9dhgwZYozh/KsId+eQc9BVaXP3+/8jjOE8LM+FzJ+nzkHb/y8AAADAMvgaPAAAsBwCEAAAsBwCEAAAsBwCEAAAsBwCEAAAsBwCEAAAsBwCEAAAsBwCEAAAsBwCEGAxEydO1A033OBcHzp0qPr06eOzes46ty5/06FDB40ZM8bXZVwQf64duFAEIMCPZGVladSoUapfv77sdrtiYmLUq1cvffnll74uzS02m02ffPKJS9u4ceP87ji8JS4uTsuWLSu1r7S5A+A+S7wNHrgc7Nu3T23bttWVV16pqVOnqkmTJiosLNTnn3+uESNG6Pvvv/d1iRelZs2aqlmzpq/L8LktW7bo8OHD6tixo69LAS5rXAEC/MQjjzwim82mb775RnfeeacaNmyo66+/XmPHjtVXX33lHLd//37dfvvtqlmzpkJCQtS/f3/98ssvFf6c/Px8jR49WuHh4apatapuvvlmpaWluYzZtm2bbrvtNoWEhCg4OFi33HKL9u7dK0lKS0vTrbfeqrCwMDkcDrVv317p6enObevVqydJ6tu3r2w2m3P93FtgxcXFeu6553T11VfLbrfrhhtucLkqsm/fPtlsNi1cuFAdO3ZU9erV1bRpU61fv77c47PZbHr33XfVt29fVa9eXfHx8VqyZInLmNWrV6tly5ay2+2KiorShAkTdPr0aWf/iRMnNHjwYNWsWVNRUVGaNm1aic8pKCjQ+PHjddVVV6lGjRpq1aqVUlJSyq1NkhYvXqxu3brJbreX6Ctr7vbu3avbb79dERERqlmzplq0aKEVK1a4bPvWW28pPj5eVatWVUREhO68884ya1i2bJkcDofmzJlz3noBf0UAAvzAkSNHtGzZMo0YMUI1atQo0X/llVdKkowx6tOnj44cOaLVq1dr+fLl2rt3rwYMGFDhzxo/frwWLFig9957T+np6WrQoIG6deumI0eOSJJ++ukntWvXTlWrVtXKlSu1ceNGPfDAA86AcPz4cQ0ZMkRr167VV199pfj4ePXo0UPHjx+XJGeYmjVrlg4ePFgiXJ31l7/8RdOmTdMrr7yiLVu2qFu3burdu7d2797tMu6pp57SuHHjtHnzZjVs2FB33323S1gpzaRJk9S/f39t2bJFPXr00KBBg1yOr0ePHmrRooW+/fZbzZgxQzNnztQLL7zg3P6JJ57QqlWrtGjRIn3xxRdKSUnRxo0bXT7j/vvv13/+8x/NmzdPW7Zs0V133aXu3buXqP9cS5Ys0e23315qX1lzl5eXpx49emjFihXatGmTunXrpl69emn//v2SpA0bNmj06NF67rnntHPnTi1btkzt2rUr9TPmzZun/v37a86cORo8eHC5tQJ+7WJfZQ/A+77++msjySxcuLDccV988YWpUqWK2b9/v7Nt27ZtRpL55ptvjDHGPPvss6Zp06bO/iFDhpjbb7/dGGNMXl6eCQwMNHPnznX2FxQUmOjoaDN16lRjjDHJyckmLi7OFBQUVKj206dPm+DgYPOvf/3L2SbJLFq0yGXcuXVFR0ebP//5zy5jWrRoYR555BFjjDEZGRlGknn33XdLHOuOHTvKrEeSefrpp53reXl5xmazmX//+9/GGGOefPJJ06hRI1NcXOwc8+abb5qaNWuaoqIic/z4cRMUFGTmzZvn7D98+LCpVq2aefTRR40xxuzZs8fYbDbz008/uXx2586dTXJycpm1HThwwAQGBprDhw+XW/+5c1ea6667zrzxxhvGGGMWLFhgQkJCTG5ubqlj27dvbx599FHz5ptvGofDYVauXHne/QP+jmeAAD9gjJF05vZNeXbs2KGYmBjFxMQ426677jpdeeWV2rFjh1q0aFHu9nv37lVhYaHatm3rbAsMDFTLli21Y8cOSdLmzZt1yy23KDAwsNR9ZGdn65lnntHKlSv1yy+/qKioSCdPnnRejaiI3Nxc/fzzzy51SFLbtm317bffurQ1adLE+XdUVJSzhmuvvbbM/f9+mxo1aig4OFjZ2dmSzsxhmzZtXOa6bdu2ysvL04EDB3T06FEVFBSoTZs2zv7Q0FA1atTIuZ6eni5jjBo2bOjyufn5+apdu3aZdS1ZskRt27ZVaGhomWNKc+LECU2aNEmffvqpfv75Z50+fVqnTp1yzvmtt96q2NhY1a9fX927d1f37t2dtwDPWrBggX755RetW7dOLVu2dOvzAX9EAAL8QHx8vGw2m3bs2FHuV9aNMaWGpLLaSxsnlQxav9++WrVq5e5j6NCh+vXXX/X6668rNjZWdrtdbdq0UUFBwXk//1zl1XHW74PY2b7i4uJy93tueLPZbM5tSvuM38/L2b/LU1xcrCpVqmjjxo2qUqWKS195D3qXd/urPE888YQ+//xzvfLKK2rQoIGqVaumO++80znnwcHBSk9PV0pKir744gs988wzmjhxotLS0py3T2+44Qalp6dr1qxZatGiRYXOF8Cf8QwQ4AdCQ0PVrVs3vfnmmzpx4kSJ/mPHjkk6c7Vn//79yszMdPZt375dOTk5+sMf/nDez2nQoIGCgoK0bt06Z1thYaE2bNjg3L5JkyZau3atCgsLS93H2rVrNXr0aPXo0UPXX3+97Ha7Dh065DImMDBQRUVFZdYREhKi6OholzokKTU1tULHcTGuu+46paamugSd1NRUBQcH66qrrlKDBg0UGBjo8uD50aNHtWvXLud6s2bNVFRUpOzsbDVo0MBliYyMLPVz8/LytGrVKvXu3bvc+kqbu7Vr12ro0KHq27evGjdurMjISO3bt89lTEBAgLp06aKpU6dqy5Yt2rdvn1auXOnsv+aaa7Rq1SotXrxYo0aNOu88Af6OAAT4ibfeektFRUVq2bKlFixYoN27d2vHjh3661//6rwd06VLFzVp0kSDBg1Senq6vvnmGw0ePFjt27dXYmLieT+jRo0aevjhh/XEE09o2bJl2r59u4YPH66TJ09q2LBhkqSRI0cqNzdXAwcO1IYNG7R79269//772rlzp6QzIer999/Xjh079PXXX2vQoEElrhrVq1dPX375pbKysnT06NFSa3niiSf00ksvaf78+dq5c6cmTJigzZs369FHH72YaTyvRx55RJmZmRo1apS+//57LV68WM8++6zGjh2rK664QjVr1tSwYcP0xBNP6Msvv9TWrVs1dOhQXXHFf/85bdiwoQYNGqTBgwdr4cKFysjIUFpaml566SV99tlnpX7usmXLFB8fr/r165dbX2lz16BBAy1cuFCbN2/Wt99+q3vuucflKtinn36qv/71r9q8ebN+/PFHzZkzR8XFxS637c7WvWrVKi1YsIAfRsTlz0fPHgG4AD///LMZMWKEiY2NNUFBQeaqq64yvXv3NqtWrXKO+fHHH03v3r1NjRo1THBwsLnrrrtMVlaWs7+8h6CNMebUqVNm1KhRJiwszNjtdtO2bVvnA9Rnffvtt6Zr166mevXqJjg42Nxyyy1m7969xhhj0tPTTWJiorHb7SY+Pt58/PHHJjY21rz22mvO7ZcsWWIaNGhgAgICTGxsbKl1FRUVmUmTJpmrrrrKBAYGmqZNmzofVDbmvw9Bb9q0ydl29OhRI8llPs6lUh4idjgcZtasWc71lJQU06JFCxMUFGQiIyPNn/70J1NYWOjsP378uLn33ntN9erVTUREhJk6darzQeKzCgoKzDPPPGPq1atnAgMDTWRkpOnbt6/ZsmVLqXXde++95qmnniqz7rNKm7uMjAzTsWNHU61aNRMTE2OmT5/uUs/atWtN+/btTa1atUy1atVMkyZNzPz58537PLf27du3m/DwcDN27Njz1gP4K5sxFbihDQDwmqKiIoWHh+vf//43DyADlwi3wADAxw4fPqzHHnvsvN/SA+A5XAECAACWwxUgAABgOQQgAABgOQQgAABgOQQgAABgOQQgAABgOQQgAABgOQQgAABgOQQgAABgOQQgAABgOf8PqV2Q8zOgP80AAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# get values on the diagonal of the preconditioner\n", + "sweeper = controller.MS[0].levels[0].sweep\n", + "diag_vals = [sweeper.QI[i + 1, i + 1] for i in range(sweeper.coll.num_nodes)]\n", + "\n", + "# get the keys for gmres iterations at various step sizes from this\n", + "work_keys = [f'work_gmres-{sweeper.level.dt * me:.2e}' for me in diag_vals]\n", + "\n", + "# compute the total number of gmres iterations per task\n", + "gmres_iter_per_task = [sum(me[1] for me in get_sorted(stats, type=key)) for key in work_keys]\n", + "total_gmres_iterations = sum(gmres_iter_per_task)\n", + "\n", + "# compute speedup in implicit solves\n", + "max_rel_gmres_per_task = max(gmres_iter_per_task) / total_gmres_iterations\n", + "speedup = 1 / max_rel_gmres_per_task\n", + "parallel_efficiency = speedup / sweeper.coll.num_nodes\n", + "\n", + "# visualize and print the results\n", + "import matplotlib.pyplot as plt\n", + "\n", + "plt.bar(np.arange(sweeper.coll.num_nodes), gmres_iter_per_task)\n", + "plt.ylabel('GMRES iterations')\n", + "plt.xlabel('Collocation node / task')\n", + "\n", + "print(\n", + " f'Performed {max_rel_gmres_per_task*100:.0f}% of GMRES iterations on task {np.argmax(gmres_iter_per_task)}. This translates to speedup of {speedup:.2f} and {parallel_efficiency*100:.0f}% parallel efficiency in the implicit solves.'\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "9ad06cef-6c26-40ed-9676-3d3f6197efa3", + "metadata": {}, + "source": [ + "We see that we have some load balancing issues here that prevent perfect scaling even without considering communication.\n", + "The $f$ evaluations, on the other hand, cost the same regardless of the node and don't suffer from load balancing issues.\n", + "What is then the expected speedup?\n", + "At this point, we should switch to the MPI sweeper and measure run time, which is a much more solid measure of speedup, but we are not going to do this in this notebook.\n", + "However, counting things is still crucial to understanding what's going on and to developments.\n", + "For instance, we could now add some inexactness and restrict the number of GMRES iterations per SDC iteration to some low value.\n", + "Then maybe we need more SDC iterations, but don't have load balancing issues.\n", + "Is it a good idea for this problem?\n", + "Who knows.\n", + "Finding out such things is precisely what pySDC is made for.\n", + "You can run it in a notebook and on your laptop, but you can also run it on HPC machines.\n", + "\n", + "Hopefully, at this point, you are convinced that it is somewhat easy to implement problems into pySDC and to do some analysis on the data.\n", + "Of course, this problem was easy to implement because of its simple nature and other problems can get arbitrarily difficult to implement.\n", + "But, if you have the problem implemented somewhere, it should be reasonable to transfer.\n", + "In particular, libraries like [Firedrake](https://github.com/firedrakeproject/firedrake), [Fenics](https://fenicsproject.org), and [Dedalus](https://dedalus-project.org) make it easy to set up complex problems with finite element or spectral discretizations.\n", + "Next we will look at how to use pySDC together with Firedrake and even how to use pySDC as a time stepper within the geophysical fluid dynamics code [Gusto](https://www.firedrakeproject.org/gusto/)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "pySDC_tutorial", + "language": "python", + "name": "pysdc_tutorial" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/pySDC/playgrounds/PinT_Workshop_2025/2_Coupling_pySDC_to_libraries.ipynb b/pySDC/playgrounds/PinT_Workshop_2025/2_Coupling_pySDC_to_libraries.ipynb new file mode 100644 index 0000000000..2f36dab66b --- /dev/null +++ b/pySDC/playgrounds/PinT_Workshop_2025/2_Coupling_pySDC_to_libraries.ipynb @@ -0,0 +1,824 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f4eecf63-f897-4d06-9f18-0105c0103052", + "metadata": {}, + "source": [ + "# Coupling pySDC to libraries for spatial discretization\n", + "pySDC is a library for time integration.\n", + "But we need something to integrate in the first place...\n", + "We can make life hard and code up some problem discretization from scratch yet again, or we use our time more efficiently by building on other peoples work.\n", + "This tutorial will show how to do the latter by coupling pySDC to Firedrake.\n", + "Coupling other libraries requires the same steps taken here.\n", + "If you are working with compiled code, keep in mind that Python is designed to work with compiled code and you can interface via well established frameworks such as [pybind11](https://github.com/pybind/pybind11) or [F2PY](https://numpy.org/doc/stable/f2py/index.html).\n", + "\n", + "## What is Firedrake?\n", + "[Firedrake](https://www.firedrakeproject.org) is an elaborate framework for discretizing PDEs using the finite element method (FEM), sharing many characteristics with FEniCS.\n", + "Firedrake code looks a lot like mathematical equations, so once you have a PDE in weak form, it is easy to implement with Firedrake.\n", + "This is an immensely powerful tool for setting up complicated problems on complicated meshes while relying on a lot of existing automation rather than coding everything yourself.\n", + "For a few basic examples and explanations see the [Firedrake documentation](https://www.firedrakeproject.org/documentation.html).\n", + "\n", + "\n", + "## Installing Firedrake\n", + "Installing Firedrake can be non-trivial, but it keeps improving.\n", + "Please consult the respective [documentation](https://www.firedrakeproject.org/install.html#id9).\n", + "If you get stuck, open a discussion on the [GitHub page](https://github.com/firedrakeproject/firedrake/discussions), where the Firedrake developers are responsive and helpful.\n", + "\n", + "To run this notebook, you need make a kernel with your Firedrake installation.\n", + "Activate a virtual environment with Firedrake installed, then install pySDC and jupyter stuff with\n", + "```\n", + "pip install -e \n", + "pip install ipykernel\n", + "pip install jupyter\n", + "pip install jdc\n", + "```\n", + "\n", + "Afterwards, generate the kernel and start the notebook:\n", + "```\n", + "python -m ipykernel install --user --name=pySDC_Firedrake\n", + "python -m jupyter lab\n", + "```\n", + "\n", + "## Coupling pySDC to Firedrake\n", + "pySDC is already coupled to Firedrake, so don't worry.\n", + "However, we will briefly go through the existing coupling here to illustrate how to couple pySDC to any library.\n", + "pySDC abstracts away specifics of the type of data is working with via a datatype interface and coupling to another library essentially means writing a datatype that connects pySDC and the library.\n", + "This datatype needs to respect only a few properties, have a rule for generating empty data containers, communication is also handled there.\n", + "\n", + "With `u`, `v` data as used in the library you want to couple to and `a` a float, the following must hold:\n", + " - `abs(u)` must return a float with the norm across the entire spatial domain\n", + " - `a*u + v - v*u` must be implemented and the result must be of the same type as `u`.\n", + " \n", + "For communication, the following functions familiar from MPI need implementing in the datatypes:\n", + " - `bcast`: Broadcast data\n", + " - `Isend`: Non-blocking send of the data\n", + " - `Irecv`: Non-blocking receive of the data\n", + "\n", + "Communication is only needed if you intend to actually run pySDC in parallel with a diagonal sweeper or PFASST.\n", + "You can always not implement any communication and run the parallel algorithms in serial.\n", + "\n", + "So, coupling pySDC to a library typically means writing a wrapper for the data or subclassing the datatype and adding only the above mentioned functionality.\n", + "\n", + "In the case of Firedrake, we write a wrapper for `Firedrake.Function` which we call `firedrake_mesh` here.\n", + "We start with the `__init__` function, which is called during object instantiation.\n", + "Here, we will make use of `__mro__`, which returns the [method resolution order](https://docs.python.org/3/howto/mro.html) of the object we call this on, which tells us the tree of inheritance relationships of the objects class.\n", + "The inheritance relationship we are looking for here depends on some intricacies of Firedrake that you needn't worry about now." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "14891203-77f9-4696-8247-6a03646a7e79", + "metadata": {}, + "outputs": [], + "source": [ + "import jdc # required to split the class definition into multiple cells...\n", + "\n", + "import firedrake as fd\n", + "\n", + "from pySDC.core.errors import DataError\n", + "from pySDC.helpers.firedrake_ensemble_communicator import FiredrakeEnsembleCommunicator\n", + "\n", + "\n", + "class firedrake_mesh(object):\n", + " \"\"\"\n", + " Wrapper for firedrake function data.\n", + "\n", + " Attributes:\n", + " functionspace (firedrake.Function): firedrake data\n", + " \"\"\"\n", + "\n", + " def __init__(self, init, val=0.0):\n", + " # init tells us how to instantiate a new mesh\n", + " if fd.functionspaceimpl.WithGeometry in type(init).__mro__:\n", + " self.functionspace = fd.Function(init)\n", + " self.functionspace.assign(val)\n", + " elif fd.Function in type(init).__mro__:\n", + " self.functionspace = fd.Function(init)\n", + " elif type(init) == firedrake_mesh:\n", + " self.functionspace = init.functionspace.copy(deepcopy=True)\n", + " else:\n", + " raise DataError('something went wrong during %s initialization' % type(init))" + ] + }, + { + "cell_type": "markdown", + "id": "ca9e4115-b5b1-45fd-9e15-2a39847ad5e9", + "metadata": {}, + "source": [ + "Next, we make this a wrapper for `firedrake.Function` via the `__getattr__` method.\n", + "If you call `a.key` on an object `a` and `a` does not have an attribute or function `key`, `a.__getattr__(key)` will be called, which makes it easy to pass on requests." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "6f80840c-deb0-4697-a88e-892d64703b77", + "metadata": {}, + "outputs": [], + "source": [ + "%%add_to firedrake_mesh\n", + "\n", + "def __getattr__(self, key):\n", + " return getattr(self.functionspace, key)" + ] + }, + { + "cell_type": "markdown", + "id": "8b953a63-a1ec-4f9a-95c4-1a780bb10d0e", + "metadata": {}, + "source": [ + "Next up, we define addition, subtraction and right mupltiplication for the new datatype:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "34504b69-f62a-4a2f-8503-360a1b43e28f", + "metadata": {}, + "outputs": [], + "source": [ + "%%add_to firedrake_mesh\n", + "\n", + "def __add__(self, other):\n", + " if isinstance(other, type(self)):\n", + " me = firedrake_mesh(other)\n", + " me.functionspace.assign(self.functionspace + other.functionspace)\n", + " return me\n", + " else:\n", + " raise DataError(\"Type error: cannot add %s to %s\" % (type(other), type(self)))\n", + "\n", + "def __sub__(self, other):\n", + " if isinstance(other, type(self)):\n", + " me = firedrake_mesh(other)\n", + " me.functionspace.assign(self.functionspace - other.functionspace)\n", + " return me\n", + " else:\n", + " raise DataError(\"Type error: cannot add %s to %s\" % (type(other), type(self)))\n", + "\n", + "def __rmul__(self, other):\n", + " \"\"\"\n", + " Overloading the right multiply by scalar factor\n", + "\n", + " Args:\n", + " other (float): factor\n", + " Raises:\n", + " DataError: if other is not a float\n", + " Returns:\n", + " fenics_mesh: copy of original values scaled by factor\n", + " \"\"\"\n", + "\n", + " try:\n", + " me = firedrake_mesh(self)\n", + " me.functionspace.assign(other * self.functionspace)\n", + " return me\n", + " except TypeError as e:\n", + " raise DataError(\"Type error: cannot multiply %s to %s\" % (type(other), type(self))) from e" + ] + }, + { + "cell_type": "markdown", + "id": "11b7fe3d-c75a-40bd-8cac-f205214f4f06", + "metadata": {}, + "source": [ + "Now, we take care of the norm, which is as simple as calling a Firedrake function:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "5300c596-e468-4261-b2ae-59170d88d2b5", + "metadata": {}, + "outputs": [], + "source": [ + "%%add_to firedrake_mesh\n", + "\n", + "def __abs__(self):\n", + " \"\"\"\n", + " Overloading the abs operator for mesh types\n", + "\n", + " Returns:\n", + " float: L2 norm\n", + " \"\"\"\n", + "\n", + " return fd.norm(self.functionspace, 'L2')" + ] + }, + { + "cell_type": "markdown", + "id": "0b721cbc-5fa0-4ff8-abe1-ed69dc126956", + "metadata": {}, + "source": [ + "Finally: Communication.\n", + "Firedrake has \"ensemble communicators\", which were built by Josh for space-time parallelism.\n", + "In pySDC, we have written a wrapper for them that makes a few things easier, but, essentially, communication is again handled using Firedrake functions." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "83e92525-5443-4eda-844e-b350f5d7b3b6", + "metadata": {}, + "outputs": [], + "source": [ + "%%add_to firedrake_mesh\n", + "\n", + "def isend(self, dest=None, tag=None, comm=None):\n", + " \"\"\"\n", + " Routine for sending data forward in time (non-blocking)\n", + "\n", + " Args:\n", + " dest (int): target rank\n", + " tag (int): communication tag\n", + " comm: communicator\n", + "\n", + " Returns:\n", + " request handle\n", + " \"\"\"\n", + " assert (\n", + " type(comm) == FiredrakeEnsembleCommunicator\n", + " ), f'Need to give a FiredrakeEnsembleCommunicator here, not {type(comm)}'\n", + " return comm.Isend(self.functionspace, dest=dest, tag=tag)\n", + "\n", + "def irecv(self, source=None, tag=None, comm=None):\n", + " \"\"\"\n", + " Routine for receiving in time\n", + "\n", + " Args:\n", + " source (int): source rank\n", + " tag (int): communication tag\n", + " comm: communicator\n", + "\n", + " Returns:\n", + " None\n", + " \"\"\"\n", + " assert (\n", + " type(comm) == FiredrakeEnsembleCommunicator\n", + " ), f'Need to give a FiredrakeEnsembleCommunicator here, not {type(comm)}'\n", + " return comm.Irecv(self.functionspace, source=source, tag=tag)\n", + "\n", + "def bcast(self, root=None, comm=None):\n", + " \"\"\"\n", + " Routine for broadcasting values\n", + "\n", + " Args:\n", + " root (int): process with value to broadcast\n", + " comm: communicator\n", + "\n", + " Returns:\n", + " broadcasted values\n", + " \"\"\"\n", + " assert (\n", + " type(comm) == FiredrakeEnsembleCommunicator\n", + " ), f'Need to give a FiredrakeEnsembleCommunicator here, not {type(comm)}'\n", + " comm.Bcast(self.functionspace, root=root)\n", + " return self" + ] + }, + { + "cell_type": "markdown", + "id": "6b1c45ac-11f6-4743-a95d-c491351f70a4", + "metadata": {}, + "source": [ + "As you can see, we didn't do a whole lot.\n", + "Adding a coupling is just about bridging the gap between the pySDC interface and the library interface, not about implementing any new functionality.\n", + "You will need intimate knowledge of the library you are coupling to, you may need to do some workarounds that are not optimal performance-wise, but it is no Hexenwerk (rocket science in German).\n", + "You definitely needn't know a lot about pySDC." + ] + }, + { + "cell_type": "markdown", + "id": "a12cdbb5-e5c7-4584-8b57-2320d56845c6", + "metadata": {}, + "source": [ + "## Using Firedrake to discretize the heat equation in pySDC\n", + "\n", + "We will now implement a simple 1d heat equation $u_t=\\nu\\Delta u$.\n", + "First, we start writing a class and set the new Firedrake datatype via the class attributes `dtype_u` and `dtype_f`.\n", + "We set up some infrastructure such as a dictionary for storing Firedrake solvers, which are expensive to assemble and cheaper to reuse, input and output buffers for the solvers, and the mesh and function space." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "f1c869f1-a95e-4134-9afa-6e5954f57675", + "metadata": {}, + "outputs": [], + "source": [ + "from pySDC.core.problem import Problem, WorkCounter\n", + "import numpy as np\n", + "\n", + "\n", + "class HeatEquation(Problem):\n", + " dtype_u = firedrake_mesh\n", + " dtype_f = firedrake_mesh\n", + "\n", + " def __init__(self, n, nu, order, comm):\n", + " # prepare Firedrake mesh and function space\n", + " self.mesh = fd.UnitIntervalMesh(n, comm=comm)\n", + " self.V = fd.FunctionSpace(self.mesh, \"CG\", order) # we can use this to instantiate new `firedrake_mesh`\n", + "\n", + " # prepare pySDC problem class infrastructure by passing the function space to super init\n", + " super().__init__(self.V)\n", + " self._makeAttributeAndRegister('n', 'nu', 'order', 'comm', localVars=locals(), readOnly=True)\n", + "\n", + " # prepare caches and IO variables for solvers\n", + " self.solvers = {}\n", + " self.tmp_in = fd.Function(self.V)\n", + " self.tmp_out = fd.Function(self.V)\n", + "\n", + " self.work_counters['solver_setup'] = WorkCounter()\n", + " self.work_counters['solves'] = WorkCounter()\n", + " self.work_counters['rhs'] = WorkCounter()" + ] + }, + { + "cell_type": "markdown", + "id": "e28a159c-68e3-4409-a205-7c7a1b0de2a7", + "metadata": { + "jp-MarkdownHeadingCollapsed": true + }, + "source": [ + "Next up, we define a function for evaluating the right hand side.\n", + "In FEM, the PDE is translated into a variational problem via multiplication by test functions.\n", + "We start with $f(y) = \\nu\\Delta y$ and multiply by the test functions $v$ to obtain $f(y)v = \\nu(\\Delta y) v$.\n", + "Then we integrate by parts and obtain $\\int_\\Omega f(y)v dx = \\int_\\Omega \\nu (\\Delta y) v dx=-\\int_\\Omega \\nabla y \\nabla v dx$, due to $v$ vanishing on the boundary.\n", + "In order to evaluate $f(y)$, we set up a linear variational problem $\\int_\\Omega uv dx = -\\int_\\Omega \\nabla y \\nabla v dx$ and solve for $u$.\n", + "Remember that $y$ is the known solution at which we evaluate $f(y)$ and $v$ are the test functions from some function space.\n", + "\n", + "In the following implementation, we assemble a Firedrake solver once and then just reuse it in future calls to `eval_f`.\n", + "Note that the implementation is very similar to the equations.\n", + "For instance, $\\nabla$ becomes `firedrake.nabla_grad` and $\\int_\\Omega [\\dots]dx$ becomes `firedrake.dx`." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "3b482054-c7bc-4e64-853c-f9ae8a2d6225", + "metadata": {}, + "outputs": [], + "source": [ + "%%add_to HeatEquation\n", + "\n", + "def eval_f(self, u, t):\n", + " # construct and cache a solver for evaluating the Laplacian\n", + " if not hasattr(self, '__solv_eval_f'):\n", + " v = fd.TestFunction(self.V)\n", + " u_trial = fd.TrialFunction(self.V)\n", + "\n", + " a = u_trial * v * fd.dx\n", + " L = -fd.inner(self.nu * fd.nabla_grad(self.tmp_in), fd.nabla_grad(v)) * fd.dx\n", + "\n", + " bcs = [fd.bcs.DirichletBC(self.V, fd.Constant(0), area) for area in [1, 2]]\n", + "\n", + " prob = fd.LinearVariationalProblem(a, L, self.tmp_out, bcs=bcs)\n", + " self.__solv_eval_f = fd.LinearVariationalSolver(prob)\n", + "\n", + " # copy the solution we want to evaluate at into the input buffer\n", + " self.tmp_in.assign(u.functionspace)\n", + "\n", + " # perform the solve using the cached solver\n", + " self.__solv_eval_f.solve()\n", + "\n", + " # instantiate an empty data container\n", + " me = self.dtype_f(self.init)\n", + "\n", + " # copy the result of the solver from the output buffer to the variable this function returns\n", + " me.assign(self.tmp_out)\n", + "\n", + " self.work_counters['rhs']()\n", + "\n", + " return me" + ] + }, + { + "cell_type": "markdown", + "id": "e3ceb832-c703-4cbd-b330-277bf4288d16", + "metadata": {}, + "source": [ + "Next, we implement the implicit Euler step, which proceeds very similar to evaluating the right hand side, but with a different variational problem.\n", + "Now, multiplying by the test functions $v$ and integrating by parts gives the linear variational problem $\\int_\\Omega u v + \\Delta t \\nu \\nabla u \\nabla v dx = \\int_\\Omega y v dx$.\n", + "Again, we cache the solvers.\n", + "This time, we store a new one for each `dt`." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "c1260aae-1972-4ffc-be30-c4079ec507f2", + "metadata": {}, + "outputs": [], + "source": [ + "%%add_to HeatEquation\n", + "\n", + "def solve_system(self, rhs, dt, u0, t):\n", + " r\"\"\"\n", + " Solver for :math:`(M - dt * nu * Lap) u = rhs`.\n", + " \"\"\"\n", + "\n", + " # construct and cache a solver for the current dt (preconditioner entry times step size)\n", + " if dt not in self.solvers.keys():\n", + "\n", + " u = fd.TrialFunction(self.V)\n", + " v = fd.TestFunction(self.V)\n", + "\n", + " a = u * v * fd.dx + fd.Constant(dt) * fd.inner(self.nu * fd.nabla_grad(u), fd.nabla_grad(v)) * fd.dx\n", + " L = fd.inner(self.tmp_in, v) * fd.dx\n", + "\n", + " bcs = [fd.bcs.DirichletBC(self.V, fd.Constant(0), area) for area in [1, 2]]\n", + "\n", + " prob = fd.LinearVariationalProblem(a, L, self.tmp_out, bcs=bcs)\n", + " self.solvers[dt] = fd.LinearVariationalSolver(prob)\n", + "\n", + " self.work_counters['solver_setup']()\n", + "\n", + " # copy solver rhs to the input buffer. Copying also to the output buffer uses it as initial guess\n", + " self.tmp_in.assign(rhs.functionspace)\n", + " self.tmp_out.assign(u0.functionspace)\n", + "\n", + " # call the cached solver\n", + " self.solvers[dt].solve()\n", + "\n", + " # copy from output buffer to return variable\n", + " me = self.dtype_u(self.init)\n", + " me.assign(self.tmp_out)\n", + "\n", + " self.work_counters['solves']()\n", + " return me" + ] + }, + { + "cell_type": "markdown", + "id": "1ab69bfd-f462-4fc1-8be5-9d096ce56866", + "metadata": {}, + "source": [ + "Finally, we add functions for obtaining the grid and exact solution" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "2848d099-8805-4211-a9ba-ee02a61c3243", + "metadata": {}, + "outputs": [], + "source": [ + "%%add_to HeatEquation\n", + "\n", + "def x(self):\n", + " return fd.SpatialCoordinate(self.mesh)\n", + "\n", + "def u_exact(self, t):\n", + " me = self.u_init\n", + " me.interpolate(np.exp(-self.nu* np.pi**2*t) * fd.sin(np.pi * self.x()[0]))\n", + " return me" + ] + }, + { + "cell_type": "markdown", + "id": "444274ef-2c2e-4b08-8ec8-c5d662fe2c9c", + "metadata": {}, + "source": [ + "Time to test the implementation!\n", + "First, we compare the right hand side evaluation and then the implicit Euler step with the respective exact solution." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "192b20b5-eaa7-43de-b376-1d87a3755b1c", + "metadata": {}, + "outputs": [], + "source": [ + "from mpi4py import MPI\n", + "\n", + "prob = HeatEquation(n=128, nu=1e-2, order=4, comm=MPI.COMM_WORLD)\n", + "\n", + "u0 = prob.u_exact(0)\n", + "\n", + "f_expect = -prob.nu * np.pi**2 * u0\n", + "f = prob.eval_f(u0, 0)\n", + "assert abs(f - f_expect) < 1e-8\n", + "\n", + "dt = 1e-2\n", + "assert abs(prob.solve_system(u0, dt, u0, 0) - prob.u_exact(dt)) < 1e-6" + ] + }, + { + "cell_type": "markdown", + "id": "ce613a2e-5206-4f81-9b05-b0eef3f32ba5", + "metadata": {}, + "source": [ + "## Running pySDC with Firedrake problem classes\n", + "\n", + "Once we set up the problem class, using it in pySDC is very easy.\n", + "We now set up a bunch of parameters and run a short simulation:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "90ed2503-1ae7-49cc-8284-367303befc69", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "controller - INFO: Welcome to the one and only, really very astonishing and 87.3% bug free\n", + " _____ _____ _____ \n", + " / ____| __ \\ / ____|\n", + " _ __ _ _| (___ | | | | | \n", + " | '_ \\| | | |\\___ \\| | | | | \n", + " | |_) | |_| |____) | |__| | |____ \n", + " | .__/ \\__, |_____/|_____/ \\_____|\n", + " | | __/ | \n", + " |_| |___/ \n", + " \n", + "controller - INFO: Setup overview (--> user-defined, -> dependency) -- BEGIN\n", + "controller - INFO: ----------------------------------------------------------------------------------------------------\n", + "\n", + "Controller: \n", + " all_to_done = False\n", + " dump_setup = True\n", + " fname = run_pid94057.log\n", + "--> hook_class = [, , ]\n", + " log_to_file = False\n", + "--> logger_level = 15\n", + " mssdc_jac = True\n", + " predict_type = None\n", + " use_iteration_estimator = False\n", + "\n", + "Step: \n", + "--> maxiter = 9\n", + " Number of steps: None\n", + " Level: \n", + " Level 0\n", + "--> dt = 1\n", + " dt_initial = 1.0\n", + " nsweeps = 1\n", + " residual_type = full_abs\n", + "--> restol = 1e-07\n", + "--> Problem: \n", + "--> comm = \n", + "--> n = 128\n", + "--> nu = 0.1\n", + "--> order = 4\n", + "--> Data type u: \n", + "--> Data type f: \n", + "--> Sweeper: \n", + "--> QI = MIN-SR-S\n", + " do_coll_update = False\n", + " initial_guess = spread\n", + "--> num_nodes = 3\n", + "--> quad_type = RADAU-RIGHT\n", + " skip_residual_computation = ()\n", + "--> Collocation: \n", + "\n", + "Active convergence controllers:\n", + " | # | order | convergence controller\n", + "----+----+-------+---------------------------------------------------------------------------------------\n", + " | 0 | 95 | BasicRestartingNonMPI\n", + " -> | 1 | 100 | SpreadStepSizesBlockwiseNonMPI\n", + " | 2 | 200 | CheckConvergence\n", + "\n", + "controller - INFO: ----------------------------------------------------------------------------------------------------\n", + "controller - INFO: Setup overview (--> user-defined, -> dependency) -- END\n", + "\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 3.49644770e-02\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 7.30643743e-03\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 1.18535196e-03\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 1.81139704e-04\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 5 -- Sweep: 1 -- residual: 2.72106803e-05\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 6 -- Sweep: 1 -- residual: 4.06654580e-06\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 7 -- Sweep: 1 -- residual: 6.06775516e-07\n", + "hooks - INFO: Process 0 on time 0.000000 at stage IT_FINE: Level: 0 -- Iteration: 8 -- Sweep: 1 -- residual: 9.04941838e-08\n", + "hooks - INFO: Process 0 on time 1.000000 at stage IT_FINE: Level: 0 -- Iteration: 1 -- Sweep: 1 -- residual: 1.30330088e-02\n", + "hooks - INFO: Process 0 on time 1.000000 at stage IT_FINE: Level: 0 -- Iteration: 2 -- Sweep: 1 -- residual: 2.72347455e-03\n", + "hooks - INFO: Process 0 on time 1.000000 at stage IT_FINE: Level: 0 -- Iteration: 3 -- Sweep: 1 -- residual: 4.41839941e-04\n", + "hooks - INFO: Process 0 on time 1.000000 at stage IT_FINE: Level: 0 -- Iteration: 4 -- Sweep: 1 -- residual: 6.75198247e-05\n", + "hooks - INFO: Process 0 on time 1.000000 at stage IT_FINE: Level: 0 -- Iteration: 5 -- Sweep: 1 -- residual: 1.01427811e-05\n", + "hooks - INFO: Process 0 on time 1.000000 at stage IT_FINE: Level: 0 -- Iteration: 6 -- Sweep: 1 -- residual: 1.51580495e-06\n", + "hooks - INFO: Process 0 on time 1.000000 at stage IT_FINE: Level: 0 -- Iteration: 7 -- Sweep: 1 -- residual: 2.26175573e-07\n", + "hooks - INFO: Process 0 on time 1.000000 at stage IT_FINE: Level: 0 -- Iteration: 8 -- Sweep: 1 -- residual: 3.37317070e-08\n", + "hooks - INFO: Finished run after 1.02e+00s\n" + ] + } + ], + "source": [ + "from pySDC.implementations.sweeper_classes.generic_implicit import generic_implicit\n", + "from pySDC.implementations.controller_classes.controller_nonMPI import controller_nonMPI\n", + "from pySDC.implementations.hooks.log_errors import LogGlobalErrorPostIter\n", + "\n", + "# level and step parameters contain general parameters such as how many SDC iterations will be done\n", + "level_params = {}\n", + "level_params['dt'] = 1\n", + "level_params['restol'] = 1e-7\n", + "\n", + "step_params = {}\n", + "step_params['maxiter'] = 9\n", + "\n", + "# the sweeper parameters describe the collocation problem and the preconditioner\n", + "sweeper_params = {}\n", + "sweeper_params['quad_type'] = 'RADAU-RIGHT'\n", + "sweeper_params['num_nodes'] = 3\n", + "sweeper_params['QI'] = 'MIN-SR-S'\n", + "\n", + "problem_params = {'n': 128, 'nu': 1e-1, 'order': 4, 'comm': MPI.COMM_WORLD}\n", + "\n", + "# gather all parameters in one dictionary and add problem and sweeper classes\n", + "description = {}\n", + "description['problem_class'] = HeatEquation\n", + "description['problem_params'] = problem_params\n", + "description['sweeper_class'] = generic_implicit\n", + "description['sweeper_params'] = sweeper_params\n", + "description['level_params'] = level_params\n", + "description['step_params'] = step_params\n", + "\n", + "# parameters for the controller\n", + "controller_params = {}\n", + "controller_params['logger_level'] = 15\n", + "controller_params['hook_class'] = LogGlobalErrorPostIter\n", + "\n", + "# setup controller\n", + "controller = controller_nonMPI(controller_params=controller_params, description=description, num_procs=1)\n", + "\n", + "# get initial coniditions\n", + "P = controller.MS[0].levels[0].prob\n", + "uinit = P.u_exact(0)\n", + "\n", + "uend, stats = controller.run(u0=uinit, t0=0, Tend=2)" + ] + }, + { + "cell_type": "markdown", + "id": "d770ecc8-fae3-40c2-bf41-49b5cc99998e", + "metadata": {}, + "source": [ + "Ok. Great.\n", + "Let's investigate further how error and residual behave between iterations.\n", + "First, let's see what kind of statistics we recorded.\n", + "Note that we used the `LogGlobalErrorPostIter` hook, which uses the `u_exact` function to compute the error." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "84a4887d-e697-4d1e-9495-f201fda8c5c1", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['residual_post_sweep', 'residual_post_iteration', 'niter', 'residual_post_step', '_recomputed', 'timing_setup', 'timing_comm', 'timing_sweep', 'timing_iteration', 'timing_step', 'timing_run', 'e_global_post_iteration', 'e_global_rel_post_iteration', 'restart']\n" + ] + } + ], + "source": [ + "from pySDC.helpers.stats_helper import get_list_of_types\n", + "\n", + "print(get_list_of_types(stats))" + ] + }, + { + "cell_type": "markdown", + "id": "de02dd33-2f36-49e7-a1d6-a86977529a3e", + "metadata": {}, + "source": [ + "Alright. Now we use the function `get_sorted` to extract only the values from the first time step and sort them by iteration.\n", + "Note that the residual is logged at the beginning of the step and the error at the end.\n", + "So to get the values from the same step, we filter for `t=0` for residual and `t=1` for error." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "63d7750f-2403-4a51-9129-714dcd8752ec", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGwCAYAAABhDIVPAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAXlRJREFUeJzt3Qd8VMXexvFfeiB0AoTQe5Heq4CAIAgiVUW6NBEL1wJ6FdCrWF8VAenSLEgXUZAWeu+9S+8t1PT3M+cQiqASSLLt+d7PXvZszu6ZjSlPZv4z4xUXFxeHiIiIiIvwdnQDRERERBJC4UVERERcisKLiIiIuBSFFxEREXEpCi8iIiLiUhReRERExKUovIiIiIhL8cXNxMbGcuzYMVKnTo2Xl5ejmyMiIiL3wSw7d+nSJUJDQ/H29vas8GKCS44cORzdDBEREXkAhw8fJnv27J4VXkyPS/ybT5MmjaObIyIiIvchPDzc6nyI/z3uUeElfqjIBBeFFxEREddyPyUfblOwO3jwYIoWLUr58uUd3RQRERFJQl7utjGj6XZKmzYtFy9eVM+LiIiIG/7+dpueFxEREfEMCi8iIiLiUhReRERExKW4TXhRwa6IiIhnUMGuiIiIOJwKdkVERMRtKbyIiIiIS1F4EREREZfiNuFFBbsiIiKeQQW7IiIi4nAq2E0qe+dDdKSjWyEiIuLRFF7u1+E1MKEZDK9h3xcREXGAuLg4oqOj73o8MvLB/rh+0Oc5ksLL/bp2DlJmgFPbYVRdmPU6XA93dKtERORfftFfjYxO9ltCKzJiY2MZMGAAefLkIUWKFJQsWZLJkydbHwsLC8PLy4vff/+dsmXLEhAQwNKlS6lZsyYvvfQSr776KsHBwdSrV886f9GiRVSoUME6L2vWrPTu3fuOsPN3z3Mlvo5ugMsoWA9eWgtz3oFNP8CaEbBzFjT8Ago3cHTrRETkHq5FxVD0vTnJft3t79cjpf/9/4o1wWXChAkMHTqUAgUKsHjxYp5//nkyZcp08xwTQj7//HPy5s1L+vTprcfGjh1L9+7dWbZsmXV89OhRGjRoQPv27Rk3bhw7d+6kc+fOBAYG0q9fv5uv9dfnuRoV7CZARHQMAb4+sD8MZr4K5w/YHyjSGBp8BqlDEvV6IiLycEwviLOHl4iICDJkyMC8efOoXLnyzcdfeOEFrl69SpcuXahVqxbTp0/nqaeeuqMHxfzOW79+/c3H3nnnHaZMmcKOHTus3hpjyJAhvPXWW9bvRW9v73s+z9V+f6vn5T5dvBrFk4OW8Ez5nHR99FF8X1wBiz6BZQNhxy+wfxHU7Qdl2oO3RuNERJxBCj8fK0g44rr3a+/evVZIqVu37l21KKVLl755XK5cubueW7Zs2TuOTWgxASg+uBhVq1bl8uXLHDlyhJw5c97zea7G153WeTG3mJiYJHn9SesOc/jcNT6bs4vftx7n02YlKVqnHxRrBr+8DMfWw6+vweafodHXkKlQkrRDRETun/klnpDhG0cwwcKYNWsW2bJlu+Njpm5l37591v2goKC7nht0j8fux4M+z1m4TRdBjx492L59O2vWJM1MoE7V8vBFi5KkTeHH1qPhNB60lP+bu5vI4EfghXlQ/2PwC4JDK2BoNQj7GKIjkqQtIiLiPswCqyakHDp0iPz5899xy5EjR4Jeq0iRIqxYseKOgmFT15I6dWqyZ8+Ou3Cb8JIc6b1Z2ezM7fUo9R7JQnRsHAPn76HRN0vZdPQSVOoOPVZBgXoQEwlhA2BodTi4wtFNFxERJ2aCxeuvv85rr71mFdKanhZTj/LNN99Yxwnx4osvcvjwYXr27GkV686YMYO+ffvSq1cvq97FXbjPO0kmmVMHMvT5sgx+rgwZg/zZdfISTw9ZxoDfdnA9KBSemwjNv4OgzHBmF3xX3y7uvXbB0U0XEREn9cEHH/Duu+9as45M70n9+vWtYSQzdTohsmXLxm+//cbq1aut6dbdunWjU6dO/Pe//8WdaLbRQzh3JZL+M7cxY+Mx6zhPcBCfNi9B+dwZ4Np5mPserB9nn5wqBBp8as9Muq2QSkREREjQ72+Fl0Qwd/tJ3pm2hVOXIqxc0q5ybt6oV4igAF84sAR+fRXO7rVPLtTQnlad9s6iLBEREU8WrvCS/BszXrwWxUezdjBx7WHrOHv6FHzctATVCgRD1HVY8jks/RJio8E/NdR+D8p3Au/7n04nIiLirhReHLir9OLdp+kzdQtHL1yzjp8pn4O3GxYhTaAfnNwOM1+BI6vtk7OXh0YDIUvRZG+niIiIM1F4cWB4MS5HRPPp7J2MW3HQOg5JE8iHTxejdpEsZgMLWDsK5vWHyEvg7QtVX4VH3wC/QIe0V0RExNEUXhwcXuKt2n+Wt6Zs5s+zV63jp0tn470ni5I+yB8uHoXf3oBds+yTM+SzF7fLU92hbRYREXEEhRcnCS/GtcgYvpy3m5FL9hMbB8Gp/Hn/qWI0KJ7VPmH7L3aIuXzCPi7dBuq+b+9gLSIi4iHCPTG83L49wO7du50mvMTbcOg8b07ezJ5T9jLQTxQLsUJMptQBcP0izOsHa0fbJwdlgic+gUeaalq1iIh4hHBPDC/O2vPy112pBy3Yy5CwfcTExpEupR99GxWlSals9iZaZjVeU9BrFrczzGq9DT+HdPZGWiIiIu4qIb+/tcJuMgrw9eE/jxdiRo+qFM2ahgtXo3ht4iY6jV3L8YvXIFdl6LYEar4NPv6wZw4MrgQrhkBs0mw4KSIi7qd9+/Y0adLkH8+pWbMmr776aqJet1+/fpQqVYqkpvDiAMWypWXGS1Wthez8fbxZsPMUj//fYn5cfYg4E1pqvgXdlkLOyhB1Beb0gZG14cQWRzddRERcwNdff82YMWNwVwovDuLn402PWvmZ9XI1SuVIx6WIaGt9mOdHreLwuauQqRC0/w2e/AoC0sKxDTCsBsztC5H27CUREXFPkZGRD/X8tGnTki5dOtyVwouDFciSmindq/DfhkUI8PVm2d6zPP7lYsYsO0AsXlCuA7y0Goo+BXExsOwr+LYy7Fvo6KaLiEgiMUM4L730kjWMExwcTL169di6dStPPPEEqVKlIkuWLLRp04YzZ87cfM7kyZMpXrw4KVKkIGPGjNSpU4crV67cc9jIPN62bVvrtbJmzcoXX3xxVxtM7eX06dPveMwEoNt7cN566y0KFixIypQpyZs3r7WZZFRUFMlN4cUJ+Hh78UL1vMx59VEq5snAtagY+s3cTsthK9h/+jKkDoGW4+CZHyFNNjj/J4xvAtO6wZWzjm6+iIjzMnNSIq8k/+0B5sKMHTsWf39/li1bxscff8xjjz1G6dKlWbt2LbNnz+bkyZO0bNnSOvf48eM8++yzdOzYkR07dhAWFkbTpk35uzk4b7zxBosWLWLGjBn88ccf1vnr169PcBtTp05thZnt27dbQ1MjRozgyy+/JLn5JvsV5W/lDg7ix86V+H71IT7+bQdrD56n/tdL6FW3IC9Uy4Nv4QaQuxos+ABWj4BNP8KeP6DeACjRUtOqRUT+KuoqfBSa/Nd9+xj4ByXoKQUKFODTTz+17v/vf/+zgstHH3108+OjR48mR44c1nIgly9fJjo62gosuXLlsj5uemHuxZw7atQoJkyYQO3atW8GpezZsyf4bf33v/+9eT937ty8/vrr/PTTT7z55pskJ/W8OBlvby/aVMrFnNcepXqBYCKjY/n49500/XY5O0+EQ2Aae1fqTnMhc1G4ehamdYEJTeHcAUc3X0REHlDZsmVv3t+0aRMLFy60hnnib4ULF7Y+tm/fPkqWLGkFERNYWrRoYfWAnD9//p6va843NTQVK1a8+ViGDBkoVKhQgts4ceJEqlatSkhIiNUmE2YOHTpEclPPi5PKnj4l4zpWYNK6I3zw63Y2H7lIo2+W8lKtAnSvmQ//HOWh62JY9jUs+hT2LYAhlaHW21DpRfDRf1oREfxS2r0gjrhuAgUFBd3RW9KoUSM++eSTu87LmjUrPj4+zJ07l+XLl1vDQN988w3vvPMOq1atIk+ePA/UZFPz8tdhp9vrWVasWEHr1q3p37+/VZNjioJNr8u96meSmnpenJj5QmpZLgfzetWgTpEsRMXEWVsNNB60lC1HLoKPHzz6Ory4AnJXh+hrMPddGFHLnp0kIuLpzHC6Gb5J7ttDDuOXKVOGbdu2WUMz+fPnv+MWdCPkmN8RphfEhIkNGzZY9TLTpk2767Xy5cuHn5+fFWzimV4aM/x0u0yZMlm1NPH27NnD1au3ZreaoGSGqExIKleunDXMdfCgvQFxclN4cQFZ0gQyom1ZBj5bmvQp/dh54hJNhizjk9k7uR4VAxnzQbuZ8NRgCEwHJzbDiMdgzjt24ZiIiLiUHj16cO7cOasod82aNdbQz5w5c+jQoYO1DY4JIqYexhTzmmGbqVOncvr0aYoUKXLXa5nhnU6dOllFuwsWLLBmMZnZSN7ed0YAUyA8aNAgKwiZ1+3WrZsVeuKZsGKuZXpbTHsGDhx4z7DkkeHl8OHD1pSxokWLUqJECSZNmuToJjkFk7Ablwxlbq8aPFkiq7W9wLdh+2gwcAnrDp6zU37p5+GlNVCsOcTFwopB9gq9e+Y5uvkiIpIAoaGh1qwjE1Qef/xxq7bFTKM2U5e9vb2t5fMXL15MgwYNrKnLpvbEDN+YqdX38tlnn1G9enVrKMpMqa5WrdodNTaGeb4pCDbnPffcc1YxrpkSHa9x48a89tpr1pRus4qu6YkxU6Udwen2NjJdVmY6mPnEnDhxwvrkmq6t28cCXXVvo8Q0e+sJ3p2xldOXIqzc0qFKHl6vV5CU/jdqXXb/AbN6wcXD9nHxFvaspFSZHNpuERERt9vbyBQixe+LYKqZzWI9putM7lS/WAjzXqtB87LZreUERi87QP2vlrB8340FjAo+Di+uhEo9wMsbtkyCweVhw/cPtP6AiIiIs0hweDHdVKbbyXRp3Ws1PmPw4MFWkVFgYKA1NWv16tUP1Lh169ZZXWamG0vuljalH5+3KMmYDuUJTRvIoXNXeW7EKt6etoVL16MgIBXU/whemA8hxeHaeZjxIoxrDGf3Obr5IiIiyRNezBLDZn65CSh/Nwe8V69e9O3b11q9z5xrplSdOnXq5jmmZ6VYsWJ33Y4duzWdzfS2mKWMhw8f/mDvzIPULJTZWhemdcWc1vEPqw5ZWwws3HXjc56tDHReCHXfB98UcGAxfFsFlnwBMcm/rLOIiIjDal5Mz4upNL59/wTT01K+fHmrYtmIjY21ek569uxJ79697+t1IyIiqFu3Lp07d7b2cvi3c83t9jEzcz13r3n5O2bYqPeULVYvjNGsTHbefbII6VL62yec2w+/vgb7w+zjzI9A44GQvZwDWy0iIp4u3FE1L2YFPzPUYyqZb17A29s6Novb3A+TpcwULjNl69+CizFgwADrzcbfPH2IqUq+YGa/Wp1O1fJYhbxT1h+hzv8ttgp8LRnyQpvp8PQwSJEBTm2DkXXg97cg4pKjmy8iIvKvEjW8mN0uTY2K2f3ydubYzBy6H2ZqmBl6MrU0ZnjJ3LZs2fK35/fp08dKafE3M9Xa05kZR+8+WZTJ3aqQL1MQZy5H0G3COnr8sN66b6Waks/AS2uhxDMmMsKqofa06l2zHd18ERGRf+R0a8ibuedmqOl+BQQEWDdTg2NuJjyJrWyu9Mx6uToD5+9h2OL9zNp8nOV7z9Cv8SPWmjFeQRmh6TAo2QpmvgoXDsKPraBoE3jiU0h9ZwgVERFxu54XM63Z7Ldg1mm5nTk2056TejVCs0W3WYlQbgn08+HN+oWZ0aMqhUNSc/5qFK/8tJHO49ZxMvy6fVK+x+xp1VVeBi8f2D7dnla9bowpWnL0WxAREUm68GL2VTCLys2fP//mY6YXxRxXrlw5MS8lCVQsW1p+eakaveoWxM/Hi3k7TlLn/xbx85rD9kZc/inh8Q+gy0LIWgquX4SZr8DYJ+H0nftfiIiIuFR4MTtdbty40boZBw4csO7Hb4ltpkmbrbnHjh3Ljh076N69uzW92uzHkJTMkJHZUsDMdJJ78/f15uXaBfi1Z3VKZk/LpevRvDllM21Hr+bI+Rubb2Utaa8LU+8je1fUg8tgaFV75+roSEe/BRERkYRPlQ4LC6NWrVp3Pd6uXTvGjBlj3TfTpM0+CqZI1xTcms2bzBTq5OAp2wM8rOiYWEYtPcD/zd1NRHQsQf4+9H6iMK0r5sLb+8ZuqOcP2lsM7L2xN1KmwtBoIORMnv+WIiLiOcIT8Pvb6fY2elgKLwmz//Rl3pqymTV/nreOK+TJwCfNSpAn+MZeUubLY+sUmN0brpw2XzLw6OtQozf4OF29t4iIuCiFF4WXBImNjWP8yoN8MnsnVyNjCPD15vXHC9GxWh584nthrp6DOe/Aph/s45xVoNlISJvNoW0XERH34NIbMz4o1bw8ODNM1K5Kbua8+ihV82e0hpE+/G0Hzb5dzu6TNxauS5kBnv4Wmo0C/9RwaLldC7Prd0c3X0REPIx6XuQO5sth4prDfDhrB5ciovH3MUW++elaIx9+PjeyrtnUcXJHOG4XbVPpRajTD3wDHNp2ERFxXRo2Unh5aMcvXuOdaVtZsNPe3LFo1jR82ryENeXaYmYezesHK29s0GmmVzcfDRnzObDVIiLiqhReFF4ShfnSmLHxGP1mbuPC1Sir/qV7jXy8UqfArV4Ys53A9G5w7bw9nNToKyje3NFNFxERF6OaF0kUZtfwJqWzMfe1GjQoHkJMbByDFu6l1bAVHLtwzT6pUH3otswu4I28BFM6wYweEHnF0c0XERE3pZ4XuW+/bj5Gn6lbrMXt0qX048uWpahVOLP9wZhoWPQJLP7M3ugxuBC0GANZijq62SIi4gI8sudFkt6TJUKZ1bM6xbOltYaROoxZw8e/7yQqJtZe8+Wxd6DtDEiVBc7sghG1YO139loxIiIiiUThRRIkZ8aUTO5emXaVc1nHQxft49nhK60CX0veGvYwUv46EH0dfn0VJnew90oSERFJBAovkmABvj70f6oYQ1qXIXWAL2sPnqfB10tYuMuemUSqTPDcJKj7AXj7wrZpMOxROLrO0U0XERE34DbhRQW7ya9B8azM7FmNR0LTcN4MI323hk9n77T2TcLbG6q+DB3nQLqccP5PGPU4LP/GLOnr6KaLiIgLU8GuPLTrUTHWonZmiwGjQu4MDHy2NCFpA+0Trl2AmS/D9hn2cYHHocm3EBTswFaLiIgzUcGuJKtAPx8+aFKMb54tTaoAX1b/eY4GA5ewaLfZyBFIkQ5ajIWG/wc+AbDnDxhaDQ4scXTTRUTEBSm8SKJpVDLUGkYyq/GeuxJJ++9W8/mcXfYwkpcXlO8EnRdAcEG4dBzGNYaFAyA2xtFNFxERF6LwIokqT3AQU1+sQuuKOa0Z0mZRu9YjV3Eq/Lp9Qkgx6BIGpVpDXCws+hjGNobwY45uuoiIuAiFF0mSYaQPny7O18+UIsjfh1UH7GGkJXtuDCP5B0GTIfD0cPBPBQeX2sNIu/9wdNNFRMQFuE140Wwj5/NUqWz80rMahUNSc+ZyJG1Hr+b//thlbTNgKdkKui6GkBJw9Sz80ALmvGNv+igiIvI3NNtIkmU2Uv+Z2/hx9WHruHLejHz9bCkyp74xGyk6Aua+B6uG2sehZewdqjPkcWCrRUQkOWm2kTjdMNKApiX4qlUpUvr7sGL/WRp8vZRle8/YJ/gGwBOfwDM/QGA6OLbeXtRu6xRHN11ERJyQwoskG7ND9S8vVaNQFjOMFMHzo1bx1bzdt4aRCjeEbkshRyWICIfJHeGXlyHyqqObLiIiTkThRZJV/sypmN6jKq3K5bBmI301bw9tR6/i9KUI+4R0OaD9LKj+HzOqCevHwojH4NRORzddRESchMKLJLsU/j580rwEX7QoSQo/H5btPWvNRlqx76x9gtmhuvZ70GYaBGWG0ztgeE1YP047VIuIiMKLOE6zstmZ2bMqBbOksnpeWo9cyTfz99waRspXC7ovg7y1IPoa/NITprwA18Md3XQREXEgtwkvmirtmvJnTm0NIzUvmx2TWb6Yu9tamdfUxFhSZYbnp0LtvuDlA1sn28W8xzY4uukiIuIgmiotTmPS2sO8O2Mr16NiyZw6wNrcsVLejLdOOLzaLuK9eBi8/aDu+1Cpu731gIiIuDRNlRaX1KJcDms2kinqPXUpgudGrGTwwr3Exg8j5agA3ZZA4SchNgrm9IEfn4Gr5xzddBERSUYKL+JUCmZJzS8vVaVpmWzWMNJnc3bRfswazsYPI6VID60mQIPPwccfds+Gb6vCweWObrqIiCQThRdxOin9fa2ZSJ82L0GgnzeLd5+2ZiOtPnCjh8UME1XoDC/Mh4z54dIxGNMQFn2qHapFRDyAwos4JS8vL1qWy8GMHtXIlymIk+ERPDtiJUPCbhtGyloCuiyCEs/YO1Qv/BDGN4FLJxzdfBERSUIKL+LUCoWYYaRqNCkVak2h/nT2LjqOXcO5Kzc2bwxIBU2HQZOh4BcEBxbbw0h75jm66SIikkQUXsTpBQX48mWrUnzctDgBvt6E7TpNw4FLWPvnbYW6pZ6FrosgS3G4ega+b2Zv9hgT5cimi4hIElB4EZcZRnqmQk5rTZi8wUEcv3idVsNXMnTRvlvDSMEF4IV5UL6zfbzsaxhdH87/6dC2i4hI4lJ4EZdSJGsafulZjcYl7WGkj3/fyQvj1nI+fhjJLxAafg4tx0NgWji6FoY+CtumO7rpIiKSSBRexOWkCvDl62dK8dHTxfH39WbBzlPWMNK6g+dvnVS0MXRdAtnLQ8RFmNQOfn0Noq45sukiIpII3Ca8aHsAzxtGeq5iTqa9WIU8wUEcM8NIw1YwYvF+bi4anT4XdPgdqr5qH68dDSPrwOndDm27iIg8HG0PIC7v0vUo+kzdwq+bj1vHdYpk5vMWJUmX0v/WSXvnwdSudjGvX0p7kbtSz2lrARERJ6HtAcSjpA7045tnS/NBk2L4+3gzb4cZRlrKhkO3DSPlr2PvUJ2nBkRdhRkvwrSuEHHJkU0XEZEHoPAibjOM1KZSLqa+WIVcGVNy9MI1Wgxdwcgltw0jpQ6BNtPgsf/aO1RvngjDasDxTY5uvoiIJIDCi7iVYtnS8mvPajQsnpXo2Dj+N2sHXcav4+LVG+u9ePvAo29A+1mQJjuc22fXwawaBu41gioi4rYUXsQth5EGPVea9596xBpGmrv9JA2/WcLGwxdunZSrsr1DdaEGEBMJv78JP7XWDtUiIi5A4UXcdhipbeXcTOlehRwZUnDkvBlGWs7opQduDSOlzADP/AD1P7F3qN41C4ZWh0MrHd18ERH5Bwov4taKZzfDSNV5olgIUTFxvP/rdrpPWM/FazeGkcxso0rdoNNcyJAXwo/Adw1g8ecQG+vo5ouIyD0ovIjbS5vCjyGty9CvUVH8fLyYve0ET36zhM1HbhtGCi0FXRdD8RYQFwMLPoAJT8Olk45suoiI3IPCi3jMMFL7qnmY3K0K2dOn4PC5azT/dgVjl/95axgpIDU0HQFPDbbXgtkfBkOrwr4Fjm6+iIjcRuFFPErJHOmY9XJ16j2ShciYWPr+so0eP6wn/Pptw0iln4cuYZD5EbhyGsY3hXn9tEO1iIiTUHgRjxxGGvp8Wd570h5G+m3LCRp9s5StRy/eOilTIeg8H8p1BOJg6Zd2LcyFQ45suoiIOGN4uXDhAuXKlaNUqVIUK1aMESNGOLpJ4qbDSB2r5WFStypkS5eCg2ev0nTIcsavuG0YyS8FPPkltBgDAWngyGoYWg12znJ080VEPJrT7W0UExNDREQEKVOm5MqVK1aAWbt2LRkzZryv52tvI0moC1cjeX3SZubtsItzG5bIysdNi1vrxdx0/k+Y3BGOrrOPK78EdfqBz23niIiIZ+5t5OPjYwUXw4QYk62cLF+JmzEbOI5oW5b/NiyCr7cXszYft4aRth27bRgpfW7oMBsq9bCPVwyCMU9C+DGHtVtExFMlOLwsXryYRo0aERoaanW9T58+/a5zBg8eTO7cuQkMDKRixYqsXr06wUNHJUuWJHv27LzxxhsEBwcntJkiCWK+ll+onpefu1W2hpH+PHuVp4cs5/tVB2+FZ19/qP8RtBxvDyMdXmkPI2k2koiIc4cXM5RjgoUJKPcyceJEevXqRd++fVm/fr11br169Th16tTNc+LrWf56O3bM/is2Xbp0bNq0iQMHDvDDDz9w8qTW2pDkUSZnema9XI3ahTMTGR3LO9O28spPG7kcEX3rpKKN7dlIIcXh6ll7NtLCARAb48imi4h4jIeqeTF/rU6bNo0mTZrcfMz0tJQvX55BgwZZx7GxseTIkYOePXvSu3fvBF/jxRdf5LHHHqN58+b3/LgZWjK328fMzPVU8yIPIzY2jpFL9/PJ7F3ExMaRJziIwc+VoWjobV9TUdfg97dg/Vj7OG9NaDoSUmVyWLtFRFyVw2peIiMjWbduHXXq1Ll1AW9v63jFihX39Rqml+XSpUvWffMGzDBVoUKF/vb8AQMGWG82/maCi8jD8vb2osuj+fi5ayWypg3kwJkrPD1kGT+uPnTnbKTGA+HpYbcWtRtWHQ7e39e6iIg8mEQNL2fOnLFmC2XJkuWOx83xiRMn7us1Dh48SPXq1a3hJvOv6bEpXrz4357fp08fK+TE3w4fPvzQ70MkXtlcGfjt5erUKpSJiOhY+kzdwpuTN1tDSjeVfAY6L4DggnDpOIxpCMsGggrNRUSShC9OpkKFCmzcuPG+zw8ICLBuIkklfZA/o9qVZ9ji/Xw2ZyeT1h3h4LmrDHu+rPUxS+Yi0HkhzHwFtk6Gue/au1M3GQIp0jn6LYiIuJVE7Xkxs4LMVOe/Ftia45CQEJKSKSAuWrSoVW8jkhTDSN1r5mN0+/KkCvBl9YFzNBmyjL2nLt86KSAVNBsJDb8AH3/YNQuGPQrHNjiy6SIibidRw4u/vz9ly5Zl/vz5Nx8zBbvmuHLlyiSlHj16sH37dtasWZOk1xHPVrNQZqa+aG/uaK/Ku4xle8/cOsHsjVT+Beg4B9LlhAsHYdTjsGaUhpFERBwVXi5fvmwN68QP7ZjpzOb+oUP2ni9mmrRZ0n/s2LHs2LGD7t27W9OrO3TokFhtFnGogllSM71HVcrmSk/49Wjajl5trQdzh2xloOtiKNQAYiJhVi+Y2hkibuupERGR5JkqHRYWRq1ate56vF27dowZM8a6b6ZJf/bZZ1aRrlnTZeDAgdYU6qQeNjI3UzC8e/duTZWWJHc9KobeUzYzfaO9PlHHqnl4p2ERfLy9bp1kvr2WD4R5/SEuBoILQctxkLmw4xouIuLiU6Wdbm+jh6W9jSQ5mW+fQQv28sXc3dbxY4UzM/DZ0lZdzB0OLrf3RjKzkcy06ie/gpKtHNNoEREn5NJ7G4m4ErNQY8/aBRj0XGkCfL1ZsPMUzb9dzpHzV+88MVcV6LoE8tSAqKswrYs9MynquqOaLiListwmvGi2kTjSkyVCmdi1MsGpAth54hJNBi9nw6Hzd55kVt5tMw1qvGViD6wbA6Pqwrn9jmq2iIhL0rCRSCI6euEancassQKMv683n7coSeOSoXefuHe+XcBr9kYKSAtNBkORRo5osoiIU9CwkYiDmB2pJ3evQp0i9saOL/+4ga/m7b61pUC8/LXtYaQcFSHiIkx8Hua8AzFRjmq6iIjLUHgRSWSmWHdYm3J0rp7HOv5q3h5rZ2ozO+kOabNB+1lQ+SX7eMUge2uBi0cd0GoREdfhNuFFNS/iTMx06XcaFmVA0+L4envxy6ZjPDdiJacvRfzlRD+o9yG0mmAPHx1eZW/uaIaVRETknlTzIpLElu89Q7cJ66wF7cywktlioFBI6rtPNIW7P7eDE5vtgt4ab9rFvd4+jmi2iEiyUs2LiBOpkj/YWpE3d8aUVkFvs2+Xs3DnqbtPzJAXOs2FsmY16jhY9AlMaAqXTzui2SIiTkvhRSQZ5M2UimkvVqVS3gxcjoim09g1fLfswN2FvH6B0OgreHq4vZjd/jB7GMksciciIhaFF5Fkkj7In3EdK9KyXHZi46D/zO28O2MrUTGxd59sVt/tvNDeTsCsyjvmSVj6lTZ3FBFxp/Cigl1xBWbtl0+aleDtBoWtDagnrDxExzFruHjtHlOkzf5HnRdA8Rb2vkjz+sJPz8G1vyx+JyLiYVSwK+Igf2w7YU2hvhYVQ75MQVYhb66MQXefaL5F146G2b3tHarT5bQ3dwwt7Yhmi4gkCRXsiriAxx8JYVK3ymRNG8i+01doMngZqw+cu/tE00VTvhN0+gPS5YILh2DU47BmpIaRRMQjKbyIOFCxbGmZ0aMqJbKn5fzVKFqPXMnkdUfufbLpaem6CAo1tHtgZv0HprwAEZeTu9kiIg6l8CLiYJnTBDKxS2UaFA8hKiaO1ydt4tPZO4k1Vb1/lSI9PPM9PP4/8PKBrZNhRC04tcMRTRcRcQiFFxEnkMLfh0HPluGlWvmt4yFh+3jx+/VcjYy+9zBSlZ7Q4TdIHQpndsOIx2DTT8nfcBERB3Cb8KLZRuLqvL29eL1eIf6vZUn8fbyZve0ErYat5GT49Xs/IWcl6LYE8taCqKswrSv88jJE/c35IiJuQrONRJzQmj/P0XX8Os5diSQkTSAj25Wz6mPuKTYGFn8GYR/bK/OGFIcWYyFjvuRutojIA9NsIxEXVz53Bqa/WJUCmVNxIvw6LYauYM62E/c+2ex9VLM3tJkKKTPCiS0wvCZs/yW5my0ikiwUXkScVM6MKZnyYhWqFwi21oIxmzsOXbTv7i0F4uV7DLothRyVICIcfm4Ds/tAdGRyN11EJEkpvIg4sTSBfnzXvjxtK+eylnT5+PedvDl5M5HRsX/zhFBo/6td0GusHAJjGsLFv5l+LSLighReRJycr4837z9VjP6NH8HbCyatO8Lzo1Zx/srf9Kj4+NlTqVt9DwFp4chqGFod9sxL7qaLiCQJhRcRF9GuSm5rC4FUAb7WSrxNhixj3+l/WKCuyJP2onYhJeDaOfi+OSz40C7wFRFxYW4TXjRVWjxBzUKZmfpiFbKnT8HBs1d5evAylu098/dPyJAHOs2Fch3tmUiLP4XxT8PlU8nZbBGRRKWp0iIu6MzlCGsq9bqD5/Hx9uKDp4rxXMWc//ykzZNg5isQdQVShUCL7yBXleRqsojIP9JUaRE3F5wqgO9fqEiTUqHExMbx9rQtfPDrduv+3yrRAroshEyF4fIJGPMkLP0KYv+m+FdExEkpvIi4qEA/H75sVYr/1C1oHY9aeoDO49ZyOeIeWwrEy1QIOi+AEq0gLgbm9YWfnoOr99jNWkTESSm8iLgwLy8vetYuwKDnShPg682Cnado/u1yjl649vdP8g+Cp4fBk1+BTwDs/h2G1YCj65Kz6SIiD0zhRcQNPFkilIldK1vDSTtPXOKpQcvYcOj83z/BbO5YrgN0+gPS54aLh2B0fVg9AmtBGRERJ6bwIuImSuVIx4yXqlI4JLVV0Ntq+Epmbjr2z08KLQVdFkHhJyEmEn57HaZ0gohLydVsEZEEU3gRcSPZ0qVgcvcq1CmS2VqFt+ePG/h63p6/31LASJEOWk2Axz8Eb1/YOgWG14KT25Oz6SIi903hRcTNmEXshrUpR+fqeazjL+ft5pWfNnI9Kuafh5GqvATtZ0HqUDi7B0Y8Bht/SL6Gi4jcJ4UXETdk1n55p2FRBjQtjq+3F79sOsZzI1Zy+lLEPz8xZyXotsTe5DH6GkzvDjNegqh/KAAWEUlmCi8ibuzZCjkZ17ECaQJ9WX/oAk0GL2PXiX+pZwkKhtaToebbpksGNoyHkXXh7L7karaIiGeEF20PIHJvVfIHM71HVfIEB1lTqJt9u5yFO/9lewBvH6j5FrSZBimD4eQWezr19hnJ1WwRkb+l7QFEPMSFq5F0m7COlfvPWbtTv/tkUdpXyW2tFfOPwo/B5I5waIV9XLE71H0ffP2Tpd0i4hnCtT2AiPxVupT+jOtYkZblsmN2Eeg/czvvzthKVMy/bA+QJhTazYQqL9vHq76FMQ3h4tFkabeIyF8pvIh4EH9fbz5pVoK3GxS2JhhNWHmIjmPWcPFa1D8/0ccPHv8AnvkBAtLCkdUwrDrsW5hcTRcRuUnhRcTDmGGiLo/mY9jzZUnh58OSPWdoOmQZB89e+fcnF24IXcMgpDhcPQvjn4ZFn2pzRxFJVgovIh7q8UdCmNStMlnTBrLv9BVrJtLqA/exQWOGvNBpLpRpC8TBwg/hh5ba3FFEko3Ci4gHK5YtLTN6VKVE9rScvxpF65ErmbzuyL8/0S8FNP4GnhoCvoGwdy4Me1SbO4pIslB4EfFwmdMEMrFLZRoUDyEqJo7XJ23i09k7iTVVvf+mdGt4YZ7dG3PxsDZ3FJFkofAiIqTw92HQs2V4qVZ+63hI2D5e/H491yL/YUuBeKb+pUvYnZs7Tu0MEZeTvuEi4pEUXkTE4u3txev1CvF/LUvi7+PN7G0naDlsBSfDr//7kwPT3tjc8X/g5QNbJsHI2nB6V3I0XUQ8jMKLiNyhaZnsfN+5IhmC/Nly9CJPDVrG1qMX//2J1uaOPaH9r5AqBE7vtHenNrtUi4gkIoUXEblL+dwZmP5iVQpkTsWJ8Ou0GLqCOdtO3N+Tc1WBroshd3WIumKvzvvbmxAdmdTNFhEPofAiIveUM2NKprxYheoFgrkWFWNtLTB00T7ua0eR1FmgzXSo1ss+Xj0MvnsCLt7HTCYREVcNL1evXiVXrly8/vrrjm6KiMdKE+jHd+3L07ZyLmsC0ce/76T3lC3E3M9MJB9fqNMXnv3Jrok5uhaGVoe985Oj6SLixpw2vHz44YdUqlTJ0c0Q8Xi+Pt68/1Qx+jd+xNrQceLaw7w3Y+v99cAYhZ6ALosga0m4dg4mNIOwj7Uqr4i4V3jZs2cPO3fu5IknnnB0U0TkhnZVcvP1M6WtutzvVx3i0zkJmEmUIQ90/APKtLNX5Q0bAN83hytnk7LJIuKmEhxeFi9eTKNGjQgNDbX2SJk+ffpd5wwePJjcuXMTGBhIxYoVWb16dYKuYYaKBgwYkNCmiUgSa1QylI+eLm7d/zZsn1UDc9/8AqHxQGjyLfimgH3z7VV5j6xNugaLiFtKcHi5cuUKJUuWtALKvUycOJFevXrRt29f1q9fb51br149Tp06dfOcUqVKUaxYsbtux44dY8aMGRQsWNC63Y+IiAjCw8PvuIlI0nm2Qk76PFHYum9qYH5YdShhL1DquRur8uaD8CP2qryrhmtVXhG5b15x9z1wfY8ne3kxbdo0mjRpcvMx09NSvnx5Bg0aZB3HxsaSI0cOevbsSe/evf/1Nfv06cOECRPw8fHh8uXLREVF8Z///If33nvvnuf369eP/v373/X4xYsXSZMmzYO+NRH5F2YLAbMSrxlGMsNJjUuGJuwFrl+EGT1gx0z7uFgzaDQQAlIlSXtFxLmZzoe0adPe1+/vRA0vkZGRpEyZksmTJ98RaNq1a8eFCxesXpWEGDNmDFu3buXzzz//x54Xc7v9zZuwpPAikrTMj453Z2xlwspD+Hp7MaJtOWoVzpzQF4GVQ2DuexAbDcGFoNV4yFQoqZotIm4QXhK1YPfMmTPExMSQJUuWOx43xydO3OcCVwkUEBBgvcnbbyKS9MwfL+83LsZTpUKJjo2z1oFZtT+BBbim26ZyD2g/C1JnhTO77FV5t0xOqmaLiBtwytlG8dq3b/+PvS63MzU4RYsWtYasRCT59kP6vEVJahfOTER0LC+MXXt/Wwn8Vc5K0HUJ5HnUXpV3SieY9R+IvtWrKiKSJOElODjYqlU5efLkHY+b45CQEJJSjx492L59O2vWrEnS64jInfx8vBncugwV82TgUkQ0bUevZu+pB9hROlUme1Xe6jcWplwz0l6V90ICC4JFxO0lanjx9/enbNmyzJ9/awVNU7BrjitXrpyYlxIRJxLo58PIduUokT0t565E0mbUKo6cv5rwF/L2gdrvwnM/Q2A6OLrOnk69Z15SNFtEPCW8mBlAGzdutG7GgQMHrPuHDtl/HZlp0iNGjGDs2LHs2LGD7t27W9OrO3ToQFLSsJGIY6UO9GNMhwrkz5yK4xev8/zIVZy+9IDDPgXr2Zs7Zi0F187bC9ot/AhiYxK72SLighI82ygsLIxatWrd9biZUWRmBxlmmvRnn31mFemaNV0GDhxoTaF2tmplEUl8Jy5ep/nQ5Rw5f40iWdPwU5dKpE3h92AvFnUd5vSBtaPt47y1oNlICApO1DaLiAdNlXZGCi8ijvfnmSs0H7qCM5cjKJsrPeM7VSClv++Dv+Cmn2DmqxB9DdJkgxZjIEeFxGyyiHjqVGlH0rCRiPPIHRxkBZY0gb6sO3ieruPXERH9EEM+JZ+BzgsgY34IP2oX8q4cqlV5RTyUel5EJMmsP3Teqn25GhlDg+IhfPNsGXzM1tQP6no4/PISbL+x4OUjT0PjbyAgdaK1WUQcwyN7XkTE+ZTJmZ7hbcrh7+PNb1tO0GfqZmtl3gcWmAZajIX6H4O3L2ybZi9qd2pHYjZbRJycwouIJKlqBYIZ+GwpTIfLz2uP8OGsHQ8XYMyqvJW6Q/vfIHUonN0DIx6DzT8nZrNFxIm5TXhRzYuI86pfLCufNCth3R+59ACDFux9+BfNWdGeTp2nBkRdhamd4ddeWpVXxAOo5kVEks3opQd4/9ft1v3+jR+hXZXcD/+iZu2XsAGw+DP7OLQMtBwL6XI+/GuLSLJRzYuIOKWO1fLwSu0C1v2+v2xj6vojD/+iZlXex/4LrSdDivRwbD0MrQ67/3j41xYRp6TwIiLJ6tU6BehQ1e5xeWPyZv7Ylkg7zheoaw8jhZaG6xfghxaw4H9alVfEDSm8iEiy8vLy4t2GRWleNjsxsXG89OMGlu89kzgvboaKOs6Bcp3sYzOUNP5puHw6cV5fRJyC24QXFeyKuA5vby8+blqceo9kITI6lhfGrWXj4QuJ8+K+AfDk/0HTEeCXEg4ssjd3PLQqcV5fRBxOBbsi4jBm1d1OY9aydO8Z0qX0Y2KXyhQKScQF58z6LxPb2NOpzbowdT+wp1mb6dYi4lRUsCsiLiHA14dhbcpSOmc6LlyNos2oVRw6ezXxLpC5CHRZCI80hdhoe5PHSe3slXpFxGUpvIiIQwUF+PJd+/IUDknNqUsRtB61kpPh1xPvAmbrgOaj4YlPwdvP3lpgRC04uS3xriEiyUrhRUQcLl1Kf8Z1qkCujCk5fO6a1QNz/kpk4l3ADBNV7Aodfrd3pT67F0bUtnerFhGXo/AiIk4hc+pAJnSqSJY0Aew+eZn2363mckR04l4kR3l7OnXeWhB9DaZ1hZmvQlQi9vSISJJzm/Ci2UYiri9HhpRWgEmf0o9NRy7Seexarkcl8jotQcHw/BSo0dt0ycC672B0PTj/Z+JeR0SSjGYbiYjT2XzkAs+NWGX1vNQtmoUhrcvg55MEf2vtmQdTX4Br5yEwHTQdDgXrJf51RORfabaRiLi0EtnTMaJtOfx9vZm7/SRvTt5MbGwS/J1VoA50XQLZyt5YlbclzH9fq/KKODmFFxFxSpXzZeTb1mXw9fZi2oaj9J+5jSTpKE6Xwy7kLd/ZPl7yBYxvApdPJf61RCRRKLyIiNOqXSQLX7QsaU0WGrviIF/O3Z00FzKr8jb8HJqNAr8gOLDYXpX34IqkuZ6IPBSFFxFxak+Vysb7TxWz7g9csJeRS/Yn3cWKN4fOCyC4EFw6DmMawvJB4F6lgSIuTwW7IuISBi/cy2dzdln3P21WgpblcyTdxSIuw8yXYesU+7hII3hqMASmTbprilMyvyIjY2K5FhnDtagY69+I6Fhi4+JuZtr4+9a/N55jH9+4f+Mc4h8j7o6P/fV8u7zrtsduPz/+ONZ6uTsfs8651Q5zcPtr3nmt+GvfaJtpY+yttsb9zfnx7yNfplSJ/j2YkN/fvrjRVGlzi4lRoZ2IO3qxZj7Cr0UxbPF+ek/dTKpAXxoUz5o0FwtIZQ8h5agEc96GHTPtFXlbjocQuxdIHM/8YjVBwkynjw8WVyNj7ji+49+oGK7fOOfmcdSN49ued/M1bpyXFLXirq5moUxJ+wfEv1DPi4i4DPPj6u1pW/hx9WH8fLwY2a48NQpmStqLHlkLP7eD8CPgm8LesbrUc0l7TTcLFjfDwj8EhL8e/zV83P46jgoWPt5epPTzIcDPGy8vL7NKEN7mXy/7X8Pb26we5IW3l1nY2f6Y+Yi5bz1m/nfjY/Y5N17DPPnmObde917X8bqxr+gdj924hn18ow032sJt59zZrlttiD/f3Ptru269bvyx3fPyTIWcDvv9rfAiIi4lJjaOl3/awKzNx0nh58OEFypQNleGpL3olbMwtTPsm28fl2ln75XkF4gnOXHxOj+sOsiR89fuGTDu6MmIiknWUiETZgP9fKyviRT+f/nXz4dAfx8reMQ/Zs5Nae772/fjzzOPmXPvdZwkaw3JTQovCi8ibi0yOpYu49cStus0qQN9mdilMkVDk/j73az9svgzCPvYqkcga0loOQ7S58bdHTx7haGL9jNl3RGr/uNBg0XK24KDCQ13HMcHi9uCQ3ywSPnXEPKXYwUL96DwovAi4vbMX/ptR69izZ/nCU7lz89dK5M3U6qkv/DeeTClM1w7ZxfwNh3htqvy7jpxiSFhe5m56djN4ZkKuTPwWJHMBMX3WPy1l+Mv/5pzFCzkfii8KLyIeITw61E8O3wl246Fky1dCiZ1q0xouhRJf+ELh2FSezi61j6u/h+o9Q54++AONh6+YM3uMqsb316g+WLN/FTIk8RDdOKxwhVeFF5EPMWZyxG0HLqC/WeukDdTEJO6ViZjqoCkv3B0JPzxDqwebh/nqWHPUEqVxAXEScT8Klix/yxDFu5j6d4z1mOmMPOJYiFWaCmWTdPEJWkpvCi8iHiUoxeu0eLb5Ry7eJ1HQtPwY5dKpAn0S56Lb5kMv/SEqKuQOhRajIGcFXEV5lfAgp2nGLRwLxsOXbg5q6ZJqWx0r5mP/JmTYShOBIUXhRcRD7T/9GVaDF3B2SuRVl3G2I4VrLqLZHFqJ/zcBs7sBm9fqPsBVOpud1048aytWVuOM2ThXnaeuGQ9ZjbCbFUuB10ezUuODCkd3UTxMOEKLwovIp5o27GLPDN8JZeuR1s1GsPb2DtTJ4uIS3YPzLZp9nHRJvDUIAhIjbPN1Jq24Qjfhu3jz7NXrcdM8e3zlXPRqVoeMqf2rOnf4jwUXhReRDzW2j/P8fyoVVyPiuXJEln5+pnS1jBIsjA/TlcNs2thYqMhYwFoNR4yF8EZZmf9tOYQwxfv5/jF69Zj6VL60aFKHtpXyU3alMk0zCbyNzwyvNy+PcDu3bsVXkQ8WNiuU3Qet5aomDierZCTj54uZq0SmmwOr7ZX5b10DPxSQqOvoURLHDUja/yKg4xeesAaUjMypw6gc/W8PFcxJ0EBbrNLjLg4jwwv8dTzIiKGWYG354/rrfVJutbIS58nkrn348oZmNIJ9ofZx+VfgHofgW8yzIQCzl6O4LtlfzJ2xZ/WMJqRI0MKutXIR7My2a31V0ScicKLwouIAD+tPkTvqVus+2/WL2RN+U1WZlXesAH2yrxGtrLQYiykS7oN7Y5fvGYNDf24+pA1dGYUyJyKF2vlo1GJUHy1YJw4KYUXhRcRuWHE4v18+NsO6/7/mhTj+Uq5kr8Ru+fA1C5w/QKkyADNRkD+Ool6iT/PmCX89zFl/RFruMwoni0tPWrl5/GiWfBOrrofkQek8KLwIiK3+XzOLmsdE1P28lWrUjxVKlvyN+L8Qfi5LRzfaO3cS423oMabD70q784T4dbCcr9uvrWEf8U8GXjpsfxUyx+cvLU+Ig9B4UXhRURuY37M9f1lG+NWHMTX24vhbcvyWOEsyd+QqOswuzes+84+zvcYNB0JQRkT/FIbDp23lvCft+PUzcdqFcpk9bSUy60l/MX1KLwovIjIX8TGxvGfSZuYtuEoAb7e1iJ2lfImPDQkio0/wq+vQfQ1SJPd3p06e9l/fZr5cb1831krtJh/DdOx0qB4Vl6smY9HQrWEv7guhReFFxG5h6iYWLpPWGf1VqQK8OWHzhUpkT2dYxpzchtMbAPn9oG3H9QfYM9Iuscwjwle83eeskKL2TTRMD1IT5fORrea+ciXHLtpiyQxhReFFxH5G9ejYmj/3WpW7j9H+pR+1k7U+TM7aBXc6xdhRg/YMdM+Lt7CXhPGP8g6jI6JtZbwN6vhxi/hb3qNnimfgy418lk7aYu4C4UXhRcR+QeXI6JpPWIlm45cJCRNoBVgHLaXj/kRvGIQzO0LcTGQqTCRzcYw9VAQ3y7ax8EbS/ibnqI2lXPRsWoeMqVOnrViRJKTwovCi4j8i/NXImk5bAV7Tl0mV8aUVoBx6L4+B5cTO6k93pdPcoVA3ojswm+xlazeIRNY2lbWEv7i3sIT8PtbqxWJiEdKH+TP+E4VyZ4+hdW70XbUai5ejXJIWy5ei2LQvkzUu/o/VsQUJYjrDPEfyK8FfmXp69XoWbuAgovIbdTzIiIe7eDZKzQfuoLTlyIokzOdFWiSa7+fM5cjrD2HzN5DlyLsJfzzpA9gcNZZFN0/2j4pewVoMQbSOmBtGpFk5PLDRrlz57Ya7u3tTfr06Vm4cOF9P1fhRUQSateJS9YQkukBqV4gmJHtyhHgm3R7/xy7YC/hb3Z5jl/Cv2CWVNYaLQ2LZ7WX8N/5G0zrBhEXIWUwNB8FeWsmWZtEHM0twsvWrVtJlSrh0/8UXkTkQZhF31qPXMXVyBjqPxLCoOdKJ/o+QAfOXOHbsL3WWjPxS/iXzG4v4V+nyD2W8D+3Hya2hZNbwMsbar0N1f4D3hrxF/ej8KLwIiIPYNneM3T4bg2RMbE0L5udT5uVSJQ9gbYfC2dI2F5+23L85hL+lfNmtJbwr5Iv4z8v4R91DX57HTZMsI8L1IOmwyBF+odul4jHFOwuXryYRo0aERoaan3DTZ8+/a5zBg8ebAWQwMBAKlasyOrVqxN0DfO6NWrUoHz58nz//fcJbaKIyAOpmj+Yb54rjY+3F5PXHeF/s3ZYq9o+qHUHz9NpzBoaDFzCr5vt4FK7cGamdK/Cj10qWdf7172H/FLAU4Oh8SDwDYQ9c2DYo3BswwO3S8TVJbgq7cqVK5QsWZKOHTvStGnTuz4+ceJEevXqxdChQ63g8tVXX1GvXj127dpF5syZrXNKlSpFdLRdnHa7P/74wwpFS5cuJVu2bBw/fpw6depQvHhxSpQocc/2REREWLfbk5uIyIOq90iI1eNithIYvewAaVP48UqdAvf9fBN2lu09y6CFe6yF8AzTedOwRCjda+SjaOgD9giXaQNZS9ibO57/E0bVgwafQpl291yVV8SdPdSwkfmLYdq0aTRp0uTmYyawmB6TQYMGWcexsbHkyJGDnj170rt37wRf44033uCRRx6hffv29/x4v3796N+//12Pa9hIRB7Gd8sO0H/mdut+30ZF6VA1zz+eb5bwn7fjJIPD9rHpxhL+fj5eNC2d3VrCP0+wvWruQ7t2AaZ3h12/2ccln4OGX4C/gxbZE3H1dV4iIyNZt26d1Vty8wLe3tbxihUr7rtn59Ilexnsy5cvs2DBAiu8/J0+ffpYbzT+dvjw4UR4JyLi6UxYea1OQeu+CTFmGOlezBL+0zccpf7Xi+kyfp0VXAL9vGlfJTeL3qjFJ81LJF5wMVKkg1bfQ51+dhHvph9gVF04uy/xriHi5BJ1MYMzZ84QExNDlix3bjVvjnfu3Hlfr3Hy5Emefvpp6755rc6dO1s9OX8nICDAuomIJLaXa+cn/HoUo5Ye4M3Jm6wl+usXC7E+FhEdw5R1Rxm6aB+HztlL+KcO8KVtlVxW8AlOlYQ/l8xso2qvQbayMLkjnNwKw2tCkyFQpFHSXVfESSTPSkwJkDdvXjZt2pTg55kiYXMzgUdEJDGYofH/NixC+LUoJq07wss/bmBw6zLWwnYjluznZLhdb5chyJ9O1fLwfKVcVo1MssnzKHRdApPaw+GVMPF5qNITavcDH6f78S6SaBL1qzs4OBgfHx+r9+R25jgkxP5rJan06NHDusWPmYmIJFaAGdC0OJeuRzN72wk6j1t782NmU8cuj+blmQo5SOnvoLCQJiu0/xXm9bM3eFz+DRxdD81HQ+qk/bkr4iiJWvPi7+9P2bJlmT9//s3HTMGuOa5cuXJiXkpEJNmYxeq+fraUtfqukTtjSj5pVpxFb9akY7U8jgsu8Xz8oN6H0HIc+KeGg8tgaHX4c6lj2yWSRBL8HWeKaPfu3Xvz+MCBA2zcuJEMGTKQM2dOa5p0u3btKFeuHBUqVLCmSpsi3A4dOpCUNGwkIknJbBfwXfvy7Dh+iSJZUyf66ruJouhTkPkR+LkNnNoOYxtDnb5Q5WVNpxbPniodFhZGrVq17nrcBJYxY8ZY98006c8++4wTJ05Ya7oMHDjQmkKdHLTCroh4vMgr8Gsv2PyTfVz4SbuYN1BD6uK8XH57gIeh8CIiYq2WB+u+g9/fgphISJ8HWo2HkOKObpmIc63z4khmyKho0aL/OK1aRMRjmGGich2h4xxImxPOH4CRdWCDtlwR16eeFxERd3f1HEztAnvn2sdl2sITn4FfoKNbJuLZPS8iIvI3UmaA536GWv81f7PC+nH2qrznDji6ZSIPROFFRMQTmFV5a7wBbaZCyoxwYjMMrwG7Zju6ZSKeG15U8yIich/yPQZdF0P28nD9IvzYCua/D7FaZkJch2peREQ8UXQk/PFfWD3s1lYDzUZDqkyObpl4qHDVvIiIyD/y9YcGn0KzUeAXBAcWw7DqcGilo1sm8q8UXkREPFnx5tBlIQQXgkvHYUxDWDHYXidGxEkpvIiIeLpMhaDzAijWHGKjYc7bMKkdXA93dMtE3Du8qGBXROQhBKSCZiPt9V+8/WD7DBhRC05ud3TLRO6igl0REbnT4TV2z0v4UfBLCY2+hhItHd0qcXPhKtgVEZEHlqO8PZ06by2IugpTO9sbPUZHOLplIhaFFxERuVtQMDw/BWq8Za/Ku3YUjK4PFw45umUiCi8iIvI3vH2g1tvQejKkSA/H1sOwR2HPPEe3TDycwouIiPyzAnXsYaTQ0nDtPHzfHBYO0Kq84jBuE14020hEJAmlywkd50C5jkAcLPrYDjFXzjq6ZeKBNNtIREQSZtNEmPkKRF+DNNmh5TjIXtbRrRIXp9lGIiKSdEq2she1y5gfwo/A6HqweoRW5ZVko/AiIiIJl6UodF4IRRpDbBT89ro9pTryiqNbJh5A4UVERB5MYBp7yKjeR+DlA1smwYjH4PRuR7dM3JzCi4iIPDgvL6jcA9rPglQhcHqnva3AtmmObpm4MYUXERF5eLkq29Opc1eHyMswqT3M7gMxUY5umbghtwkvmiotIuJgqbNAm+lQ9VX7eOUQGNMQwo85umXiZjRVWkREEt/O32BaN4i4CEGZoNkoyFvD0a0SJ6ap0iIi4liFG0DXMMhSHK6chvFNYMkXEBvr6JaJG1B4ERGRpJEhL7wwF0o/D3GxMP99+OlZe4sBkYeg8CIiIknHLwU8NRgafwM+AbB7NgyrAcc3Obpl4sIUXkREJOmVaWv3wqTPDRcOwsi6sH6co1slLkrhRUREkkfWktAlDAo+ATER8EtPmN4Doq45umXiYhReREQk+aRID8/8ALXfAy9v2DjB7oU5u8/RLRMXovAiIiLJy9sbqv/HXhMmZTCc3ALDa8HOWY5umbgIhRcREXEMs+5LtyWQo6K9HsxPz8Hc9yAm2tEtEyen8CIiIo6TJtTeF6lSD/t42df2mjCXTjq6ZeLE3Ca8aHsAEREX5eMH9T+CFmPAPxX8uQSGPQoHVzi6ZeKktD2AiIg4jzN7YGIbOL0DvHyg7vv2rtVm92pxa+HaHkBERFxScAHoPB+Kt4S4GPjjHfi5LVwPd3TLxIkovIiIiHPxD4Kmw6HhF+DtBzt+geE14eQ2R7dMnITCi4iIOB8zTFT+Beg4G9Jkh3P7YERt2PSTo1smTkDhRUREnFf2ctB1MeR7DKKvwbSu8OtrEB3h6JaJAym8iIiIcwvKCK0nQ43epksG1o6G0fXg/EFHt0wcROFFREScn7cP1OpjhxizxcCxDTC8BuyZ6+iWiQMovIiIiOsoUMceRgotA9fOw/ctYOFHEBvj6JZJMlJ4ERER15Iup13Iawp6iYNFn8D3zeHKWUe3TJKJwouIiLge3wB7KnXTEeCXEvYtsFflPbLW0S2TZKDwIiIirqtES3hhPmTMD+FHYHR9WD0C3GvxePkLhRcREXFtWYpC54VQpDHERsFvr8OUFyDisqNbJp4UXg4cOECtWrWsjRaLFy/OlStXHN0kERFxZoFpoOU4qPeRvSfS1skwsjac3u3olomnhJf27dvz/vvvs337dhYtWkRAQICjmyQiIq6wKq/ZxLH9LEgVAqd3wohasHWqo1sm7h5etm3bhp+fH9WrV7eOM2TIgK+vr6ObJSIiriJXZei2BHJXh8jLMLkD/N4boiMd3TJxVHhZvHgxjRo1IjQ0FC8vL6ZPn37XOYMHDyZ37twEBgZSsWJFVq9efd+vv2fPHlKlSmVdo0yZMnz00UcJbaKIiHi6VJmhzXSo9pp9vOpbGNMQLh51dMskESS4S8PUn5QsWZKOHTvStGnTuz4+ceJEevXqxdChQ63g8tVXX1GvXj127dpF5syZrXNKlSpFdHT0Xc/9448/rMeXLFnCxo0brfPr169P+fLlqVu37j3bExERYd3ihYdr23QREQF8fKFOP8heAaZ1gyOr7enUzUdB3pqObp08BK+4uAefT2Z6XqZNm0aTJk1uPmYCiwkbgwYNso5jY2PJkSMHPXv2pHdvsy/FP1uxYgX9+vVjzpw51vFnn31m/fvGG2/c83xzbv/+/e96/OLFi6RJk+ZB35qIiLiTc/vh57ZwYgt4eUOtd6BaL/B2uuoJjxUeHk7atGnv6/d3ov5Xi4yMZN26ddSpU+fWBby9rWMTSu6HCT6nTp3i/PnzVvAxw1RFihT52/P79OljvdH42+HDhxPlvYiIiBvJkBc6zYXSz0NcLCz4AH561t5iQFxOooaXM2fOEBMTQ5YsWe543ByfOHHivl7DFOeaOpdHH32UEiVKUKBAAZ588sm/Pd/MRDIJ7fabiIjIXfxSwFODofE34BMAu2fDsBpwbKOjWyYJ5JTTeJ544gnrlhCmSNjcTHgSERH5W2XaQtaS9jDS+T9h1OPQ4DP7cTPdWjyr5yU4OBgfHx9Onjx5x+PmOCQkhKTUo0cPa12YNWvWJOl1RETEDZjw0iUMCj4BMREw82WY0QMirzq6ZZLc4cXf35+yZcsyf/78m4+ZuhVzXLly5cS8lIiIyMNJkR6e+QFq97WLeDd+b/fCnN3n6JZJYoeXy5cvW9OYzS1+KX9z/9ChQ9axmSY9YsQIxo4dy44dO+jevbs1vbpDhw4kJTNkZLYTMAW/IiIi98XMNqreC9rOgKBMcHILDK8JO351dMskMadKh4WFWfsO/VW7du0YM2aMdd9MkzZTnE2RrlnTZeDAgdYUamebaiUiInJT+DGY1AEOr7SPq74Cj71nrxcjTvX7+6HWeXFGCi8iIvLAYqJgbl9YOdg+zlUNmo+G1HfOohU3WufFkTRsJCIiD83HD+p/BC3Ggn8qOLgUhlWHg8sd3TK5jXpeRERE7uXMHpjYBk7vAC8fqNsfKr+k6dRJxCN7XkRERBJVcAHoPB+Kt4S4GPjjv/BzG7h+0dEt83gKLyIiIn/HPwiaDoeGX4C3H+yYCcNrwYmtjm6ZR1N4ERER+SdmmKj8C9BxDqTNAef2wcg6sOknR7fMY7lNeFHBroiIJKnsZaHrYshXG6KvwbSuMPNViLru6JZ5HBXsioiIJERsDCz+DMI+BuIgtLQ9Oyl9Lke3zKWpYFdERCSpePtAzd7w/GR7i4FjG2B4Ddgz19Et8xgKLyIiIg8ifx17GCm0DFw7D9+3gAUf2j0zkqTcJryo5kVERJJdupzQcbZd0GuGkBZ/ChOawZWzjm6ZW1PNi4iISGLY/DPMfAWirkKabHYdTA79QX2/VPMiIiKS3Eq0hBfmQ8b8EH4UvnsCVo8A9+ojcAoKLyIiIoklS1HovBCKNIbYKPjtdZjyAkRcdnTL3IrCi4iISGIKTAMtx0G9j+w9kbZOhpG14fRuR7fMbSi8iIiIJMWqvJV7QPtZkCoETu+EEbVg61RHt8wtuE140WwjERFxOrkqQ7clkLs6RF6GyR3g994QHenolrk0zTYSERFJajHRsPB/sPRL+zhHRWgxBtKEOrplTkOzjURERJyJjy/U6QfP/AgBaeHwKhhaHfaHObplLknhRUREJLkUbgBdwyBLcbh6BsY/DYs/h9hYR7fMpSi8iIiIJKcMeeGFuVD6eYiLhQUfwE/P2lsMyH1ReBEREUlufingqcHQ+BvwCYDds2FYDTi20dEtcwkKLyIiIo5Spq3dC5MuF1w4CKMeh/XjHN0qp6fwIiIi4khZS0LXRVDwCYiJgF96wvQeEHXN0S1zWm4TXrTOi4iIuKwU6eGZH6D2e+DlDRsnwMi6cHafo1vmlLTOi4iIiDPZvwgmd7RnI5lp1U9/C4Ub4u7Ctc6LiIiIi8pbw16V1yxkF3ERfnoO5va1F7oTi8KLiIiIszEr75p9kSq9aB8v+wrGN4HLpxzdMqeg8CIiIuKMfPyg/gBo/h34p4I/l9ir8h5cgadTeBEREXFmxZpC54WQqTBcPgFjGsLyQeBeJasJovAiIiLi7DIVhBfmQ/EWEBcDf7wDP7eF6+F4IoUXERERVxCQCpqOgAafg7cf7PgFhteEk9vxNAovIiIirsLLCyp0ho6zIU12OLcPRtaGTRPxJAovIiIiriZ7Oei6GPI9BlFXYVoX+LUXREfgCdwmvGiFXRER8ShBGaH1ZKjxlumSgbWjYHR9uHAId6cVdkVERFzdnnkw9QW4dt7eaqDpSChQB1eiFXZFREQ8SYE69jBSaGk7wHzfHBYOgNgY3JHCi4iIiDtIlxM6zoFyHYE4WPSxHWKunMXdKLyIiIi4C98AePJLeHoY+KaAfQtg2KNwZB3uROFFRETE3ZR8BjrPhwz5IPwIjK4Hq0e4zaq8Ci8iIiLuKMsj0CUMijSC2Cj47XWY2hkir+DqFF5ERETcVWAaaDkeHv8QvHxgyyQYURvO7MGVKbyIiIi4+6q8VV6C9r9CqhA4vcPeVmDbdFyVwouIiIgnyFXFnk6dqxpEXoZJ7WB2H4iJwtUovIiIiHiK1Fmg7Qyo+qp9vHIIjHkSwo/hShReREREPImPL9TtD8/8AAFp4fBKezr1/kW4CoUXERERT1S4IXRZCFmKwZXTML4JLPk/iI3F2TldeNm1axelSpW6eUuRIgXTp7tuUZGIiIjTypgPOs2FUq0hLhbm94efnrO3GHBiTr0x4+XLl8mdOzcHDx4kKCjovp6jjRlFREQSyESB9ePgtzcgJgLS5YJW4yFrSZKL22zM+Msvv1C7du37Di4iIiLygNOpy7aDTn/YeyRdOAgj69qBxgklOLwsXryYRo0aERoaipeX1z2HdAYPHmz1mAQGBlKxYkVWr179QI37+eefadWq1QM9V0RERBIotJQ9nbpgfbsH5peeMKMHRF3DpcPLlStXKFmypBVQ7mXixIn06tWLvn37sn79euvcevXqcerUqZvnmFqWYsWK3XU7duzYHd1Hy5cvp0GDBv/YnoiICOvc228iIiLygFKkh2d+hMfeBS9v2DABRtWFcwdwi5oX0/Mybdo0mjRpcvMx09NSvnx5Bg0aZB3HxsaSI0cOevbsSe/eve/7tcePH8+cOXOYMGHCP57Xr18/+vfvf9fjqnkRERF5SPvDYHInuHrGnlb99FAo/M+dCi5X8xIZGcm6deuoU6fOrQt4e1vHK1asSJIhoz59+lhvNP52+PDhB2q7iIiI/EXemvYwUvYKEHERfnoW5vWDmGgcKVHDy5kzZ4iJiSFLlix3PG6OT5w4cd+vY0KIqZMxw03/JiAgwEpot99EREQkkaTNBu1nQcXu9vHSL+01YRy4rYBTzjYy3UYnT57E39//vp9janCKFi1qDVmJiIhIIvL1hyc+hubfgV8QhJYGHz/HNScxXyw4OBgfHx8reNzOHIeEhJCUevToYd3ix8xEREQkkRVras9ISpsTR0rUnhfTU1K2bFnmz59/8zFTsGuOK1eunJiXEhEREUfIkNfeH8mBfB9k1du9e/fePD5w4AAbN24kQ4YM5MyZ05om3a5dO8qVK0eFChX46quvrOnVHTp0ICmZYSNzMzU3IiIi4r4SPFU6LCyMWrVq3fW4CSxjxoyx7ptp0p999plVpGvWdBk4cKA1hTo5aHsAERER15OQ399OvbfRg1B4ERERcT1us7eRiIiIiNuGF02VFhER8QwaNhIRERGH07CRiIiIuC2FFxEREXEpbhNeVPMiIiLiGVTzIiIiIg6nmhcRERFxWwovIiIi4lLcJryo5kVERMQzuF3NixkrS5cuHYcPH1bNi4iIiAvVvOTIkYMLFy5YtS//xLF7WieBS5cuWf+aT4CIiIi43u/xfwsvbtfzEhsby7Fjx0idOjVeXl5Jkgo9tVfH09+/4emfA71/z37/hqd/Djz9/Sfl58DEERNcQkND8fb29qyeF/OGs2fPnqTXMP+xPPWL1vD09294+udA79+z37/h6Z8DT3//SfU5+LceF7cr2BURERHPoPAiIiIiLkXhJQECAgLo27ev9a8n8vT3b3j650Dv37Pfv+HpnwNPf//O8jlwu4JdERERcW/qeRERERGXovAiIiIiLkXhRURERFyKwouIiIi4FIWX+7B48WIaNWpkrfpnVu2dPn06nmTAgAHWhpdm1eLMmTPTpEkTdu3ahaf49ttvKVGixM0FmSpXrszvv/+Op/r444+t74NXX30VT9GvXz/rPd9+K1y4MJ7k6NGjPP/882TMmJEUKVJQvHhx1q5di6fInTv3XV8D5tajRw88QUxMDO+++y558uSx/vvny5ePDz74wFoV1xHcboXdpHDlyhVKlixJx44dadq0KZ5m0aJF1jeoCTDR0dG8/fbbPP7442zfvp2goCDcnVmx2fzCLlCggPWNOnbsWJ566ik2bNjAI488gidZs2YNw4YNs8KcpzH/refNm3fz2NfXc358nj9/nqpVq1KrVi0ruGfKlIk9e/aQPn16POlr3/wCj7d161bq1q1LixYt8ASffPKJ9Yec+flnvhdMcO3QoYO1Iu7LL7+c7O3xnO++h/DEE09YN081e/bsO47HjBlj9cCsW7eORx99FHdnet1u9+GHH1rfxCtXrvSo8HL58mVat27NiBEj+N///oenMWElJCQET2R+cZm9bL777rubj5m/wD2JCWy3M3/QmN6HGjVq4AmWL19u/dHWsGHDmz1RP/74I6tXr3ZIezRsJAl28eJF698MGTLgacxfXj/99JPVG2eGjzyJ6X0zP7jq1KmDJzI9DWboOG/evFaIO3ToEJ7il19+oVy5clYvg/nDpXTp0laI9VSRkZFMmDDB6o1P7A2AnVWVKlWYP38+u3fvto43bdrE0qVLHfaHvXpeJMG7dptaB9OFXKxYMTzFli1brLBy/fp1UqVKxbRp0yhatCiewgS29evXW13nnqhixYpWj2OhQoU4fvw4/fv3p3r16tbQgakFc3f79++3eht79eplDRubrwMzVODv70+7du3wNKbu8cKFC7Rv3x5P0bt3b2s3aVPr5ePjY/0hZ3qhTZB3BIUXSfBf3+YHtkncnsT80tq4caPV6zR58mTrB7apBfKEAGO2vX/llVeYO3cugYGBeKLb/7o09T4mzOTKlYuff/6ZTp064Ql/tJiel48++sg6Nj0v5ufA0KFDPTK8jBo1yvqaMD1xnuLnn3/m+++/54cffrCGy83PQ/OHrPkcOOJrQOFF7ttLL73Er7/+as2+MkWsnsT8hZk/f37rftmyZa2/PL/++mureNXdmdqmU6dOUaZMmZuPmb+6zNfBoEGDiIiIsP4S8yTp0qWjYMGC7N27F0+QNWvWu4J6kSJFmDJlCp7m4MGDVuH21KlT8SRvvPGG1fvyzDPPWMdmtpn5XJjZqAov4pTMDJuePXtaQyVhYWEeV6j3d3+Jml/anqB27drWsNntzCwD03381ltveVxwiS9e3rdvH23atMETmGHivy6PYGofTO+TpzFFy6buJ75w1VNcvXoVb+87y2TN9775WegICi/3+YPq9r+wDhw4YHWZmYLVnDlz4glDRaarcMaMGdb4/okTJ6zHzRQ5M9/f3fXp08fqIjb/rS9dumR9LkyImzNnDp7A/Df/a32TmSJv1vvwlLqn119/3Zp1Zn5ZHzt2zNpR1/zgfvbZZ/EEr732mlWwaYaNWrZsac0wGT58uHXzJOYXtQkvpqfBk6bKG+br39S4mJ+DZtjILBXxf//3f1bRskOYXaXlny1cuNCswnPXrV27dnGe4F7v3dy+++67OE/QsWPHuFy5csX5+/vHZcqUKa527dpxf/zxR5wnq1GjRtwrr7wS5ylatWoVlzVrVutrIFu2bNbx3r174zzJzJkz44oVKxYXEBAQV7hw4bjhw4fHeZo5c+ZYP/t27doV52nCw8Ot7/mcOXPGBQYGxuXNmzfunXfeiYuIiHBIe7zM/zkmNomIiIgknNZ5EREREZei8CIiIiIuReFFREREXIrCi4iIiLgUhRcRERFxKQovIiIi4lIUXkRERMSlKLyIiIiIS1F4EZGHUrNmTWt3WWfi5eXF9OnTHd0MEUkiWmFXRB7KuXPn8PPzs/ZAyp07txVkkivM9OvXzwopZq+x25n9t9KnT09AQECytENEkpdn7SwlIonObFCa2CIjI/H393/g54eEhCRqe0TEuWjYSEQSZdjI/Hvw4EFrB2IzbGNu8ZYuXUr16tWtXchz5MjByy+/zJUrV25+3PTYfPDBB7Rt25Y0adLQpUsX6/G33nqLggULkjJlSvLmzcu7775LVFSU9bExY8bQv39/Nm3adPN65rF7DRtt2bKFxx57zLq+2Q3bvL7ZLT5e+/btadKkCZ9//jlZs2a1zjG7qcdfS0Sci8KLiCSKqVOnkj17dt5//32OHz9u3Yx9+/ZRv359mjVrxubNm5k4caIVZl566aU7nm+CQ8mSJdmwYYMVUgwzFGUCyfbt2/n6668ZMWIEX375pfWxVq1a8Z///IdHHnnk5vXMY39lQlK9evWsYaQ1a9YwadIk5s2bd9f1Fy5caLXV/Dt27FjruvFhSESci4aNRCTRho98fHyswHH7sM2AAQNo3br1zTqYAgUKMHDgQGrUqMG3335LYGCg9bjpGTFh5Hb//e9/7+idef311/npp5948803rV6UVKlS4evr+4/DRD/88APXr19n3LhxBAUFWY8NGjSIRo0a8cknn5AlSxbrMRNuzOPmPRQuXJiGDRsyf/58OnfunMifKRF5WAovIpKkzLCO6XH5/vvvbz5m5gnExsZy4MABihQpYj1Wrly5u55remlM0DE9ImaYJzo62hpWSogdO3ZYPTrxwcWoWrWqdf1du3bdDC+mB8cEl3hm+MgMN4mI81F4EZEkZUJH165drTqXv8qZM+fN+7eHC2PFihVWj42pazHDPmnTprV6Xb744oskaaeZMXU7UzdjAo6IOB+FFxFJNGaGUExMzB2PlSlTxqpZyZ8/f4Jea/ny5eTKlYt33nnn5mOmIPjfrvdXpmfH1K6Y2pf4gLRs2TK8vb0pVKhQgtokIs5BBbsikmhMXcrixYs5evQoZ86cuTljyAQRUyBr1mPZs2cPM2bMuKtg9q9MbcyhQ4es3hYzbGSGj6ZNm3bX9czQk3ldc72IiIi7Xsf03pi6mnbt2rF161arILdnz560adPm5pCRiLgWhRcRSTRmptGff/5Jvnz5yJQpk/VYiRIlWLRoEbt377amS5cuXZr33nuP0NDQf3ytxo0bW9OuTcgpVaqUFYDiZyHFMzOYzEymWrVqWdf78ccf73odM816zpw51mJ65cuXp3nz5tSuXdsqzhUR16QVdkVERMSlqOdFREREXIrCi4iIiLgUhRcRERFxKQovIiIi4lIUXkRERMSlKLyIiIiIS1F4EREREZei8CIiIiIuReFFREREXIrCi4iIiLgUhRcRERHBlfw/7F4MSZvhBtQAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from pySDC.helpers.stats_helper import get_sorted\n", + "\n", + "error_first_step_storted = get_sorted(stats, sortby='iter', type='e_global_post_iteration', time=1)\n", + "residual_first_step_storted = get_sorted(stats, sortby='iter', type='residual_post_iteration', time=0)\n", + "\n", + "import matplotlib.pyplot as plt\n", + "\n", + "plt.plot([me[0] for me in error_first_step_storted], [me[1] for me in error_first_step_storted], label='error')\n", + "plt.plot([me[0] for me in residual_first_step_storted], [me[1] for me in residual_first_step_storted], label='residual')\n", + "plt.xlabel('iteration')\n", + "plt.yscale('log')\n", + "plt.legend(frameon=False)\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "id": "185f4de0-7bbd-4b7a-92b2-1371ff66ebcd", + "metadata": {}, + "source": [ + "Ups! The residual continues to go down, but the error does not.\n", + "Seems we solved the collocation problem unnecessary accurately.\n", + "The collocation problem is the discretization in time.\n", + "Try increasing the number of collocation nodes or reducing the step size to increase the temporal resolution and achieving lower error.\n", + "Once more we see that pySDC allows to collect vast amounts of data and perform detailed analysis.\n", + "\n", + "Finally, let's check about the caching of solvers.\n", + "Looking at the timings, we should see that the first step takes much longer than subsequent ones." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "31d019c6-1b5d-45d3-a0a2-329feef3bf60", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Took 6.88e-01s for the first step and 3.319e-01s for the second step\n" + ] + } + ], + "source": [ + "timings = get_sorted(stats, type='timing_step')\n", + "print(f'Took {timings[0][1]:.2e}s for the first step and {timings[1][1]:.3e}s for the second step')" + ] + }, + { + "cell_type": "markdown", + "id": "7bc27a1f-87a5-456d-b07f-d1cff17f554a", + "metadata": {}, + "source": [ + "## Using pySDC as an integrator within another library\n", + "Once you have coupled a datatype to pySDC, you can implement pySDC as an integrator for libraries building on the same datatype.\n", + "For instance, we have implemented this for [Gusto](https://www.firedrakeproject.org/gusto/), which builds on Firedrake to assemble dynamical cores as used in numerical weather prediction.\n", + "The way you go about this is to look at the interface for other integrators in the library you want to couple to.\n", + "Chances are an integrator accepts some initial conditions and returns the solution at the end of the interval.\n", + "In this case, you need to assemble a pySDC controller to do what you want and then, in the library you want to couple to, pass the solution between pySDC and the library.\n", + "In code, this looks approximataly like so:\n", + "\n", + "```\n", + "class pySDC_integrator(my_library_integrator):\n", + "\n", + " def __init__(self, description):\n", + " super().__init__()\n", + " self.controller = controller_nonMPI(description, ...)\n", + "\n", + " def integrate(u0, dt, t):\n", + " P = self.controller.MS[0].levels[0].prob\n", + " _u = P.u_init(u0)\n", + " u_end, _ = self.controller.run(u0=_u, t0=t, Tend=t+dt)\n", + " return uend.as_my_library_dtype()\n", + "```\n", + "\n", + "Of course, you need to take care that `dt` is divisible by the pySDC time step and so on.\n", + "Also, you need to implement the coupling in a problem class.\n", + "That is to say, you need an interface that can be called by pySDC and which calls the respective functions for evaluating and inverting $f$ in the library you want to couple to.\n", + "Since the library may not be designed to be used like this, this may cause some issues.\n", + "But if the library you want to couple to is well coded, it should be feasible to use pySDC as an integrator within this library.\n", + "\n", + "## Summary\n", + "In this notebook you saw how to couple pySDC with another library, in this case Firedrake.\n", + "The main part of this is to write a datatype that connects the interfaces of pySDC and the library.\n", + "Then, we implemented the heat equation to be integrated in time with pySDC and discretized in space with Firedrake.\n", + "The problem classes add a little pySDC-specific infrastructure, but they are only as difficult to write as it is to use the library you are coupling to.\n", + "Once the data type and problem implementations are done, you can readily plug this into pySDC and enjoy access to all sorts of SDC-related things." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "pySDC_firedrake", + "language": "python", + "name": "pysdc_firedrake" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/pySDC/playgrounds/PinT_Workshop_2025/README.rst b/pySDC/playgrounds/PinT_Workshop_2025/README.rst new file mode 100644 index 0000000000..04feffe4c9 --- /dev/null +++ b/pySDC/playgrounds/PinT_Workshop_2025/README.rst @@ -0,0 +1,83 @@ +pySDC Tutorial at 14th PinT Workshop +==================================== +**Time**: July 7, 2025 + +**Place**: ICMS, Edinburgh + +Installation +------------ +In order to start playing, install `pySDC` and its dependencies, ideally in developer mode. +First, we need to download the repository. +There are multiple ways to do that, but if you plan to work with `pySDC` directly, the best way is to +(1) `fork `_ +the main repository to your Github account and then +(2) `clone `_ it from there. +This way you can work on a separate repository while being able to pull updates from the main one and +starting pull requests to merge back your ideas. +You can also clone the main repository, but this will not accept your pushes. +Downloading `pySDC` as a tarball is the easiest, but also the least favorable solution. +Finally, the code can also be obtained using ``pip install``, but then sources are not that easily accessible. + +So, please go ahead and clone from your fork on Github: + +.. code-block:: bash + + git clone https://github.com//pySDC.git + +Next, navigate to the directory that contains this file and setup up a virtual environment, e.g. by using `Micromamba `_. +From the root directory of `pySDC`, you can run + +.. code-block:: bash + + cd /pySDC/playgrounds/PinT_Workshop_2025 + +Now, create the virtual environment with the following command. If you are using ``conda`` instead of ``micromamba``, you can just replace ``micromamba`` with ``conda`` in the commands, or run first run ``conda install -c conda-forge micromamba``. + +.. code-block:: bash + + micromamba env create -f environment-tutorial.yml + micromamba activate pySDC_tutorial + + +This may take a while... +Note that this folder and all changes in it will remain even if you leave the virtual environment. +Only installations made with ``micromamba`` or ``pip`` are affected by changing the environment. +Use `branches `_ to isolate development work. +**Please make sure to perform all the following steps inside the virtual environment!** + +Finally, you can install `pySDC` via ``pip`` in editable mode using: + +.. code-block:: bash + + pip install -e + +You don't need to do this, but it can make life easier when it comes to setting path variables etc. + +Testing +------- + +Change to `pySDC`'s root directory and run + +.. code-block:: bash + + pytest pySDC/tests -m "base and not slow" + +This will check if "all" went well with the installation you just created. +Note that at the time of creating this tutorial we had to remove `mpi4py_fft` from the dependencies. +Therefore, 7 out of the 39 selected test cases may fail! +Anyway, you are now ready to play with `pySDC`. + +Jupyter +------- +In order to use our virtual environment within Jupyter, we make a kernel for it with all our nice packages. +We do that with + +.. code-block:: bash + + python -m ipykernel install --user --name=pySDC_tutorial + +Now, start jupyter and have a look at the notebooks! + +.. code-block:: bash + + jupyter notebook diff --git a/pySDC/playgrounds/PinT_Workshop_2025/environment-tutorial.yml b/pySDC/playgrounds/PinT_Workshop_2025/environment-tutorial.yml new file mode 100644 index 0000000000..4cff458b1f --- /dev/null +++ b/pySDC/playgrounds/PinT_Workshop_2025/environment-tutorial.yml @@ -0,0 +1,23 @@ +--- + +name: pySDC_tutorial +channels: + - conda-forge +dependencies: + - numpy>=1.15.4 + - scipy>=0.17.1 + - matplotlib>=3.0 + - dill>=0.2.6 + - vtk + - mpich + - mpi4py-fft + - mpi4py>=3.0.0 + - pytest + - pytest-cov + - jupyter + - ipyparallel + - pip + - ipython + - pip: + - jdc + - qmat diff --git a/pySDC/playgrounds/PinT_Workshop_2025/figs/3D_RBC_3.png b/pySDC/playgrounds/PinT_Workshop_2025/figs/3D_RBC_3.png new file mode 100644 index 0000000000..f5eaa0313f Binary files /dev/null and b/pySDC/playgrounds/PinT_Workshop_2025/figs/3D_RBC_3.png differ diff --git a/pySDC/playgrounds/PinT_Workshop_2025/figs/ADVECTION_steps_vs_iteration_hf_7x7_INTERP.png b/pySDC/playgrounds/PinT_Workshop_2025/figs/ADVECTION_steps_vs_iteration_hf_7x7_INTERP.png new file mode 100644 index 0000000000..3a9cd01bae Binary files /dev/null and b/pySDC/playgrounds/PinT_Workshop_2025/figs/ADVECTION_steps_vs_iteration_hf_7x7_INTERP.png differ diff --git a/pySDC/playgrounds/PinT_Workshop_2025/figs/GS3D_000040.png b/pySDC/playgrounds/PinT_Workshop_2025/figs/GS3D_000040.png new file mode 100644 index 0000000000..552c98be9c Binary files /dev/null and b/pySDC/playgrounds/PinT_Workshop_2025/figs/GS3D_000040.png differ diff --git a/pySDC/playgrounds/PinT_Workshop_2025/figs/compression_order_time_advection_d=1.00e-06_n=4_MPI=True.png b/pySDC/playgrounds/PinT_Workshop_2025/figs/compression_order_time_advection_d=1.00e-06_n=4_MPI=True.png new file mode 100644 index 0000000000..d24dc8659c Binary files /dev/null and b/pySDC/playgrounds/PinT_Workshop_2025/figs/compression_order_time_advection_d=1.00e-06_n=4_MPI=True.png differ diff --git a/pySDC/playgrounds/PinT_Workshop_2025/figs/parallelSDC_preconditioner_vanderpol.png b/pySDC/playgrounds/PinT_Workshop_2025/figs/parallelSDC_preconditioner_vanderpol.png new file mode 100644 index 0000000000..7a7fcc4e85 Binary files /dev/null and b/pySDC/playgrounds/PinT_Workshop_2025/figs/parallelSDC_preconditioner_vanderpol.png differ diff --git a/pySDC/playgrounds/PinT_Workshop_2025/figs/scaling_GS3D_time.png b/pySDC/playgrounds/PinT_Workshop_2025/figs/scaling_GS3D_time.png new file mode 100644 index 0000000000..e7de5ac2b5 Binary files /dev/null and b/pySDC/playgrounds/PinT_Workshop_2025/figs/scaling_GS3D_time.png differ diff --git a/pySDC/playgrounds/PinT_Workshop_2025/figs/timings_SDC_variants_GrayScott.png b/pySDC/playgrounds/PinT_Workshop_2025/figs/timings_SDC_variants_GrayScott.png new file mode 100644 index 0000000000..e0dcc66e3f Binary files /dev/null and b/pySDC/playgrounds/PinT_Workshop_2025/figs/timings_SDC_variants_GrayScott.png differ diff --git a/pySDC/playgrounds/PinT_Workshop_2025/figs/wp-run_RBC-RK_comp-t-e_global_rel.png b/pySDC/playgrounds/PinT_Workshop_2025/figs/wp-run_RBC-RK_comp-t-e_global_rel.png new file mode 100644 index 0000000000..8bd1e8540a Binary files /dev/null and b/pySDC/playgrounds/PinT_Workshop_2025/figs/wp-run_RBC-RK_comp-t-e_global_rel.png differ