Merge pull request #2649 from dweindl/release_0.31.0

Release 0.31.0

Author: dweindl

.pre-commit-config.yaml

@@ -2,7 +2,7 @@
 # See https://pre-commit.com/hooks.html for more hooks
 repos:
 - repo: https://github.com/pre-commit/pre-commit-hooks
-  rev: v4.6.0
+  rev: v5.0.0
   hooks:
   - id: check-added-large-files
   - id: check-merge-conflict
@@ -12,7 +12,7 @@ repos:
   - id: trailing-whitespace
 - repo: https://github.com/astral-sh/ruff-pre-commit
   # Ruff version.
-  rev: v0.6.7
+  rev: v0.8.6
   hooks:
     # Run the linter.
     - id: ruff
@@ -28,7 +28,7 @@ repos:
         - python/sdist/pyproject.toml
 
 -   repo: https://github.com/asottile/pyupgrade
-    rev: v3.17.0
+    rev: v3.19.1
     hooks:
     -   id: pyupgrade
         args: ["--py310-plus"]

CHANGELOG.md

@@ -4,6 +4,27 @@ See also our [versioning policy](https://amici.readthedocs.io/en/latest/versioni
 
 ## v0.X Series
 
+### v0.31.0 (2025-02-18)
+
+* Added `RDataReporting::observables_likelihood` for computing observables,
+  likelihood and the respective sensitivities
+  by @dweindl in https://github.com/AMICI-dev/AMICI/pull/2627,
+  https://github.com/AMICI-dev/AMICI/pull/2633
+* JAX:
+  * Updated diffrax & jaxlib
+    by @FFroehlich in https://github.com/AMICI-dev/AMICI/pull/2632
+  * Avoid silent preequilibration failure in JAX
+    by @FFroehlich in https://github.com/AMICI-dev/AMICI/pull/2631
+  * jax vectorisation
+    by @FFroehlich in https://github.com/AMICI-dev/AMICI/pull/2636
+  * No flattening of timepoint specific overrides in jax
+    by @FFroehlich in https://github.com/AMICI-dev/AMICI/pull/2641
+* Faster PEtab parameter mapping
+  by @dweindl in https://github.com/AMICI-dev/AMICI/pull/2638,
+  https://github.com/AMICI-dev/AMICI/pull/2640
+
+**Full Changelog**: https://github.com/AMICI-dev/AMICI/compare/v0.30.1...v0.31.0
+
 ### v0.30.1 (2025-02-18)
 
 Bugfix-only release.

documentation/amici_refs.bib

@@ -1312,17 +1312,6 @@ @Article{MerktAli2024
   publisher        = {Springer Science and Business Media LLC},
 }
 
-@Misc{LakrisenkoPat2024,
-  author           = {Polina Lakrisenko and Dilan Pathirana and Daniel Weindl and Jan Hasenauer},
-  title            = {Exploration of methods for computing sensitivities in ODE models at dynamic and steady states},
-  year             = {2024},
-  archiveprefix    = {arXiv},
-  creationdate     = {2024-05-30T09:48:00},
-  eprint           = {2405.16524},
-  modificationdate = {2024-05-30T09:48:00},
-  primaryclass     = {q-bio.QM},
-}
-
 @PhdThesis{Mutsuddy2024,
   author           = {Mutsuddy, Arnab},
   school           = {Clemson University},
@@ -1435,6 +1424,38 @@ @Article{ArmisteadHoe2024
   publisher        = {Springer Science and Business Media LLC},
 }
 
+@Article{LakrisenkoPat2024,
+  author           = {Lakrisenko, Polina and Pathirana, Dilan and Weindl, Daniel and Hasenauer, Jan},
+  journal          = {PLOS ONE},
+  title            = {Benchmarking methods for computing local sensitivities in ordinary differential equation models at dynamic and steady states},
+  year             = {2024},
+  month            = {10},
+  number           = {10},
+  pages            = {1-19},
+  volume           = {19},
+  abstract         = {Estimating parameters of dynamic models from experimental data is a challenging, and often computationally-demanding task. It requires a large number of model simulations and objective function gradient computations, if gradient-based optimization is used. In many cases, steady-state computation is a part of model simulation, either due to steady-state data or an assumption that the system is at steady state at the initial time point. Various methods are available for steady-state and gradient computation. Yet, the most efficient pair of methods (one for steady states, one for gradients) for a particular model is often not clear. In order to facilitate the selection of methods, we explore six method pairs for computing the steady state and sensitivities at steady state using six real-world problems. The method pairs involve numerical integration or Newton’s method to compute the steady-state, and—for both forward and adjoint sensitivity analysis—numerical integration or a tailored method to compute the sensitivities at steady-state. Our evaluation shows that all method pairs provide accurate steady-state and gradient values, and that the two method pairs that combine numerical integration for the steady-state with a tailored method for the sensitivities at steady-state were the most robust, and amongst the most computationally-efficient. We also observed that while Newton’s method for steady-state computation yields a substantial speedup compared to numerical integration, it may lead to a large number of simulation failures. Overall, our study provides a concise overview across current methods for computing sensitivities at steady state. While our study shows that there is no universally-best method pair, it also provides guidance to modelers in choosing the right methods for a problem at hand.},
+  creationdate     = {2025-01-06T14:49:28},
+  doi              = {10.1371/journal.pone.0312148},
+  modificationdate = {2025-01-06T14:49:48},
+  publisher        = {Public Library of Science},
+  url              = {https://doi.org/10.1371/journal.pone.0312148},
+}
+
+@Article{SmithMal2025,
+  author           = {Smith, Lucian and Malik-Sheriff, Rahuman S. and Nguyen, Tung V. N. and Hermjakob, Henning and Karr, Jonathan and Shaikh, Bilal and Drescher, Logan and Moraru, Ion I. and Schaff, James C. and Agmon, Eran and Patrie, Alexander A. and Blinov, Michael L. and Hellerstein, Joseph L. and May, Elebeoba E. and Nickerson, David P. and Gennari, John H. and Sauro, Herbert M.},
+  journal          = {bioRxiv},
+  title            = {Using {SED-ML} for reproducible curation: Verifying {BioModels} across multiple simulation engines},
+  year             = {2025},
+  abstract         = {The BioModels Repository contains over 1000 manually curated mechanistic models drawn from published literature, most of which are encoded in the Systems Biology Markup Language (SBML). This community-based standard formally specifies each model, but does not describe the computational experimental conditions to run a simulation. Therefore, it can be challenging to reproduce any given figure or result from a publication with an SBML model alone. The Simulation Experiment Description Markup Language (SED-ML) provides a solution: a standard way to specify exactly how to run a specific experiment that corresponds to a specific figure or result. BioModels was established years before SED-ML, and both systems evolved over time, both in content and acceptance. Hence, only about half of the entries in BioModels contained SED-ML files, and these files reflected the version of SED-ML that was available at the time. Additionally, almost all of these SED-ML files had at least one minor mistake that made them invalid. To make these models and their results more reproducible, we report here on our work updating, correcting and providing new SED-ML files for 1055 curated mechanistic models in BioModels. In addition, because SED-ML is implementation-independent, it can be used for verification, demonstrating that results hold across multiple simulation engines. Here, we use a wrapper architecture for interpreting SED-ML, and report verification results across five different ODE-based biosimulation engines. Our work with SED-ML and the BioModels collection aims to improve the utility of these models by making them more reproducible and credible.Author summary Reproducing computationally-derived scientific results seems like it should be straightforward, but is often elusive. Code is lost, file formats change, and knowledge of what was done is only partially recorded and/or forgotten. Model repositories such as BioModels address this failing in the Systems Biology domain by encoding models in a standard format that can reproduce a figure from the paper from which it was drawn. Here, we delved into the BioModels repository to ensure that every curated model additionally contained instructions on what to do with that model, and then tested those instructions on a variety of simulation platforms. Not only did this improve the BioModels repository itself, but also improved the infrastructure necessary to run these validation comparisons in the future.Author contributions LS: Writing, Conceptualization, Data Curation, Investigation, Methodology, Project Administration, Software, Validation. RMS: Reading, Writing, Data Curation, Methodology TN: Reading, Data Curation, Methodology HH: Reading JK: Conceptualization, Data Curation, Investigation, Methodology, Software. BS: Software LD: Software IIM: Reading, Conceptualization, Funding JCS: Software, Methodology EA: Reading, Writing AAP: Software MLB: Reading, Writing JH: Writing, Methodology EM: Reading, Writing DPN: Reading, Writing, Methodology JG: Reading, Writing, Methodology HMS: Reading, Writing, FundingCompeting Interest StatementThe authors have declared no competing interest.},
+  creationdate     = {2025-01-27T09:08:10},
+  doi              = {10.1101/2025.01.16.633337},
+  elocation-id     = {2025.01.16.633337},
+  eprint           = {https://www.biorxiv.org/content/early/2025/01/20/2025.01.16.633337.full.pdf},
+  modificationdate = {2025-01-27T09:08:41},
+  publisher        = {Cold Spring Harbor Laboratory},
+  url              = {https://www.biorxiv.org/content/early/2025/01/20/2025.01.16.633337},
+}
+
 @Comment{jabref-meta: databaseType:bibtex;}
 
 @Comment{jabref-meta: grouping:

documentation/references.md

@@ -1,6 +1,6 @@
 # References
 
-List of publications using AMICI. Total number is 94.
+List of publications using AMICI. Total number is 95.
 
 If you applied AMICI in your work and your publication is missing, please let us know via a new
 [GitHub issue](https://github.com/AMICI-dev/AMICI/issues/new?labels=documentation&title=Add+publication&body=AMICI+was+used+in+this+manuscript:+DOI).
@@ -12,6 +12,18 @@ If you applied AMICI in your work and your publication is missing, please let us
 }
 </style>
 
+<h1 class="unnumbered" id="section">2025</h1>
+<div id="refs" class="references csl-bib-body hanging-indent"
+role="list">
+<div id="ref-SmithMal2025" class="csl-entry" role="listitem">
+Smith, Lucian, Rahuman S. Malik-Sheriff, Tung V. N. Nguyen, Henning
+Hermjakob, Jonathan Karr, Bilal Shaikh, Logan Drescher, et al. 2025.
+<span>“Using <span>SED-ML</span> for Reproducible Curation: Verifying
+<span>BioModels</span> Across Multiple Simulation Engines.”</span>
+<em>bioRxiv</em>. <a
+href="https://doi.org/10.1101/2025.01.16.633337">https://doi.org/10.1101/2025.01.16.633337</a>.
+</div>
+</div>
 <h1 class="unnumbered" id="section">2024</h1>
 <div id="refs" class="references csl-bib-body hanging-indent"
 role="list">
@@ -72,9 +84,10 @@ href="https://doi.org/10.1093/nar/gkae123">https://doi.org/10.1093/nar/gkae123</
 </div>
 <div id="ref-LakrisenkoPat2024" class="csl-entry" role="listitem">
 Lakrisenko, Polina, Dilan Pathirana, Daniel Weindl, and Jan Hasenauer.
-2024. <span>“Exploration of Methods for Computing Sensitivities in ODE
-Models at Dynamic and Steady States.”</span> <a
-href="https://arxiv.org/abs/2405.16524">https://arxiv.org/abs/2405.16524</a>.
+2024. <span>“Benchmarking Methods for Computing Local Sensitivities in
+Ordinary Differential Equation Models at Dynamic and Steady
+States.”</span> <em>PLOS ONE</em> 19 (10): 1–19. <a
+href="https://doi.org/10.1371/journal.pone.0312148">https://doi.org/10.1371/journal.pone.0312148</a>.
 </div>
 <div id="ref-LangPen2024" class="csl-entry" role="listitem">
 Lang, Paul F., David R. Penas, Julio R. Banga, Daniel Weindl, and Bela

documentation/rtd_requirements.txt

@@ -5,6 +5,7 @@ setuptools>=67.7.2
 pysb>=1.11.0
 jax>=0.4.26
 diffrax>=0.5.0
+interpax>=0.3.4
 matplotlib==3.7.1
 nbsphinx==0.9.1
 nbformat==5.8.0

include/amici/defines.h

@@ -244,6 +244,7 @@ enum class RDataReporting {
     full,
     residuals,
     likelihood,
+    observables_likelihood,
 };
 
 /** boundary conditions for splines */

include/amici/rdata.h

@@ -476,6 +476,13 @@ class ReturnData : public ModelDimensions {
      */
     void initializeLikelihoodReporting(bool quadratic_llh);
 
+    /**
+     * @brief initializes storage for observables + likelihood reporting mode
+     * @param quadratic_llh whether model defines a quadratic nllh and computing
+     * res, sres and FIM makes sense.
+     */
+    void initializeObservablesLikelihoodReporting(bool quadratic_llh);
+
     /**
      * @brief initializes storage for residual reporting mode
      * @param enable_res whether residuals are to be computed

pytest.ini

@@ -24,5 +24,8 @@ filterwarnings =
     ignore:.*PyDevIPCompleter6.*:DeprecationWarning
     # ignore numpy log(0) warnings (np.log(0) = -inf)
     ignore:divide by zero encountered in log:RuntimeWarning
+    # ignore jax deprecation warnings
+    ignore:jax.* is deprecated:DeprecationWarning
+
 
 norecursedirs = .git amici_models build doc documentation matlab models ThirdParty amici sdist examples

python/examples/example_constant_species/ExampleEquilibrationLogic.ipynb

@@ -124,8 +124,7 @@
     "for reaction in sbml_model.getListOfReactions():\n",
     "    reactants = \" + \".join(\n",
     "        [\n",
-    "            \"%s %s\"\n",
-    "            % (\n",
+    "            \"{} {}\".format(\n",
     "                int(r.getStoichiometry()) if r.getStoichiometry() > 1 else \"\",\n",
     "                r.getSpecies(),\n",
     "            )\n",
@@ -134,8 +133,7 @@
     "    )\n",
     "    products = \" + \".join(\n",
     "        [\n",
-    "            \"%s %s\"\n",
-    "            % (\n",
+    "            \"{} {}\".format(\n",
     "                int(r.getStoichiometry()) if r.getStoichiometry() > 1 else \"\",\n",
     "                r.getSpecies(),\n",
     "            )\n",
@@ -144,7 +142,7 @@
     "    )\n",
     "    reversible = \"<\" if reaction.getReversible() else \"\"\n",
     "    print(\n",
-    "        \"%3s: %10s %1s->%10s\\t\\t[%s]\"\n",
+    "        \"%3s: %10s %1s->%10s\\t\\t[%s]\"  # noqa: UP031\n",
     "        % (\n",
     "            reaction.getId(),\n",
     "            reactants,\n",
@@ -370,7 +368,7 @@
     "\n",
     "# np.set_printoptions(threshold=8, edgeitems=2)\n",
     "for key, value in rdata.items():\n",
-    "    print(\"%12s: \" % key, value)"
+    "    print(f\"{key:12s}: \", value)"
    ]
   },
   {
@@ -664,15 +662,15 @@
     "Since the Eigenvalues of the Jacobian are negative and since the Jacobian at steady state is a fixed matrix, this system has a simple algebraic solution:\n",
     "\n",
     "   $$p(t) = e^{t J(x^*, \\theta)^T} p_{\\text{end}}.$$\n",
-    "   \n",
+    "\n",
     "As a consequence, the quadratures in adjoint computation also reduce to a matrix-vector product:\n",
     "\n",
     "   $$Q(x, \\theta) = Q(x^*, \\theta) = p_{\\text{integral}} * \\frac{\\partial f}{\\partial \\theta}$$\n",
     "\n",
     "with\n",
     "\n",
-    "   $$p_{\\text{integral}} = \\int_0^\\infty p(s) ds = (J(x^*, \\theta)^T)^{-1} p_{\\text{end}}.$$ \n",
-    "   \n",
+    "   $$p_{\\text{integral}} = \\int_0^\\infty p(s) ds = (J(x^*, \\theta)^T)^{-1} p_{\\text{end}}.$$\n",
+    "\n",
     "However, this solution is given in terms of a linear system of equations defined by the transposed Jacobian of the right hand side. Hence, if the (transposed) Jacobian is singular, it is not applicable.\n",
     "In this case, standard integration must be carried out."
    ]
@@ -922,7 +920,7 @@
     "\n",
     "for key, value in rdata.items():\n",
     "    if key[0:6] == \"preeq_\":\n",
-    "        print(\"%20s: \" % key, value)"
+    "        print(f\"{key:20s}:\", value)"
    ]
   },
   {
@@ -960,7 +958,7 @@
     "\n",
     "for key, value in rdata.items():\n",
     "    if key[0:6] == \"preeq_\":\n",
-    "        print(\"%20s: \" % key, value)"
+    "        print(f\"{key:20s}:\", value)"
    ]
   },
   {
@@ -993,7 +991,7 @@
     "\n",
     "for key, value in rdata_reduced.items():\n",
     "    if key[0:6] == \"preeq_\":\n",
-    "        print(\"%20s: \" % key, value)"
+    "        print(f\"{key:20s}:\", value)"
    ]
   },
   {
@@ -1044,7 +1042,7 @@
     "\n",
     "for key, value in rdata_reduced.items():\n",
     "    if key[0:6] == \"preeq_\":\n",
-    "        print(\"%20s: \" % key, value)\n",
+    "        print(f\"{key:20s}:\", value)\n",
     "print(\"Gradient:\", rdata_reduced[\"sllh\"])"
    ]
   },
@@ -1079,7 +1077,7 @@
     "\n",
     "for key, value in rdata_reduced.items():\n",
     "    if key[0:6] == \"preeq_\":\n",
-    "        print(\"%20s: \" % key, value)\n",
+    "        print(f\"{key:20s}:\", value)\n",
     "print(\"Gradient:\", rdata_reduced[\"sllh\"])"
    ]
   },
@@ -1117,7 +1115,7 @@
     "\n",
     "for key, value in rdata_reduced.items():\n",
     "    if key[0:6] == \"preeq_\":\n",
-    "        print(\"%20s: \" % key, value)\n",
+    "        print(f\"{key:20s}:\", value)\n",
     "print(\"Gradient:\", rdata_reduced[\"sllh\"])"
    ]
   },
@@ -1160,7 +1158,7 @@
     "\n",
     "for key, value in rdata.items():\n",
     "    if key[0:6] == \"preeq_\":\n",
-    "        print(\"%20s: \" % key, value)\n",
+    "        print(f\"{key:20s}:\", value)\n",
     "print(\"Gradient:\", rdata[\"sllh\"])"
    ]
   },

python/examples/example_errors.ipynb

@@ -76,9 +76,7 @@
    "outputs": [],
    "source": [
     "petab_problem = benchmark_models_petab.get_problem(\"Fujita_SciSignal2010\")\n",
-    "amici_model = import_petab_problem(\n",
-    "    petab_problem, verbose=False, compile_=None\n",
-    ")\n",
+    "amici_model = import_petab_problem(petab_problem, verbose=False, compile_=None)\n",
     "\n",
     "np.random.seed(2991)\n",
     "problem_parameters = dict(\n",
@@ -422,9 +420,7 @@
    "outputs": [],
    "source": [
     "petab_problem = benchmark_models_petab.get_problem(\"Weber_BMC2015\")\n",
-    "amici_model = import_petab_problem(\n",
-    "    petab_problem, verbose=False, compile_=None\n",
-    ")\n",
+    "amici_model = import_petab_problem(petab_problem, verbose=False, compile_=None)\n",
     "\n",
     "np.random.seed(4)\n",
     "problem_parameters = dict(\n",