Failed biomodels

[luciansmith]

Attached are a list of amici failures on the current suite of biomodels from https://github.com/sys-bio/temp-biomodels/tree/main/omex_files

Also included is the list of tellurium failures, because anything that fails on tellurium is probably also going to fail on amici, for the reasons listed.

tellurium_failures.xlsx
amici_failures.csv

[dweindl]

Thanks for reporting, @luciansmith.

I only checked a handful of results so far, but those were all:

1. XPath issues (e.g. target /sbml:sbml/sbml:model/sbml:listOfParameters/sbml:parameter[@id='A0'] is not a valid XPath to an attribute of a model element), unrelated to amici itself. Since tellurium seems to not have those issues, it's probably something in here.
2. Unsupported SBML features / extensions. Is there a way to differentiate between expected and unexpected failures in biosimulators?

It will probably take me a couple of weeks before I can have a closer look.

[luciansmith]

I got these from running stuff from the command line in batches, so didn't have access to the logs. If the log message accurately conveys what's going on, I think that's sufficient. Tellurium has the same issue, with unsupported features:

https://run.biosimulations.org/runs/64e53123548ceb84f3ad7916#tab=log

or not supporting 'qual':

https://run.biosimulations.org/runs/64e5319b548ceb84f3ad843c#tab=log

The XPath issue is a special case. I solved it for Biosimulators_tellurium by using the 'presimulated_task'. The basic problem is that when SED-ML describes a change to the XML of a model, biosimulators_utils can manipulate the SBML document directly and re-feed the resulting document to the simulator. But in many cases, a change needs to be made to the model state, not the model document, so the biosimulators_XXXX code needs to tell the simulator to make the change, instead of making the change to the XML. These are all changes along the lines of 'set S1 to 5' or the like, and are often used for parameter scans.

For Biosimulators_tellurium, the 'roadrunner' simulator was being stored in the 'preprocessed_task' object, so there are now new routines in biosimulators_utils that will essentially ask the biosimulators_XXXX code to set a value, if it can't set it itself, by passing the preprocessed_task object around. If that still doesn't work, it'll just throw a 'not implemented error'. So if Amici indeed cannot change a value to the model state, we only need to worry about returning a reasonable error message. But if it can, I'm happy to walk you through what I did for Tellurium, to see if something similar could be done for Amici.

[dweindl]

Thanks for the explanation, Lucian. Sounds like this should all be doable in AMICI. We'll get back to you in case of further questions.

[luciansmith]

So, @CodeByDrescher has been working on updating the Copasi wrapper to accept more SED-ML variations, and might be able to update the Amici wrapper to incorporate the same fixes. For comparison, here's a complete set of Amici runs on BioModels omex files before any fixes have been applied:

amici_compare.csv