Load a model from cad manifest file

[pcrespov]

• cad manifest file https://github.com/ITISFoundation/hornet-manifest-spec
• git checkout shallow a repo
• load manifest, find files in repo and load cad model
• for now, no simulation
• integrate in osparc modeling service

[mguidon]

This is the proposal I just sent out: @ignapas @esraneufeld @pcrespov

We have drafted a proposal for a JSON Schema–based manifest file that contains all the necessary information to enable CAD visualization in oSPARC. I think this was discussed an approved by K-CORE. This manifest will live in the original GitHub repository at a predefined path, most likely .hornet/*manifest.json.
To import these CAD files into oSPARC, we will need two things: the URL to the GitHub repository and a reference to a release/tag or commit hash. With that, we can locate the hornet manifest and fetch the CAD models for import.
I just took a closer look at this example device, and it seems that the relevant metadata is already available in the manifest.json generated during the curation process. It's also publicly accessible from api.pennsieve.io via a URL that is easy to derive.
Assuming that holds, the steps forward would be:

• Add an “Import in oSPARC” button to the Dataset Landing Page (wire-frame needed?)
• On the oSPARC side: parse the hornet manifest, download the required files, and launch the visualization service.

[ignapas]

Requirements after discussion with Pedro:

• This is only affecting datasets under "Devices" https://staging.sparc.science/data?type=device
• The dataset's metadata file contains a property pointing to a repository containing the .hornet/*manifest.json which itself points to the CAD models in that same repository. See the property release in that metadata file.
• IMPORTANT QUESTION: is the presence of that release property in the dataset's metadata enough to conclude that there will be some .hornet/manifest.json in the repository? Should it also have a release.hasHornet: boolean property or be under hornetRepository instead of just release? This is necessary to easily let the frontend know that some button "Open HORNET models in oSPARC" needs to be rendered.
• Once the user clicks the button: the /view endpoint is used from the portal, specifying the metadata's URL as downloadLink. The other parameters of the endpoint are currently required but don't necessarily make sense, so an appropriate value for them needs to be decided by @pcrespov
• oSPARC downloads the metadata file and parses it, checkouts the repo, opens the manifest file and parses it, downloads the models (check Github's download restrictions), creates a study with the model viewer and feeds it with the models for visualization.

From SPARC's perspective there are also some requirements:

• Dataset's metadata contains the URL of the repo containing the .hornet/manifest.json file and the models.
• In the metadata, consider adding a hasHornet flag or a property named differently the indicate the special behavior.
• The target repository needs to contain a valid .hornet/manifest.json file and the models. This should happen ideally before the dataset is published.
• Frontend changes:
  • Checking of the metadata file
  • Adding a dedicated button (all available models will be opened)
  • Calling oSPARC's /view endpoint with the metadata download link

[pcrespov]

After NIH hornet meeting. Next steps

• Chris (and Alex if possible) to forward datasets once cleaned up to Esra and Pedro
• SIM-Core/ Pedro to work with Yomi regarding tickets to be submitted to the CCB to outline the changes that will need to be made to the Source Code tab of the Device Dataset Landing page such that CAD files can be opened in o2S2PARC directly from the Portal
• DAT-Core/ Chris/ Alex to publish the datasets as test datasets to the Portal