This code is provided for research and development use only. This code is not intended for use in clinical decision-making or for any other clinical use and the performance of the code for clinical use has not been established. This source code requires selection of a reference and test datasets by the user. Microsoft does not warrant the suitability or accuracy of any predictive model generated using this source code. You bear sole responsibility for selection of a training dataset and for evaluation and use of any resulting model
Make sure you have a working AzureML Workspace, you have uploaded and created a dataset from the PadChest data, and uploaded our pretrained classifer and VAE (or trained your own). If not, please complete our setup tutorial.
You will need to generate VAE vectors and predicted probabilities for each of the images in the PadChest data.
Please follow these guides:
Note: Our VAE and predictions are available upon request.
Once you have vectors and predicted probabilities you will need to calculate individual metrics using that data.
To this, you must create a result dataset in AzureML and to place the jsonl files, and PadChest metadata csv, then run a script to calculate metrics for PadChest.
Please follow this guide:
- Calculating individual drift metrics: Notebook
Note: Our individual metrics are provided as part of this release.
After you have generated individual metrics, we use these to generate our unified metric, . Please see this guide:
- Calculate and plot
: Notebook