sidewalk-ai-api

This serves as an API implementation of the Project Sidewalk AI models for tagging and validating labels.

This README contains the basic steps for setting up the API, but for a detailed usage guide on using the API, please see API_USAGE.md

Requirements

• Docker

• NVIDIA GPU drivers

  • If you aren't sure whether you have these, check if the nvidia-smi command works

• NVIDIA Container Toolkit for Docker

  • https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

  • Don't forget this step!!

• A good amount of disk space - the docker image is quite large

• Lots of VRAM (9-10 GB MINIMUM)

After cloning, don't forget to download the submodules

git submodule init
git submodule update

Download checkpoints

./download_models.sh

Build

docker  build  -t  sidewalk-ai-api  .

Run

docker  run  --gpus  all  --runtime  nvidia  -d  -p  5000:5000  sidewalk-ai-api

Try an example

curl -X POST -F "label_type=curbramp" -F "panorama_id=3-WpZU8MDYwe_9edeLw30w" -F "x=0.18981933593" -F "y=0.63134765625" http://127.0.0.1:5000/process

Please note that x and y are normalized coordinates (between 0 and 1) on the equirectangular image.

{"label_type":"curbramp","tag_scores":{"missing-tactile-warning":0.9999904632568359,"narrow":2.0490285805863095e-06,"not-enough-landing-space":9.689169928606134e-07,"not-level-with-street":2.722915048991581e-08,"parallel-lines":8.671155690381078e-18,"points-into-traffic":0.0051573594100773335,"pooled-water":2.6774074513014057e-07,"steep":1.7215750176546862e-06,"surface-problem":4.324794744547944e-08,"tactile-warning":1.704332674989928e-07},"tags":["missing-tactile-warning"],"validation_estimated_accuracy":0.941747572815534,"validation_result":"correct","validation_score":0.9998575448989868}

Please note that tag_scores and tags will not be returned if the label_type does not match the following: ["crosswalk", "curbramp", "obstacle", "surfaceproblem"]

Likewise, validation_result, validation_score, and validation_estimated_accuracy will not be returned if the label_type does not match the following: ["crosswalk", "curbramp", "obstacle", "surfaceproblem", "nocurbramp"]

This is because we do not have models for other label types yet.

For a more detailed guide on using the API, please see API_USAGE.md