|
1 | 1 | # SageMaker Model Inference |
2 | 2 |
|
3 | | -This is a small example about how you can use LocalStack to host your PyTorch ML models. |
| 3 | +This is a small example about how you can use LocalStack to host your PyTorch ML models. It does the following: |
| 4 | + |
| 5 | +* Create MNIST model in SageMaker |
| 6 | +* Create a SageMaker Endpoint for accessing the model |
| 7 | +* Invoke the endpoint |
| 8 | + * directly on the container |
| 9 | + * via boto |
| 10 | + |
| 11 | +## Requirements |
| 12 | + |
| 13 | +* Python 3.8+ |
| 14 | + * boto3 |
| 15 | + * numpy |
| 16 | + * mypy |
| 17 | +* LocalStack |
| 18 | +* Docker |
| 19 | + |
| 20 | +## How To |
| 21 | + |
| 22 | +### Obtain Deep Learning image |
4 | 23 |
|
5 | 24 | Before using this example you should set up your Docker Client to pull the AWS Deep Learning images ([more info here](https://github.com/aws/deep-learning-containers/blob/master/available_images.md)): |
6 | 25 |
|
7 | 26 | ```bash |
8 | | - aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 763104351884.dkr.ecr.us-east-1.amazonaws.com |
| 27 | +aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 763104351884.dkr.ecr.us-east-1.amazonaws.com |
9 | 28 | ``` |
10 | 29 |
|
11 | | -Because the images tend to be heavy (multiple GB), you might want to `docker pull` them beforehand. |
| 30 | +Because the images tend to be heavy (multiple GB), you might want to `docker pull` them beforehand: |
| 31 | + |
| 32 | +```bash |
| 33 | +docker pull 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:1.5.0-cpu-py3 |
| 34 | +``` |
| 35 | + |
| 36 | +### Test the application |
| 37 | + |
| 38 | +Afterwards you can start localstack: |
| 39 | + |
| 40 | +```bash |
| 41 | +localstack start |
| 42 | +``` |
| 43 | + |
| 44 | +And execute the example with: |
| 45 | + |
| 46 | +```bash |
| 47 | +python main.py |
| 48 | +``` |
| 49 | + |
| 50 | +You should see an output like this: |
| 51 | +``` |
| 52 | +Creating bucket... |
| 53 | +Uploading model data to bucket... |
| 54 | +Creating model in SageMaker... |
| 55 | +Adding endpoint configuration... |
| 56 | +Creating endpoint... |
| 57 | +Checking endpoint status... |
| 58 | +Endpoint not ready - waiting... |
| 59 | +Checking endpoint status... |
| 60 | +Endpoint ready! |
| 61 | +Invoking via boto... |
| 62 | +Predicted digits: [7, 3] |
| 63 | +Invoking endpoint directly... |
| 64 | +Predicted digits: [2, 6] |
| 65 | +``` |
0 commit comments