fix: anchor in pytorch 1.6 (#4)

* fix: anchor in pytorch 1.6

* fix: fastcore version for fastai<2.1

* chore: dockerfile reformatting

* add: address first API call latency

Author: sunbc0120

Dockerfile

@@ -1,4 +1,4 @@
-FROM pytorch/pytorch
+FROM pytorch/pytorch:1.6.0-cuda10.1-cudnn7-runtime
 
 ENV PYTHONUNBUFFERED TRUE
 
@@ -40,8 +40,8 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
     && rm -rf /var/lib/apt/lists/*
 
 # FASTAI
-RUN git clone https://github.com/fastai/fastai.git --depth 1  && git clone https://github.com/fastai/fastcore.git --depth 1
-RUN /bin/bash -c "cd fastai && pip install . && cd ../fastcore && pip install ."
+RUN git clone https://github.com/fastai/fastai.git && git clone https://github.com/fastai/fastcore.git
+RUN /bin/bash -c "cd fastai && git checkout 2.0.18 && pip install . && cd ../fastcore && git checkout 1.1.0 && pip install ."
 
 # TORCHSERVE
 RUN git clone https://github.com/pytorch/serve.git
@@ -53,7 +53,6 @@ RUN chmod +x /usr/local/bin/dockerd-entrypoint.sh
 RUN mkdir -p /home/model-server/ && mkdir -p /home/model-server/tmp
 COPY ./deployment/config.properties /home/model-server/config.properties
 
-
 WORKDIR /home/model-server
 ENV TEMP=/home/model-server/tmp
 ENTRYPOINT ["/usr/local/bin/dockerd-entrypoint.sh"]

README.md

@@ -515,7 +515,7 @@ user    0m0.280s
 sys     0m0.039s
 ```
 
-For more details about TorchServe setup and usage, please refere to `notebook/03_TorchServe.ipynb` [[link](notebook/03_TorchServe.ipynb)].
+The first call would have longer latency due to model weights loading defined in `initialize`, but this will be moderated from the second call onward. For more details about TorchServe setup and usage, please refere to `notebook/03_TorchServe.ipynb` [[link](notebook/03_TorchServe.ipynb)].
 
 ## Deployment to Amazon SageMaker Inference Endpoint
 

deployment/handler.py

@@ -33,6 +33,8 @@ def initialize(self, ctx):
             if torch.cuda.is_available()
             else "cpu"
         )
+
+        logger.info(f"Device on initialization is: {self.device}")
         model_dir = properties.get("model_dir")
 
         manifest = ctx.manifest
@@ -84,6 +86,7 @@ def inference(self, img):
         """
         Predict the chip stack mask of an image using a trained deep learning model.
         """
+        logger.info(f"Device on inference is: {self.device}")
         self.model.eval()
         inputs = Variable(img).to(self.device)
         outputs = self.model.forward(inputs)