v0.8

Author: tripathiarpan20

HF-only/README.md

@@ -141,10 +141,11 @@ The high level approach to use any of the 🤗 pipelines for a desired [*task*](
 
 * Decide a name (variable `$modelName` below) for the task (variable `$task` below) supported by a 🤗 model that supports the task (its repo link as variable `$repoUrl` below), then run the below commands to download the models and create the `.mar` file:
 ```
-$modelName="vitxxsmall"
-$task="image-classification"
-$repoUrl="https://huggingface.co/apple/mobilevit-xx-small/tree/main"
+modelName="vitxxsmall"
+task="image-classification"
+repoUrl="https://huggingface.co/apple/mobilevit-xx-small"
 
+chmod +x prepare_mar_from_hf.sh 
 ./prepare_mar_from_hf.sh -t $task -n $modelName -u $repoUrl
 ```
 

HF-only/prepare_mar_from_hf.sh

@@ -5,12 +5,13 @@ for arg in "$@"
 do
     case $arg in
         -t|--task)
-            sed -i "s/task=\".*/task=\"$2\"/g" HF-only/scripts/torchserve_vitxxsmall_handler.py
-            shift
+            sed -i "s/task=\".*/task=\"$2\"/g" scripts/torchserve_vitxxsmall_handler.py
+            shift 2
             ;;
         -n|--model-name)
             MODEL_NAME="$2"
-            shift
+            echo "MODEL_NAME is $MODEL_NAME"
+            shift 2
             ;;
         -u|--hf-hub-link)
             HF_REPO_URL="$2"
@@ -23,7 +24,7 @@ do
             touch dummy_file.pth
             torch-model-archiver --model-name $MODEL_NAME --serialized-file dummy_file.pth --version 1.0 --handler scripts/torchserve_vitxxsmall_handler.py --export-path model-store -r requirements.txt
             rm -f dummy_file.pth
-            shift
+            shift 2
             ;;
     esac
 done

README.md

@@ -4,6 +4,8 @@ This repository contains an example to deploy models with third-party dependenci
 
 For the context of this repository, we would deploy the models on an AWS [`t2.micro`](https://aws.amazon.com/ec2/instance-types/) instance which can be used for free (for 750 hours) on a new AWS account. We work with a 🤗 MobileViT Transformer [model](https://huggingface.co/apple/mobilevit-xx-small) for the task of image classification by using its [`pipeline`](https://huggingface.co/docs/transformers/main_classes/pipelines) feature, the handler code in `scripts` can also be used as a simplistic template to deploy an 🤗 `pipeline`.
 
+This work can also be [extended to deploy *any* 🤗 `pipeline` for *any* supported task with Torchserve](https://github.com/tripathiarpan20/HF-torchserve-pipeline/tree/main/HF-only#instructions-to-use-any--model-from-the-hub-for-any-task-supported-by-the--pipeline).
+
 This work *may* also be extended to deploy the Torchserve Docker containers with HF models at scale with [AWS Cloudformation](https://github.com/pytorch/serve/tree/master/examples/cloudformation) & [AWS EKS](https://github.com/pytorch/serve/tree/master/kubernetes/EKS) as explained in the official Torchserve repo & [AWS Sagemaker](https://github.com/tescal2/TorchServeOnAWS/tree/master/3_torchserve_byoc_with_amazon_sagemaker), incorporating utilities like AWS ELB & Cloudwatch.
 
 We would also benchmark the REST API calls in time units and compare the model performances for the following approaches: 
@@ -12,7 +14,7 @@ We would also benchmark the REST API calls in time units and compare the model p
 
 
 ## Todo
-- [ ] Verify HF pipeline functionality with  AWS EC2 `t2.micro`
+- [X] Verify HF pipeline functionality with  AWS EC2 `t2.micro`
 - [ ] Add [AB Testing](https://github.com/pytorch/serve/tree/master/benchmarks) experiments with Torchserve Benchmarking utilities.
 - [ ] Add dynamic batching explanation
 - [ ] Add Torchscripted model code