TorchServe failing to batch multi-image requests about serve

[mycpuorg]

@harshbafna this should really be working. Can we please look into this?

[harshbafna]

The issue is with the curl command which sends the images in two separate requests.

curl -v -X POST http://127.0.0.1:8080/predictions/resnet-152-batch -T "{kitten.jpg,dog.jpg}"
*   Trying 127.0.0.1:8080...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> POST /predictions/resnet-152-batch HTTP/1.1
> Host: 127.0.0.1:8080
> User-Agent: curl/7.65.3
> Accept: */*
> Content-Length: 110969
> Expect: 100-continue
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 
< x-request-id: 4fb625c8-cfcc-45e4-a389-73c39007fb74
< Pragma: no-cache
< Cache-Control: no-cache; no-store, must-revalidate, private
< Expires: Thu, 01 Jan 1970 00:00:00 UTC
< content-length: 32
< connection: keep-alive
< 
[
  "n02123159",
  "tiger_cat"
* Connection #0 to host 127.0.0.1 left intact
]* Found bundle for host 127.0.0.1: 0x7f8db9f00320 [serially]
* Re-using existing connection! (#0) with host 127.0.0.1
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> POST /predictions/resnet-152-batch HTTP/1.1
> Host: 127.0.0.1:8080
> User-Agent: curl/7.65.3
> Accept: */*
> Content-Length: 78949
> Expect: 100-continue
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 
< x-request-id: 0f46f9e1-eebe-4fe6-8689-64b73f5b8ed6
< Pragma: no-cache
< Cache-Control: no-cache; no-store, must-revalidate, private
< Expires: Thu, 01 Jan 1970 00:00:00 UTC
< content-length: 41
< connection: keep-alive
< 
[
  "n02099712",
  "Labrador_retriever"
* Connection #0 to host 127.0.0.1 left intact
]

Also note that the default handlers do not support batch inferencing. We have already added an example for batch inferencing using resnet152 model in fix for #66