Why output shows only 5 record. While test data has a lot rows about flask_api

[pratos]

I've just used 5 records, below is the code where slice those 5 records:

"""Reading test batch
"""
df = pd.read_csv('../data/test.csv', encoding="utf-8-sig")
df = df.head()

I'd advise you to run through the notebook thoroughly. Hope this helps 😃

[sohamnavadiya]

Thank you for correct direction, Have receiving error:

Received error:

Traceback (most recent call last):
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/soham/Documents/flask_api/flask_api/server.py", line 40, in apicall
    predictions = loaded_model.predict(test)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/utils/metaestimators.py", line 115, in <lambda>
    out = lambda *args, **kwargs: self.fn(obj, *args, **kwargs)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/model_selection/_search.py", line 466, in predict
    return self.best_estimator_.predict(X)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/utils/metaestimators.py", line 115, in <lambda>
    out = lambda *args, **kwargs: self.fn(obj, *args, **kwargs)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/pipeline.py", line 316, in predict
    return self.steps[-1][-1].predict(Xt)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/ensemble/forest.py", line 536, in predict
    proba = self.predict_proba(X)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/ensemble/forest.py", line 576, in predict_proba
    X = self._validate_X_predict(X)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/ensemble/forest.py", line 356, in _validate_X_predict
    return self.estimators_[0]._validate_X_predict(X, check_input=True)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/tree/tree.py", line 373, in _validate_X_predict
    X = check_array(X, dtype=DTYPE, accept_sparse="csr")
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/utils/validation.py", line 422, in check_array
    _assert_all_finite(array)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/utils/validation.py", line 43, in _assert_all_finite
    " or a value too large for %r." % X.dtype)
ValueError: Input contains NaN, infinity or a value too large for dtype('float32').

[pratos]

There are 2 things you can do so that I may help out:

1. Can you run the code that is in this repository and then try to call the server?
2. If the above isn't working, can you list down the steps you followed upto the about point (till the error)?

Regards,
Prathamesh

[sohamnavadiya]

1. I have latest code from master branch.

2. Following are the steps:

To run the server:

soham@soham:~/Documents/flask_api$ git remote -v
origin	https://github.com/pratos/flask_api.git (fetch)
origin	https://github.com/pratos/flask_api.git (push)

soham@soham:~/Documents/flask_api$ source activate flask-api
Could not find conda environment: flask-api
You can list all discoverable environments with `conda info --envs`.

soham@soham:~/Documents/flask_api$ source activate flask_api
(flask_api) soham@soham:~/Documents/flask_api$ ls
data  flask_api  LICENSE  notebooks  README.md

(flask_api) soham@soham:~/Documents/flask_api$ cd flask_api/

(flask_api) soham@soham:~/Documents/flask_api/flask_api$ l

call.py  flask_api.yml  hello-world.py  __init__.py  models/  __pycache__/  readme.txt  requirements.txt  server.py  utils.py

(flask_api) soham@soham:~/Documents/flask_api/flask_api$ gunicorn --bind 0.0.0.0:5000 server:app
[2018-06-20 16:43:54 +0530] [14959] [INFO] Starting gunicorn 19.7.1
[2018-06-20 16:43:54 +0530] [14959] [INFO] Listening at: http://0.0.0.0:5000 (14959)
[2018-06-20 16:43:54 +0530] [14959] [INFO] Using worker: sync
[2018-06-20 16:43:54 +0530] [14962] [INFO] Booting worker with pid: 14962

In new tab, I executed server.py for create model file

soham@soham:~/Documents/flask_api/flask_api$ source activate flask_api
(flask_api) soham@soham:~/Documents/flask_api/flask_api$ ls
call.py  flask_api.yml  hello-world.py  __init__.py  models  __pycache__  readme.txt  requirements.txt  server.py  utils.py
(flask_api) soham@soham:~/Documents/flask_api/flask_api$ python3 server.py 
(flask_api) soham@soham:~/Documents/flask_api/flask_api$ 

Here is my call.py which I created for predict the data.

(flask_api) soham@soham:~/Documents/flask_api/flask_api$ cat call.py 
import json
import requests
import pandas as pd
"""Setting the headers to send and accept json responses
"""
header = {'Content-Type': 'application/json', \
                  'Accept': 'application/json'}

"""Reading test batch
"""
df = pd.read_csv('../data/test.csv', encoding="utf-8-sig")
# df = df.head()

"""Converting Pandas Dataframe to json
"""
data = df.to_json(orient='records')

# print(data)

"""POST <url>/predict
"""
try:
	resp = requests.post("http://0.0.0.0:5000/predict", \
                    data = json.dumps(data),\
                    headers= header)

except Exception as e:
	print(e)

print(resp.status_code)

"""The final response we get is as follows:
"""
print(resp.json())

Here is the response of execution of error:

(flask_api) soham@soham:~/Documents/flask_api/flask_api$ python3 call.py 
500
Traceback (most recent call last):
  File "call.py", line 34, in <module>
    print(resp.json())
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/requests/models.py", line 892, in json
    return complexjson.loads(self.text, **kwargs)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[sohamnavadiya]

Here is the test.csv file.
test.csv.zip

[sohamnavadiya]

Reponse from server

Loading the model...
The model has been loaded...doing predictions now...
[2018-06-20 17:00:15,720] ERROR in app: Exception on /predict [POST]
Traceback (most recent call last):
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/soham/Documents/flask_api/flask_api/server.py", line 40, in apicall
    predictions = loaded_model.predict(test)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/utils/metaestimators.py", line 115, in <lambda>
    out = lambda *args, **kwargs: self.fn(obj, *args, **kwargs)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/model_selection/_search.py", line 466, in predict
    return self.best_estimator_.predict(X)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/utils/metaestimators.py", line 115, in <lambda>
    out = lambda *args, **kwargs: self.fn(obj, *args, **kwargs)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/pipeline.py", line 316, in predict
    return self.steps[-1][-1].predict(Xt)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/ensemble/forest.py", line 536, in predict
    proba = self.predict_proba(X)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/ensemble/forest.py", line 576, in predict_proba
    X = self._validate_X_predict(X)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/ensemble/forest.py", line 356, in _validate_X_predict
    return self.estimators_[0]._validate_X_predict(X, check_input=True)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/tree/tree.py", line 373, in _validate_X_predict
    X = check_array(X, dtype=DTYPE, accept_sparse="csr")
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/utils/validation.py", line 422, in check_array
    _assert_all_finite(array)
  File "/home/soham/anaconda3/envs/flask_api/lib/python3.6/site-packages/sklearn/utils/validation.py", line 43, in _assert_all_finite
    " or a value too large for %r." % X.dtype)
ValueError: Input contains NaN, infinity or a value too large for dtype('float32').

[sohamnavadiya]

the issue has been resolved, The issue was there because of blank value in data.csv.

By the way, thank you so much for sharing such a nice example of machine learning. If you have any other examples like this then please share with me.