ai4os / deepaas

A REST API to serve machine learning and deep learning models

DEEPaaS

DEEP as a Service API (DEEPaaS API) is a REST API built on aiohttp that allows to provide easy access to machine learning, deep learning and artificial intelligence models. By using the DEEPaaS API users can easily run a REST API in front of their model, thus accessing its functionality via HTTP calls. DEEPaaS API leverages the OpenAPI specification.

Documentation

The DEEPaaS documentation is hosted on Read the Docs.

Quickstart

The best way to quickly try the DEEPaaS API is through:

make run

This command will install a virtualenv (in the virtualenv directory) with DEEPaaS and all its dependencies and will run the DEEPaaS REST API, listening on 127.0.0.1:5000. If you browse to http://127.0.0.1:5000 you will get the Swagger documentation page (i.e. the Swagger web UI).

Develop mode

If you want to run the code in develop mode (i.e. pip install -e), you can issue the following command before:

make develop

Citing

If you are using this software and want to cite it in any work, please use the following:

Lopez Garcia, A. "DEEPaaS API: a REST API for Machine Learning and Deep Learning models". In: Journal of Open Source Software 4(42) (2019), pp. 1517. ISSN: 2475-9066. DOI: 10.21105/joss.01517

You can also use the following BibTeX entry:

@article{Lopez2019DEEPaaS,
    journal = {Journal of Open Source Software},
    doi = {10.21105/joss.01517},
    issn = {2475-9066},
    number = {42},
    publisher = {The Open Journal},
    title = {DEEPaaS API: a REST API for Machine Learning and Deep Learning models},
    url = {http://dx.doi.org/10.21105/joss.01517},
    volume = {4},
    author = {L{\'o}pez Garc{\'i}a, {\'A}lvaro},
    pages = {1517},
    date = {2019-10-25},
    year = {2019},
    month = {10},
    day = {25},}

Acknowledgements

This software has been developed within the DEEP-Hybrid-DataCloud (Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud) project that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 777435.

deepaas's Issues

Integrate with a good UI tool (Gradio)

Change Swagger URL path from /ui to /docs or /api

Description

A few users got confused by the "/ui" URL path for the Swagger interface, and understanding it as a full-features GUI. Since Swagger is still the API, not GUI, we better update the URL to something more conventional like "/docs" or "/api" (latter would be my preferred :-) )

Allow to load non installable models

Description

After using the package for a year and helping a few users, I find that one of the most confusing issues for users is the concept of application project as installable.

Basic Python users do not understand most of the consequences of installing with pip install . , e.g. how imports work with relative paths, models not being copied to the installation folder (unless include_package_data). And even more complicated to understand is pip install -e . (editable mode). For example, tox does not support -e editable_mode installation, and some of the complaints come in that tox fails because it cannot find package metadata (e.g. by using skipsdist without relying that you need to install).

In addition, other applications use submodules or dependencies that are not in PyPI. This also complicates things exponentially.

Expected behavior:

I think DEEPaaS needs to rethink how the project interface works. I understand that previously the installation was required to support multiple models at the same time. If this is no longer the case (and even if it is still the case, there may be better approaches), perhaps it should be oriented towards the way other web servers work. For example, FastAPI or Flask do not require installation, but rather configuration of a "server" object instance. See https://fastapi.tiangolo.com/#example

Unless projects are intended to be distributed via PyPI, distribution and installation make things unnecessarily complicated for users.

Actual behavior

Instllation with "pip install -e .".

Versions

Version: 2.1.0

Other comments

One constrain of non installing the project, is that metadata from the "package" is not accesible via importlib.metadata. However, it is still under discussion between users, which metadata should be deliver, each case is special. Probably this information should just be computed in the "api.get_metadata" method.

Add codemeta.json and CITATION.cff

Add support for bearer authentication

Add "conventional commits" specification

Some options for configuration files are not working

Some options from: configurations are not working. For example:

Config file:

[DEFAULT]
listen_ip = 0.0.0.0
listen_port = 5000
debug_endpoint = false
workers = 1
client_max_size = 0
warm = true
debug = true
#log_config_append = <None>
#log_date_format = %Y-%m-%d %H:%M:%S
#log_dir = <None>
#watch_log_file = false
use_syslog = false
#use_journal = false
#syslog_log_facility = LOG_USER
use_json = false
use_stderr = true
use_eventlog = false
#log_rotate_interval = 1
#log_rotate_interval_type = days
#max_logfile_count = 30
#max_logfile_size_mb = 200
#log_rotation_type = none
default_log_levels = websocket=DEBUG,requests=DEBUG
publish_errors = true
rate_limit_interval = 0
rate_limit_except_level = CRITICAL
fatal_deprecations = false

Error message:

$  cd /home/borja/projects/demo-advanced-api ; /usr/bin/env /home/borja/miniconda3/envs/DEEPaaS/bin/python /home/borja/.vscode-server/extensions/ms-python.python-2023.20.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher 57883 -- -m deepaas.cmd.run --config-file=deepaas.conf --debug 
Traceback (most recent call last):
  File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/borja/.vscode-server/extensions/ms-python.python-2023.20.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/borja/.vscode-server/extensions/ms-python.python-2023.20.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/borja/.vscode-server/extensions/ms-python.python-2023.20.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 317, in run_module
    run_module_as_main(options.target, alter_argv=True)
  File "/home/borja/.vscode-server/extensions/ms-python.python-2023.20.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 238, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/borja/.vscode-server/extensions/ms-python.python-2023.20.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/deepaas/cmd/run.py", line 124, in <module>
    main()
  File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/deepaas/cmd/run.py", line 96, in main
    config.config_and_logging(sys.argv)
  File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/deepaas/config.py", line 81, in config_and_logging
    log.setup(cfg.CONF, "deepaas")
  File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/oslo_log/log.py", line 273, in setup
    _setup_logging_from_conf(conf, product_name, version)
  File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/oslo_log/log.py", line 412, in _setup_logging_from_conf
    handler = importutils.import_object(
  File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/oslo_utils/importutils.py", line 44, in import_object
    return import_class(import_str)(*args, **kwargs)
  File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/oslo_utils/importutils.py", line 30, in import_class
    __import__(mod_str)
ModuleNotFoundError: No module named 'oslo_messaging'

Additional info

Maybe missing oslo_messaging from requirements at deepaas?

Add support for energy consumption estimation

I would be nice to support an energy consumption estimator like Perun, that wraps deepaas methods. Another option is Codecarbon but it gives the carbon estimation instead of the energy, so it gives us less flexibility to compute carbon emissions.

Support for asynchronous training

Allow to perform asynchronous training.

Add deprecation warning when various models are present

URL prediction error

Description

Deepaas returns an error when trying to predict an url after having predicted a file.
If this same url is being predicted at the beginning, just after having launched deepaas (and before having predicted a local file), the prediction runs just fine.

Steps to Reproduce

This error has been found while using the image classification model but this should be model independent as it is a deepaas error.

1. Launch deepaas
2. Try to predict a local file. You should get a json dict with your prediction.
3. Try to predict a URL.You get the following error:

400Undocumented | Error: BAD REQUEST
Response bodyDownload{ "message": "You must provide either 'url' or 'data' in the payload" }

Expected behavior:

If the url corresponded to an image you should get a json dict with the prediction from your model.

Actual behavior

Got a Bad Request error instead,

Versions

deepaas 0.2.1.dev1

Remove and deprecate the loading of various models

More routes/methods to implement pre-post processing

Hi,
As you know, in most Deep Learning developments, preprocessing plays a crucial role in preparing the data for training. For example, in one of my developments where I want to add an API to an object detection model, image conversion in tfrecord is required. I would like to include this preprocessing step in my API. Users will have the option to perform this step themselves during model training. Those unfamiliar with this process can use this built-in preprocessing step to easily convert their data into tfrecord. I would like to see this method included in the API.

Another valuable feature would be an evaluation method. After users train a model on the new dataset, they may want to compare its performance to the older version of the model on a new test dataset using certain metrics they define. By integrating this method into the API, users can efficiently evaluate and compare the performance of the model on the test dataset.

Thank you for taking the time and effort to include these additions

predict-train or train-predict fails with tensorflow and GPU

Description

Doing sequence of predict-train or train-predict methods fails while using deepaas with Tensorflow on GPU. Executing predict-predict and/or train-train works.

Steps to Reproduce

1. If after deploying the container I start only predict, it works and I can repeat it.
2. If I start only training after deployment, it works. I can also repeat it.
3. However, if I first start predict and then train or vice versa, it fails.

Expected behavior:

That whatever order of executed functions, they work fine.

Actual behavior

predict-train or train-predict fails, could be Tensorflow specific. The reason seems to be that predict and train are two different processes in Linux. First started process occupies GPU and the second one simply has not enough GPU memory to perform the task.

Versions

DEEPaaS 1.0.1 and 1.2.0
Tensorflow 1.12.0 and 1.14.0
Nvidia driver 418.56 on one site and 440.33.01 on another

Deprecate `deepaas-predict`

Functionality relies on a method predict_data() that was deprecated long ago. deepaas-cli already supports that functionality.

Code located in deepaas/cmd/execute.py

Exception when running in Python3.8

On some Python versions replacing standard Pool to custom can raise error: AssertionError: group argument must be None for now.

This is somehow related with nipy/nipype#2754

Allow to load models in other ways apart from entrypoints

Request object missing in parameters?

Description

For use with flaat, the request object of aiohttp needs to be passed along as the first unnamed parameter.
I find this in my examples of flaat (see fix-aio-problems branch) but nowhere in deepaas api.

Steps to Reproduce

define a predict function like this:

@flaat.login_required() # Require only authorized people to do training
def predict(*args, **kwargs):
    
    logger.error("starting predict")
    print(F"predict: got {len(args)} args")
    for arg in args:
        print(F"    arg: {arg}")
    print(F"predict: got {len(kwargs)} kwargs")
    for arg in kwargs:
        print(F"    kwarg: {arg}: {kwargs[arg]}")

And obser the output. Expected is to see a line like:

    arg <Request GET /valid_user_no_args >
    ...

Versions

1.3.0

Bug with webargs 6.0.0

Description

When trying to predict an image with the DEEPaaS API an error arises saying that no files were read. We figured out that it is caused by the new webargs version, released yesterday.

Steps to Reproduce

1. Deployed the container.
2. Ran deepaas-run.
3. Tried to upload an image for prediction.

Expected behavior:

Return a json with the results.

Actual behavior

Got as a response:
{'files': ['Missing data for required field.']}

Jupyter:

2020-02-28 12:56:02.350 3571 ERROR aiohttp.server [-] Error handling request: aiohttp.http_exceptions.BadStatusLine: invalid HTTP method

2020-02-28 12:56:02.350 3571 ERROR aiohttp.server Traceback (most recent call last):

2020-02-28 12:56:02.350 3571 ERROR aiohttp.server File "/opt/conda/lib/python3.7/site-packages/aiohttp/web_protocol.py", line 275, in data_received

2020-02-28 12:56:02.350 3571 ERROR aiohttp.server messages, upgraded, tail = self._request_parser.feed_data(data)

2020-02-28 12:56:02.350 3571 ERROR aiohttp.server File "aiohttp/_http_parser.pyx", line 523, in aiohttp._http_parser.HttpParser.feed_data

2020-02-28 12:56:02.350 3571 ERROR aiohttp.server aiohttp.http_exceptions.BadStatusLine: invalid HTTP method

2020-02-28 12:56:02.350 3571 ERROR aiohttp.server

2020-02-28 12:56:47.433 3571 INFO aiohttp.access [-] 147.213.75.181 [28/Feb/2020:12:56:47 +0000] "GET /ui HTTP/1.1" 200 1615 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:73.0) Gecko/20100101 Firefox/73.0"

Solution

The bug disappeared by downgrading to webargs 5.5.3 with pip install webargs==5.5.3.

raise HTTPException in train function never changes status from 'running'

Description

Some raised exceptions in the train function, do not change the status of the training.

Steps to Reproduce

Consider the following api.py for the demo example:

from aiohttp.web import HTTPException

def get_metadata():
    return {"status": "All ok"}

def get_train_args():
    return {}

def train(**kwargs):
    raise HTTPException(reason="Some error")

After calling:

curl -X 'POST' \
  'http://127.0.0.1:5000/v2/models/demo_app/train/' \
  -H 'accept: application/json' \
  -d ''

The server replies:

{
  "date": "2023-10-01 10:02:05.370140",
  "args": {},
  "uuid": "2631a11696c44cfea4a16d7095c64205",
  "status": "running"
}

However, after calling GET:

curl -X 'GET' \
  'http://127.0.0.1:5000/v2/models/demo_app/train/' \
  -H 'accept: application/json'

returns:

[
  {
    "date": "2023-10-01 10:02:05.370140",
    "args": {},
    "uuid": "2631a11696c44cfea4a16d7095c64205",
    "status": "running"
  }
]

Which is clearly not possible that status is running as an exception was raised.

Expected behavior:

It should be described http exceptions cannot be used in the documentation.
There should be some protection mechanisms to avoid the usage of this exceptions.
At worse, the GET method train should return:

[
  {
    "date": "2023-10-01 10:06:58.097097",
    "args": {},
    "uuid": "0fd0d21a8457482d885ee50d5aeed364",
    "status": "error",
    "message": "Some error"
  }
]

Ideally, the POST method train should return:

500 Internal Server Error

Versions

$ pip show deepaas
Name: deepaas
Version: 2.2.0
....
Location: /home/borja/miniconda3/envs/demo/lib/python3.10/site-packages
Requires: aiohttp, aiohttp-apispec, jsonschema, marshmallow, oslo.config, oslo.log, pbr, six, stevedore, typer, webargs, werkzeug
Required-by: demo-app

OpenWhisk support

Add support to execute DEEPaaS as an OpenWhisk Docker action, without extra software or modifications.

Support for cancelling a training

Ability to define "basePath"

Description

Sometimes, especially when behind the reverse proxy, it might be useful to have "basePath" for the API not as / but as something else, e.g /api. It would be nice, if a user could (re-)define this "basePath"

In the case of OAS3, in swagger.yaml this would correspond to:

servers:
  - url: /api

Ref: https://swagger.io/docs/specification/api-host-and-base-path/

Add training duration once the training is completed

If the training is completed, return a duration field (alongside status, date or uuid) stating how long did it took to the training to complete.

Refactor code to remove v2 organization

Since V1 code is deprecated, we can safely drop the existing codebase.

Stop using "oslo" libraries

Transfer repo to `ai4os` organization?

Should we transfer the repo to ai4os organization?

`POST /v2/models/{model}` raises `FileField is not JSON serializable` with file argumetns

Description

Error in train when implementing file arguments in get_train_args:
ERROR aiohttp.server TypeError: Object of type FileField is not JSON serializable

Steps to Reproduce

1. Download demo_app (or any other use case).
2. Add a file type into get_train_args:

def get_train_args():
    arg_dict = {
        "demo-image": fields.Field(
            required=True,
            type="file",
            location="form",
            description="image",
        ),
    }
    return arg_dict

3. Run deepaas: deepaas-run ...
4. Call train method providing a dummy txt or any other file.

Expected behavior:

Get the file as argument on train function (as in predict).

Actual behavior

2023-05-24 11:57:55.606 14073 INFO aiohttp.access [-] 127.0.0.1 [24/May/2023:09:57:55 +0000] "GET /swagger.json HTTP/1.1" 200 11513 "http://127.0.0.1:5000/ui" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server [-] Error handling request: TypeError: Object of type FileField is not JSON serializable
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server Traceback (most recent call last):
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server   File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server     resp = await request_handler(request)
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server   File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server     resp = await handler(request)
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server   File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server     return await handler(request)
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server   File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 108, in impl
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server     return await handler(request)
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server   File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server     return await handler(request)
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server   File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/webargs/asyncparser.py", line 157, in wrapper
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server     return await func(*new_args, **kwargs)
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server   File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/deepaas/api/v2/train.py", line 91, in post
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server     return web.json_response(ret)
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server   File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages/aiohttp/web_response.py", line 817, in json_response
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server     text = dumps(data)
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server   File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/json/__init__.py", line 231, in dumps
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server     return _default_encoder.encode(obj)
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server   File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/json/encoder.py", line 199, in encode
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server     chunks = self.iterencode(o, _one_shot=True)
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server   File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/json/encoder.py", line 257, in iterencode
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server     return _iterencode(o, 0)
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server   File "/home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/json/encoder.py", line 179, in default
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server     raise TypeError(f'Object of type {o.__class__.__name__} '
2023-05-24 11:58:02.030 14073 ERROR aiohttp.server TypeError: Object of type FileField is not JSON serializable

Versions

$ pip show deepaas
Name: deepaas
Version: 2.1.0
Summary: DEEPaaS is a REST API to expose a machine learning model.
Home-page: https://github.com/indigo-dc/deepaas
...
License: Apache-2
Location: /home/borja/miniconda3/envs/DEEPaaS/lib/python3.10/site-packages
Requires: aiohttp, aiohttp-apispec, jsonschema, marshmallow, oslo.config, oslo.log, pbr, six, stevedore, webargs, werkzeug
Required-by: demo-app

Implement type hints

Manual install fails: `pbr.version.SemanticVersion`

Manual installation is failing. Installation from PyPi works though.

❯ pip install -e .   
Obtaining file:///home/iheredia/ignacio/projects/deephdc/deepaas
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      ERROR:root:Error parsing
      Traceback (most recent call last):
        File "/home/iheredia/anaconda3/lib/python3.8/site-packages/pbr/core.py", line 105, in pbr
          attrs = util.cfg_to_args(path, dist.script_args)
        File "/home/iheredia/anaconda3/lib/python3.8/site-packages/pbr/util.py", line 272, in cfg_to_args
          pbr.hooks.setup_hook(config)
        File "/home/iheredia/anaconda3/lib/python3.8/site-packages/pbr/hooks/__init__.py", line 25, in setup_hook
          metadata_config.run()
        File "/home/iheredia/anaconda3/lib/python3.8/site-packages/pbr/hooks/base.py", line 27, in run
          self.hook()
        File "/home/iheredia/anaconda3/lib/python3.8/site-packages/pbr/hooks/metadata.py", line 25, in hook
          self.config['version'] = packaging.get_version(
        File "/home/iheredia/anaconda3/lib/python3.8/site-packages/pbr/packaging.py", line 857, in get_version
          version = _get_version_from_git(pre_version)
        File "/home/iheredia/anaconda3/lib/python3.8/site-packages/pbr/packaging.py", line 800, in _get_version_from_git
          result = _get_version_from_git_target(git_dir, target_version)
        File "/home/iheredia/anaconda3/lib/python3.8/site-packages/pbr/packaging.py", line 759, in _get_version_from_git_target
          raise ValueError(
      ValueError: git history requires a target version of pbr.version.SemanticVersion(2.3.2), but target version is pbr.version.SemanticVersion(2.0.0)
      error in setup command: Error parsing /home/iheredia/ignacio/projects/deephdc/deepaas/setup.cfg: ValueError: git history requires a target version of pbr.version.SemanticVersion(2.3.2), but target version is pbr.version.SemanticVersion(2.0.0)
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

It looks there is a mismatch in versions (ref, ref), due to Release Please not GPG signing the tags (issue).

Temporary fix: revert to last signed tag and install from there:

git checkout tags/2.2.0
pip install -e .
git checkout master

Default DEEPaaS bind address

When 127.0.0.1 is used as the default bind address, the docker container running DEEPaaS cannot be accessed on published port (-p 5000:5000). If this address is changed to 0.0.0.0 it works ok.

https://github.com/indigo-dc/DEEPaaS/blob/0c0cadd67b5dab5d4e7966463efb881575ec558b/deepaas/cmd/run.py#L29

`sterror` not correctly piped

Description

Errors are not correctly piped. An error in the module code that prevents deepaas from launching is piped to stout, not sterr.

Steps to Reproduce

1. Introduce error in module (raise Exception('some error'))
2. Launch deepaas: deepaas-run

stout:

...
2023-07-26 14:05:38.595 149055 ERROR deepaas   File "/home/iheredia/ignacio/projects/deephdc/apps/demo_app/demo_app/api.py", line 103, in get_predict_args
2023-07-26 14:05:38.595 149055 ERROR deepaas     raise Exception('some error')
2023-07-26 14:05:38.595 149055 ERROR deepaas Exception: some error
2023-07-26 14:05:38.595 149055 ERROR deepaas 

sterr:

Two things might occur:

• if Exception is raised in get_*_args, the deepaas process ends (with empty sterr)
• if Exception was raised outside get_*_args, deepaas process does not end (an ERROR is just shown in stout).

In both cases, behaviour is incorrect.

Expected behavior:

Error message is in sterr.
deepaas should not launch if an error is raised outside get_*_args (though is coorectly ending if error is inside).

Actual behavior

Error message is in stout.
deepaas launching despite error.

Versions

2.0.0

Leverage dask.distributed to run training/predict tasks

Instead of using our own task pool, we should leverage Dask distributed, as this will allow us to better consume resources from existing clusters.

Debug level is not propagated to service threads

Description

Logger output on predict and train functions with lower level than WARN is not printed on stderr.

Steps to Reproduce

1. Download demo_app (or any other use case).
2. Create a logger in the api module and log in get_meatada and predict using debug, for example:

import logging
logger = logging.getLogger(__name__)

def get_metadata():
    logger.debug("Some debug information")
    return {"status": "All ok, debug was printed"}

def get_predict_args():
    return {}

def predict(**kwargs):
    logger.debug("Some debug information")
    return {"status": "Debug was not printed"}

3. Create a configuration file which set debug level to "DEBUG" (./deepaas.conf):

[DEFAULT]
debug = true
log_date_format = %Y-%m-%d %H:%M:%S

4. Run using the confg file: deepaas-run --config-file deepaas.conf

Expected behavior:

When calling predict / train method, debug information should be printed.
(like in ´get_metadata`)

Actual behavior

Only logger print levels higher or equal than WARN are printed on predict / train method.

Versions

$ deepaas-run --version
2.1.0

Extra information

I see the functions predict and train are executed by workers. However, the logger level at those workers seems to not be inherited from the original logger level.

Implement Dask Distributed task Actor

Remove "deepaas-test" model

If no model is present, we should not load a test model.

Failed to load model

Hi @alvarolopez,

We are using the Deepaas library to deploy our machine learning model following instructions in this repo README and also your medium article below:
https://towardsdatascience.com/publish-your-models-with-a-standard-rest-api-94de1e1da682

Both approaches failed to load our model and the test-deepaas model is always loaded.
Can you please elaborate on the path where to place the .py files?
And also how shall we specify the entry point?

cc @reemhammoud @huda96

Empty file `Content-type` error

When a module has an predict arg which is a file (eg. image), but the arg is not required, deepaas fails to run:

        "demo-image": fields.Field(
            required=False,
            type="file",
            location="form",
            description="image",  # description needed to be parsed by UI
        ),

Seems to be related to the Content-type header in the aiohttp call. I opened a branch in the demo-app to test this: empty-file-form.

This issue was originally raised by @emvollmer (Nov. 2023)

Error logs

2024-04-30 15:46:17.946 162826 ERROR aiohttp.server [-] Error handling request: ValueError: boundary missed for Content-Type: multipart/form-data
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server Traceback (most recent call last):
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     resp = await request_handler(request)
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/aiohttp/web_app.py", line 504, in _handle
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     resp = await handler(request)
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/aiohttp/web_middlewares.py", line 117, in impl
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     return await handler(request)
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/aiohttp/web_middlewares.py", line 108, in impl
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     return await handler(request)
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/aiohttp/web_middlewares.py", line 117, in impl
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     return await handler(request)
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/ignacio/projects/deephdc/deepaas/deepaas/api/v2/predict.py", line 66, in post
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     args = await aiohttpparser.parser.parse(handler_args, request)
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/webargs/asyncparser.py", line 87, in parse
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     parsed = await self._parse_request(
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/webargs/asyncparser.py", line 59, in _parse_request
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     parsed_value = await self.parse_arg(
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/webargs/asyncparser.py", line 206, in parse_arg
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     value = await self._get_value(name, field, req=req, location=location)
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/webargs/asyncparser.py", line 217, in _get_value
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     value = await function(req, name, argobj)
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/webargs/aiohttpparser.py", line 89, in parse_form
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     self._cache["post"] = await req.post()
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/aiohttp/web_request.py", line 689, in post
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     multipart = await self.multipart()
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/aiohttp/web_request.py", line 667, in multipart
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     return MultipartReader(self._headers, self._payload)
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/aiohttp/multipart.py", line 537, in __init__
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     self._boundary = ("--" + self._get_boundary()).encode()
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server   File "/home/iheredia/anaconda3/lib/python3.8/site-packages/aiohttp/multipart.py", line 636, in _get_boundary
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server     raise ValueError(
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server ValueError: boundary missed for Content-Type: multipart/form-data
2024-04-30 15:46:17.946 162826 ERROR aiohttp.server 
2024-04-30 15:46:17.948 162826 INFO aiohttp.access [-] 127.0.0.1 [30/Apr/2024:13:46:17 +0000] "POST /v2/models/ai4os_demo_app/predict/?demo-str=some-string&demo-str-choice=choice2&demo-int=1&demo-int-range=50&demo-float=0.1&demo-bool=true&demo-dict=%7B%22a%22%3A%200%2C%20%22b%22%3A%201%7D&demo-list-of-floats=0.1&demo-list-of-floats=0.2&demo-list-of-floats=0.3 HTTP/1.1" 500 244 "http://0.0.0.0:5000/api" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"

Schema validation from methods?

Description

From experience, conventions intriduce problems unless forced or directed.
In this particular case, I would use Schemas to ensure the API methods return valid types and fields.

For example, that the return from the function get_metadata returns a lis or, from doc mandatory are ‘description’ and ‘name’ that it returns a dict with at least the fields ‘description’ and ‘name’. Or even some schemas might not accept extra parameters.

In case method return unexpected types or dictionary fields (or missing), it should raise and exception so user is forced to modify it.

Additionally, it should be possible to bypass this in a setting or somethig (i.e. env=development) so when developing the framework it we can test the edited behavior.

In some cases I get:

def get_metadata():
    return [{"name": "rnacontactmap"}]

# API returns
500 Internal Server Error
Server got itself in trouble

Which is not very traceable. It might be easier to raise an Exception to get "Unexpected return value from get_metadata, missing x,x,x"

Versions

2.1.0

Implement model loading through decorators and type hints

Apart from supporting old-style loading, we should support loading of models through decorators and type hints, so that users do not need to deal with entrypoints and external libraries.

System requirements

Can you provide minimum and recommended system requirements (e.g., CPU and memory) to deploy a machine learning model using DEEPaaS?

Method `GET /v2/models/` return `[ModelMeta]` instead of `{ models: [ModelMeta] }`

Description

Method GET /v2/models/ could return [ModelMeta] instead of { models: [ModelMeta]}, that might be more intuitive?

Steps to Reproduce

Run the application, and in the ui, you can see the example and Model in Swagger is ModelMeta. However, as the method url is plural, I suspect users might expect a list containing models, not a JSON.

Versions

You can get this information from executing caso --version.
caso --version fails, probably deepaas-run --version? -> 2.1.0

From API:

{
  "version": "stable",
  "id": "v2",
  "links": [
    {
      "rel": "self",
      "type": "application/json",
      "href": "/v2/"
    }
  ]
}