datatorch / python

:snake: Python/CLI library for interacting with DataTorch programmatically

DataTorch Python

DataTorch CLI and Python API libary for programmatic access.

Installation

Note: Newer versions (>=0.4.8.2) of DataTorch client use libmagic for more accurate mimetype guessing. This falls back to guessing via extension. Consider installing libmagic if you have issues with file types uploaded to DataTorch.

pip install datatorch

Development

python3 -m pip install --editable .
pip3 install -r requirements.txt

VSCode Configuration

Formatter

Open your VSCode settings, by going Code -> Preferences -> Settings. Search for "python formatting provider" and select "black" from the dropdown menu.

python's Issues

fresh install of latest release does not work with python 3.9

to reproduce:

pip install datatorch

then

dt

results in this error

# rave at rave-desktop in ~/datatorch on git:master ✖︎ [12:07:45]
→ datatorch
Traceback (most recent call last):
  File "/home/rave/miniconda3/envs/datatorchtest/bin/datatorch", line 5, in <module>
    from datatorch.cli import main
  File "/home/rave/miniconda3/envs/datatorchtest/lib/python3.9/site-packages/datatorch/__init__.py", line 4, in <module>
    from datatorch.api import ApiClient
  File "/home/rave/miniconda3/envs/datatorchtest/lib/python3.9/site-packages/datatorch/api/__init__.py", line 1, in <module>
    from .api import ApiClient
  File "/home/rave/miniconda3/envs/datatorchtest/lib/python3.9/site-packages/datatorch/api/api.py", line 7, in <module>
    from .client import Client
  File "/home/rave/miniconda3/envs/datatorchtest/lib/python3.9/site-packages/datatorch/api/client.py", line 5, in <module>
    from gql.transport.websockets import WebsocketsTransport
  File "/home/rave/miniconda3/envs/datatorchtest/lib/python3.9/site-packages/gql/transport/websockets.py", line 11, in <module>
    from websockets.http import HeadersLike
ImportError: cannot import name 'HeadersLike' from 'websockets.http' (/home/rave/miniconda3/envs/datatorchtest/lib/python3.9/site-packages/websockets/http.py)
(datatorchtest) 

I ran this in a fresh python environment with python 3.9

NotImplementedError - sys:1: RuntimeWarning: coroutine 'stop' was never awaited

New conda environment on Windows 10 64bit, Python-3.8.5 with the following requirements.txt:

numpy>=1.15.2
scipy>=1.1.0
scikit-learn>=0.20.0
scikit-image>=0.14.1
opencv-python
pylint
rope
datatorch
datatorch[agent]

And the following error:

datatorch agent start
Traceback (most recent call last):
  File "c:\users\user\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\user\anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\user\Anaconda3\Scripts\datatorch.exe\__main__.py", line 7, in <module>
  File "c:\users\user\anaconda3\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\user\anaconda3\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\user\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\user\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\user\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\user\anaconda3\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\user\anaconda3\lib\site-packages\datatorch\cli\agent\start.py", line 16, in start
    loop.add_signal_handler(SIGINT, exit_func)
  File "c:\users\user\anaconda3\lib\asyncio\events.py", line 540, in add_signal_handler
    raise NotImplementedError
NotImplementedError
sys:1: RuntimeWarning: coroutine 'stop' was never awaited