iotaledger / pyota-ccurl

C Curl extension for PyOTA

C Curl Extension for PyOTA

This is an extension for the crypto functionality in PyOTA, the Python API for IOTA. When installed, it will significantly boost the performance of PyOTA's crypto functionality.

This extension is installed as an add-on to the pyota package:

pip install pyota[ccurl]

After running the above command, you can verify that the C extension is installed correctly by running the check_ccurl command in your shell:

> check_ccurl
😸  CCurl is installed correctly!
For support, visit the #iota-libs-pyota channel on the IOTA Slack.
https://slack.iota.org/

Compatibility

This extension is compatible with Python 3.6, 3.5 and 2.7.

Testing

The easiest way to test the extension is to install it and then run the PyOTA unit tests.

Here's a modified tox.ini file that for PyOTA that will install the C extension before running unit tests:

[tox]
envlist = py27, py35, py36

[testenv]
commands =
  pip install --pre pyota-ccurl
  nosetests
deps =
  mock
  nose

Note that this file needs to be installed in the PyOTA codebase!

pyota-ccurl's Issues

Does not compile in Python 2.7

The extension won't compile in Python 2.7:

    src/ccurlmodule.c:205:8: error: unknown type name 'PyModuleDef'
    static PyModuleDef ccurlmodule = {
           ^
    src/ccurlmodule.c:206:3: error: use of undeclared identifier 'PyModuleDef_HEAD_INIT'
      PyModuleDef_HEAD_INIT,
      ^
    src/ccurlmodule.c:220:5: error: void function 'PyInit_ccurl' should not return a value [-Wreturn-type]
        return NULL;
        ^      ~~~~
    src/ccurlmodule.c:222:7: warning: implicit declaration of function 'PyModule_Create' is invalid in C99 [-Wimplicit-function-declaration]
      m = PyModule_Create(&ccurlmodule);
          ^
    src/ccurlmodule.c:224:5: error: void function 'PyInit_ccurl' should not return a value [-Wreturn-type]
        return NULL;
        ^      ~~~~
    src/ccurlmodule.c:228:5: error: void function 'PyInit_ccurl' should not return a value [-Wreturn-type]
        return NULL;
        ^      ~~~~
    src/ccurlmodule.c:238:3: error: void function 'PyInit_ccurl' should not return a value [-Wreturn-type]
      return m;
      ^      ~
    1 warning and 6 errors generated.
    error: command '/usr/bin/clang' failed with exit status 1

This appears to be due to changes introduced in the Python C API during the transition to version 3:
https://docs.python.org/3/howto/cporting.html

Add command to check if C extension is installed correctly

Add a Python function that checks if issubclass(iota.crypto.Curl, CCurl).

Bonus points: Include a console_scripts entry point for the function.

This will help us to kill two stones with one bird:

• Easy way for developers to verify that the extension is installed correctly.
• Workaround for meshy/pythonwheels#46 (comment)

Cannot install pyota-ccurl on Arch Linux

Hi, have you seen this?

iotaledger/iota.py#227

I'm unable to install it on Arch systems - tried on various PCs. Works on Fedora and others, but Arch does not seem to work. Any idea how to fix it?

UnicodeDecodeError when installing on Windows 10, Python 3.6.4

C:\Python36\Scripts> .\pip.exe install pyota[ccurl]
...
Installing collected packages: pyota-ccurl
  Running setup.py install for pyota-ccurl ... error
Exception:
Traceback (most recent call last):
  File "c:\python36\lib\site-packages\pip\compat\__init__.py", line 73, in console_to_str
    return s.decode(sys.__stdout__.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x94 in position 90: invalid start byte
 
During handling of the above exception, another exception occurred:
 
Traceback (most recent call last):
  File "c:\python36\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "c:\python36\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "c:\python36\lib\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "c:\python36\lib\site-packages\pip\req\req_install.py", line 878, in install
    spinner=spinner,
  File "c:\python36\lib\site-packages\pip\utils\__init__.py", line 676, in call_subprocess
    line = console_to_str(proc.stdout.readline())
  File "c:\python36\lib\site-packages\pip\compat\__init__.py", line 75, in console_to_str
    return s.decode('utf_8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x94 in position 90: invalid start byte

See https://pastebin.com/kTAsr3MH for full output.

Note: Works OK on same system with Python 2.7

Add `offset` and `length` arguments to `absorb()` and `squeeze()`

For compatibility with iotaledger/iota.py#92 absorb() and squeeze() need offset and length arguments.

Convert to wrapper for ccurl library

Convert the extension into a wrapper for the official CCurl library.

This gets us a ton of goodies for free:

• Crypto code is already/regularly vetted.
• Any enhancements or fixes made to the CCurl library would automatically apply to us.
• We'd be able to seamlessly incorporate PoW.

Automated PyPI distribution

Add deployment instructions to .travis.yml so that new releases are automatically deployed to PyPI.

References:

• https://github.com/eflglobal/filters/blob/master/.travis.yml#L8-L15
• https://docs.travis-ci.com/user/deployment/pypi/

Bonus Points

Create binaries for major OS distros as part of the deployment process, so that end users don't have to compile anything.