pysftp dependency is not being recognized about auto_backup

[lukebranch]

FWIW on another instance i've run into the following issue while attempting this again from another web hosting company:

Traceback (most recent call last):
File "/usr/bin/pip", line 5, in
from pkg_resources import load_entry_point
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 70, in
import packaging.version
ImportError: No module named packaging.version

I have resolved it using the following method (as root user):
apt-get purge -y python-pip
wget https://bootstrap.pypa.io/get-pip.py
python ./get-pip.py
apt-get install python-pip

thanks to the suggestions on this blog:
https://odedrabhavesh.blogspot.hk/2017/02/importerror-no-module-named.html

[lukebranch]

@wangdaze ,

I haven't encountered your specific issue personally, however I believe it is still due to missing dependencies on your system. Try removing pysftp first and then do this:

sudo apt-get install -y build-essential libssl-dev libffi-dev python-dev

then reinstall pysftp and see if you still encounter the same issues. I'm pretty sure that specific error is related to the build-essentials dependency.

[lukebranch]

@beatrizlana-br ,

I am in no way involved with this project, however as another user I did find that I needed to:
sudo pip uninstall pysftp

to remove the library first. then:
sudo apt-get install libpq-dev python-dev libxml2-dev libxslt1-dev libldap2-dev libsasl2-dev libffi-dev

to install some required tools and libraries related to installing pysftp.

and then:
sudo pip install pysftp

to reinstall pysftp. Once you've done this restart the odoo service, or your server and see if you still run into this issue again.

In my case pysftp was not being compiled properly on a clean Ubuntu 14.04 instance, and these were the steps I needed to follow to get it working.

[wangdaze]

@lukebranch Thanks!

[Yenthe666]

Hi @beatrizlana-br,

Looks like your Ubuntu has issues with figuring out what is installed or not installed.
The easiest fix is a full reboot of your server, then the system should know the exact state.

[beatrizlana-br]

Thanks for all the answers guys!
I rebooted the server and nothing happened, but I found out the solution. I'll put it here in case someone has the same issues:

I typed python in the terminal, then import pysftp and I got the message

File "/usr/local/lib/python2.7/dist-packages/paramiko/transport.py", line 33, in
from cryptography.hazmat.backends import default_backend
ImportError: No module named cryptography.hazmat.backends

Then I exited python and typed sudo pip install cryptography . After that I entered python again and typed import pysftp and it worked.

[wangdaze]

@beatrizlana-br Thanks for your sharing！It works,first I install cryptography ,It not work until reboot .Thanks for all

[Yenthe666]

Hi all,

I've decided to drop the pysftp library in the upcoming V11 and change it directly to paramenko. This means there'll be less dependencies downloaded and installed (since pysftp itself depends on a lot of other stuff too). I'm slowly changing the behaviour (locally, not yet on Github) and only face a major problem on the paraminko transfer methods in case there is no folder. The goal is to have this new dependency in place before V11.

@lukebranch yes it would be possible to hook into other libraries and execute methods (for example through a shell or virtual environment). I do believe that this is a bit tricky / extreme to accomplish this though as it'll most likely have more chance of breaking than just a python library as it works now.
Hope you agree / understand 😉

Regards,
Yenthe

[treviser]

@Yenthe666 I was able to install pysftp on my older servers (for 8.0 and 9.0) because they were initially on Ubuntu 14.04. It is only on my newly created 16.04 instances running Odoo 10.0. I'll give your master version a try, thanks!

[treviser]

You don't have to understand, but Google is your friend ;-)

[Yenthe666]

Thanks for answering @lukebranch 😉

[lukebranch]

@Yenthe666 ,

No problem!

[wangdaze]

Hi, @beatrizlana-br ,How u fix the issue?after I followed the following method,It not work also,
and show:
Downloading/unpacking pysftp
Downloading pysftp-0.2.9.tar.gz
Running setup.py (path:/tmp/pip_build_root/pysftp/setup.py) egg_info for package pysftp

no previously-included directories found matching 'docs/_build'

Requirement already up-to-date: paramiko>=1.17 in /usr/local/lib/python2.7/dist-packages (from pysftp)
Installing collected packages: pysftp
Running setup.py install for pysftp

no previously-included directories found matching 'docs/_build'

Successfully installed pysftp
Cleaning up...

[beatrizlana-br]

Hi guys! Thanks a million for answering!
I still have an issue with that.
I followed the two processes described by @lukebranch and I couldn't uninstall pysftp, I got

Can't uninstall 'pysftp'. No files were found to uninstall.

But when I typed sudo pip install pysftp I got

Requirement already satisfied (use --upgrade to upgrade): pysftp in /usr/local/lib/python2.7/dist-packages
Requirement already satisfied (use --upgrade to upgrade): paramiko>=1.17 in /usr/local/lib/python2.7/dist-packages (from pysftp)
Cleaning up...

Now, in the application I could update modules list, but when I try to install the module I got the message

Unable to install module "auto_backup" because an external dependency is not met: No module named pysftp

Thanks a lot for the help!

[Yenthe666]

Hmm this really makes me think about changing pySFTP for another library again. There are quite often dependency and versions issues, which is annoying to use this module..

[lukebranch]

@Yenthe666 ,

Yes, it may be worthwhile looking into. I had initially had a bit of trouble myself getting it to work due to dependency related issues around that library. If there are other options you have in mind it may be worth a look.

Regarding that Duplicity library I mentioned before, do you have any idea if it's possible to interact with the command line from an odoo module in any way?

Here's a few terminal commands for the utility for reference:

http://duplicity.nongnu.org/docs.html

If it was possible I believe adding support for Duplicity would be relatively easy to implement, and it would allow you a number of other options in terms of storage backends (cloud buckets, SFTP, google drive, dropbox, etc.) as well as incremental backups among other things.

Full WIP docs are here if you're interested:

http://duplicity.readthedocs.io/en/latest/

I've experimented with the utility standalone in the terminal, and am planning on just executing some commands on cron jobs, however if there was any way to interact with this through an Odoo instance itself, it would make open up a huge range of possibilities.

In regards to the sftp library, i'll keep my eyes open and ping you here if I come across anything that looks like it may be suitable as a drop-in replacement for pysftp.

[lukebranch]

@Yenthe666 ,

Thanks for the info!

[treviser]

@Yenthe666 It would be nice if you could provide a simple version without sftp in the meantime. I did not succeed to properly install pysftp on Ubuntu 16.04 LTS (I did try every possible way I could find in the internet) and the result is that the whole Odoo instance is completely unrecoverable, it's not an 500 internal error, no, the instance is not reachable at all (deleting the module and anything else do not help) :-(

[Yenthe666]

@treviser wow that is very invasive, I've never heard from anybody running that stuck yet. I'm going to guess that your service (the planner) ran stuck, which causes this. Killing it by command and restarting should work.
As for a version without pysftp: https://github.com/Yenthe666/auto_backup/tree/master the master version should be working fine (except for SFTP side in some cases). If you need this on a 10.0 you should be fine out of the box I believe. If you need this for 8.0 or 9.0 you would need to alter some things, such as the manifest.py becomes __openerp__.py for example. I'm not sure if this helps you enough?

[treviser]

@Yenthe666 paramiko seems to be the same beast, I did destroy my instance again the same way...

[Yenthe666]

@treviser so you mean that you have the same issues with paramiko as with pysftp? Can you give all the feedback you have about what happens, what is logged, etc?

[treviser]

This is what I see when I try to start Odoo from CLI:

ubuntu@ip-192-168-110-144:~$ sudo -H -u odoo /usr/bin/odoo -c /etc/odoo/odoo.conf -u all
Traceback (most recent call last):
  File "/usr/bin/odoo", line 5, in <module>
    __import__('pkg_resources').declare_namespace('odoo.addons')
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2081, in declare_namespace
    declare_namespace(parent)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2097, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2046, in _handle_ns
    loader.load_module(packageName)
  File "/usr/lib/python2.7/pkgutil.py", line 246, in load_module
    mod = imp.load_module(fullname, self.file, self.filename, self.etc)
  File "/usr/lib/python2.7/dist-packages/odoo/__init__.py", line 81, in <module>
    import cli
  File "/usr/lib/python2.7/dist-packages/odoo/cli/__init__.py", line 9, in <module>
    import deploy
  File "/usr/lib/python2.7/dist-packages/odoo/cli/deploy.py", line 5, in <module>
    import requests
  File "/usr/lib/python2.7/dist-packages/requests/__init__.py", line 53, in <module>
    from .packages.urllib3.contrib import pyopenssl
  File "/usr/lib/python2.7/dist-packages/urllib3/contrib/pyopenssl.py", line 54, in <module>
    import OpenSSL.SSL
  File "/usr/lib/python2.7/dist-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import rand, crypto, SSL
  File "/usr/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 118, in <module>
    SSL_ST_INIT = _lib.SSL_ST_INIT
AttributeError: 'module' object has no attribute 'SSL_ST_INIT'

[treviser]

I have solved the problem (and this is maybe the same for pysftp):

Ubuntu 16.04 LTS obviously has pyOpenSSL 0.15.1 installed by default. I did uninstall it:

ubuntu@ip-192-168-110-144:~$ sudo -H pip uninstall pyOpenSSL
Uninstalling pyOpenSSL-0.15.1:
  /usr/lib/python2.7/dist-packages/OpenSSL
  /usr/lib/python2.7/dist-packages/pyOpenSSL-0.15.1.egg-info
Proceed (y/n)? y
  Successfully uninstalled pyOpenSSL-0.15.1

And then I have installed pyOpenSSL-16.2.0:
ubuntu@ip-192-168-110-144:~$ sudo -H pip install pyOpenSSL==16.2.0

and now Odoo is working again.

[Yenthe666]

Hmm, this kinda blows my mind. I've never seen this around.. not sure what I should do with this one 😉

[Highcooley]

Sorry for warming up this thread, but has anybody been able to successfully get pysftp installed and recognised by auto-backup with the official odoo 10.0 docker container? I get pysftp installed in the version 0.2.8 or 0.2.9 together with all dependencies (some warnings along the way but no errors), but no luck with it being recognised by odoo.

[Yenthe666]

@Highcooley do you have errors in your logfile?

[Highcooley]

Despite Mixed content HTTP 304 problems of the webpage, which are not related to this problem, no.

[Yenthe666]

Hmm, odd. Then I guess you have deployment issues. I barely ever used Docker and have no deployments for Odoo so I'm afraid I can't really help you with this one.

[darm-torres]

I had similar issue. I fix it using
sudo pip install pysftp

Note: OS: Ubuntu19

[Teja-unique]

Hi all,

I am also facing an issue with pysftp.

I am getting below error with the
command " import pysftp"
Error - module not found error for pysftp

It's a monthly activity.. it worked last month also but this type we are facing the above error.

Kindly help to found out the issue

I have also tried to install pysftp through pip command but here also facing an error - pip is not recognised as internal command