jbardin / scp.py Goto Github PK
View Code? Open in Web Editor NEWscp module for paramiko
License: Other
scp module for paramiko
License: Other
I'd like to add two utility functions to the scp module -- get and put, something like;
def get(transport, remote_path, local_path='',
recursive=False, preserve_times=False):
return SCPClient(transport).get(...)
In many one-off use cases, creating an SCPClient object is more syntactic noise than necessary.
I'd be happy to work on patches if this sounds reasonable.
Im using the lib in a automatic testing scripts and notice that if send a config file via scp and using it immediately after (in my case, usinga api service that used this conf file) the OS dont notice the changes in file.
I solved this using a wait() after the scp finish
Running tests on Mac OS X with Python 2
....FF
======================================================================
FAIL: test_put_bytes (__main__.TestUpload)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 228, in test_put_bytes
self.upload_test(b'cl\xC3\xA9/r\xC3\xA9mi', False, [b'r\xC3\xA9mi'])
File "test.py", line 219, in upload_test
self.assertEqual(set(out_list), set(expected))
AssertionError: Items in the first set but not the second:
're\xcc\x81mi'
Items in the second set but not the first:
'r\xc3\xa9mi'
======================================================================
FAIL: test_put_unicode (__main__.TestUpload)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 238, in test_put_unicode
self.upload_test(u'cl\xE9/r\xE9mi', False, [b'r\xC3\xA9mi'])
File "test.py", line 219, in upload_test
self.assertEqual(set(out_list), set(expected))
AssertionError: Items in the first set but not the second:
're\xcc\x81mi'
Items in the second set but not the first:
'r\xc3\xa9mi'
print('initializing ssh client')
#initialize ssh client
key = paramiko.RSAKey.from_private_key_file('/home/edward/workspace/waka/mykeypair.pem')
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
for instance_id in instance_ids:
instance = ec2_resource.Instance(instance_id)
ssh.connect(instance.public_ip_address, username='ubuntu', pkey=key)
scp = SCPClient(ssh.get_transport())
scp.put('cluster_ips.txt', 'flock.jar')
scp.close()
no idea why it happened. the txt file went through alright, but the jar file was corrupted, different sums
from paramiko import SSHClient
from scp import SCPClient
ssh = SSHClient()
ssh.load_system_host_keys()
ssh.connect('example.com')
"""SCPCLient takes a paramiko transport as its only argument"""
scp = SCPClient(ssh.get_transport())
scp.put('test.txt', 'test2.txt') ----------line 11
print "1" ----------line 12
scp.get('test2.txt') ----------line 13
print "2" ----------line 14
scp.put('test.txt', 'test3.txt') ----------line 15
print "3"
scp.close()
:::::::::::::::::::::result::::::::::::::::::::::::::::
1
Channel closed.
At line 13. will raise Channel closed. I found the exception is raised by paramiko/channel.py:_wait_for_event().Because "if self.event_ready:" is False.
We has called exec_command in scp.put. Then I see the exec_command doc_string. It writed """When the command finishes executing, the channel will be closed and can't be reused. You must open a new channel if you wish to execute another command."""
I don't known how about your tests. It run error in my pc. wish your help.
Hi! Thanks for writing scp; it's made my life a lot easier!
I am running into a problem with recursive gets that I think may be unicode related, but I'm not entirely sure. I'm running scp 0.10.2 on Python 2.7.6. In my code, I have:
scp.get(runtime_dir + "/*.txt", local_path=args.test_id)
Indeed, this works fine via scp on the command line (scp [email protected]:/home/ubuntu/testdir/module/*.txt .
). However, when I try using the module, I get:
Traceback (most recent call last):
File "./script.py", line 116, in <module>
scp.get(runtime_dir + "/*.txt", local_path=args.test_id)
File "/Users/patcable/Virtualenvs/osclient/lib/python2.7/site-packages/scp.py", line 198, in get
self._recv_all()
File "/Users/patcable/Virtualenvs/osclient/lib/python2.7/site-packages/scp.py", line 348, in _recv_all
raise SCPException(asunicode(msg[1:]))
scp.SCPException: scp: /home/ubuntu/testdir/module/*.txt: No such file or directory
Am I missing something simple? It seems like this should work; for now I just work around by using os.subprocess, but figured it'd be neat if it worked in the module. In any case, thanks for your time!
os.utime is called with (path, (mtime, atime)), but the actual parameters should be (path, (atime, mtime)).
This issue is very similar to #32. I actually found another similar scenario that throws an SCPException and breaks the process. This one is when you are copying a file to a destination that has a directory that is the same name as the file that you are copying. Here is an example where I try to recursively copy one directory to a remote device. In the local directory, there is a file gres-tp
, and in the remote directory there already existed a directory gres-tp/
, creating the following stack trace:
Traceback (most recent call last):23123123
File "jaide.py", line 345, in copy_file
scp.put(scp_source, scp_dest, recursive=True, preserve_times=True)
File "build/bdist.macosx-10.9-intel/egg/scp.py", line 101, in put
File "build/bdist.macosx-10.9-intel/egg/scp.py", line 217, in _send_recursive
File "build/bdist.macosx-10.9-intel/egg/scp.py", line 170, in _send_files
File "build/bdist.macosx-10.9-intel/egg/scp.py", line 251, in _recv_confirm
scp.SCPException: scp: /var/tmp///gres-tp: Is a directory
The recursion stops and any other files/folders are not copied. For completeness, here is an example of the scp
command handling this:
$ scp -r ~/desktop-link/scp/tmp/* [email protected]:/var/tmp
[email protected]'s password:
asdf 100% 0 0.0KB/s 00:00
asdf123 100% 0 0.0KB/s 00:00
asdf123123123 100% 0 0.0KB/s 00:00
scp: /var/tmp/gres-tp: Is a directory
hello-world 100% 0 0.0KB/s 00:00
marking-scheme 100% 0 0.0KB/s 00:00
template 100% 281 0.3KB/s 00:00
ssh.connect("the name of my server goes here")
Traceback (most recent call last):
File "", line 1, in
File "build/bdist.macosx-10.9-x86_64/egg/paramiko/client.py", line 307, in connect
File "build/bdist.macosx-10.9-x86_64/egg/paramiko/client.py", line 520, in _auth
paramiko.ssh_exception.SSHException: No authentication methods available
Please include classifiers in the setup.py
, so it is easier to know what versions are supported when browsing PyPI:
For example:
classifiers=[
'Programming Language :: Python :: 3.3',
...
SCP'ing single file to Cisco Device fails in 0.7.0 but working fine in 0.5.1.
Error is: No such file or directory.
Hello,
I am facing issue during transfer of a file of size ~ 200MB. scp.put fails with below error:
Traceback (most recent call last): File "C:\Python27\lib\site-packages\robot\running\timeouts.py", line 85, in run return runner.get_result()
File "C:\Python27\lib\site-packages\robot\utils\robotthread.py", line 37, in run self._result = self.runnable()
File "C:\Python27\lib\site-packages\robot\utils\robotthread.py", line 28, in self.runnable = lambda: runnable((args or ()), *(kwargs or {}))
File "C:\Documents and Settings\test1\libs\UpgradeBuild.py", line 12, in UpgradeBuild scp.put(BuildLocation,DestLocation )
File "build\bdist.win32\egg\scp.py", line 103, in put self._send_files(files)
File "build\bdist.win32\egg\scp.py", line 181, in _send_files chan.sendall(file_hdl.read(buff_size)) File "C:\Python27\lib\site-packages\paramiko\channel.py", line 765, in sendall sent = self.send(s) File "C:\Python27\lib\site-packages\paramiko\channel.py", line 690, in send size = self._wait_for_send_window(size)
File "C:\Python27\lib\site-packages\paramiko\channel.py", line 1177, in _wait_for_send_window raise socket.timeout()
https://bitbucket.org/pypa/pypi/issues/349/get-of-scpscpclient-not-accepting-wildcard
I recently upgraded the scp to 0.10.2 after years, and the get() no longer accept wildcard "*" to get all the files from a specific folder.
I have updated details there as well.
Digging through the code base I found the progress call back you can pass to the scpclient class. I found that pretty helpful and it would be a good thing to document in the readme.
Hello there,
Would you mind updating pypi. Thanks.
This thing is great, except you cannot specify wildcards in get() ie, get("/var/log/messages*") will say that it doesn't exist, because it's looking for a literal single file named that. specifying a single file that exists, with the same code works great, so it looks to me like it's definitely a wildcarding issue. Is there a workaround, is this a bug, or is this not how the code is intended to be used and won't be fixed/changed?
This PR (a57e683) was merged which updated the documentation according to that new code but a new package was never built.
Hi,
I'm afraid I'm pretty confused by this one -- when calling scp.put with a remote path parameter specified, I'm getting a "got multiple values for keyword argument" error, which doesn't seem to make any sense:
scp.put(zipname, zipname, remote_path='./deposit_here')
TypeError: put() got multiple values for keyword argument 'remote_path'
I tried both ~/deposit_here and ./deposit_here and both are throwing the same issue. Anything dumb I'm missing? Thanks!
It is misleading when your tests fail on the Travis build because it no longer supports Python 2.6, which is depreciated now anyway.
Hi, thanks for this great package! I'm having an issue but I'm not sure if the problem is in scp or paramiko, perhaps you have some insight.
First, this works:
ssh = paramiko.SSHClient()
ssh.load_system_host_keys()
ssh.connect(hostname=HOST, username=USER, key_filename=KEY)
x = scp.SCPClient(ssh.get_transport())
with x as c:
print(c)
But identical code where I have just moved the connection creation to a separate function doesn't work:
def get_client():
ssh = paramiko.SSHClient()
ssh.load_system_host_keys()
ssh.connect(hostname=HOST, username=USER, key_filename=KEY)
return scp.SCPClient(ssh.get_transport())
with get_client() as c:
print(c)
I get the following error:
SSHException: SSH session not active
Am I doing something wrong here or is there a bug in scp or paramiko? Thanks for any help!
Last update to PyPI is from May 2015, a PyPI update would be nice (especially for putfo)
Paramiko has a putfo method that allows uploading file-like objects.
Shall we add a similar method to this library as well? I need this feature to enable openwisp2 to push configurations to small routers that are not running an SFTP server by default. OpenWISP 2 generates the configuration directly in memory, it can write to disk, but we avoid that as much as possible because it wouldn't scale for thousands of devices.
I'm willing to put effort in adding this feature, I hope to hear back from you and receive some guidance about the best way to do it.
As a starter, I'm looking at the putfo method implementation in paramiko although I'm not sure it can be replicated when I look at the implementation of _send_files in scp.py.
Currently scp doesn't show the progress of a file transfer. It would be helpful if it can display the progress, especially in the context of large files
Hi, not sure what this is about:
bag_dir_parent
'C:\Users\Alex\AppData\Local\Temp\tmpxniltb'
remote_path
'~/deposit_here/324-444'
scp.put(bag_dir_parent, remote_path, recursive="True")
Traceback (most recent call last):
File "", line 1, in
File "c:\Users\Alex\Dropbox\Python27\lib\site-packages\scp.py", line 143, in put
scp_command = (b'scp -t ', b'scp -r -t ')[recursive]
TypeError: tuple indices must be integers, not str
This is newest scp.py from pip, in Python 2.7.3. If you don't have any idea what's going on I can try to update Python version or duplicate in OSX/Linux later on...
Looks like py3 support was added. Could you update the release version?
Not sure what the best way to handle this is, but it seems like scp doesn't do it implicitly...
If I want to get a whole directory full of objects, but its parent directory name has a UUID appended to the known string, like:
scp /path/to/thing-I-know*/objects/
Is that a use case this module is intended to support? Or do I have to ssh in there first and try to figure out the directory name?
Hi,
I've been using this library for a little Python GUI client I've written for some of our depositors (I even managed to package it up into an executable for use on Windows, which was a real coup), and I'm generally very happy with it, but is there any way to make the transfer more resilient? A recursive directory transfer seems to be dying after around 64gb across a very fast local network. I know we're dealing with the limits of scp here, and it probably wouldn't be any easier to try to pack in an rsync or something, but if you have any ideas, I'd appreciate it!
I'm debugging a few problems I have with this module, and if I understand correctly, if there's an exception in downloading or uploading files, you can't use the client object again. You get a "this channel is not open" error. If I'm not mistaken, that's because you try to reuse the dead channel instead of creating a new one. Could it be because you haven't put self.close()
inside a finally
clause?
How can i copy directory from a remote machine using scp.py??
On Thu, Nov 7, 2013 at 8:04 PM, chubinou [email protected] wrote:
Hello,
I've made some tests with OpenSSH and Dropbear, and it seems that filenames should not be escaped when sending files (*)
I wrote a test suite to test this behavior against different version of paramiko and it runs good without the "shquote", the shquote has the effect of adding unnecessary " ' ", your split(" ", 2) seems correct though
test.py
import unittest
import paramiko
from paramiko import SSHClient
from scp import SCPClient
import os
class TestPut(unittest.TestCase):
def setUp(self):
self.ssh = SSHClient()
pkey = paramiko.DSSKey.from_private_key_file(
os.path.expanduser("~/.ssh/id_dsa")
)
self.ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
self.ssh.connect("127.0.0.1",pkey=pkey)
self.scp = SCPClient(self.ssh.get_transport())
def tearDown(self):
os.remove("/tmp/good ol' spaces #!${")
os.remove("./good ol' spaces #!${")
self.ssh.close()
def test_put(self):
with open("good ol' spaces #!${", "w") as fd:
fd.write("data")
self.scp.put("good ol' spaces #!${","/tmp/")
self.assertTrue(os.path.isfile("/tmp/good ol' spaces #!${"))
def test_get(self):
with open("/tmp/good ol' spaces #!${", "w") as fd:
fd.write("data")
self.scp.get("/tmp/good ol' spaces #!${","./")
self.assertTrue(os.path.isfile("./good ol' spaces #!${"))
Hello!
Thank you for this module. I noticed the transfer time can be quite slow (compared to command line scp). An order of magnitude or more slower. Is this an issue of the scp module or perhaps of the underlying paramiko/ssh settings?
Obviously scp.get("File with spaces.mov")
fails.
I want to scp a file(having a space in the file-name itself) from a remote machine. Here is the script which i use to scp a file.
import paramiko
from paramiko import SSHClient
from scp import SCPClient
ssh = SSHClient()
pkey = paramiko.RSAKey.from_private_key_file("/home/dummy/.ssh/id_rsa")
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect("example.com", username="root",pkey=pkey)
scp = SCPClient(ssh.get_transport())
filename='testing out'
scp.get(filename)
I am getting the following error
scp.SCPException: '\x01scp: testing: No such file or directory\n\x01scp: out: No such file or directory\n'
I have changed the scp.py script to match my criteria.
110,115c110
< remote_path=remote_path.replace(" "," ")
< remote_path='"'+remote_path+'"'
< print remote_path
< cmd = 'scp%s%s -f %s' % (rcsv, prsv, remote_path)
< print cmd
< self.channel.exec_command(cmd)
--- self.channel.exec_command('scp%s%s -f %s' % (rcsv, prsv, remote_path))
However,with this change the file is getting saved locally as "testing" instead of "testing out" file.
HEllo,
it would be cool if you could include CHANGELOG.md in the next releases on PyPI
There is a put
method that takes paths and a putfo
method that takes a file object instead. On the other hand there is only a get
method and no getfo
method. Could it be added?
It looks like the SCPClient context accepts a transport internally and then executes on a channel, but it doesn't appear to provide a mechanism for accessing the channel's return code.
with scp.SCPClient(transport) as scp_transport:
scp_transport.put(localpath, remotepath)
With the above code, the scp_transport.channel
is None
after put
is executed and scp_transport
does not contain the return code.
Proposal here is to capture return code information prior to running a self.close
at the end of put
. New code is below:
class SCPClient(object):
"""
An scp1 implementation, compatible with openssh scp.
Raises SCPException for all transport related errors. Local filesystem
and OS errors pass through.
Main public methods are .put and .get
The get method is controlled by the remote scp instance, and behaves
accordingly. This means that symlinks are resolved, and the transfer is
halted after too many levels of symlinks are detected.
The put method uses os.walk for recursion, and sends files accordingly.
Since scp doesn't support symlinks, we send file symlinks as the file
(matching scp behaviour), but we make no attempt at symlinked directories.
"""
def __init__(self, transport, buff_size=16384, socket_timeout=5.0,
progress=None, sanitize=_sh_quote):
"""
Create an scp1 client.
@param transport: an existing paramiko L{Transport}
@type transport: L{Transport}
@param buff_size: size of the scp send buffer.
@type buff_size: int
@param socket_timeout: channel socket timeout in seconds
@type socket_timeout: float
@param progress: callback - called with (filename, size, sent) during
transfers
@param sanitize: function - called with filename, should return
safe or escaped string. Uses _sh_quote by default.
@type progress: function(string, int, int)
"""
self.transport = transport
self.buff_size = buff_size
self.socket_timeout = socket_timeout
self.channel = None
self.preserve_times = False
self._progress = progress
self._recv_dir = b''
self._rename = False
self._utime = None
self.sanitize = sanitize
self._dirtimes = {}
self.return_code
def close(self):
"""close scp channel"""
if self.channel is not None:
self.return_code = self.channel.exit_status
self.channel.close()
self.channel = None
I need to send file content directly to a remote name over SCP.
I currently use this module by first creating a temporary file with the content and then copying said file, but it would be much nicer if I could do away with the logic and just pass the data directly.
Assuming this is not impossible for some reason, would you accept patches to that end?
I got an issue copying a .tar.gz from a server to local and afterwardy copying that .tar.gz to another server. It seems like get command is not blocking or returns before the whole file is copied, so in the next step a file which is not fully copied yet gets copied to another server, which results in having a corrupted file.
I guess directly copying between the remote machiens would work - still trying to figure out how.
Thanks for this -- it's useful! Would you be willing to package it and upload to pypi, so it can be installed & managed with standard tools?
Tested on 0.8.0 and 0.10.0 on OSX, python 3.4
$ python3
Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 00:54:21)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import paramiko
>>> import scp
>>> sshClient = paramiko.SSHClient()
>>> sshClient.load_system_host_keys()
>>> sshClient.set_missing_host_key_policy(paramiko.AutoAddPolicy())
>>> sshClient.connect("127.0.0.1", username="root", password="alpine", port=3333)
>>> scpClient = scp.SCPClient(sshClient.get_transport())
>>> scpClient.get("decryptedAPPs/RefMe")
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/scp.py", line 323, in _recv_all
command[code](msg[1:])
KeyError: b'\x01'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/scp.py", line 190, in get
self._recv_all()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/scp.py", line 325, in _recv_all
raise SCPException(str(msg).strip())
scp.SCPException: b'\x01scp: decryptedAPPs/RefMe: No such file or directory'
>>>
If I try to install this module i get a few errors:
Traceback (most recent call last):
File "C:\Python27\Scripts\pip-script.py", line 9, in <module>
load_entry_point('pip==1.5.6', 'console_scripts', 'pip')()
File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 542, in l
oad_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2569, in
load_entry_point
return ep.load()
File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2229, in
load
return self.resolve()
File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2235, in
resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\Python27\lib\site-packages\pip\__init__.py", line 13, in <module>
from pip.commands import commands, get_summaries, get_similar_commands
File "C:\Python27\lib\site-packages\pip\commands\__init__.py", line 6, in <mod
ule>
from pip.commands.bundle import BundleCommand
File "C:\Python27\lib\site-packages\pip\commands\bundle.py", line 6, in <modul
e>
from pip.commands.install import InstallCommand
File "C:\Python27\lib\site-packages\pip\commands\install.py", line 5, in <modu
le>
from pip.req import InstallRequirement, RequirementSet, parse_requirements
File "C:\Python27\lib\site-packages\pip\req\__init__.py", line 3, in <module>
from .req_install import InstallRequirement
File "C:\Python27\lib\site-packages\pip\req\req_install.py", line 33, in <modu
le>
from pip.utils import (
File "C:\Python27\lib\site-packages\pip\utils\__init__.py", line 100, in <modu
le>
def rmtree(dir, ignore_errors=False):
File "C:\Python27\lib\site-packages\pip\_vendor\retrying.py", line 47, in wrap
@six.wraps(f)
AttributeError: 'module' object has no attribute 'wraps'
or with python setup.py install:
C:\Users\User1\Desktop\scp.py-master>python setup.
py install
running install
running bdist_egg
running egg_info
writing requirements to scp.egg-info\requires.txt
writing scp.egg-info\PKG-INFO
writing top-level names to scp.egg-info\top_level.txt
writing dependency_links to scp.egg-info\dependency_links.txt
reading manifest file 'scp.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'scp.egg-info\SOURCES.txt'
installing library code to build\bdist.win32\egg
running install_lib
running build_py
creating build\bdist.win32\egg
copying build\lib\scp.py -> build\bdist.win32\egg
byte-compiling build\bdist.win32\egg\scp.py to scp.pyc
creating build\bdist.win32\egg\EGG-INFO
copying scp.egg-info\PKG-INFO -> build\bdist.win32\egg\EGG-INFO
copying scp.egg-info\SOURCES.txt -> build\bdist.win32\egg\EGG-INFO
copying scp.egg-info\dependency_links.txt -> build\bdist.win32\egg\EGG-INFO
copying scp.egg-info\requires.txt -> build\bdist.win32\egg\EGG-INFO
copying scp.egg-info\top_level.txt -> build\bdist.win32\egg\EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist\scp-0.10.2-py2.7.egg' and adding 'build\bdist.win32\egg' to it
removing 'build\bdist.win32\egg' (and everything under it)
Processing scp-0.10.2-py2.7.egg
Removing c:\python27\lib\site-packages\scp-0.10.2-py2.7.egg
Copying scp-0.10.2-py2.7.egg to c:\python27\lib\site-packages
scp 0.10.2 is already the active version in easy-install.pth
Installed c:\python27\lib\site-packages\scp-0.10.2-py2.7.egg
Traceback (most recent call last):
File "setup.py", line 21, in <module>
install_requires = ['paramiko'],
File "C:\Python27\lib\distutils\core.py", line 151, in setup
dist.run_commands()
File "C:\Python27\lib\distutils\dist.py", line 953, in run_commands
self.run_command(cmd)
File "C:\Python27\lib\distutils\dist.py", line 972, in run_command
cmd_obj.run()
File "C:\Python27\lib\site-packages\setuptools\command\install.py", line 67, i
n run
self.do_egg_install()
File "C:\Python27\lib\site-packages\setuptools\command\install.py", line 117,
in do_egg_install
cmd.run()
File "C:\Python27\lib\site-packages\setuptools\command\easy_install.py", line
370, in run
self.easy_install(spec, not self.no_deps)
File "C:\Python27\lib\site-packages\setuptools\command\easy_install.py", line
594, in easy_install
return self.install_item(None, spec, tmpdir, deps, True)
File "C:\Python27\lib\site-packages\setuptools\command\easy_install.py", line
645, in install_item
self.process_distribution(spec, dist, deps)
File "C:\Python27\lib\site-packages\setuptools\command\easy_install.py", line
688, in process_distribution
distreq.project_name, distreq.specs, requirement.extras
TypeError: __init__() takes exactly 2 arguments (4 given)
Is it possible to emulate the native scp
command's handling for when it comes across a permission error on a file within a recursive transfer? For example, the operate
user in these cases doesn't have access to the op-script.log file (only root
has read access). The scp
command handles permissions errors natively on mac OS X by just continuing with the next file:
$ scp -r [email protected]:/var/log ~/desktop-link/scp/
[email protected]'s password:
utmp 100% 0 0.0KB/s 00:00
pppoed_era_jpppoed_era_in_progress.log 100% 610 0.6KB/s 00:00
pppoed_era_jpppoed_era_in_progress.log.2 100% 611 0.6KB/s 00:00
**scp: /var/log/op-script.log: Permission denied**
pim_log 100% 14KB 14.2KB/s 00:00
pim_log.0.gz 100% 12KB 11.9KB/s 00:00
[...truncated for brevity...]
Here is an implementation of the SCP module for python:
Traceback (most recent call last):s/nprintz/desktop-link/scp/log/log/pppoed_era_jpppoed_era_in_progress.log.2
File "jaide.py", line 302, in copy_file
scp.get(scp_source, destination_file, recursive=True, preserve_times=True)
File "build/bdist.macosx-10.9-intel/egg/scp.py", line 143, in get
File "build/bdist.macosx-10.9-intel/egg/scp.py", line 276, in _recv_all
scp.SCPException: scp: /var/log/op-script.log: Permission denied
T1351695717 0 1351695717 0
I removed the error catching for SCPException so you can see the stack trace. This implementation is trying to receive a directory from a Juniper device (which runs on FreeBSD).
I didn't know if this was a known limitation that can't be helped, or if there is a way to skip and continue on to the next file. Perhaps an optional parameter could be passed to get()
and put()
to allow for skipping permission error files, or throwing an SCPException if the implementation desires it?
Local paths may be either PurePath
or Path
, while remote paths must be PurePath
.
def get(self, remote_path, local_path='',
recursive=False, preserve_times=False):
"""
Transfer files from remote host to localhost
@param remote_path: path to retreive from remote host. since this is
evaluated by scp on the remote host, shell wildcards and
Then call.
scp.get('/var/tmp/user-action_._', ".")
Hi,
this line https://github.com/jbardin/scp.py/blob/master/scp.py#L348 masks IOError exception when no file or directory exists, which makes it hard to catch the exception:
File "/home/ngoldin/virtualenv/lago-venv/lib/python2.7/site-packages/scp.py", line 198, in get
self._recv_all()
File "/home/ngoldin/virtualenv/lago-venv/lib/python2.7/site-packages/scp.py", line 348, in _recv_all
raise SCPException(asunicode(msg[1:]))
SCPException: scp: /var/koko/bobo: No such file or directory
If I understand correctly, the docs state that all IOError errors should be thrown.
This is not compatible with Python 3 because it mixes bytes and unicode (str).
Current put() method executes "scp -t " on remote server and receives file then being sent via chan.sendall() over the channel.
I'm getting intermittent failures and I'd like to get verbose output from scp to help trace issue. I can manually change scp.py to execute "scp -v -t " however no idea where the stdout is being sent to ?
Any ideas ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.