korniichuk / rk Goto Github PK

I know what the documentation says about this issue however my remote server does require a password and there is no way I can change that (ssh keys or other auth means are not supported). My users will be ok with it.

Is there any technical reasons (such as this) that apply here and would prevent this from working? I see most of the ssh invocations are happening via pexpect and that should be able to handle the password, however I've never done it. If you think this could work, I'll be happy to work on a PR

Hi,

I tried to run rk install-template on Windows 7 and it failed with the following error:

C:\Anaconda>rk install-template
Traceback (most recent call last):
  File "c:\anaconda\lib\runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "c:\anaconda\lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Anaconda\Scripts\rk.exe\__main__.py", line 5, in <module>
  File "c:\anaconda\lib\site-packages\rk\rk.py", line 7, in <module>
    from os import link, listdir, makedirs, remove, strerror
ImportError: cannot import name link

(As you can see, I'm using Anaconda distribution)

Is the utility is compatible with Windows? What would you recommend?

Cheers,
Denis

Just a few sentences would be helpful. I got here by a link, and I'm not sure if it solves the needs I have.

wouldn't it be possible to save kernelspecs in ~/.ipython/kernels?

Hi,

I followed the instructions, using iPython 3.1.0. When creating the new "Template" notebook, it will connect for a few seconds, and then begin a loop between "Dead kernel" and "Connected" notifications.

Thank you in advance.

While trying to use rk, I run into the following problems:

• python3.5 does not have raw_input(), instead it only has input() so I changed the corresponding 3 lines in the rk.py file.
• In my local install of jupyter, the kernel folder is located in ~/.local/share/jupyter/kernels/ which is in a different place than the documentation gives.
• For rk installed in user space, the rkscript should look for the configuration file with getusersitepackages() so I modified for my own purpose the rkscript to have

module_location = join(getusersitepackages(), module_name)

• Then I got the ImportError: No module named IPython.kernel.zmq.kernelapp which is already reported. The solution was not immediate to me but eventually I found that I had to change argv in kernel.json to python3 to use my local python3 and not the global python2.7 install on the remote machine.

Did it work eventually ? Nope. Now I'm facing an strange issue. I can ssh into the remote machine without being asked for passphrase or password since the authentication is based on kerberos. Strangely enough, rk asks me first my passphrase, then my password and both eventually fail. Not sure what's causing this. Here is part of the log:

[I 12:04:27.899 NotebookApp] KernelRestarter: restarting kernel (4/5)
WARNING:root:kernel a251e120-4f06-485e-938f-5e60c1e11505 restarted
/usr/lib/python3.5/getpass.py:92: GetPassWarning: Can not control echo on the terminal.
  passwd = fallback_getpass(prompt, stream)
Warning: Password input may be echoed.
remote's password: Traceback (most recent call last):
  File "/usr/lib/python3.5/getpass.py", line 70, in unix_getpass
    old = termios.tcgetattr(fd)     # a copy to save
termios.error: (25, 'Inappropriate ioctl for device')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "~/.local/bin/rkscript", line 230, in <module>
    paramiko_tunnel(v, remote_ports[k], remote_username_at_remote_host)
  File "~/.local/lib/python3.5/site-packages/rk/ssh/tunnel.py", line 321, in paramiko_tunnel
    password = getpass("%s's password: "%(server))
  File "/usr/lib/python3.5/getpass.py", line 92, in unix_getpass
    passwd = fallback_getpass(prompt, stream)
  File "/usr/lib/python3.5/getpass.py", line 127, in fallback_getpass
    return _raw_input(prompt, stream)
  File "/usr/lib/python3.5/getpass.py", line 149, in _raw_input
    raise EOFError
EOFError
[5897] Warning: unhandled RemoteError: Traceback (most recent call last):
  File "<string>", line 1072, in executetask
  File "<string>", line 1, in do_exec
  File "<remote exec>", line 55, in <module>
PermissionError: [Errno 13] Permission denied: '~/.ipython/profile_default/security/kernel-a251e120-4f06-485e-938f-5e60c1e11505.json'

Cheers.

I've followed the setup steps, and confirmed that I can log in to my remote host without a password, but when I run Jupyter and switch to the remote kernel, I get the following:

Traceback (most recent call last):
  File "/usr/local/bin/rkscript", line 218, in <module>
    remote_ports = ch.receive()
  File "/usr/local/lib/python3.4/site-packages/execnet/gateway_base.py", line 711, in receive
    raise self._getremoteerror() or EOFError()
execnet.gateway_base.RemoteError: Traceback (most recent call last):
  File "<string>", line 1039, in executetask
  File "<string>", line 1, in do_exec
  File "<remote exec>", line 11, in <module>
ImportError: No module named IPython.kernel.zmq.kernelapp

Perhaps a PYTHONPATH issue?

Running Python 3.4 on OX X 10.11 locally.

I usually set ssh to listen to different ports than the default 22, so is there a way to specify the port number?

First of all, great package, that's exactly what I was looking for. I still have an issue with using the remote conda environment: I am not sure the conda environment on the remote is used properly.

Ideally I would like to source it but the only option I found was to indicate the path of the remote conda environment python binary in kernel spec.

So when importing sys in the local notebook using the remote kernel, i get the correct environment python version (3.5):

sys.version
`3.5.1`

But clearly, the environment variables are not right (conda environment was clearly not sourced):

!python --version
Python 2.7.11

Is there a way to properly source the whole remote conda environment instead of just using its associated python binary?

@korniichuk any thoughts what's going wrong here?

Michaels-MacBook:~ michael$ jupyter notebook
[I 09:56:43.382 NotebookApp] Serving notebooks from local directory: /Users/michael
[I 09:56:43.382 NotebookApp] 0 active kernels
[I 09:56:43.382 NotebookApp] The IPython Notebook is running at: http://localhost:8888/
[I 09:56:43.382 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 09:56:57.407 NotebookApp] Creating new notebook in
[I 09:56:58.257 NotebookApp] Kernel started: 8cab0703-0950-4334-96c6-334f3a3f2d56
[W 09:56:58.362 NotebookApp] 404 GET /kernelspecs/desktop%20ipython%203/logo-64x64.png (::1) 31.53ms referer=http://localhost:8888/notebooks/Untitled3.ipynb?kernel_name=desktop%20ipython%203
Traceback (most recent call last):
File "/Applications/anaconda/bin/rkscript", line 230, in
paramiko_tunnel(v, remote_ports[k], remote_username_at_remote_host)
File "/Applications/anaconda/lib/python2.7/site-packages/rk/ssh/tunnel.py", line 320, in paramiko_tunnel
if not _try_passwordless_paramiko(server, keyfile):
File "/Applications/anaconda/lib/python2.7/site-packages/rk/ssh/tunnel.py", line 118, in _try_passwordless_paramiko
client.load_system_host_keys()
File "/Applications/anaconda/lib/python2.7/site-packages/paramiko/client.py", line 101, in load_system_host_keys
self._system_host_keys.load(filename)
File "/Applications/anaconda/lib/python2.7/site-packages/paramiko/hostkeys.py", line 101, in load
e = HostKeyEntry.from_line(line, lineno)
File "/Applications/anaconda/lib/python2.7/site-packages/paramiko/hostkeys.py", line 331, in from_line
key = RSAKey(data=decodebytes(key))
File "/Applications/anaconda/lib/python2.7/site-packages/paramiko/rsakey.py", line 64, in init
if msg.get_text() != 'ssh-rsa':
File "/Applications/anaconda/lib/python2.7/site-packages/paramiko/message.py", line 186, in get_text
return u(self.get_bytes(self.get_int()))
File "/Applications/anaconda/lib/python2.7/site-packages/paramiko/py3compat.py", line 53, in u
return s.decode(encoding)
File "/Applications/anaconda/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xf7 in position 1: invalid start byte
[1437] Warning: unhandled RemoteError: Traceback (most recent call last):
File "", line 1072, in executetask
File "", line 1, in do_exec
File "", line 66, in
File "D:\Users\Michael\Anaconda3\lib\site-packages\traitlets\config\application.py", line 587, in launch_instance
app = cls.instance(**kwargs)
File "D:\Users\Michael\Anaconda3\lib\site-packages\traitlets\config\configurable.py", line 415, in instance
'%s are being created.' % cls.name
traitlets.config.configurable.MultipleInstanceError: Multiple incompatible subclass instances of IPKernelApp are being created.

[W 09:57:08.386 NotebookApp] Timeout waiting for kernel_info reply from 8cab0703-0950-4334-96c6-334f3a3f2d56
[I 09:57:10.257 NotebookApp] KernelRestarter: restarting kernel (1/5)
WARNING:root:kernel 8cab0703-0950-4334-96c6-334f3a3f2d56 restarted
^C[I 09:57:13.542 NotebookApp] interrupted
Serving notebooks from local directory: /Users/michael
1 active kernels
The IPython Notebook is running at: http://localhost:8888/
Shutdown this notebook server (y/[n])? y
[C 09:57:14.778 NotebookApp] Shutdown confirmed
[I 09:57:14.779 NotebookApp] Shutting down kernels
[I 09:57:15.825 NotebookApp] Kernel shutdown: 8cab0703-0950-4334-96c6-334f3a3f2d56

So... I've been racking my brain about this and can't figure it out, please help?

I've seen the video on how to set it up... I am even getting connection errors when I specify a bad name in kernel.json. For example it asks me for my password if I specify the wrong user... all fine. Then I open up the terminal and do ipython console --kernel fionn, where fionn is (from kernel.json):

{
 "argv": [
  "rkscript",
  "python3",
  "{connection_file}",
  "[email protected]"
 ],
 "display_name": "Fionn",
 "language": "python"
}

I had to change one thing in rkscript, that is, I'm calling: makegateway("popen//service .profile//ssh=%s//python=%s"..., without the popen//service .profile// it wouldn't work cause this machine uses loaded modules and a lot of environment variable setting up. Anyway... one more proof that I am connected properly, because without popen//service .profile it wouldn't work (so I think it's a good practice to read the .profile or .bashrc or both...) But then I go to the console and write import sys; sys.version and I get 3.4.3 which is the version on my local computer!... the office machine to which I want to connect to and run the kernel has 3.4.0. What am I missing here?!... Please help! (I've also tried adding popen//service .profile after ssh=%s, but that didn't change anything...

edit: oh yeah... to be sure I also checked sys.path so this returns the local path... instead of the office machine path.

edit: I noticed one more thing. I'm getting: Port forwarding stopped uncleanly: [Errno 98] Address already in use. Might this mean that the connection isn't properly set up and I end up with the commands being executed on my local machine?

some more details
So the way I connect to my work machine is through VPN and another machine so, my .ssh/config looks like this:

ControlMaster auto
ControlPath /home/razvan/.ssh/tmp/%h_%p_%r

Host office
    HostName IP-MACHINE-1

Host fionn
    ProxyCommand ssh razvan@office nc -q0 IP-MACHINE-2 %p
    LocalForward 8888 127.0.0.1:8888

And I connect to fionn to work...

from site import getsitepackages

returns an ImportError when ran in a virtualenv. It seems to be an open bug since 2012…

Is there an easy workaround, to be able to launch jupyter-notebook with remote kernels, from within a virtualenv?

Thanks

Starting a remote kernel (as per you example) or even running rkscript manually always results in this error:

Traceback (most recent call last):
  File "/root/.local/bin/rkscript", line 110, in <module>
    create_messages()
  File "/root/.local/bin/rkscript", line 103, in create_messages
    config_messages_rel_path = config["config_messages_rel_path"]
  File "/root/.local/lib/python2.7/site-packages/configobj.py", line 554, in __getitem__
    val = dict.__getitem__(self, key)
KeyError: 'config_messages_rel_path'

Installed rk via pip-2.7 in a FreeBSD 10.3 jail