ncar / pangeo

NCAR-specific tools, scripts, and other information relating to the Pangeo project

NCAR-Specific Pangeo Tools

NCAR-specific tools, scripts, and other information relating to the Pangeo project

pangeo's Issues

Archive this repo?

Wondering if it is time to archive this repo? JupyterHub services (https://jupyterhub.ucar.edu/) are now live for both Cheyenne and Casper. We also have globally available dask configuration files (DASK_ROOT_CONFIG=/glade/u/apps/config/dask). Besides launching Jupyter and Dask, the main thing I see in this repo is a demo notebook. This seems like it could be moved to a new location.

What defaults do we want in jobqueue.yaml?

Right now, jobqueue.yaml includes memory: "100 GB" -- is that necessary, or should we only include that in example notebooks that explicitly need the additional memory?

Wait for user before starting ssh tunnel

It would be nice to have a Press any key to continue... prompt before attempting to open the ssh tunnel -- I often run this script and then do something else for the 30 seconds or minute that it takes to connect. By the time I come back to my terminal window, the ssh request to open the tunnel has timed out so I enter my yubikey token at

Running create_tunnel...
ssh -N -L localhost:8888:r6i6n33:8890 [email protected]
Tunnel PID: 66102
Use ^C to kill ssh tunnel
Waiting for tunnel to open
appending output to nohup.out
Token_Response:

and get

PID 66102 died!

Does not work for users with tcsh as default on cheyenne

  (ssh -t ${REM_USER}@cheyenne.ucar.edu "$(declare -f launch_job); launch_job ${REM_USER} ${WALLTIME} ${CONDA_ENV} ${ACCOUNT}" ; echo $? > .ret_code) | tee .last_run.log

Assumes the user's shell on cheyenne is bash; need a way to force ssh to use bash even if default shell for user is tcsh

launch-pangeo-on-cheyenne script host name issue

In the launch-pangeo-on-cheyenne script, it is not capturing the host name correctly (the port # is fine). In the tunnel command to ssh in:

cmd="ssh -N -L localhost:${LOCALPORT}:{REMOTE HOST}:${PORT} ${REM_USER}@cheyenne.ucar.edu"

it is not able to connect to http://localhost:8888/ without hardcoding the correct host name, opening a new terminal, and correctly 'ssh'ing in. There was a suggestion to have a unique log file each time you run from @mnlevy1981

Support dav cluster?

Right now the client-side script only connects to cheyenne (not geyser or caldera), and I don't know if the server-side instructions will require any additional steps for dav support. I assume users will have to install conda on the dav side as well?

Instead of providing conda environment, point to ncar-pangeo-tutorial

Instead of providing server-side/, this package should specify that users follow the instructions in NCAR/ncar-pangeo-tutorial to set up proper environments on cheyenne and then the default environment in the client-side script will be base (that will let users switch to analysis or any other)