Accessing your account using Mac terminal:
ssh [email protected] (NYU Langone HPC)
ssh [email protected] (NYU Tandon HPC)
Then enter your password.
(The setting up process for both HPCs are the same)
To avoid entering password each time you want to use your HPC account, generate a SSH key:
ssh-keygen -t rsa -b 4096 -C "[email protected]"
Replace "[email protected]" with your actual email address. This email is used as a label for your key, making it easier to identify its purpose later.
Press Enter to Accept the Default File Location.
Enter a Secure Passphrase (Press Enter to skip this part)
Display your public SSH key:
cat ~/.ssh/id_rsa.pub
Copy it to the clipboard:
pbcopy < ~/.ssh/id_rsa.pub
Now log in to your HPC account. Before appending your public SSH key to the ~/.ssh/authorized_keys file, ensure that the .ssh directory exists and has the correct permissions:
mkdir -p ~/.ssh
chmod 700 ~/.ssh
Open the authorized_keys file with a text editor available on the HPC system, such as 'nano' or 'vi':
vi ~/.ssh/authorized_keys
Go to the end of the line press i then Enter and paste the public SSH key there and go back by typing :wq
type the following to check if the key is there:
cat ~/.ssh/id_rsa.pub
Check if you can login without password.
First make sure your VPN is connected to the NYU.
For Greene you need Cisco Any Connect and for Bigpurple you need BIG-IP Edge.
Search for "Remote - SSH" and install the extension.
Create a "New remote".
Enter SSH [email protected] in the search bar.
Select directory /.ssh/config
Add the following information in the new window and save it
Host nyu-langone-hpc (Optional name)
HostName bigpurple.nyumc.org
User NetID
ForwardAgent yes
IdentityFile <private key>
Press "fn + F1" and look for "Remote-SSH: Connect to host..." and choose the config you created earlier.
In the new window which is connected to the HPC you should be able to access your folders through search bar.
You submit the job with sbatch:
$ sbatch myscript.s or myscript.sh
And monitor its progress with:
$ squeue -u UserName
Cancel a specific job if you know the ID:
scancel JOB_ID
Cancel all your jobs that you have submitted:
scancel -u YOUR_USERNAME
When you open your notebook (.ipynb) on VScode on top right there is an option which allows you to choose your environment.
You either create one or use the existing environments.
If the environmnet you are looking to use is not in the list of search bar follow the following instruction:
- Make sure you already installed Python and Anaconda on your system.
- press F1 and type "Python: Select Interpreter".
- In the opened window select "Enter interpreter Path...".
- Paste the sys path to the environment you wish to use.
To install a package within an environment, follow these three steps:
module purge
module load condaenvs/new/YourEnvironment
pip install Package
To install a package within an existing container, do the following: (you can check the slurm file for correct paths)
module purge
module load singularity/3.7.1 # Depending on the singularity you are using
singularity exec --bind /gpfs/scratch/pa2297:/gpfs/scratch/pa2297
> /gpfs/scratch/pa2297/Singularity/tf_lu…sif\
> /bin/bash
Apptainer> python -m pip install --user Package
exit
Parisima Abdali: [email protected]