I wrote this set of instructions to help my fellow grad students in the STAT department at UC - Santa Cruz better utilize the available resources in our department. I have tried to make this tutorial general, but may at times refer to resources (servers) specific to our department.

Feel free to leave a comment below!

STAT Servers at UCSC

There are several Linux interactive servers available at UCSC. You can log in to an interactive server and execute commands, like you would on your local machine, in a terminal.

(The most powerful interactive servers at UCSC STAT are jerez, mencia, and muscat.)

In order to efficiently use the resources on the Linux server, it helps to know how to efficiently navigate a UNIX terminal. If you have a MAC or LINUX machine, you have a UNIX terminal. If you have Windows 10, you can now access a LINUX BASH shell natively. Though the setup is quite straight-forward, it is out of the scope of this post.

In this post, I will review the basics of navigating and executing commands on a UNIX terminal. I will then show you how to do it on a server. It’s often good to start with the end in mind. So I’ll start with a motivating example.

Motivating Example

The following shows one way to run an R-script on an interactive server:

1. Launch SFTP (secure file transfer protocol) to send / receive files from a server (say jerez)
   • sftp myUserName@jerez.soe.ucsc.edu
2. Send your script to the server
   • put myFile.R
3. Log on to the server
   • ssh -Y myUserName@jerez.soe.ucsc.edu
   • The -Y option allows you to open graphics on the server. It’s not necessary if you don’t intend to view graphs over the network.
4. Execute the command on your script
   • Rscript myFile.R &
   • Note here that the command Rscript acts on the script myFile.R.
   • The ampersand & at the end is optional. It allows you to execute your script “in the background”. Meaning, you can do other things in your terminal after executing a command (e.g. Rscript). You can even log off the server while the job runs. Without the ampersand, you cannot use the terminal until the job is finished. If you exit the server before the job is done, your job stops and your results will not be saved. Therefore, add & at the end of your commands when you have long-running jobs.
   • Also, if you plan to run many jobs, you should add nice before your command. (e.g. nice Rscript myFile.R &) This basically prevents you from becoming a server hog who uses most of the resources on the server.
5. Send results back to your computer
   • get myOutput.rda

Avoid Excessive Typing

You can create an alias for a server to reduce typing username@server every time Just add this to ~/.ssh/config, after replacing the tags.

#! This file belongs in ~/.ssh/config
Host <shortName>
  HostName <server>
  User <username>
  ForwardX11 yes

For example,

#! This file belongs in ~/.ssh/config
Host jerez
  HostName jerez.soe.ucsc.edu
  User myUserName
  ForwardX11 yes

Now, to log in to the server jerez.soe.ucsc.edu, you only need to type ssh jerez. You can do this for multiple servers, so your config file might look like this.

#! This file belongs in ~/.ssh/config
Host jerez
  HostName jerez.soe.ucsc.edu
  User myUserName
  ForwardX11 yes

Host mencia
  HostName mencia.soe.ucsc.edu
  User myUserName
  ForwardX11 yes

Host muscay
  HostName muscat.soe.ucsc.edu
  User myUserName
  ForwardX11 yes

To avoid re-entering your password every time when you log in to the same server though ssh or sftp, you can do the following.

1. Generate an ssh key locally by executing ssh-keygen. Then hit enter whenever you’re prompted to do something.
2. Then run the following line in your terminal to send the key to your server. Enter your user-name and password when prompted. Replace the tags accordingly.

   cat ~/.ssh/id_rsa.pub | ssh <username>@<server> 'cat >> ~/.ssh/authorized_keys'
   

   For example,

   cat ~/.ssh/id_rsa.pub | ssh myUserName@jerez.soe.ucsc.edu 'cat >> ~/.ssh/authorized_keys'
   

   All this does is send the public key you generated before to the server in a specific location. Also this is all very safe. You’re not saving your password on the server. Your just allowing the server to recognize your machine. The only problem is if someone stole your computer and you had it left on, someone could log in to your server without typing a password.

Now, you won’t be prompted for your password every time you log in from the machine you’re currently on.

Note: For some reason, sometimes, you need to send your public key to another location. Another common location is ~/.ssh/authorized_keys2. So, change the previous command to:

cat ~/.ssh/id_rsa.pub | ssh myUserName@jerez.soe.ucsc.edu 'cat >> ~/.ssh/authorized_keys2'

Git

SFTP is a great tool, but the process above can be tedious for managing multiple files. There’s a better way if you have a lot more files to move around. You could put your project in a Git repository and pull your repository onto the server.

Using Vim with TMUX

TMUX is a terminal multiplexer. It basically allows you to open multiple screens in a terminal at a time. Say you start a tmux session on a server. You can start a job in tmux, leave it running, leave your tmux session, leave the server, and the job will still be running.

Vim is a text editor. It has strange key bindings that can be very convenient when you get used to them.

Using Vim with tmux has increased my productivity as a developer. Getting familiar with these tools also greatly enhances productivity in a terminal, where there is no GUI, but where Vim and TMUX are installed.

Vim is installed on most compute servers (including the STAT servers).

tmux is not installed on the STAT servers, but you can install it following these instructions. On your personal devices, where you have root access, you can install tmux using a package manager.

I usually configure my tmux and vim by adding two configuration files so that I can navigate my terminal a little better, and run scripts line by line.

Add these two files to their respective locations:

• Add tmux.conf to ~/.tmux.conf
  • note the . before tmux.conf
  • If you already have a ~/.tmux.conf, you may want to either back it up and replace it, or edit it by appending the new content.
• Add line-feeder.vim ~/.vim/plugin/line-feeder.vim
  • Similarly, if you don’t have a ~/.vim/plugin/ folder, you will need to create it.

Now, you should be set. Here’s a demo of what you can do with vim and tmux, and these additional config files.

Credits:

• I got tmux.conf from my adviser, David Dahl, at BYU during my time there as a Statistics masters student.
• The video was made using a screen-caster for Linux called kazam. The key-presses were captured and displayed using screenkey.

• ← Previous Post