p1-stats

CAEN Linux

This tutorial will copy source code to CAEN Linux and make sure that it works on a computer that is a lot like the autograder. We’ll also check for problems like uninitialized variables with valgrind.

Prerequisites

We’re assuming that you already have a folder with starter source code in it, e.g., p1-stats/. You’ve created new files and added function stubs to make the code base compile, e.g., stats.cpp and main.cpp. We expect that your unit tests will fail because many functions are stubs.

$ pwd
/Users/awdeorio/src/eecs280/p1-stats
$ ls
Makefile      main_test.out.correct  p1_library.h  stats_public_test.cpp
main.cpp      main_test_data.tsv     stats.cpp     stats_tests.cpp
main_test.in  p1_library.cpp         stats.h
$ make clean
rm -rvf *.exe *~ *.out *.dSYM *.stackdump
$ make test
g++ -Wall -Werror -pedantic -g --std=c++11 main.cpp stats.cpp p1_library.cpp -o main.exe
g++ -Wall -Werror -pedantic -g --std=c++11 stats_tests.cpp stats.cpp p1_library.cpp -o stats_tests.exe
g++ -Wall -Werror -pedantic -g --std=c++11 stats_public_test.cpp stats.cpp p1_library.cpp -o stats_public_test.exe
./stats_public_test.exe
Assertion failed: (false), function count, file stats.cpp, line 12.
countmake: *** [test] Abort trap: 6

You have installed ssh and rsync. Your versions might be different.

$ ssh -V
OpenSSH_7.4p1, LibreSSL 2.5.0
$ rsync --version
rsync  version 2.6.9  protocol version 29

Test log in

CAEN is the information technology (IT) services department for the University of Michigan (U-M) College of Engineering. Everyone who registers for an EECS class (like EECS 280) should receive a CAEN account automatically.

You’ll need a two factor authentication app set up on your mobile device. Make sure that you have the Duo Mobile app installed and configured according the ITCS documentation.

Test if you have access to CAEN Linux computers. Be sure to change awdeorio to your own uniqname. If this is the first time you’re logging in, you’ll need to confirm that you want to continue connecting.

Tip: If you’re having trouble accessing your account, see the CAEN Hotline https://caen.engin.umich.edu/hotline/.

$ ssh awdeorio@login.engin.umich.edu
The authenticity of host 'login.engin.umich.edu (141.213.74.65)' can't be established.
ECDSA key fingerprint is SHA256:LL0GPTtaVGa6gvv2kVpGq4ZULA1l5pw2wXC4dK3ymIk.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'login.engin.umich.edu,141.213.74.65' (ECDSA) to the list of known hosts.
Password: 
Duo two-factor login for awdeorio

Enter a passcode or select one of the following options:

 1. Duo Push to XXX-XXX-2735
 2. Phone call to XXX-XXX-2735
 3. SMS passcodes to XXX-XXX-2735

Passcode or option (1-3): 1
Success. Logging you in...
...
-bash-4.2$ 

Exit as soon as your test is successful.

$ exit
Connection to login.engin.umich.edu closed.

Copy files with rsync

Next, we will copy our source code to CAEN Linux using the rsync command line program.

We don’t want to copy any compiled binary files, so clean up first.

$ make clean
rm -rvf *.exe *~ *.out *.dSYM *.stackdump
removed 'main.exe'
removed 'stats_tests.exe'
removed 'stats_public_test.exe'

Next, copy files using rsync. Remember to change awdeorio to your username.

$ pwd
/Users/awdeorio/src/eecs280/p1-stats
$ rsync -rtv --exclude '.git*' ./ awdeorio@login.engin.umich.edu:p1-stats-copy/
building file list ... done
created directory p1-stats-copy
./
Makefile
main.cpp
main_test.in
main_test.out.correct
main_test_data.tsv
p1_library.cpp
p1_library.h
stats.cpp
stats.h
stats_public_test.cpp
stats_tests.cpp

sent 9557 bytes  received 268 bytes  19650.00 bytes/sec
total size is 8818  speedup is 0.90

Connect with ssh

Now connect to CAEN Linux. We’re going to get a shell on a remote computer. Don’t forget to change awdeorio to your own uniqname.

$ pwd                                  # folder on awdeorio's laptop
/Users/awdeorio/src/eecs280/p1-stats
$ hostname                             # name of awdeorio's laptop
manzana.local
$ ssh awdeorio@login.engin.umich.edu   # connect to CAEN
$ pwd                                  # folder on CAEN computer
/home/awdeorio
$ hostname                             # name of a CAEN computer
caen-vnc-vm16.engin.umich.edu

Notice that the folder we copied is there on the CAEN Linux machine.

$ ls
p1-stats-copy

Change directory into the copied folder and double-check that all binary files are cleaned up.

$ cd p1-stats-copy
$ make clean

Compile and run main, just like we did before using the shell on our local machine.

$ make main.exe
$ ./main.exe
hello from main!

Run the regression test. It fails on the assertion we added, just like it did on our local machine.

$ make test
g++ -Wall -Werror -pedantic -g --std=c++11 main.cpp stats.cpp p1_library.cpp -o main.exe
g++ -Wall -Werror -pedantic -g --std=c++11 stats_tests.cpp stats.cpp p1_library.cpp -o stats_tests.exe
g++ -Wall -Werror -pedantic -g --std=c++11 stats_public_test.cpp stats.cpp p1_library.cpp -o stats_public_test.exe
./stats_public_test.exe
stats_public_test.exe: stats.cpp:12: int count(std::vector<double>): Assertion `false' failed.
make: *** [test] Aborted

Log out.

$ hostname
caen-vnc-vm16.engin.umich.edu
$ exit
$ hostname
manzana.local

Avoiding repeated 2FA

You’ve noticed that each time we use ssh or rsync, CAEN requires us to re-authenticate with 2FA (two factor authentication, using our phone). You can configure SSH to share one connection and only authenticate once. Note that rsync uses SSH under the hood.

NOTE: this approach does not work on Cygwin. Consider using the Windows Subsystem for Linux (WSL) if you want this feature on Windows. Refer back the section above on Windows.

On your laptop, navigate to your home directory. The tilde (~) symbol means “my home directory” at the command line. Then, navigate to the hidden .ssh directory. Note: files and directories that start with a dot (.) are hidden.

$ cd ~         # This will move to your home directory
$ ls           # list files
src ...
$ ls -A        # list files, including hidden files
.ssh src  ...
$ cd .ssh/
$ cd ~/.ssh/   # Alternative: do it in one command

Add some lines to the SSH config file, which lives in ~/.ssh/config. Alternatively, you can use a text editor to make the changes.

$ echo -e '# SSH multiplexing\nHost *\n  ControlMaster auto\n  ControlPath ~/.ssh/master-%r@%h:%p' >> ~/.ssh/config
$ chmod 600 ~/.ssh/config

Let’s double-check and make sure you see this chunk in your ~/.ssh/config file. You can find a more in depth explanation at the Linux Journal.

$ cat ~/.ssh/config
# SSH multiplexing
Host *
  ControlMaster auto
  ControlPath ~/.ssh/master-%r@%h:%p

SSH into CAEN Linux. You’ll need to use 2FA.

$ ssh awdeorio@login.engin.umich.edu
Password:
Duo two-factor login for awdeorio

Enter a passcode or select one of the following options:

 1. Duo Push to XXX-XXX-2735
 2. Phone call to XXX-XXX-2735
 3. SMS passcodes to XXX-XXX-2735

Passcode or option (1-3): 1
Success. Logging you in...

$

Now, open a second terminal. We’re going to use SSH again, this time via rsync. Notice that no authentication is required. Cool!

$ pwd
/Users/awdeorio/src/eecs280/p1-stats
$ rsync -rtv --exclude '.git*' ./ awdeorio@login.engin.umich.edu:p1-stats-copy/
building file list ... done

sent 273 bytes  received 20 bytes  586.00 bytes/sec
total size is 13015  speedup is 44.42

Version control on CAEN Linux

We can also check out a copy of our committed code on CAEN Linux.

SSH to a CAEN Linux machine and see the copy we made earlier using rsync.

$ ssh awdeorio@login.engin.umich.edu
$ ls
p1-stats-copy  # this is from our rsync'ed copy earlier

Notice that the copy is not under version control.

$ cd p1-stats-copy
$ git status
fatal: Not a git repository (or any of the parent directories): .git

Change directory and git clone your repo.

$ cd ~  # This will move to your home directory
$ git clone https://gitlab.eecs.umich.edu/awdeorio/p1-stats.git
$ ls
p1-stats p1-stats-copy

Pro-tips

Synchronizing deleted files

Tell rsync to synchronize deleted files. In other words, if it’s gone on your laptop, delete it on CAEN.

$ rsync -rtv --delete --exclude '.git*' ./ awdeorio@login.engin.umich.edu:p1-stats-copy/

Don’t synchronize Git-ignored files

Tell rsync not to synchronize files ignored by Git. You can also combine this option with --delete.

$ rsync -rtv --exclude '.git*' --filter=':- .gitignore' ./ awdeorio@login.engin.umich.edu:p1-stats-copy/

make sync shortcut

Avoid typing a long rsync command by adding a Makefile target. Add these lines to the end of your Makefile. Notice that’s a TAB character before rsync.

# Copy files to CAEN Linux
sync :
	rsync \
  -rtv \
  --delete \
  --exclude '.git*' \
  --filter=':- .gitignore' \
  ./ \
  awdeorio@login.engin.umich.edu:p1-stats-copy/

Now you can type make sync as a short cut.

$ make sync
rsync \
  -rtv \
  --delete \
  --exclude '.git*' \
  --filter=':- .gitignore' \
  ./ \
  awdeorio@login.engin.umich.edu:p1-stats-copy/
building file list ... done
./
Makefile

sent 446 bytes  received 60 bytes  1012.00 bytes/sec
total size is 8939  speedup is 17.67

PITFALL Some IDEs (Xcode, for example) will automatically insert several spaces instead of a TAB. Makefiles must have a TAB just before a command (rsync in this case). On macOS, Option-Tab will insert a literal TAB. Here’s an example of the problem:

$ make sync
make: Nothing to be done for `sync'.

Next steps

Return to the main set up tutorial.