This tutorial will show you how to copy source code from your Laptop to CAEN Linux. CAEN Linux is a server in the University data center that runs the Linux operating system. It’s useful for making sure that your code works on a computer that is a lot like the autograder.
Why use CAEN Linux? One reason is that it has the same compiler used by the autograder, which is useful for making sure you’ll get the same results as on your laptop. For example, on an Apple laptop,
g++ is actually an alias for the Clang compiler supplied by Apple. Similar things can happen on Windows.
$ g++ --version Apple clang version 12.0.0 (clang-1220.127.116.11)
After logging in to CAEN Linux, you can use the same compiler used by the autograder.
$ g++ g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
We’re assuming that you already have a folder with starter source code in it, e.g.,
p1-stats/. You’ve created new files and added function stubs to make the code base compile, e.g.,
main.cpp. We expect that your unit tests will fail because many functions are stubs.
$ pwd /Users/awdeorio/src/eecs280/p1-stats $ ls Makefile main_test.out.correct p1_library.h stats_public_test.cpp main.cpp main_test_data.tsv stats.cpp stats_tests.cpp main_test.in p1_library.cpp stats.h $ make clean rm -rvf *.exe *~ *.out *.dSYM *.stackdump $ make test g++ -Wall -Werror -pedantic -g --std=c++11 main.cpp stats.cpp p1_library.cpp -o main.exe g++ -Wall -Werror -pedantic -g --std=c++11 stats_tests.cpp stats.cpp p1_library.cpp -o stats_tests.exe g++ -Wall -Werror -pedantic -g --std=c++11 stats_public_test.cpp stats.cpp p1_library.cpp -o stats_public_test.exe ./stats_public_test.exe Assertion failed: (false), function count, file stats.cpp, line 12. countmake: *** [test] Abort trap: 6
You have installed
rsync. Your versions might be different.
$ ssh -V OpenSSH_7.4p1, LibreSSL 2.5.0 $ rsync --version rsync version 2.6.9 protocol version 29
Test log in
CAEN is the information technology (IT) services department for the University of Michigan (U-M) College of Engineering. Everyone who registers for an EECS class (like EECS 280) should receive a CAEN account automatically by the first day class. If you register after the first day of class, you should get your account within 24 hours of registration.
You’ll need a two factor authentication app set up on your mobile device. Make sure that you have the Duo Mobile app installed and configured according the ITCS documentation.
Test if you have access to CAEN Linux computers. Be sure to change
awdeorio to your own uniqname. If this is the first time you’re logging in, you’ll need to confirm that you want to continue connecting.
$ ssh firstname.lastname@example.org The authenticity of host 'login.engin.umich.edu (18.104.22.168)' can't be established. ECDSA key fingerprint is SHA256:LL0GPTtaVGa6gvv2kVpGq4ZULA1l5pw2wXC4dK3ymIk. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'login.engin.umich.edu,22.214.171.124' (ECDSA) to the list of known hosts. Password: Duo two-factor login for awdeorio Enter a passcode or select one of the following options: 1. Duo Push to XXX-XXX-2735 2. Phone call to XXX-XXX-2735 3. SMS passcodes to XXX-XXX-2735 Passcode or option (1-3): 1 Success. Logging you in... ... -bash-4.2$
If you see an error like
Could not chdir to home directory /home/awdeorio: No such file or directory, a common reason is that you don’t have a home directory set up yet. Request one at the IFS Storage Request page. Note that it may take a few hours before your home directory is available across all CAEN systems. If you already have one, you should see “You already have IFS Storage!”.
If you’re still having trouble accessing your account, see the CAEN Help Desk.
Exit as soon as your test is successful.
$ exit Connection to login.engin.umich.edu closed.
Copy files with
Next, we will copy our source code to CAEN Linux using the
rsync command line program.
We don’t want to copy any compiled binary files, so clean up first.
$ make clean rm -rvf *.exe *~ *.out *.dSYM *.stackdump removed 'main.exe' removed 'stats_tests.exe' removed 'stats_public_test.exe'
Next, copy files using
rsync. Remember to change
awdeorio to your username.
$ pwd /Users/awdeorio/src/eecs280/p1-stats $ rsync -rtv --exclude '.git*' ../p1-stats/ email@example.com:p1-stats-copy/ building file list ... done created directory p1-stats-copy ./ Makefile main.cpp main_test.in main_test.out.correct main_test_data.tsv p1_library.cpp p1_library.h stats.cpp stats.h stats_public_test.cpp stats_tests.cpp sent 9557 bytes received 268 bytes 19650.00 bytes/sec total size is 8818 speedup is 0.90
Now connect to CAEN Linux. We’re going to get a shell on a remote computer. Don’t forget to change
awdeorio to your own uniqname.
$ pwd # folder on awdeorio's laptop /Users/awdeorio/src/eecs280/p1-stats $ hostname # name of awdeorio's laptop manzana.local $ ssh firstname.lastname@example.org # connect to CAEN $ pwd # folder on CAEN computer /home/awdeorio $ hostname # name of a CAEN computer caen-vnc-vm16.engin.umich.edu
Notice that the folder we copied is there on the CAEN Linux machine.
$ ls p1-stats-copy
Change directory into the copied folder and double-check that all binary files are cleaned up.
$ cd p1-stats-copy $ make clean
Compile and run main, just like we did before using the shell on our local machine.
$ make main.exe $ ./main.exe hello from main!
Run the regression test. It fails on the assertion we added, just like it did on our local machine.
$ make test g++ -Wall -Werror -pedantic -g --std=c++11 main.cpp stats.cpp p1_library.cpp -o main.exe g++ -Wall -Werror -pedantic -g --std=c++11 stats_tests.cpp stats.cpp p1_library.cpp -o stats_tests.exe g++ -Wall -Werror -pedantic -g --std=c++11 stats_public_test.cpp stats.cpp p1_library.cpp -o stats_public_test.exe ./stats_public_test.exe stats_public_test.exe: stats.cpp:12: int count(std::vector<double>): Assertion `false' failed. make: *** [test] Aborted
$ hostname caen-vnc-vm16.engin.umich.edu $ exit $ hostname manzana.local
Avoiding repeated 2FA
You’ve noticed that each time we use
rsync, CAEN requires us to re-authenticate with 2FA (two factor authentication, using our phone). You can configure SSH to share one connection and only authenticate once. Note that
rsync uses SSH under the hood.
NOTE: This approach does not work on Cygwin. Consider using the Windows Subsystem for Linux (WSL) if you want this feature on Windows. Refer back the section above on Windows.
On your laptop, navigate to your home directory. The tilde (
~) symbol means “my home directory” at the command line. Then, navigate to the hidden
.ssh directory. Note: files and directories that start with a dot (
.) are hidden.
$ cd ~ # This will move to your home directory $ ls # list files src ... $ ls -A # list files, including hidden files .ssh src ... $ cd .ssh/ $ cd ~/.ssh/ # Alternative: do it in one command
Add some lines to the SSH config file, which lives in
~/.ssh/config. Alternatively, you can use a text editor to make the changes.
$ echo -e '# SSH multiplexing\nHost *\n ControlMaster auto\n ControlPath ~/.ssh/master-%r@%h:%p' >> ~/.ssh/config $ chmod 600 ~/.ssh/config
Let’s double-check and make sure you see this chunk in your
~/.ssh/config file. You can find a more in depth explanation at the Linux Journal.
$ cat ~/.ssh/config # SSH multiplexing Host * ControlMaster auto ControlPath ~/.ssh/master-%r@%h:%p
SSH into CAEN Linux. You’ll need to use 2FA.
$ ssh email@example.com Password: Duo two-factor login for awdeorio Enter a passcode or select one of the following options: 1. Duo Push to XXX-XXX-2735 2. Phone call to XXX-XXX-2735 3. SMS passcodes to XXX-XXX-2735 Passcode or option (1-3): 1 Success. Logging you in... $
Now, open a second terminal. We’re going to use SSH again, this time via
rsync. Notice that no authentication is required. Cool!
$ pwd /Users/awdeorio/src/eecs280/p1-stats $ rsync -rtv --exclude '.git*' ../p1-stats/ firstname.lastname@example.org:p1-stats-copy/ building file list ... done sent 273 bytes received 20 bytes 586.00 bytes/sec total size is 13015 speedup is 44.42
Version control on CAEN Linux
We can also check out a copy of our committed code on CAEN Linux.
SSH to a CAEN Linux machine and see the copy we made earlier using
$ ssh email@example.com $ ls p1-stats-copy # this is from our rsync'ed copy earlier
Notice that the copy is not under version control.
$ cd p1-stats-copy $ git status fatal: Not a git repository (or any of the parent directories): .git
Change directory and
git clone your repo.
$ cd ~ # This will move to your home directory $ git clone https://gitlab.eecs.umich.edu/awdeorio/p1-stats.git $ ls p1-stats p1-stats-copy
Synchronizing deleted files
rsync to synchronize deleted files. In other words, if it’s gone on your laptop, delete it on CAEN.
$ rsync -rtv --delete --exclude '.git*' ../p1-stats/ firstname.lastname@example.org:p1-stats-copy/
Don’t synchronize Git-ignored files
rsync not to synchronize files ignored by Git. You can also combine this option with
$ rsync -rtv --exclude '.git*' --filter=':- .gitignore' ../p1-stats/ email@example.com:p1-stats-copy/
make sync shortcut
Avoid typing a long
rsync command by adding a Makefile target. Add these lines to the end of your
Makefile. Notice that’s a TAB character before
# Copy files to CAEN Linux sync : rsync \ -rtv \ --delete \ --exclude '.git*' \ --filter=':- .gitignore' \ ../p1-stats/ \ firstname.lastname@example.org:p1-stats-copy/
Now you can type
make sync as a short cut.
$ make sync rsync \ -rtv \ --delete \ --exclude '.git*' \ --filter=':- .gitignore' \ ../p1-stats/ \ email@example.com:p1-stats-copy/ building file list ... done ./ Makefile sent 446 bytes received 60 bytes 1012.00 bytes/sec total size is 8939 speedup is 17.67
PITFALL Some IDEs (Xcode, for example) will automatically insert several spaces instead of a TAB. Makefiles must have a TAB just before a command (
rsync in this case). On macOS, Option-Tab will insert a literal TAB. Here’s an example of the problem:
$ make sync make: Nothing to be done for `sync'.