This tutorial will run the Valgrind utility to check for undefined behavior. Valgrind can find problems like:
Table of Contents
We recommend running Valgrind on CAEN Linux because it can give different results on different machines and is difficult to configure (more details).
We’re assuming that you already have a folder with starter source code in it, e.g., p1-stats/.
$ pwd
/Users/awdeorio/src/eecs280/p1-stats
$ ls
Makefile main_test.out.correct p1_library.h stats_public_test.cpp
main.cpp main_test_data.tsv stats.cpp stats_tests.cpp
main_test.in p1_library.cpp stats.h
Your tests compile and run with no assertion failures. It’s OK if your tests don’t pass. If your test fails on an assertion, fix that before using Valgrind. Only use Valgrind on tests that produce no assertion failures.
$ make clean
rm -rvf *.exe *~ *.out *.dSYM *.stackdump
$ make test
g++ -Wall -Werror -pedantic -g --std=c++11 main.cpp stats.cpp p1_library.cpp -o main.exe
g++ -Wall -Werror -pedantic -g --std=c++11 stats_tests.cpp stats.cpp p1_library.cpp -o stats_tests.exe
g++ -Wall -Werror -pedantic -g --std=c++11 stats_public_test.cpp stats.cpp p1_library.cpp -o stats_public_test.exe
./stats_public_test.exe
count3
sum6
mean2
median2
mode1
min1
max3
stdev1
percentile2
./stats_tests.exe
test_sum_small_data_set
PASS!
./main.exe < main_test.in > main_test.out
diff main_test.out main_test.out.correct
You can connect to CAEN Linux. See the CAEN Linux tutorial for help.
$ ssh awdeorio@login.engin.umich.edu
The authenticity of host 'login.engin.umich.edu (141.213.74.65)' can't be established.
ECDSA key fingerprint is SHA256:LL0GPTtaVGa6gvv2kVpGq4ZULA1l5pw2wXC4dK3ymIk.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'login.engin.umich.edu,141.213.74.65' (ECDSA) to the list of known hosts.
Password:
Duo two-factor login for awdeorio
Enter a passcode or select one of the following options:
1. Duo Push to XXX-XXX-2735
2. Phone call to XXX-XXX-2735
3. SMS passcodes to XXX-XXX-2735
Passcode or option (1-3): 1
Success. Logging you in...
...
-bash-4.2$ exit
Connection to login.engin.umich.edu closed.
We’re going to synchronize our code to CAEN Linux many times, so be sure to set up these short cuts.
Be sure that you can avoid repeated two-factor authentication (2FA). In a separate terminal, connect to CAEN Linux with SSH so you won’t have to enter your password again.
$ ssh awdeorio@login.engin.umich.edu
Password:
Duo two-factor login for awdeorio
Enter a passcode or select one of the following options:
1. Duo Push to XXX-XXX-2735
2. Phone call to XXX-XXX-2735
3. SMS passcodes to XXX-XXX-2735
Passcode or option (1-3): 1
Success. Logging you in...
$
If you didn’t already, add a make sync shortcut to your Makefile.
Clean up (from your laptop).
$ hostname
manzana.local
$ make clean
rm -rvf *.exe *~ *.out *.dSYM *.stackdump
removed 'main.exe'
removed 'stats_tests.exe'
removed 'stats_public_test.exe'
Copy (from your laptop).
$ hostname
manzana.local
$ make sync
rsync \
-rtv \
--delete \
--exclude '.git*' \
--filter=':- .gitignore' \
./ \
awdeorio@login.engin.umich.edu:p1-stats-copy/
building file list ... done
created directory p1-stats-copy
./
Makefile
main.cpp
main_test.in
main_test.out.correct
main_test_data.tsv
p1_library.cpp
p1_library.h
stats.cpp
stats.h
stats_public_test.cpp
stats_tests.cpp
sent 9557 bytes received 268 bytes 19650.00 bytes/sec
total size is 8818 speedup is 0.90
Connect to CAEN Linux via ssh. If you’re already connected, you don’t need to connect again.
$ ssh awdeorio@login.engin.umich.edu
Success. Logging you in...
...
$ hostname
caen-vnc-vm16.engin.umich.edu
Change directory to the folder containing your source code. Depending on whether you copied with rsync or cloned with git, your folder name might be different.
$ ls
p1-stats
$ cd p1-stats
Reminder: you should be SSH’ed into CAEN Linux, and in your project directory.
$ hostname
caen-vnc-vm16.engin.umich.edu
$ pwd
/home/awdeorio/p1-stats
Clean and build.
$ make clean
rm -rvf *.exe *~ *.out *.dSYM *.stackdump
$ make stats_tests.exe
g++ -Wall -Werror -pedantic -g --std=c++11 stats_tests.cpp stats.cpp p1_library.cpp -o stats_tests.exe
Run without Valgrind. Make sure that no assertions fail. If your program fails on an assertion, stop and fix that first.
$ ./stats_tests.exe
test_sum_small_data_set
PASS!
Run with Valgrind. Notice the output said 0 errors from 0 contexts, and didn’t generated any errors.
$ valgrind ./stats_tests.exe
==24672== Memcheck, a memory error detector
==24672== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==24672== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==24672== Command: ./stats_tests.exe
==24672==
test_sum_small_data_set
PASS!
==24672==
==24672== HEAP SUMMARY:
==24672== in use at exit: 0 bytes in 0 blocks
==24672== total heap usage: 4 allocs, 4 frees, 80 bytes allocated
==24672==
==24672== All heap blocks were freed -- no leaks are possible
==24672==
==24672== For counts of detected and suppressed errors, rerun with: -v
==24672== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Reminder: you should be SSH’ed into CAEN Linux, and in your project directory.
$ hostname
caen-vnc-vm16.engin.umich.edu
$ pwd
/home/awdeorio/p1-stats
Clean and build.
$ make clean
rm -rvf *.exe *~ *.out *.dSYM *.stackdump
$ make main.exe
g++ -Wall -Werror -pedantic -g --std=c++11 main.cpp stats.cpp p1_library.cpp -o main.exe
Run without Valgrind. Make sure that no assertions fail. If your program fails on an assertion, stop and fix that first.
$ ./main.exe
enter a filename
main_test_data.tsv
enter a column name
B
reading column B from main_test_data.tsv
Summary (value: frequency)
6: 1
7: 1
8: 1
9: 1
10: 1
count = 5
sum = 40
mean = 8
stdev = 1.58114
median = 8
mode = 6
min = 6
max = 10
0th percentile = 6
25th percentile = 7
50th percentile = 8
75th percentile = 9
100th percentile = 10
Run with Valgrind. Enter user input if the program prompts you. Verify that the output said 0 errors from 0 contexts, and didn’t generated any errors.
$ valgrind ./main.exe
==25049== Memcheck, a memory error detector
==25049== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==25049== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==25049== Command: ./main.exe
==25049==
enter a filename
main_test_data.tsv
enter a column name
B
reading column B from main_test_data.tsv
Summary (value: frequency)
6: 1
7: 1
8: 1
9: 1
10: 1
count = 5
sum = 40
mean = 8
stdev = 1.58114
median = 8
mode = 6
min = 6
max = 10
0th percentile = 6
25th percentile = 7
50th percentile = 8
75th percentile = 9
100th percentile = 10
==25049==
==25049== HEAP SUMMARY:
==25049== in use at exit: 0 bytes in 0 blocks
==25049== total heap usage: 75 allocs, 75 frees, 11,226 bytes allocated
==25049==
==25049== All heap blocks were freed -- no leaks are possible
==25049==
==25049== For counts of detected and suppressed errors, rerun with: -v
==25049== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Different program inputs can produce different Valgrind output. Let’s run it again on the Real Data Example from project 1.
$ valgrind ./main.exe
==11464== Memcheck, a memory error detector
==11464== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==11464== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==11464== Command: ./main.exe
==11464==
enter a filename
ICPSR_30103/DS0001/30103-0001-Data.tsv
enter a column name
QFLAG
reading column QFLAG from ICPSR_30103/DS0001/30103-0001-Data.tsv
Summary (value: frequency)
1: 3009
2: 993
count = 4002
sum = 4995
mean = 1.24813
stdev = 0.431979
median = 1
mode = 1
min = 1
max = 2
0th percentile = 1
25th percentile = 1
50th percentile = 1
75th percentile = 1
100th percentile = 2
==11464==
==11464== HEAP SUMMARY:
==11464== in use at exit: 0 bytes in 0 blocks
==11464== total heap usage: 18,474 allocs, 18,474 frees, 5,396,190 bytes allocated
==11464==
==11464== All heap blocks were freed -- no leaks are possible
==11464==
==11464== For counts of detected and suppressed errors, rerun with: -v
==11464== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Let’s say we made a mistake in the sum() function in stats.cpp. We accidentally went off the end of a vector (notice <= instead of <).
double sum(vector<double> v) {
assert(!v.empty());
double sum = 0;
for (size_t i = 0; i <= v.size(); ++i) {
sum += v[i];
}
return sum;
}
Build and run without Valgrind. It passes on CAEN Linux, but because going off the end of a vector (or array) is undefined, it could get the wrong answer on the autograder!
$ make stats_tests.exe
g++ -Wall -Werror -pedantic -g --std=c++11 stats_tests.cpp stats.cpp p1_library.cpp -o stats_tests.exe
master awdeorio@caen-vnc-vm04 p1-stats-copy
$ ./stats_tests.exe
test_sum_small_data_set
PASS!
Run with Valgrind.
$ valgrind ./stats_tests.exe
==12081== Memcheck, a memory error detector
==12081== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==12081== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==12081== Command: ./stats_tests.exe
==12081==
test_sum_small_data_set
==12081== Invalid read of size 8
==12081== at 0x402A61: sum(std::vector<double, std::allocator<double> >) (stats.cpp:55)
==12081== by 0x4016FF: test_sum_small_data_set() (stats_tests.cpp:44)
==12081== by 0x401645: main (stats_tests.cpp:29)
==12081== Address 0x5a19158 is 0 bytes after a block of size 24 alloc'd
==12081== at 0x4C2A203: operator new(unsigned long) (vg_replace_malloc.c:334)
==12081== by 0x402369: __gnu_cxx::new_allocator<double>::allocate(unsigned long, void const*) (new_allocator.h:104)
==12081== by 0x4021AC: std::_Vector_base<double, std::allocator<double> >::_M_allocate(unsigned long) (stl_vector.h:168)
==12081== by 0x401FB4: std::_Vector_base<double, std::allocator<double> >::_M_create_storage(unsigned long) (stl_vector.h:181)
==12081== by 0x401B80: std::_Vector_base<double, std::allocator<double> >::_Vector_base(unsigned long, std::allocator<double> const&) (stl_vector.h:136)
==12081== by 0x4018EA: std::vector<double, std::allocator<double> >::vector(std::vector<double, std::allocator<double> > const&) (stl_vector.h:312)
==12081== by 0x4016F3: test_sum_small_data_set() (stats_tests.cpp:44)
==12081== by 0x401645: main (stats_tests.cpp:29)
==12081==
PASS!
==12081==
==12081== HEAP SUMMARY:
==12081== in use at exit: 0 bytes in 0 blocks
==12081== total heap usage: 4 allocs, 4 frees, 80 bytes allocated
==12081==
==12081== All heap blocks were freed -- no leaks are possible
==12081==
==12081== For counts of detected and suppressed errors, rerun with: -v
==12081== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
To debug this issue, look in the Valgrind output for line numbers from your own source code. In this example, stats.cpp:55 is the line which goes off the end of the vector.
for (size_t i = 0; i <= v.size(); ++i) { // line 54
sum += v[i]; // line 55
}
Let’s look at another example, where we forgot to initialize the
variable i:
for (size_t i; i < v.size(); ++i) { // line 54
sum += v[i]; // line 55
}
When running stats_tests.exe with Valgrind, the test case fails, but
before it does so, we get a Valgrind error:
==208== Conditional jump or move depends on uninitialised value(s)
==208== at 0x402F94: sum(std::vector<double, std::allocator<double> >) (stats.cpp:54)
==208== by 0x401924: test_sum_small_data_set() (stats_tests.cpp:44)
==208== by 0x40185E: main (stats_tests.cpp:29)
This tells us that line 54 in stats.cpp depends on the value of an
unitialized variable.
Skip this section for project 1. We’ll use dynamic memory (new and delete) on projects 2 through 5.
Use Valgrind to check for memory leaks. If you use the new operator
in your code, you should run Valgrind with the --leak-check=full
argument.
$ valgrind --leak-check=full ./main.exe
Let’s comment out the deletion of the Matrix object in
test_fill_basic in Matrix_tests.cpp from project 2.
TEST(test_fill_basic) {
Matrix *mat = new Matrix; // create a Matrix in dynamic memory
const int width = 3;
const int height = 5;
const int value = 42;
Matrix_init(mat, 3, 5);
Matrix_fill(mat, value);
for(int r = 0; r < height; ++r){
for(int c = 0; c < width; ++c){
ASSERT_EQUAL(*Matrix_at(mat, r, c), value);
}
}
//delete mat; // delete the Matrix
}
Recompile the code.
$ make Matrix_tests.exe
g++ --std=c++11 -Wall -Werror -pedantic -g Matrix_tests.cpp Matrix.cpp Matrix_test_helpers.cpp unit_test_framework.cpp -o Matrix_tests.exe
Run Valgrind with the --leak-check=full argument.
$ valgrind --leak-check=full ./Matrix_tests.exe
==100== Memcheck, a memory error detector
==100== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==100== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==100== Command: ./Matrix_tests.exe
==100==
Running test: test_fill_basic
PASS
*** Results ***
** Test case 'test_fill_basic': PASS
*** Summary ***
Out of 1 tests run:
0 failure(s), 0 error(s)
==100==
==100== HEAP SUMMARY:
==100== in use at exit: 1,072,712 bytes in 2 blocks
==100== total heap usage: 6 allocs, 4 frees, 1,073,992 bytes allocated
==100==
==100== 1,000,008 bytes in 1 blocks are definitely lost in loss record 2 of 2
==100== at 0x4C2E0EF: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==100== by 0x401F47: test_fill_basic() (Matrix_tests.cpp:15)
==100== by 0x403B46: TestCase::run(bool) (unit_test_framework.cpp:28)
==100== by 0x4041FB: TestSuite::run_tests(int, char**) (unit_test_framework.cpp:102)
==100== by 0x401FFF: main (Matrix_tests.cpp:42)
==100==
==100== LEAK SUMMARY:
==100== definitely lost: 1,000,008 bytes in 1 blocks
==100== indirectly lost: 0 bytes in 0 blocks
==100== possibly lost: 0 bytes in 0 blocks
==100== still reachable: 72,704 bytes in 1 blocks
==100== suppressed: 0 bytes in 0 blocks
==100== Reachable blocks (those to which a pointer was found) are not shown.
==100== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==100==
==100== For counts of detected and suppressed errors, rerun with: -v
==100== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Valgrid reports a leak, and the output includes the stack trace of
where the leaked object is created. The first line of the trace
indicates that the object was created by the new operator:
==100== at 0x4C2E0EF: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
The second line tells us that it occurred in line 15 of
Matrix_tests.cpp, in test_fill_basic:
==100== by 0x401F47: test_fill_basic() (Matrix_tests.cpp:15)
Line 15 of Matrix_test.cpp is the following:
Matrix *mat = new Matrix; // create a Matrix in dynamic memory
The remaining lines in the stack trace are internal to the unit test framework.
Let’s run Valgrind again, this time with the deletion line included.
$ valgrind --leak-check=full ./Matrix_tests.exe
==123== Memcheck, a memory error detector
==123== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==123== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==123== Command: ./Matrix_tests.exe
==123==
Running test: test_fill_basic
PASS
*** Results ***
** Test case 'test_fill_basic': PASS
*** Summary ***
Out of 1 tests run:
0 failure(s), 0 error(s)
==123==
==123== HEAP SUMMARY:
==123== in use at exit: 72,704 bytes in 1 blocks
==123== total heap usage: 6 allocs, 5 frees, 1,073,992 bytes allocated
==123==
==123== LEAK SUMMARY:
==123== definitely lost: 0 bytes in 0 blocks
==123== indirectly lost: 0 bytes in 0 blocks
==123== possibly lost: 0 bytes in 0 blocks
==123== still reachable: 72,704 bytes in 1 blocks
==123== suppressed: 0 bytes in 0 blocks
==123== Reachable blocks (those to which a pointer was found) are not shown.
==123== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==123==
==123== For counts of detected and suppressed errors, rerun with: -v
==123== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Valgrind no longer reports an error.