---

Generating Coverage Reports using gcov

I wrote this tutorial in 2023 before I realized that the Makefiles in all subsequent projects can easily be configured to use clang. Nevertheless, it is interesting to know that gcc can add coverage instrumentation to your binary just like clang can. This tutorial will show you how to set that up.

Prerequisites

This tutorial is based off of the Makefiles and test directory structure used in the starter code. In particular, we will make changes to p*_files/Makefile and p*_files/p*_tests/Makefile, and we will put our test cases in p*_files/p*_tests/.

Creating a test suite

We first need to add files to our test suite in the p*_tests directory. The Makefile has a rule to automaticlaly run every test in the directory. Look at the test cases included in the starter code for each project to see which files the Makefile is expecting for each case. For project 4, each case needs the following files:

• *.a
• *.unparse.expected
• *.err.expected

Modifying make test

p*_files/p*tests/Makefile will exit as soon as a test case fails. We do not necessarily want this behavior for coverage reporting. To ensure that make test will continue to run tests even after a failed case, we prepend '-' to the line which runs ac on your test case.

Example for Project 4: p4_tests/Makefile:

TESTFILES := $(wildcard *.a)
TESTS := $(TESTFILES:.a=.test)

.PHONY: all

all: $(TESTS)

%.test:
	@rm -f $*.unparse $*.err
	@touch $*.unparse
	@echo "TEST $*"
	# The '-' tells make to carry on even if this command exits with an error
	-@../ac $*.a -u $*.unparse 2> $*.err ;\
	PROG_EXIT_CODE=$$?;\
	if [ $$PROG_EXIT_CODE != 0 ]; then \
		echo "ac error:"; \
		cat $*.err; \
		exit 1; \
	fi; \
	diff -B --ignore-all-space $*.unparse $*.unparse.expected; \
	STDOUT_DIFF_EXIT=$$?;\
	exit $$STDOUT_DIFF_EXIT || echo "Tests passed"

clean:
	rm -f *.unparse *.err

Invoking g++ with --coverage

The --coverage option tells g++ to instrument the resulting binary for coverage analysis. We can add this option to the FLAGS variable in the top-level Makefile.

Example for Project 4: p4_files/Makefile:

FLAGS=--coverage -pedantic -Wall -Wextra -Wcast-align -Wcast-qual -Wctor-dtor-privacy -Wdisabled-optimization -Wformat=2 -Wuninitialized -Winit-self -Wmissing-declarations -Wmissing-include-dirs -Wold-style-cast -Woverloaded-virtual -Wredundant-decls -Wsign-conversion -Wsign-promo -Wstrict-overflow=5 -Wundef -Werror -Wno-unused -Wno-unused-parameter

Running tests

We are now ready to build the instrumented binary and run our test suite. From the project directory, run:

make clean
make test

Running the tests will cause the binary to generate some data in *.gcda files. This is where the coverage information comes from.

Generating and viewing the coverage report

Now we can generate and view the coverage reports for source files we're interested in using gcov. For example, if we are interested in unparse.cpp, run the following command in the project directory:

gcov unparse

This command prints the percent coverage in unparse.cpp and included files to the standard output. It also creates the file unparse.cpp.gcov, which has the counts of how many times each source line is executed throughout the test suite.

Updating `make clean`

We're done instrumenting our binary, now let's clean up. Add *.gcno *.gcda *.gcov *.d to the clean rule.

Example for Project 4: p4_files/Makefile:

clean:
	rm -rf *.output *.o *.cc *.hh *.gcno *.gcda *.gcov $(DEPS) ac parser.dot parser.png

---