Value-Parameterized GTest

Introduction

Last time I talked about gtest in the gtest primer, and in this post we discuss the usage of “value-paramterized” tests. In general, when you write a simple test case and verified it is working, what is the next step? Maybe you want to add more test cases! But how? Adding them by “hard-coding” the logics in the source files? It’s not a good idea since cases may evolve and we may later add new corner cases, maybe months later. At that time, you may already forgot how to write a test case using gtest.

Value-parameterized tests are a good way to organize your test cases and group them in a logical manner e.g. test cases for a particular API. I use value-parameterized tests and a YAML configuration file to dynamically add/reduce my test cases, so that adding or removing a test case is as simple as updating the YAML file.

Problem setting

I finished writing test cases for the new APIs using gtest and I felt great! During the meeting, my mentor raised a question: “How can I add a new test case when you finished your internship? Do I need to learn gtest from scratch to do that?“. Definitely “yes” is not a good option, so I decided to update existing test implementations and use an easier approach to add/remove test cases. That is using value-parameterized tests together with a test config file written in YAML.

An example test config file should be like:

Foo:
  # Read-oriented API tests
  GetBarTest:
    - case_name: TestEmptyParams
      case_assertion: EXPECT_EQ
      case_description: "Test if right value returned when no arg is given"
      case_params:
        name: "barbar"
        verbose: true
      case_expected_result: []

Prepare the APIs

In this example we will add a simple getter method under the namespace of “Foo”. The only function we have is called GetBar which returns a vector of strings.

The header file foo.h contains:

namespace foo {

std::vector<std::string> GetBar();

}  // namespace foo

And its implementation foo.cpp is:

#include "foo.h"

namespace foo {

std::vector<std::string> GetBar() {
  return std::vector<std::string>{"hello", "world"};
}

}  // namespace foo

Setup the gtest

I want to be as general as possible. So I setup a separate fixture for namespace Foo. If we later added more APIs under Foo, then the adaptation will be easier.

Note that we use yaml-cpp to parse the config files and I will assume you already built and installed the library.

I created a header file called test_foo_core.h to setup the test cases for Foo and defined a value-parameterized fixture class called FooTest. All the API tests can afterwards be derived from the FooTest class to reuse the settings. The value type will be a YAML::Node that include the configurations for a specific test case.

The SetUp method of FooTest will retrieve the YAML::Node by using gtest’s GetParam method, from which we read the case_params and case_assertion.

We defined a function ReadTestCasesFromYaml to help us parse a specific test case from the test config file. You provide the fucntion test name (e.g. GetBarTest in the above example config file) and it will read all the test cases under that name and use them to run the tests.

Enough introduction and explanations! Go clone the tutorial repo and play around the tests!

Commands to run the build/test/coverage

In the tutorial repo, I also included the option to produce a code coverage report using gcov and lcov. Check it out by running make coverage under the build directory and read the reports in the coverage folder.

Clone the repo

git clone [email protected]:mcao2/value-parameterized-gtest.git

Build the source files and tests

mkdir build && cd build
cmake -DBUILD_TEST=ON .. && make -j4

Run the gtest

ctest -V

Sample output

Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
    Start 1: test_foo

1: Test command: value-parameterized-gtest/build/test/test_foo
1: Environment variables:
1:  TEST_FOO_CONFIG_PATH=value-parameterized-gtest/test/test_foo.yaml
1:  GTEST_OUTPUT=xml:value-parameterized-gtest/build/reports/gtest_test_foo.xml
1: Test timeout computed to be: 10000000
1: [GetBarTest] Num cases: 1
1: Read case TestEmptyParams
1: [==========] Running 1 test from 1 test suite.
1: [----------] Global test environment set-up.
1: [----------] 1 test from GetBarTestInstantiation/GetBarTest
1: [ RUN      ] GetBarTestInstantiation/GetBarTest.GetBar/TestEmptyParams
1: [       OK ] GetBarTestInstantiation/GetBarTest.GetBar/TestEmptyParams (0 ms)
1: [----------] 1 test from GetBarTestInstantiation/GetBarTest (0 ms total)
1:
1: [----------] Global test environment tear-down
1: [==========] 1 test from 1 test suite ran. (1 ms total)
1: [  PASSED  ] 1 test.
1/1 Test #1: test_foo .........................   Passed    0.48 sec

Create code coverage report

make coverage

To view the report, go to coverage and run:

python3 -m http.server

And you can view your report from 127.0.0.1:8000.

Sample output

Thank you and I hope this is useful :)