Last modified: 2025-02-21 @ 83c48cf
Roll Your Own GTest
Something I keep telling myself and others about when it comes to programming is “There is no such thing as magic”. By this I mean that anything you encounter in the world of programming can be understood. And you should try to make an effort to understand things that you currently don’t.
One such thing I’ve been thinking about is the
GoogleTest
framework for writing tests
in C++. On the face of it, it’s pretty simple. You define a test suite as a
class with all the data and setup needed for your suite of tests, and with some
macros you instantiate a bunch of tests:
#include <gtest/gtest.h>
class FooSuite : public ::testing::Test
{
// test suite data goes here
};
TEST_F(FooSuite, Foo)
{
// test some stuff
}
TEST_F(FooSuite, Bar)
{
// test some more stuff
}
And in your main
function you just do something like
#include <gtest/gtest.h>
int main(int argc, char **argv)
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
You don’t even have to write this main test runner yourself, GoogleTest
provides a program like this that you can simply add as a linked library in your
CMakeLists.txt
. This will run every TEST_F
macro you defined earlier and if
they all pass, you’re happy. Really simple stuff.
But there’s something hidden here. How does GoogleTest
actually find all the
tests you defined? You never explicitly call them anywhere. What would you even
call? What magic1 is happening here that they are called automatically
simply by defining them? Does the test executable somehow open itself, getting
addresses of every constructor and test body through
dlsym(3)
or something?
As it turns out, there is no such thing as magic. The trick is actually pretty simple.
Some code blocks below will have a path attached to them. They are relative paths into the companion repo to this post.
The knowledge of the secrets of Nature
To illustrate what is going on behind the scenes, here’s a little example
program featuring everyone’s favorite class: Widget
!
[examples/01-widget.cpp]
#include <iostream>
class Widget
{
public:
Widget() { std::cout << "Widget constructor" << std::endl; }
~Widget() { std::cout << "Widget destructor" << std::endl; }
};
int main()
{
std::cout << "Hello, world!" << std::endl;
auto widget = Widget{};
return 0;
}
If we compile and run this little program, we get the following expected output:
$ make 01-widget && ./01-widget
Hello, world!
Widget constructor
Widget destructor
No surprises here. In main
, we print “Hello, world!”, then the Widget
constructor is called when we create our widget, which prints “Widget
constructor”. Then as the program exits and cleans up its stack, the Widget
destructor is called, printing “Widget destructor”. Simple stuff.
Now watch what happens if I move the widget outside the main
function, into
the global scope.
[examples/02-widget.cpp]
#include <iostream>
class Widget
{
public:
Widget() { std::cout << "Widget constructor" << std::endl; }
~Widget() { std::cout << "Widget destructor" << std::endl; }
};
auto widget = Widget{};
int main()
{
std::cout << "Hello, world!" << std::endl;
return 0;
}
Compile and run:
$ make 02-widget && ./02-widget
Widget constructor
Hello, world!
Widget destructor
“Widget constructor” prints before “Hello, world!”! This means that we ran
code before entering main
. This is because variables in non-local variables
with static storage duration (such as our global widget
) are initialized as
part of program startup. Before entering main
2.
This is the trick that makes GoogleTest
tick, and why you don’t have to
manually call the tests you define. Their constructors register themselves under
the hood as instances of non-local variables to whatever handles the actual
execution of tests, which itself can be a non-local variable such that
everything is set up automatically.
The second example can actually be simplified even further by having our static global be defined as part of the class definition:
[examples/03-widget.cpp]
#include <iostream>
class Widget
{
public:
Widget() { std::cout << "Widget constructor" << std::endl; }
~Widget() { std::cout << "Widget destructor" << std::endl; }
} widget;
int main()
{
std::cout << "Hello, world!" << std::endl;
return 0;
}
This compiles and prints exactly the same as example 02.
Rolling your own
Now GoogleTest
has a lot of features and to keep this post to a reasonable
length, I’m going to implement a much simpler unit testing framework inspired by
GoogleTest
. But it will only be able to define test cases. No test suites,
parameterized test suites, type-parameterized test cases, etc. Extremely simple
stuff. And only a couple of helper macros to use for tests. Specifically:
UNIT_TEST(test_name)
: The main macro defining a testEXPECT(expr)
: Expectsexpr
to be true. Test fails but continues if it’s false.ASSERT(expr)
: Expectsexpr
to be true. Test fails and terminates if it’s false.
I will also not take special care to do everything “properly”. This is a learning thing, not production code.
The basics
Let’s start with a base class called UnitTest
. I’ll put this in a namespace
called smalltest
as the name of this micro-framework:
[src/smalltest.hpp]
#pragma once
#include <string>
namespace smalltest {
class UnitTest
{
protected:
virtual void Run() = 0;
};
}
It’s an abstract class with a single method, Run()
, which is what the user
will define as the main body of the test. Every instance of this class (every
call to UNIT_TEST(test_name)
) will define a class that inherits from this base
class, creates a static global, which calls the constructor that registers the
test in the test executor.
The registry
Now we need a central registry to hold all the tests and execute them when it’s
time. And there should only be one instance of this class. And it should be
static-initialized. Sounds like a singleton to me. Let’s call it TestRegistry
[src/smalltest.hpp]
namespace smalltest {
// ---<snip>---
class TestRegistry
{
public:
static TestRegistry *GetInstance()
{
if (instance_ == nullptr)
{
instance_ = new TestRegistry{};
}
return instance_;
}
private:
TestRegistry() {}
static TestRegistry *instance_;
};
}
Simple singleton. Now we need to hold on to all UnitTest
objects somehow. How
about each test having a name that we map from a pointer to the test instance,
and also keeping track of whether the test passed?
[src/smalltest.hpp]
class TestRegistry
{
public:
// ---<snip>---
void RegisterTest(UnitTest *test, std::string name)
{
tests_.insert({test, {name, true}});
}
private:
TestRegistry() {}
using TestData = std::tuple<std::string, bool>;
std::map<UnitTest*, TestData> tests_{};
static TestRegistry *instance_;
};
}
No magic here. But let’s get back to the trick now.
The main macro
We want the UNIT_TEST(test_name)
macro to expand to a class derived from
UnitTest
that instantiates a static global and registers itself in to the
TestRegistry
in the constructor. Basically we want it to expand to something
like this:
class test_name : public ::smalltest::UnitTest
{
public:
test_name() : name_{"test_name"}
{
auto* registry =
::smalltest::TestRegistry::GetInstance();
registry->RegisterTest(name_, this);
}
protected:
void Run() override; // to be defined
private:
std::string name_;
} test_name; // here is the instantiation trick
And since we want to use it like
UNIT_TEST(foo)
{
EXPECT(true);
}
the last line of the macro should implement the Run
method. Some minor tweaks
to the above gets the macro implemented quickly:
[src/smalltest.hpp]
#define UNIT_TEST(test_name) \
class test_name : public ::smalltest::UnitTest \
{ \
public: \
test_name() : name_{ #test_name } \
{ \
auto* registry = \
::smalltest::TestRegistry::GetInstance();\
registry->RegisterTest(name_, this); \
} \
protected: \
void Run() override; \
private: \
std::string name_; \
} test_name; \
void test_name::Run() // the "magic" line (no such thing!)
Now the macro instantiates a new UnitTest
-derived class that automatically
registers itself in the TestRegistry
, and the final line means that the block
following the macro is the definition of the Run
method.
Now we need to make the TestRegistry
execute all the Run
methods. To do
this, it first needs to become a friend of UnitTest
, since I figured Run
should be a protected method:
[src/smalltest.hpp]
class UnitTest
{
protected:
virtual void Run() = 0;
friend class TestRegistry;
};
And running the test is a simple for-each loop:
[src/smalltest.hpp]
class TestRegistry
{
public:
// ---<snip>---
void RunTests()
{
size_t failed_tests = 0;
for (auto& [test, test_data] : tests_)
{
auto& [name, passed] = test_data;
test->Run();
if (!passed)
{
failed_tests++;
}
}
if (failed_tests == 0)
{
return EXIT_SUCCESS;
}
return EXIT_FAILURE;
}
private:
// ---<snip>---
};
}
Of course, a test case needs to be able to inform the registry that it has failed. So let’s add a little method for that as well:
[src/smalltest.hpp]
class TestRegistry
{
public:
// ---<snip>---
void FailTest(UnitTest *test)
{
auto& [name, passed] = tests_[test];
passed = false;
}
private:
// ---<snip>---
};
}
And we’re almost done, actually! We just need a couple of small details.
The helper macros
I mentioned that we wanted EXPECT(expr)
and ASSERT(expr)
calls inside the
tests to cause them to fail (and optionally abort the test). Since they will be
called inside the implementation of the Run
method, the aborting behavior can
be implemented with a simple return
statement. Otherwise the macros are
completely identical.
[src/smalltest.hpp]
#define EXPECT(expr) \
do \
{ \
if (!(expr)) \
{ \
auto registry = \
::smalltest::TestRegistry::GetInstance(); \
registry->FailTest(this); \
} \
} while (false)
#define ASSERT(expr) \
do \
{ \
if (!(expr)) \
{ \
auto registry = \
::smalltest::TestRegistry::GetInstance(); \
registry->FailTest(this); \
return; \
} \
} while (false)
Here I’m using the do-while(false) trick to make them behave more like function statements. Now the functionality is complete! The only thing left is initialization.
The init
The whole machinery kicks off when we define our test runner program as follows:
#include "smalltest.hpp"
smalltest::TestRegistry *smalltest::TestRegistry::instance_
= nullptr;
int main()
{
auto *registry = smalltest::TestRegistry::GetInstance();
return registry->RunTests();
}
That nullptr
line is needed because non-const static members must be
initialized out of line.
In fact, since this is what every test running executable using smalltest will look like, we can provide a convenience macro for it:
[src/smalltest.hpp]
#define RUN_SMALLTEST() \
smalltest::TestRegistry *smalltest::TestRegistry::instance_ \
= nullptr; \
int main() \
{ \
auto *registry = smalltest::TestRegistry::GetInstance();\
return registry->RunTests(); \
}
And it can be used! Here’s a simple little program:
[examples/footest.cpp]
#include "smalltest.hpp"
#include <iostream>
UNIT_TEST(foo)
{
EXPECT(true);
std::cout << "This is fine" << std::endl;
}
UNIT_TEST(bar)
{
EXPECT(false);
std::cout << "This is less fine" << std::endl;
}
UNIT_TEST(baz)
{
ASSERT(false);
std::cout << "This will never print" << std::endl;
}
RUN_SMALLTEST();
If we compile and run this program:
$ make footest && ./footest
This is fine
This is less fine
$ echo $?
1
Yeah, not much happens. But the exit code was non-zero! So we have failing
tests, which is to be expected, since both bar
and baz
are expecting
false
to be true
, which I don’t have to tell you will never be the case.
The finishing touch
We did save the name of each test. Let’s simply print out which test is about to run, and at the end of all tests print how many passed and which ones failed.
[src/smalltest.hpp]
1class TestRegistry
2{
3public:
4 // ---<snip>---
5 void RunTests()
6 {
7 size_t passed_tests = 0;
8 size_t failed_tests = 0;
9 std::vector<std::string> fails{};
10 for (auto& [test, test_data] : tests_)
11 {
12 auto& [name, passed] = test_data;
13 std::cout
14 << "RUN: "
15 << name
16 << std::endl;
17 test->Run();
18
19 if (passed)
20 {
21 passed_tests++;
22 std::cout
23 << "PASS: "
24 << name
25 << std::endl;
26 }
27 else
28 {
29 failed_tests++;
30 fails.emplace_back(name);
31 std::cout
32 << "FAIL: "
33 << name
34 << std::endl;
35 }
36 }
37
38 std::cout
39 << "\nTests run: " << tests_.size()
40 << "\nTests passed:" << passed_tests
41 << "\nTests failed:" << failed_tests
42 << std::endl;
43
44 if (failed_tests == 0)
45 {
46 return EXIT_SUCCESS;
47 }
48
49 std::cout << "\nFailed tests:\n";
50 for (const auto& test : fails)
51 {
52 std::cout << "\t" << test << "\n";
53 }
54 return EXIT_FAILURE;
55 }
56private:
57 // ---<snip>---
58};
59}
Now if we compile and run footest.cpp
from before:
$ make footest && ./footest
RUN: foo
This is fine
PASS: foo
RUN: bar
This is less fine
FAIL: bar
RUN: baz
FAIL: baz
Tests run: 3
Tests passed: 1
Tests failed: 2
Failed tests:
bar
baz
And look at that! We have a barebones unit testing framework!
Wrapping up
All code blocks from above have paths relative to this companion
repo. It contains all the
examples from this post and a couple extras. The entire framework is just above
100 lines of C++, contained in a single header file (src/smalltest.hpp
), which
makes it really simple to use for any testing purpose. This is not the only
small C++ testing framework, far from it. There are multiple projects that
implement this exact thing. And of course, GoogleTest
is the most complex
example. But this barebones demonstration contains just enough functionality to
be understandable and extendable to implement features from any other similar
testing framework you might find.
Hopefully you learned something, dear reader. I know I did. And here’s a short, incomplete list of features to implement if the mood hits you to do something similar based on this:
- More
EXPECT_
andASSERT_
helper macros - Prettier printing (e.g. in the
EXPECT
andASSERT
macros) - Exception handling
- Proper test suites, like
GoogleTest
. - Test shuffling
- Threading
The main takeaway?
There is no such thing as magic, though there is such a thing as a knowledge of the secrets of Nature.
- H. Rider Haggard, from the book “She”
(I have to admit, I have not actually read the book. I just really like the quote, as it summarizes my feelings towards topics in programming that seem mysterious at first.)