Unit testing for c code in simple way

[ARibecaJob]

Hi,

I want to share you some info on how we can add some checking (unit testing) to existing code.
There are many unit testing libraries floating around the web but I think for our needs we can use a C features to mimic unit testing usage: macros.

In C language, we can have parts of code that can be enabled by defining a symbol (keyword):

#ifdef ENABLE_TEST
puts("some test enabled!");
#endif

if the compiler during compilation will find ENABLE_TEST defined, the code inside the block will be compiled.
Let's make a working example:

#include <stdio.h>

int main(void)
{
	int a = 5;
	printf("the value of a is %d\n", a);
	
#ifdef ENABLE_TEST
	puts("test enabled!");
	printf("a is %s than 10\n", (a < 10) ? "<" : ">=");
	puts("end of test!");	
#endif

	return 0;
}

if we compile it with gcc:

gcc main.c && ./a.out

we have as output:

the value of a is 5

but if we compile it declaring ENABLE_TEST definition using -D parameter

gcc -DENABLE_TEST main.c && ./a.out

we'll have

the value of a is 5
test enabled!
a is < than 10
end of test!

As you can see, we can easily enable or disable this kind of checking.
Let's do a better example:

#include <stdio.h>

#ifdef ENABLE_TEST
#define str__(a) #a
#define STR(a) str__(a)
#define TEST(condition) \
		if (!(condition)) printf("condition '%s' fails at row %u in function %s inside file %s\n", \
		STR(condition), __LINE__, __FUNCTION__, __FILE__)
#else
#define TEST(unused)
#endif

int main(void)
{
	int a = 5;
	printf("the value of a is %d\n", a);
	
	// it will fail, condition must be true!
	TEST(a > 10);
	// nothing happens
	TEST(a < 7);

	return 0;
}

In this example the macro TEST will evaluate only if ENABLE_TEST is defined, otherwise nothing happens.
Again if we compile and run it

gcc -DENABLE_TEST main.c && ./a.out

we have

the value of a is 5
condition 'a > 10' fails at line 19 in function main inside main.c file

We can eventually adopt this to encapsulated a unit testing library too:

// please note: simplified example
#ifdef ENABLE_TEST
#include "unit_test_files.c"
#define TEST(condition) unit_test_func(condition)
#else
#define TEST(unused)
#endif

Any ideas ?

Otherwise this seems like a good candidate for unit testing: https://github.com/sheredom/utest.h

[ma595]

Hi Alessio! Thanks for the raising the issue. By 'Unit testing' we mean testing small pieces of code, possibly functions or collections of functions under different input conditions.

There are a number of advantages to adopting Unit test frameworks when testing code. They encourage modularity thereby promoting separation of concerns; we can write tests on functions, groups of functions and organise tests together. In addition, unit testing frameworks come with many helper functions (many more than simply asserts) including checks on floating point values, inequalities, fixtures (keeping data for multiple tests), which have all been well tested for us.

What is suggested above is more akin to writing assertions than unit tests and runs the risk that it may make the code more difficult to read due to extra complexity and length. I think that undertaking testing in this way would likely miss some of the benefits of unit testing described above.

The suggestion of uTest is a good one. I propose that I experiment with uTest in the ONEFlux pipeline first and submit a PR for you to review? I can then demonstrate how and why a unit testing framework has advantages. This will also give me an opportunity to assess the differences between Check and uTest and make a more informed opinion about which we should use in future. I believe Check is more feature rich which may prove useful down the line but it would probably be useful to have a discussion at the development meetings around this. We can summarise the outcome in this issue thread.

[ARibecaJob]

Hi,

as per the agreements made at our last meeting, I report a small comparative analysis between "check" and "utest.h" unit test libraries.

A brief summary:

url

check: https://libcheck.github.io/check/
utest.h: https://github.com/sheredom/utest.h

license comparison

utest.h: public domain
check: LPGPL

usage

check: a full library that needs to be compiled
utest.h: a simple header file to add to your code

analysis

Let's analyze a small example that is taken from check test package:

int main(void)
{
    int number_failed;
    Suite *s; 
    SRunner *sr;

    s = money_suite();
    sr = srunner_create(s);

    srunner_run_all(sr, CK_NORMAL);
    number_failed = srunner_ntests_failed(sr);
    srunner_free(sr);
	
    return (number_failed == 0) ? EXIT_SUCCESS : EXIT_FAILURE;
}

Suite and SRunner are two data types of check library.
They are used to create and run a suite of tests.
utest.h library instead does not have this feature so these tests are run individually.
It should be noted that with utest library we may not specify a main function and directly use the macro UTEST_MAIN();

Let's see these tests:

START_TEST(test_money_create)
{
    ck_assert_int_eq(money_amount(five_dollars), 5);
    ck_assert_str_eq(money_currency(five_dollars), "USD");
}
END_TEST

START_TEST(test_money_create_neg)
{
    Money *m = money_create(-1, "USD");

    ck_assert_msg(m == NULL,
                  "NULL should be returned on attempt to create with "
                  "a negative amount");
}
END_TEST

START_TEST(test_money_create_zero)
{
    Money *m = money_create(0, "USD");

    if (money_amount(m) != 0)
    {
        ck_abort_msg("Zero is a valid amount of money");
    }
}
END_TEST

these tests are simple and need no explanation.
utest library however does not have the feature to specify a custom message
through its functions but we can mimic it with standard assert function.

Let's see how these tests are called:

Suite * money_suite(void)
{
    Suite *s;
    TCase *tc_core;
    TCase *tc_limits;

    s = suite_create("Money");

    /* Core test case */
    tc_core = tcase_create("Core");

    tcase_add_checked_fixture(tc_core, setup, teardown);
    tcase_add_test(tc_core, test_money_create);
    suite_add_tcase(s, tc_core);

    /* Limits test case */
    tc_limits = tcase_create("Limits");

    tcase_add_test(tc_limits, test_money_create_neg);
    tcase_add_test(tc_limits, test_money_create_zero);
    suite_add_tcase(s, tc_limits);

    return s;
}

Summing up this function create one suite called Money with two test cases called Core and Limits
Core test case has 1 test: test_money_create
Limit test case has 2 tests: test_money_create_neg and test_money_create_zero

Again TCase is a check data type used to create a named test case.
We can mimic this feature easily with utest library.
Skipping to tcase_add_checked_fixture func, we read from the documentation:

Checked fixture functions are run before and after each unit test

We can use the fixtured testcase with utest, from the documentation:

A fixtured testcase is one in which there is a struct that is instantiated that can be shared across multiple testcases.

Running the example we have:

Running suite(s): Money
100%: Checks: 3, Failures: 0, Errors: 0

We can rewrite that example using _utest_in this way:

#include <stdio.h>
#include <assert.h>
#include "utest.h"

typedef struct
{
    int amount;
    char *currency;
	
}  Money;

Money *money_create(int amount, char *currency)
{
    Money *m;

    if (amount < 0)
    {
        return NULL;
    }

    m = malloc(sizeof(Money));

    if (m == NULL)
    {
        return NULL;
    }

    m->amount = amount;
    m->currency = currency;
	
    return m;
}

int money_amount(Money * m)
{
    return m->amount;
}

char *money_currency(Money * m)
{
    return m->currency;
}

void money_free(Money * m)
{
    free(m);
    return;
}

struct checked_fixture_t
{
	Money *five_dollars;
};

UTEST_F_SETUP(checked_fixture_t)
{
	utest_fixture->five_dollars = money_create(5, "USD");
	
	ASSERT_TRUE(utest_fixture->five_dollars);
}

UTEST_F_TEARDOWN(checked_fixture_t)
{
	free(utest_fixture->five_dollars);
}

UTEST_F(checked_fixture_t, test_money_create)
{	
    ASSERT_EQ(money_amount(utest_fixture->five_dollars), 5);
    ASSERT_STREQ(money_currency(utest_fixture->five_dollars), "USD");
}

UTEST(utt, test_money_create_neg)
{
    Money *m = money_create(-1, "USD");
	ASSERT_FALSE(m);

    assert((m == NULL) &&
                  "NULL should be returned on attempt to create with "
                  "a negative amount");
}

UTEST(utt, test_money_create_zero)
{
    Money *m = money_create(0, "USD");
	ASSERT_TRUE(m);

    if ( money_amount(m) != 0 )
    {
		assert(0 && "Zero is a valid amount of money");
    }
}

UTEST_MAIN()

if we compile it and run it with

gcc -o utt main.c && ./utt

we'll have

[==========] Running 3 test cases.
[ RUN ] checked_fixture_t.test_money_create
[ OK ] checked_fixture_t.test_money_create (1600ns)
[ RUN ] utt.test_money_create_neg
[ OK ] utt.test_money_create_neg (800ns)
[ RUN ] utt.test_money_create_zero
[ OK ] utt.test_money_create_zero (900ns)
[==========] 3 test cases ran.
[ PASSED ] 3 tests.

if we change line 70 slightly comparing 4 instead of 5 so that the test fails, we have

[==========] Running 3 test cases.
[ RUN ] checked_fixture_t.test_money_create
main.c:70: Failure
Expected : (money_amount(utest_fixture->five_dollars)) == (4)
Actual : 5 vs 4
[ FAILED ] checked_fixture_t.test_money_create (665400ns)
[ RUN ] utt.test_money_create_neg
[ OK ] utt.test_money_create_neg (2400ns)
[ RUN ] utt.test_money_create_zero
[ OK ] utt.test_money_create_zero (1500ns)
[==========] 3 test cases ran.
[ PASSED ] 2 tests.
[ FAILED ] 1 tests, listed below:
[ FAILED ] checked_fixture_t.test_money_create

with check library we have

Running suite(s): Money
66%: Checks: 3, Failures: 1, Errors: 0
check_money.c:41:F:Core:test_money_create:0: Assertion 'money_amount(five_dollars) == 4' failed: money_amount(five_dollars) == 5, 4 == 4

As you can see in this case it is quite simple to use the utest library but obviously not all the features of the check library have been compared.