Testing the Code

Testing is another important ongoing task for any large code base such as Open Babel. The following documentation is intended to give some idea as to the types of tests used by Open Babel and various tools available for debugging and testing.

Types of Tests

Unit tests

These tests cover specific functionality of the library.

Unit tests attempt to show that functions work as indicated (i.e. pass) and fail properly on unacceptable input. They are run after building the code by calling make test. Ideally these tests should cover every public function in the Open Babel API. Adding more tests to provide greater coverage is a high priority.

Regression tests

These tests are associated with particular bug fixes.

A regression test ensures that a bug, once fixed, stays fixed. If you have spent time fixing a bug, it’s worthwhile to write an associated regression test. In fact, writing the test first is to a good way to prove that you have in fact fixed the problem.

Roundtrip tests

Test file translation.

These are run from the test file repository and test file translation to/from file formats.

“Dogfood” tests

Using the development code.

Many of the developers use the current development snapshots for their daily use. This is sometimes called eating your own dog food and ensures that bugs which impact common functionality are caught quickly.

Scripting language tests

Perl and Python unit tests

Both the Perl and Python language bindings have additional tests to ensure that these scripting language modules work. These tests not only stress the scripting modules themselves, but also the underlying library.

Test dashboard

We have set up a test dashboard with nightly builds of Open Babel for several compilers and operating systems. The dashboard allows problems with particular SVN commits to be easily identified. Developers should check the dashboard after committing code and check whether the test suite has failed.

We thank KitWare for providing dashboard resources.

Tools for software testing

A variety of tools can be used to help diagnose problems and debug errors:

Debuggers

gdb (the GNU project debugger) is a basic debugger on UNIX systems.

• GDB can be run on the babel program like so: ./libtool gdb src/babel
• Several GDB commands are highly useful, including bt which gives a full backtrace of program execution upon error.
• The use of code optimization when compiling can make debugging with GDB and other debuggers extremely difficult. If needed, make sure Open Babel has been compiled, e.g. with CXXFLAGS="-g -O0" on UNIX.

Memory Checkers

Unlike languages like Java, which handle memory allocation and deletion, C++ requires that all code handle memory. In particular, this means if you use new or malloc or similar calls, you must be sure the memory will be properly freed when no longer needed. So-called “memory leaks” are cases where memory has been allocated, but never properly freed and are significant bugs in Open Babel.

Several tools can help find such errors:

• valgrind works on several platforms by carefully monitoring each memory access and allocation.
• LeakTracer works on a range of platforms by using C++ operator overloading.
• MallocDebug (Mac OS X only) is a graphic front-end to monitor memory use and incorrect memory usage.
• Visual Leak Detector (Windows/MSVC only) is a very easy way to find memory leaks.

Code Profiling

To improve the speed of execution, it is often helpful to rely on tools that monitor the frequency and duration of function calls. This information can help to identify performance bottlenecks.

• gprof is used with the GCC compilers to produce profiling data including the number of calls to routines and the amount of time spent executing.
• Shark/Saturn (Mac OS X only) is a graphical front-ends to gprof and can sample programs while running.
• callgrind is run through the valgrind virtual machine using valgrind --tool=callgrind ./myexe. This works best (although slower) if the executable is compiled with debugging information.

C++ Unscrambler

The symbols used by C++ compilers and exposed by debugging, profiling, and memory tools are “scrambled.” The c++filt program can be used to decode the symbol to a human-readable form.