Unit Testing Fosters Software Quality

By Roland Helbig

Today’s chips run at extremely high clock speeds, pushing the performance limits of traditional test strategies such as in-circuit emulation. At the same time, high integration is making processors less accessible to external test devices. On-chip debugging provides many of the functions of in-circuit emulation, but does not offer the performance needed for testing and debugging an entire application. The use of dynamic software unit testing can provide an alternative approach to that also helps ensure software quality.

Processors utilizing Intel XScale® technology offer superior on-chip debugging support based on special resources that are addressable through the chip’s JTAG interface. Besides providing the standard debugging features of single steps and software breakpoints that are commonly used with other processors, Intel XScale® technology offers extended debugging features such as watchpoints, trace history, trigger, sequencer and counter. All of these features can be accessed through the JTAG interface. However, even this superior on-chip debugging cannot provide the same performance as the conventional in-circuit emulation, leaving developers with a software testing challenge.

A reasonable solution to bypass the limited debug functionality of the Intel XScale core is the dynamic software unit test. This method separates an embedded application into small units with limited complexity for easy handling. After the separation of these units, the user defines appropriate test cases for each of them and performs the tests on the units individually. If a test fails, the reason for the failure can easily be identified, because it can only stem from the unit under test, not from a function way down in the calling hierarchy.

The unit test can detect whether or not a test case delivers an expected value. If it does not, the unit is small enough to be successfully debugged even with the reduced feature set of the on-chip debugger. Another benefit of this approach is that if forces systematic test planning and the inclusion of test and debug tasks very early in the software development process.

Unit Testing Demands Rigor

During unit testing of C programs, a single C-level function would be tested rigorously and in isolation from the rest of the application. Rigorous means that the test cases are specifically made for the unit in question and include the use of input errors: data that may be unexpected by the unit under test. Isolated means that the test result does not depend on the behavior of the other units in the application. Isolation from the rest of the application can be achieved by directly calling the unit under test and replacing the calls to other units by stub functions.

Figure 1: Dynamic software unit testing with tools such as Hitex�s Tessy simplifies software debugging by breaking the application into small, easily tested units

Unit testing can be conducted as soon as the unit to be tested compiles. Therefore, errors inside the unit can be detected very early in the software development effort rather than waiting until the entire software package is assembled. If executing a test case with a particular set of input values does not yield the expected result, the cause of the failure can be determined before integrating the unit with others.

Because changing source code can lead to a failure of test cases that had previously yielded the expected value, all test cases performed before fixing the bugs have to be repeated. These regression tests, re-runs of tests that have already passed, are necessary to ensure that modifications and enhancements do not lead to undesired effects. Regression testing proves that a change in the software did not introduce new errors.

Regression testing is a key to software quality. However, the practice of regression testing requires automation of tests; the effort to repeat the tests manually is too high. Tool support is indispensable for the repetition of the unit tests, but even for non-repetitive unit tests, the proper tool support will save considerable test time. Tools for the automation of software unit test, such as Hitex’s Tessy, speed the tasks of unit testing by providing a batch feature that enables extensive regression testing to take place without any user interaction, e.g. overnight. Such tools become indispensable if the repeatability of tests has to be ensured, as demanded by a large number of standards.

Tools such as Tessy can be brought into use as soon as a suitable compiler for the target hardware finishes compiling the software module containing the C function to be tested. These tools can even analyze the software module and lists all C functions it finds within it. The user can then select which function is to be tested. After analyzing the source code of the function to be tested, the tool then determines the number and type of variables in test object’s interface along with whether they are input, output, or both. Tessy also offer a Test Interface Editor (TIE) that graphically displays information about the data direction flow and allows the user to modify or complete the flow.

Figure 2: Unit testing looks for software elements to return a set of expected values. Automated software test tools help identify unexpected results then link to debuggers to find the cause.

Figure 3: A Test Interface Editor (TIE) allows developers to specify the direction of test data flow in unit testing.

Automatic Test Driver Generation

From the information gained from the interface analysis, these tools can automatically generate additional source code: the so-called “test driver”. The test driver consists of startup code for the microcontroller in question and code to call the function to be tested, i.e., the test object. Furthermore, the test driver may contain user-supplied source code or data, such as for characteristic curves, sampled data from real observations, and so on. The combination of code and data forms the test application, which the tool automatically compiles and links. The test driver also supplies the main() function for the test application.

After running the test applications, the tool generates reports on test case results at a level of detail that the user can specify. These reports also indicate if the execution of a test case yielded the expected result or not. Tools like Tessy come integrated with support for the design of test case specifications according to the Classification Tree Method. This method transforms a functional problem specification into a set of error-sensitive, non-redundant test cases. This type of unit test automation, if used in the right way, can reduce overall development and test time and therefore reduce the time-to-market.

After unit testing is complete, the application software is made up of single, fully tested units. A test of the whole application at this point will be more likely to pass. If some application tests fail, the cause is most likely somewhere in the interaction of units and not from an error inside a unit. The effort to find the failure can concentrate on interfaces, and not doubt the internals of the units.

Unit testing thus reduces the complexity of testing, finds errors early, and simplifies the final test of the whole application. Further, the technique presents a better match to the limitations of on-chip debug than traditional approaches. All of these factors help speed the development effort and lower cost while ensuring high-quality software.

Roland Helbig is the head of marketing for Hitex Development Tools in Germany. Hitex provides emulators, bus and protocol analyzers, and software tools for microcontroller and microprocessor development. Owned by Infineon Technologies AG, the company supports a wide variety of 8- and 16- bit processors and is moving into the 32-bit market.