Rocket Surgery

Are unit tests and unit testing on the decline?  Here is an interesting blog post that discusses this very question.

My $0.02:  Read the rest of this entry »

I’ve started introducing Apache JMeter at my workplace as a tool for automatic functional and load testing of Web Services. This is the first of what I plan to be the first of a multi-part series on setting up JMeter, developing tests for it, and incorporating its output as part of an overall strategy for testing our code as it moves through the lifecycle.

A bit about JMeter

At the JMeter site, they define JMeter as “…a 100% pure Java desktop application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.” One of the other types of applications it can test is a Web Service - which of course is not terribly different from testing a regular web application.
Read the rest of this entry »

I have an open-source project called Project PEA that is an initial attempt to gather metrics on how directly and intentionally classes are tested.

One of the main weaknesses of existing code coverage tools is that all code that executes during a test run is considered covered, and covered equally. If a method under test executes and calls another method for which no test was specifically written, the other method looks like it was tested, because it has coverage. The situation is exacerbated the more that one method calls another. Sometimes method calls can be chained many, many levels from one test, and a method that never had a test written for it has 100% coverage.

Project PEA watches your JUnit tests run and sees how many stack frames are between each class and the unit test being run. So for example, if test testFoo() calls foo(), and foo() calls bar(), and bar() calls baz(), it will track that testFoo() tests foo() at a “test distance” of 1, bar() at a test distance of 2, and baz() at a test distance of 3. PEA collects how many times each method is called at a given test distance, and weights the “testedness” of each method based on a formula that improves the score for methods that are tested at a shorter distance.

Here is a powerpoint discussing the aims of project PEA that I gave at the November meeting of the Southeast Virginia Java Users Group.

While preparing a training class this past month on Effective Exception Handling for a client, I came up with a list of “Ten Commandments of Exception Handling.” Some of them were the basic, standard items (”Thou shalt not catch or throw java.lang.Throwable”, “Thou shalt not throw raw java.lang.RuntimeExceptions–thou shalt throw a subclass instead”, “Thou shalt not catch an Exception unless thou art going to truly handle it, or are going to wrap it in another Exception”). Other items on the list were specifically tailored for the client. I liked one of the “commandments” better than the others, however:

“XIII: Thou shalt not catch any Exceptions in JUnit tests, unless testing that your code under test throws the Exceptions the API says it does.”

If you read this and say, “well, duh, Matt!” then you and I are thinking alike and I am preaching to the choir. If this statement has even a whiff of controversy to it, however, read on.

Read the rest of this entry »

In a previous blog entry I mentioned that I like emma for measuring code coverage in my JUnit test suites. In this entry I will discuss several items that make some JUnit test suites stronger than others. In this entry I am discussing JUnit 3.8.1.

How do you measure the quality of your JUnit tests and put a number
on it? This is a very good question with no quick and easy answers,
I’m afraid. I can tell you two answers that are NOT measures of test quality:

• “We have 100% success rate in our test suites every night.”
• “We have 100% code coverage”

Read the rest of this entry »