Rocket Surgery

I’ve started introducing Apache JMeter at my workplace as a tool for automatic functional and load testing of Web Services. This is the first of what I plan to be the first of a multi-part series on setting up JMeter, developing tests for it, and incorporating its output as part of an overall strategy for testing our code as it moves through the lifecycle.

A bit about JMeter

At the JMeter site, they define JMeter as “…a 100% pure Java desktop application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.” One of the other types of applications it can test is a Web Service - which of course is not terribly different from testing a regular web application.

JMeter, being part of the Jakarta project at Apache, has several pros and cons. Naturally one of the biggest pros is its price tag. Although not as functional as, say, LoadRunner from HP (which aquired Mercury Interactive), it can do an awful lot of the main use case, which is to send HTTP requests to some address and check that the results are what are expected. One nice feature that JMeter has that LoadRunner does not have is a batch mode, in which tests can be run without the GUI, making it ideal for use in nightly and continuous integration builds (of which I am a huge fan). Another alternative to JMeter is The Grinder, which is another open-source functional/load testing tool. For what it’s worth, I chose to use JMeter over The Grinder because of two main reasons: 1) I needed a tool where the scripts could be built (or at least read and understood) by a Business Analyst. JMeter builds scripts semi-graphically using the GUI, while The Grinder requires that you write your scripts in Jython. 2) I found the reporting capabilities of JMeter to be much more comprehensive and attractive.

Installation

Installation of JMeter is a snap. Download the ZIP file, explode it into a directory. Then put mail.jar - the JavaMail API jar - in JMeter’s lib directory (not ext/lib). If you’re using JDK 5 you also have to put activation.jar - the Java Activation Framework jar - into JMeter’s lib directory as well.

Note: You have to put mail.jar (and possibly activation.jar) into JMeter’s lib directory for SOAP processing to work. These jars (for licensing reasons) are not included in the JMeter distribution. (Without these jars you will get ClassNotFoundExceptions (that are not easy to spot) when you try and invoke a web service using JMeter.

Then run the start script in the bin directory - for Windows, it’s jmeter.bat.

Writing a Simple Script

I will be focusing here on writing scripts for Web Services, since that’s my use case, but I’m sure you could extrapolate from this how to do other kinds of projects.

When you start JMeter, you start with an empty test plan. Give your test plan a suitable name. The web service I am testing converts numbers into words (e.g., “15″ to “fifteen”), so I name the test plan “Numbers-to-Text Web Service Test” — note that this by default will be the name of your file when you save it.

Next you need to add a Thread Group to the test plan. A Thread Group is a set of requests that simulate the other systems’ requests to your web service. To add a thread group, right-click the Test Plan or select Edit from the main menu, then select Add | Thread Group. I usually don’t rename the thread group. In it I specify how many simultaneous requestors I want to simulate in the “Number of Threads (users)” box. For a functional test, one or two is sufficient. For load testing, you may want to increase the number — slowly. (We’ll come back to it.) The “Ramp-up period” box indicates the number of seconds delay between introducing each requestor to the mix. “Loop Count” is the number of times each requestor should repeat the actions that you are about to add to the Thread Group. I leave the rest alone.

Next, you need to add a Sampler. In JMeter lingo, a Sampler is anything that gets performance and success/failure data (that is, provides a “sample” for the statistics). A sampler can be a call to a web service (which is what we will do), but can also be a call to a web page, a JDBC query, a POP mail reader, etc. To add a sampler that calls a web service, right-click the Thread Group, and select Add | Sampler | WebService(SOAP) Request.

On the page that comes up, I usually change the name field to a human-readable description that indicates what I’m doing when I make the call I’m about to enter. In this case “Convert integer to words”. I then enter the URL of the WSDL that describes the web service I will be calling and hit the “Load WSDL” button. This fetches the WSDL and gets the endpoints and operations that are available, populating the “Web Methods” pulldown. I select the method I want, and click “Configure”, which fills out most of the form for me. Then, in the large text area, I enter the full XML SOAP request to be sent. I usually cheat at this point and copy/paste the SOAP request from a GUI-based SOAP tool such as StrikeIron Web Services Analyzer, or SoapBits, or the .NET Web Services Studio (which seems to have disappeared since GotDotNet has been shut down, but is available in an earlier post). The SOAP message in this case includes the integer 15 as an input parameter to the web service. Finally, I uncheck “Memory Cache” (because I want the calls to actually happen each time) and I check “Read SOAP Response” so that I can make assertions about its content. This last step is important. If you leave “Read SOAP Response” unchecked, JMeter basically ignores the SOAP envelope coming back in its entirety — not desirable at all for a functional test, but possibly very useful for a load test.

(Note that I have blurred out the name of the service I am running, because I don’t want to cause them any trouble with their site being hit by people they don’t know).

Once the Sampler is added, I add an Assertion to the Sampler - in the case of a Web Service, a Response Assertion is an obvious fit (right-click the sampler, Add | Assertion | Response Assertion). The first thing I want to assert is that I got a 200 HTTP response code, so in the panel that comes up I select “Response Code” as field to test, add “200″ to the “Patterns to Test” list (this is a list of regex’s), and select “Equals” as my pattern matching rule. I also change the assertion name to something describing (in English) what I’m asserting. Then, I add a second response assertion to the sampler, and this time, I select “Text Response” as my field to test, set “Contains” to my operation, and put the string “fifteen” in the list of patterns to test — because the input parameter to the web service was “15″, I expect to see “fifteen” in the response.

In order to see how things actually go during the test, you need a Listener. I prefer the “View Results Tree” listener, as it gives the most information in the smallest area, with the ability to drill in for more details. To add the Listener, righ-click the test itself (the top-most element) and select Add | Listener | View Results Tree.

Once this is in place, select Run | Start from the main menu. The web requests are sent, the responses recorded, the assertions made, and reported in the listener which I can see if I select it from the script tree in the left-most window.

This is a very basic functional test. To test additional invocations of your web service, or to invoke other web services, simply repeat the steps of adding samplers and assertions - the listener will pick them all up if you put it on the test itself. You can also add more assertions to your existing samplers to increase the rigor of your test.

Future blog entries on JMeter will include how to make the data of your tests variable, and how to automate the execution of the tests.

This entry was posted on Saturday, April 12th, 2008 at 8:22 pm and is filed under Software development, Web tools.You can follow any responses to this entry through the RSS 2.0 feed.You can leave a response, or trackback from your own site.