Rocket Surgery

In an earlier post I wrote about how you can use JMeter to do automated functional and regression testing for web services. In this post I will talk about how you can parameterize your tests from external files to quickly get more thorough tests.

Parameterizing tests means that you run the test with different data each time, the data being passed in as parameters to your test. Obviously it would be preferable to run the same test 50 times with different values than to explicity put 50 different tests in the script that are identical except for the value used (duh.)

JMeter has a facility for doing just that. In my opinion, it’s a bit crude, but it works, and that’s what really counts.

Let’s start with a JMeter test that calls a web service. (You can download my sample). The test calls a web service that gives you the words for an integer (e.g., “fifteen” for 15), and calls the service three times with the same value (15) hard-coded in it, and looks for a 200 response code from the service, and the word “fifteen” in the response SOAP envelope.


Now to introduce parameters. We want the script to run a bunch of times, with different input integers, and check the anticipated numbers as words. Using your favorite text editor, create two plain text files, one for the input numbers and one for the expected outputs. The row numbers need to match up and correspond. The value in line 3 of the input file must match up with line 3 of the expected results file.


Once we have both files, we need to tell JMeter about them. Right-click the Thread Group, then select Add | Pre Processors | User Parameters from the context menu. This will add a step to the test that will define and load the parameters for your test (with a little configuration). Note that the position of the new step is not important (i.e., it does not have to appear in the tree above the sampler). JMeter knows that it’s a pre-processor and will use it before the sampler regardless of order.

On the pane that comes up, optionally change the title, and check the box that says “Update Once Per Iteration” (this is important!!). Then, you will need to add two variables: one for the input integer and one for the expected output words. Click the Add Variable button at the bottom of the pane, and this adds one row to the table. In the left cell (the “Name:” column) click and type in a variable name (such as “input” — no quotes). The column labeled “User_1″ is where your value goes. (Side rant: what a horrible name for the column!! Why not “value_1″? Ah well…)


In our case the value is going to come from a function built into JMeter: the StringFromFile function. In the “User_1″ column, click and type ${_StringFromFile(fn)}. The ${} notation must be typed exactly like that, and yes, the function name is preceded by an underscore. You will also need to replace fn with the filename of the input text file. Note: the file name is not quoted or anything. (Weird huh?) Also note that you will probably want to use forward slashes and not backslashes, even on a Windows machine. Repeat the process for the output value, giving it a name (like “expected”) and a value using the StringFromFile function on the other file.


Now that we’ve defined our parameters, we need to reference them in the test. Click on the Sampler to make changes to it, and down in the SOAP envelope, change the 15 to ${input} (or whatever you named the input parameter). Then in the assertion, change the pattern to test to use the other parameter ${expected}.


Ok, one last step. You need to tell the thread group to run the test repeatedly; otherwise it will only get the first line from each file and run the test once. Click on the Thread Group, and in the pane for it, change the number of users to match the number of lines in your parameter text files. This way, each iteration will pull one number from the files, and all the numbers will get used. (By the way, if you ever use a number higher than the number of lines in the files, it starts over again from the first line…you don’t get End Of File errors or anything like that).


Now, you can run the test. In your results view, you will see the repeated invocations, with different inputs substituted in on the request, and the different values coming back on the response. The assertions run for each request, and you can see that the web service seems to be working correctly.

In a future post, I plan to show how to run JMeter tests in batch (say, as part of a Continuous Integration cycle).

This entry was posted on Sunday, May 11th, 2008 at 11:28 am and is filed under Software development, Web tools.You can follow any responses to this entry through the RSS 2.0 feed.You can leave a response, or trackback from your own site.