Saturday, November 13, 2010

Java bench - Simple and lightweight service load testing tool

At WSO2, we use various types of tools and programmatic approaches to load, stress and performance test SOA middleware products. Out of them, one particular tool is part of almost all engineers workspaces. It is Java bench (or java-ab).
Java bench is the java clone of the popular ApacheBench load generator customized for better HTTP1.1 support.

Java bench is a very simple and easy to use utility. One of the greatest advantages of using it for performance testing is, its lesser overhead on load generation machines. Most of the load generators consist of rich UIs and reporting utilities hence the load generator it self consumes considerable amount of CPU and memory when you run performance testing in single server. It is always not practical to run load generators in separate machines. Sometimes we need to run load/performance tests against a server running on a local machine in order to provide quick feedback about the product under test or reproduce perf/memory issues. In these situations, Java-bench is the ideal solution.

Lets see how we can run a basic performance test using java bench.

Pre-requisites:
Linux or windows OS
jdk1.5 or higher

Step 1
Download Java bench from here and extract the downloaded zip file.

Step 2
Go to the root of the extracted directory and run 'java -jar benchmark.jar' to see the available options.

usage: HttpBenchmark [options] [http://]hostname[:port]/path
-v Set verbosity level - 4 and above prints response
content, 3 and above prints information on headers, 2 and above prints
response codes (404, 200, etc.), 1 and above prints warnings and info.
-H
Add arbitrary header line, eg. 'Accept-Encoding:
gzip' inserted after all normal header lines. (repeatable)
-n Number of requests to perform for the benchmarking
session. The default is to just perform a single request which usually
leads to non-representative benchmarking results.
-T Content-type header to use for POST data.
-c Concurrency while performing the benchmarking
session. The default is to just use a single thread/client.
-h Display usage information.
-i Do HEAD requests instead of GET.
-k Enable the HTTP KeepAlive feature, i.e., perform
multiple requests within one HTTP session. Default is no KeepAlive
-p File containing data to POST.

Step 3

We will run a load test against one of the publicly available web services. I use a simple web service hosted in xmethods which generates a list of prime numbers that are less than a specified max value.
First you need to invoke this service using SOAPUI or some other client and capture the SOAP request message. Then save it as a xml in your file system.

Sample request:

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:web="http://microsoft.com/webservices/">
<soapenv:Header/>
<soapenv:Body>
<web:GetPrimeNumbers>
<web:max>56 </web:max>
</web:GetPrimeNumbers>
</soapenv:Body>
</soapenv:Envelope>

Step 4

Now, we can use java bench to send series of requests to the above web service as follows.

java -jar benchmark.jar -p SoapRequest.xml -n 100 -c 10 -k -H "SOAPAction: http://microsoft.com/webservices/GetPrimeNumbers" -T "text/xml; charset=UTF-8" http://www50.brinkster.com/vbfacileinpt/np.asmx

As you can see, we specify the soap request xml with -p parameter. -c specifies the no.of concurrent threads/clients used in the test. -n is used to specify the no.of requests transmitted per a keep-alive connection. In our example, each client carries 100 requests therefore 1000 total requests will be sent to the service. -k enables HTTP keep-alive so that multiple requests share the same HTTP session. -H is used to specify the HTTP headers. In our example, SOAPAction HTTP header is appended to the message. You can find the relevant SOAPAction from the WSDL of the service. -T is used to specify the Content-Type header of the request. The last parameter is the endpoint of target service.

After the test is completed, you will see the result as shown below.

Server Software: Microsoft-IIS/6.0
Server Hostname: www50.brinkster.com
Server Port: 80

Document Path: http://www50.brinkster.com/vbfacileinpt/np.asmx
Document Length: 429 bytes

Concurrency Level: 10
Time taken for tests: 59.378401 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 756000 bytes
Requests per second: 16.84 [#/sec] (mean)
Time per request: 593.784 [ms] (mean)
Time per request: 59.378 [ms] (mean, across all concurrent requests)
Transfer rate: 7.22 [Kbytes/sec] received
5.51 kb/s sent
12.73 kb/s total

Simple. Isn't it? We will look in to more interesting tests with java-bench in future posts.

8 comments:

Cataldo Spataro said...

Hi, the link to download benchmark.jar is wrong, where can i found a working link?
Thanks Aldo.

Charitha said...

Hi Aldo,
I have fixed the permission issue, now you should be able to download. Thanks for informing me.

Regards
Charitha

Cataldo Spataro said...

Hi Charitha, sorry but i have a problem, when i do the command:
java -jar benchmark.jar -p testmess.xml -n 50 -c 10 -k -T "text/xml; charset=UTF-8" http://localhost:9090/mailingServicePort

i have the following error:
Exception in thread "main" java.lang.NullPointerException
at org.apache.http.contrib.benchmark.HttpBenchmark.prepare(HttpBenchmark.java:137)
at org.apache.http.contrib.benchmark.HttpBenchmark.execute(HttpBenchmark.java:151)
at org.apache.http.contrib.benchmark.HttpBenchmark.main(HttpBenchmark.java:97)

i haven't inserted soap-action because there isen't in wsdl...
can you help me?

Charitha said...

Hi Aldo,
Due to a bug in java-bench code, you should set HTTP header (-H) parameter. Since you do not have a soapAction, can you just set an empty header as follows and try?
java -jar benchmark.jar -p testmess.xml -n 50 -c 10 -k -H "" -T "text/xml; charset=UTF-8" http://localhost:9090/mailingServicePort

Regards
Charitha

Cataldo Spataro said...

Hi, Charitha can you explane me the exact diffrence between
Time per request: 54.664,560 [ms] (mean)
Time per request: 437,316 [ms] (mean, across all concurrent requests)???
Thanks!

Asankha said...

Hi Charitha

Now there is a Swing GUI based graphical application of the original JavaBench :) This is in addition to an integrated TCP dump, raw TCP level socket client, and advanced HTTP/S client supporting basic/digest/NTLM authentications etc. The configurations can be saved and reloaded for easier QA testing too

AdroitLogic ToolBox

cheers
asankha

Charitha said...

Hi Asankha,
As always, really appreciate your valuable comments. Thanks a lot. I will surely have a look.

Regards
Charitha

Cataldo Spataro said...

hi Charita, can you answer me? I am a student, sorry if my question is stupid...