Saturday, May 11, 2013

The simplest possible way to simulate a backend service delay

When testing service integrations, we usually want to simulate the delayed responses from backend web services. For example, if you want to test how WSO2 ESB or Apache Synapse reacts when the backend service takes large amount of time for responding, there are many approaches to introduce delay to backend web service. I have observed that, most of the people modify the backend web services to add Thread.sleep() when they need to introduce a delay in service invocation.

If you are not testing the backend web service and just want to test the integration (e.g:- outgoing calls from ESB), I cannot think of a better solution than using soapUI mock services.

Step 1

 

Add a new mock service to soapUI project. This can be done at the time of creating the project or by selecting a particular interface (binding) of a wsdl based project.

Step 2

 

Select the mock service in soapUI navigator and open the mock service editor. In the mock service editor, select OnRequest script.










Step 3

 


Add the following groovy script inside the OnRequest Script editor to introduce 1 minute delay.

sleep(60000)

Now, send a request to the mock service. You will notice that all requests will respond back to the caller after 1 minute of delay.

Read chapter 6 (Web service simulation with soapUI) and 11 (Extending soapUI with Scripting) of Web Services Testing with soapUI book for more information about soapUI scripting capabilities.




Sunday, May 5, 2013

Testing one-way operations which do not return HTTP 202 responses

When you invoke a one-way (in-Only) operation of a web service over HTTP, it responds with HTTP 202 accepted message. Many web service clients such as soapUI or Jmeter waits till they receive a response from the web service.
Waiting for HTTP 202 response is always not desirable since there are situations where you do not even get a 202 response. For example, if you invoke one-way JMS operation, it does not send a reply back to the client.
Look at a scenario similar to the following.

A client sends a message over HTTP to a proxy service in WSO2 ESB. The proxy service places the message in a JMS queue and does not expect a response back. In this case, client does not even get a HTTP 202 response hence it waits and eventually timed out. This prevents you using the tools like soapUI, Apache Jmeter in these scenarios. How can we fix this so that the client always get a HTTP 202 response back?

Let's go through the procedure in detail.

Step 1:

Configure Apache ActiveMQ JMS broker with WSO2 ESB as explained here.

Step 2: 

Create a queue in ActiveMQ. You could access ActiveMQ console application through http://localhost:8161/admin and create a new queue. Name it as "onewaytest".

Step 3: 

Create a simple proxy service in WSO2 ESB as shown below. This proxy service just forwards the incoming SOAP messages into the "onewaytest" JMS queue which we have created in the previous step.

<proxy xmlns="http://ws.apache.org/ns/synapse" name="onewayProxy" transports="http" statistics="disable" trace="disable" startOnLoad="true">
   <target>
      <inSequence>
         <property name="OUT_ONLY" value="true"/>
         <send>
            <endpoint>
               <address uri="jms:/onewaytest?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory&java.naming.provider.url=tcp://localhost:61616"/>
            </endpoint>
         </send>
      </inSequence>
      <outSequence>
         <send/>
      </outSequence>
   </target>
   <description></description>
</proxy>
                                

Step 4:


Use soapUI (or SOAP/XML-RPC sampler in Apache Jmeter) and send a SOAP request to the above proxy service. You will get "java.net.SocketTimeoutException: Read timed out" in soapUI log. This explains the behavior of HTTP client waiting for HTTP 202 response.

Step 5:


There is an useful property, FORCE_SC_ACCEPTED which can be used inside the inSequence of the proxy service to send HTTP 202 Accepted response back to the client in case of one-way JMS operations.

Add this property to the inSequence of the above proxy service.

<property name="FORCE_SC_ACCEPTED" value="true" scope="axis2" />

Step 6:


Re-send a SOAP message to the proxy service using soapUI. You will get a HTTP 202 response.


HTTP/1.1 202 Accepted
Content-Type: text/xml; charset=UTF-8
Date: Sun, 05 May 2013 07:42:27 GMT
Server: WSO2-PassThrough-HTTP
Transfer-Encoding: chunked



Tuesday, February 26, 2013

How to configure soapUI to send HTTP chunked encoded requests

soapUI makes use of Apache HttpClient (based on HTTPComponents project) as the client side HTTP implementation. When submitting requests using soapUI, you may have noticed that, the requests always send Content-Length HTTP header. You can find it out in the Raw View of the request editor.

POST http://localhost:9000/services/SimpleStockQuoteService HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: "urn:getQuote"
User-Agent: Jakarta Commons-HttpClient/3.1
Host: localhost:9000
Content-Length: 416

In functional and non-functional testing, we usually need to test the servers by sending HTTP requests with both Content-Length as well as chunked transfer encoding. How can we configure soapUI to send chunked encoded requests instead of Content-Length?

Step 1

Click on Global soapUI Preferences icon in the main tool bar (Or else, click on File --> Preferences)

Step 2

HTTP Settings tab will be selected by default. You will notice Chunking Threshold option. By default this is set to empty so that the chunking is disabled for any HTTP request.















Update this field with non negative integer value (e.g:- 1) and resubmit your request. You will notice chunked encoded request as shown below.

POST http://localhost:9000/services/SimpleStockQuoteService HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: "urn:getQuote"
User-Agent: Jakarta Commons-HttpClient/3.1
Host: localhost:9000
Transfer-Encoding: chunked

For more information about various soapUI tips and tricks, read Web services Testing with soapUI book.

Saturday, February 16, 2013

How to connect WSO2 ESB to Apache ActiveMQ using simple authentication

There are different types of pluggable authentication mechanisms provided by Apache ActiveMQ message broker. One of the quickest and easiest mechanisms is to use simple authentication. As the name implies, it is as simple as adding authentication details to ACTIVEMQ_HOME/conf/activemq-security.xml


 <plugins>
        <!-- Configure authentication; Username, passwords and groups -->
        <simpleAuthenticationPlugin>
            <users>
                <authenticationUser username="system" password="${activemq.password}"
                    groups="users,admins"/>
                <authenticationUser username="user" password="${guest.password}"
                    groups="users"/>
                <authenticationUser username="guest" password="${guest.password}" groups="guests"/>
            </users>
        </simpleAuthenticationPlugin>


In this simple configuration, passwords of each user is defined in ACTIVEMQ_HOME/apache-activemq-5.7.0/conf/credential.properties file.


You can find more details about ActiveMQ simple authentication plugin in this blog.

Suppose, you have started Apache ActiveMQ with simple authentication. Then, any consumer of the destinations defined in your ActiveMQ broker should connect using the credentials defined under simple authentication plugin.

If WSO2 ESB needs to connect to ActiveMQ configured with simple authentication, we can simply update the broker configuration details in ESB_HOME/repository/conf/axis2/axis2.xml as shown below.

 <transportReceiver name="jms" class="org.apache.axis2.transport.jms.JMSListener">
        <parameter name="myTopicConnectionFactory" locked="false">
        	<parameter name="java.naming.factory.initial" locked="false">org.apache.activemq.jndi.ActiveMQInitialContextFactory</parameter>
        	<parameter name="java.naming.provider.url" locked="false">tcp://localhost:61616</parameter>
                <parameter name="transport.jms.UserName">system</parameter>
                <parameter name="transport.jms.Password">manager</parameter>
         	<parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">TopicConnectionFactory</parameter>
		    <parameter name="transport.jms.ConnectionFactoryType" locked="false">topic</parameter>
        </parameter>

        <parameter name="myQueueConnectionFactory" locked="false">
        	<parameter name="java.naming.factory.initial" locked="false">org.apache.activemq.jndi.ActiveMQInitialContextFactory</parameter>
        	<parameter name="java.naming.provider.url" locked="false">tcp://localhost:61616</parameter>
                <parameter name="transport.jms.UserName">system</parameter>
                <parameter name="transport.jms.Password">manager</parameter>
        	<parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">QueueConnectionFactory</parameter>
		    <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
        </parameter>

        <parameter name="default" locked="false">
        	<parameter name="java.naming.factory.initial" locked="false">org.apache.activemq.jndi.ActiveMQInitialContextFactory</parameter>
        	<parameter name="java.naming.provider.url" locked="false">tcp://localhost:61616</parameter>
                <parameter name="transport.jms.UserName">system</parameter>
                <parameter name="transport.jms.Password">manager</parameter>
        	<parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">QueueConnectionFactory</parameter>
		    <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
        </parameter>
    </transportReceiver>

Note that highlighted parameters (transport.jms.UserName and transport.jms.Password) which are used to connect to the broker using simple authentication.

The above configuration can be used to connect any Apache Axis2 based server (Apache Axis2, WSO2 Application Server etc..) to ActiveMQ using simple authentication.

Sunday, February 10, 2013

WSO2Con 2013 - 5 must attend talks




Anyone who missed the opportunity to attend the last WSO2Con (WSO2Con 2011 which was held in Colombo, Sri Lanka) should not miss WSO2Con 2013 which will be started on Tuesday, 12th at Park Plaza in London in sync with the live stream hosted in Colombo.
In WSO2Con 2011, I could not concentrate much on the great set of talks given by geniuses around the world. I was a speaker and I spent each minute practicing and preparing for my talk, Quality, Key to successful SOA :)
This year, I'm freed from that burden, I have the opportunity to watch and listen to the talks. Though I'm not going to miss any session from the interesting set of talks in the 2 day conference agenda, I prepared my favorite five which I should watch live from Cinnamon Lakeside, Colombo.

1. Raspberry Pi - WSO2 Style

WSO2 is THE knowledge hub in Sri Lankan software industry. I have no doubt about that. Folks at WSO2 pursuit for excellence by doing many crazy things everyday. Everyone is driven by the organizational vision of taking over the world. The perception of general public about enterprise middleware is, you always need heavy infrastructure and resources to run them. This is true for some proprietary middleware since it takes days/weeks to configure them and run a simplest use case. However, leanness is one of the primary objective of any WSO2 product. Running enterprise grade middleware in a Raspberry Pi can be a dream for some heavy weight proprietary middleware since their pre-requisites include more than 1G memory, 1G disk capacity etc.. Afkham Azeez, Sumedha Rubasinghe and  Lakmal Warusawithana are going to demonstrate how WSO2 enterprise middleware can be launched in a resource constrained environment, in the last session of conference day 1. 

2. Enterprise Identity Management and Security: Patterns and Practices

Security is a complex subject when comparing to the other aspects of SOA. However, you cannot avoid security and identity management models when building solutions. You may have come across many situations where there are enormous amount of resources publicly available for the subjects related to enterprise security, but those are explained in non-human terms using various acronyms. Prabath Siriwardhana is presenting about enterprise identity management patterns at 2PM on conference day 1. The unique ability of Prabath's is, he can explain complex subject matters in extremely simple terms. You should not miss this talk by Prabath. He will clear most of the doubts you had about enterprise security and identity management. 

3. Running operations for Carbon and Stratos

It is really important to understand the best practices associated with building operational infrastructure. It will be interesting to listen to Chamith Kumarage who will share his experience of deployment automation, scaling in the cloud, backup procedures and monitoring in enterprise middleware. Chamith will conduct his session at 2.30 PM on conference day 1.

4. Enterprise Governance with WSO2 Governance Registry and more

To achieve the true benefits of SOA, managed governance framework is a primary necessity. WSO2 Governance Registry provides users with the facilities to achieve all aspects of SOA governance including design and run-time governance, life cycle management, development process governance etc..
Senaka Fernando, the SOA governance expert at WSO2, presents how SOA governance can be adopted in an organization to realize competitive advantage, in his session at 1.30 PM on the conference day 2.  

5. Enabling a Data-Driven agile business

The talk given by Tharindu Mathew in WSO2Con 2011 was really interesting. He did that in very interactive manner asking many questions from the audience. During that session, he introduced WSO2 Business Activity Monitor and explained the key features provided by a business activity monitoring tool in general. Now, in WSO2Con 2013, Tharindu is going to demonstrate how users can get most out of data stored in your systems using WSO2 Business Activity Monitor. His session will be held at 4.30 PM of the conference day 1.


Friday, February 8, 2013

How to preserve User-Agent HTTP header when messages passing through WSO2 ESB

When WSO2 ESB forwards a request to the back-end web service, ESB alters the User-Agent of the incoming request and sets it to Synapse-HttpComponent-NIO.

Suppose, you want to avoid this and preserve User-Agent header sent by client. You can simply do it using the following method.

Step 1

Open ESB_HOME/repository/conf/nhttp.properties file

Step 2

Add the following property and save nhttp.properties file.

http.user.agent.preserve=true


Step 3

Restart ESB and send a HTTP request to ESB. Monitor the HTTP traffic between ESB and the back-end web service. You will see the User-Agent value is not altered by ESB.

HTTP headers of the inbound request (client to ESB)

POST http://localhost:8280/services/Axis2Proxy HTTP/1.1

Accept-Encoding: gzip,deflate

Content-Type: text/xml;charset=UTF-8

SOAPAction: "urn:echoString"

User-Agent: Jakarta Commons-HttpClient/3.1

Host: localhost:8280

Content-Length: 299


HTTP headers of the outbound request (ESB to back-end service)

POST /services/Axis2Service HTTP/1.1

Content-Type: text/xml; charset=UTF-8

Accept-Encoding: gzip,deflate

User-Agent: Jakarta Commons-HttpClient/3.1

SOAPAction: "urn:echoString"

Transfer-Encoding: chunked

Host: 127.0.0.1:9765

Connection: Keep-Alive






Friday, January 25, 2013

How to disable HTTP Keep-Alive connections for all out-going requests in WSO2 ESB

By default, WSO2 ESB establishes HTTP Keep-Alive connection with the back-end web service. However, in some cases, we want to close connections just after using them specially when back-end server does not properly support Keep-Alive connections .
At individual proxy service or sequence level, you can add NO_KEEPALIVE property to disable Keep-Alive connection between ESB and backend web services.

<property name="NO_KEEPALIVE" value="true"
                        scope="axis2"/>

However, if you want to disable Keep-Alive connections globally for all message mediations, the above property will not be the solution.

In such cases, you can add the following parameter to ESB_HOME/repository/conf/nhttp.properties file.

http.connection.disable.keepalive=1

Saturday, January 12, 2013

JSON pass-through in WSO2 ESB

JSON is a lightweight data exchange format used in many web applications. WSO2 ESB supports sending and receiving JSON messages out-of-the-box. Exposing a SOAP web service over JSON is explained under sample 440 in WSO2 ESB official documentation. This post is to summarize steps of invoking a pass-through proxy service which accepts JSON request, forwards it to a RESTful service and reply back with JSON response (JSON-in, JSON-out).

Pre-requisite:

  • Download and extract the binary distribution of WSO2 ESB-4.5.1
  • I will use WSO2 Application Server-5.0.1 to deploy RESTful backend web service. You can use any server as the backend. However, in order to follow the steps mentioned in this post, you should use WSO2 Application server.

Step 1:

First, we need to setup the backend web service.  We will deploy a JAX-RS based restful service which consumes and produces JSON messages. You can download this RESTful web service(jaxrs_basic.war) from here.

Step 2:

Start WSO2 Application Server, log in to management console and navigate to Manage --> Applications --> List.
You will notice the web applications which are deployed by default in WSO2 Application Server. Since we already have a JAX-RS web app with the name jaxrs_basic, delete the default jaxrs_basic web application.

Now, we can deploy our jaxrs_basic web application (Note that, jaxrs_basic is a modified version of default sample jax-rs web application which has been customized to consume and produce JSON messages).

Go to  Manage --> Applications -->Add --> JAX-WS/JAX-RS.
Upload JAX-WS/JAX-RS Applications page will be shown. Browse the downloaded jaxrs_basic.war and click on upload. New JAX-RS web service will be deployed in WSO2 Application Server.

Step 3:

The auto-generated WADL of the RESTful web service can be obtained by accessing http://<wso2_application_server_hostname>:<wso2_application_server_port>/jaxrs_basic/services/customers?_wadl
From there, you will identify the POST URL of 'customer' resource;

http://<wso2_application_server_hostname>:<wso2_application_server_port>/jaxrs_basic/services/customers/customerservice/customers

We will forward a message to the above URL from WSO2 ESB.

Step 4:

Now we have the backend web service, ready to process messages. Let's create a pass-through proxy service in ESB. Obviously, the endpoint URL will be the one mentioned above.


<proxy xmlns="http://ws.apache.org/ns/synapse" name="jsonproxy" transports="https,http" statistics="disable" trace="disable" startOnLoad="true">
   <target>
      <outSequence>
         <send/>
      </outSequence>
      <endpoint>
         <address uri="http://localhost:9764/jaxrs_basic/services/customers/customerservice/customers"/>
      </endpoint>
   </target>
   <description></description>
</proxy>

Either using Pass Through Proxy template in WSO2 ESB management console or using file system, deploy the above proxy service.

Step 5:

Let's send a JSON request to our proxy service. I will use soapUI as usual since it is the best tool out there to invoke web services.

Open soapUI project and add a HTTP request test step to any existing test suite. Change the default method as POST. Enter the proxy service endpoint URL (http://<esb_host_name>:8280/services/jsonproxy) as the Request URL.
Set the media type as application/json and add the following JSON request to POST body text area.

{
  "Customer": { "name": "charitha" }
}

Finally, your soapUI request editor will look like the following.

















Submit the request. You will get a JSON response back as shown below. WSO2 ESB will seamlessly forward JSON request to RESTful web service and forward the JSON response to soapUI.

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Server: WSO2 Carbon Server
Date: Sat, 12 Jan 2013 13:34:31 GMT
Transfer-Encoding: chunked

{"Customer":{"id":"125","name":"charitha"}}


NOTE:
In the previous versions of ESB (4.0.3 etc), you should add JSON message builders and formatters to ESB_HOME/repository/conf/axis2.xml in order to enable JSON message processing capabilities in ESB. (See http://docs.wso2.org/wiki/display/ESB403/ESB+and+JSON)
However, these builder and formatters are enabled by default in WSO2 ESB version 4.5.1 and later.  In addition to that, you may need to set the following property in IN and OUT paths of proxy services and sequences in previous ESB versions.

<property name="messageType" value="application/json" scope="axis2" />


Saturday, January 5, 2013

Effective Test Automation and WSO2 System Test Framework

The automated tests are an absolute necessity in agile software development. However,  the true benefits of test automation can only be achieved only through a systematic process which should consist of;
  • Focused automation strategy
  • Dedicated teams to maintain and manage test automation infrastructure
  • Firm commitment by everyone in engineering NOT specific to a separate testing team
For the past few years, we tried to build various types of automation test frameworks as part of WSO2 Carbon middleware platform. Our first attempt was to build a Selenium based test framework back in 2009. We were able to use that for four release cycles in that year. Later on, the drastic architectural and UI changes made the framework unmanageable hence we could not continue use it.

WSO2 System Test Framework (A.K.A WSO2 Clarity) came in to picture during 2011. The new test framework development was handled by a separate test automation team headed by Krishantha. The team overcame many challenges and worked like crazy to build a test framework which would serve not only for one or two release cycles but as a long-lasting test automation solution.

As I mentioned in many of my previous blog posts, it takes time to achieve the real advantages of test automation. The one most important thing is, everyone needs to be patient. You cannot achieve greater test coverage in overnight. WSO2 Clarity test framework is primarily driven through admin web service APIs. There is a little chance to break tests in between releases when comparing to traditional UI oriented automated tests.
The future releases of WSO2 middleware products and cloud services will evaluate the success or failure of the new WSO2 Clarity test framework. But personally, I'm experiencing the true value given by the new framework in day-to-day work which are carried out at WSO2.
WSO2 Clarity framework is an integral part of WSO2 continuous integration system (CI). The tests are automatically executed as part of the build process. However, we usually have many requirements to execute tests externally instead of gluing into build system. For example, we patch the products, we do configuration changes such as change the underlying OS, JDK or DBMS etc.. In such cases, a test framework which is tightly coupled with build system does not help much. Because of that, we wanted to have a mechanism to launch tests externally from the maven build. On the other hand, we wanted to have a simple mechanism to run tests without building massive Carbon code base.
Thanks to Krishantha and test automation team,  binary deliverable of Clarity test framework has been provided as a solution for this long outstanding requirement. A preview version of it can be downloaded from https://svn.wso2.org/repos/wso2/people/krishantha/wso2clarity-1.0.5.zip

This will surely help anyone who uses WSO2 products/services not specific to internal testers and/or developers. How?
Suppose, you download the latest version of WSO2 ESB and want to customize default settings (e.g:- you simply want to deploy the product in IBM JDK and change the default H2 registry database to DB2). Now, you want to make sure there are no adverse effects due to those changes. You can download the Clarity binary and launch all ESB integration/system tests. The test runner is completely ANT based and you just need to have Apache ant. Update the clarity configuration file by modifying the host, IP etc (as mentioned in INSTALL.txt inside Clarity binary distribution) and run the tests you need.
Note that, we still have a preview version of Clarity binary. This will eventually be part of product downloads.

Friday, January 4, 2013

TIP: How to view log statements generated by WSO2 ESB log mediator in System Logs UI?

When you use log mediator inside sequences and proxy services in WSO2 ESB , the logs are shown by default in wso2carbon.file which can be found in CARBON_HOME/repository/logs directory. The same logs are viewable from "System Logs" page in ESB management console.
However, from ESB-4.5.X releases, you should do the following simple change in order to see these logs in "System Logs" UI in management console.

Step 1
Open CARBON_HOME/repository/conf/axis2/axis2.xml

Step 2
Uncomment the following element.

             <handler class="org.wso2.carbon.utils.logging.handler.TenantDomainSetter" name="TenantDomainSetter">

Now, restart the server. Send a request to ESB and see the logs in "System Logs" UI in management console.