Popular Posts

Monday, November 21, 2011

WSO2 Deployment Synchronizer - Sharing deployment artifacts across a product cluster

This post is about a new feature in WSO2 Carbon product platform. I will use the latest versions of WSO2 Governance Registry(WSO2-G-reg-4.1.0) and WSO2 Enterprise Service Bus (ESB-4.0.2) for the demonstration.
Before going through the configuration steps, lets look at the problem which is going to be addressed using Deployment Synchronizer.
Suppose we have a WSO2 ESB product cluster with a single READ-WRITE node and a several READ-ONLY nodes which shares a common configuration registry. When we deploy a proxy service from READ-WRITE node, in order for other nodes to be synced with that new service deployment, all READ-ONLY nodes have to be restarted. This is a painful concern in a large product cluster.
WSO2 Deployment Synchronizer feature has been implemented to resolve that concern. With that, once you deploy a service (or any deployable artifact such as ESB sequence, scheduled task etc..) from one node in a product cluster, the other nodes automatically get the changes and sync up with the master (or READ-WRITE) node.
The Deployment Synchronizer makes use of two different approaches for deployment artifact synchronization.

1. SVN based synchronizer
2. Registry based synchronizer

Lets look at each of these in detail.

Download WSO2 ESB-4.0.2 and WSO2 Governance Registry-4.1.0 binary distributions from wso2 oxygen tank

Product Clustering Setup

Lets proceed through setting up a two node WSO2 Carbon product cluster as shown above.

Step 1:

Extract the downloaded WSO2ESB-4.0.2.zip and make two copies of it as wso2esb-rw and wso2esb-ro
wso2esb-rw directory is used as the master node of the cluster and wso2esb-ro node will be used as the slave node.

Step 2:

Extract the downloaded WSO2greg-4.1.0.zip into a new directory. This will be used as the central governance and configuration registry in our ESB cluster.

Step 3:
The above WSO2 G-reg instance will run on a mysql DB instead of the default H2 database. Therefore, lets create a mysql DB first.
Open a mysql prompt in your server. Type the following commands to create a database and assign user privileges.

mysql>create database reg_db;
mysql>use reg_db;
mysql>grant all on reg_db.* TO regadmin@localhost identified by "regadmin";

Edit the CARBON_HOME/repository/conf/registry.xml of the WSO2 G-reg server as follows.

<dbConfig name="mysql-reg">
<validationQuery>SELECT 1</validationQuery></dbConfig>

Now, copy mysql jdbc driver (mysql-connector-java-5.1.7-bin.jar or later) to CARBON_HOME/repository/component/lib directory of WSO2Greg server and start the server with -Dsetup switch.

sh wso2server.sh -Dsetup

This will start WSO2-Greg server on mysql.

Step 4

Now, we have started central governance and configuration registry instance of our WSO2 ESB product cluster. Now, lets proceed with configuring WSO2 ESB nodes.
Lets configure read-write node first.

We are going to run 3 carbon servers in the same machine. Therefore, we need to change the port index in CARBON_HOME/repository/conf/carbon.xml so that each of the WSO2 ESB nodes will run on their own ports without conflicting with each other.
In carbon.xml, change the following element in order to run the ESB read-write node in HTTP port 9764 and HTTPS port 9444.


We are going to store the configuration data of ESB nodes in the cluster in /_system/esbnodes space of the above registry. Also, the governance data will be stored in /_system/governance directory. Therefore, lets add the following registry mounts through CARBON_HOME/repository/conf/registry.xml.
<dbConfig name="mysql-reg">
<validationQuery>SELECT 1</validationQuery></dbConfig>

<remoteInstance url="https://localhost:9443/registry">

<!-- Governance data will be stored in /_system/governance collection of central registry instance -->
<mount overwrite="true" path="/_system/governance">

<!-- Configuration data will be stored in /_system/esbnodes collection of central registry instance -->
<mount overwrite="true" path="/_system/config">

Copy mysql jdbc driver (mysql-connector-java-5.1.7-bin.jar or later) to CARBON_HOME/repository/component/lib directory of WSO2 ESB read-write node and start the server.

sh wso2server.sh

Step 5

We have configured and started the READ-WRITE node of ESB cluster. Now, we can configure the READ-ONLY node. The configuration is almost same as READ-WRITE node except the highlighted elements given below.

First, change the port offset to 2 so that the ports will not be conflicted with the other server ports.


Change the default NIO HTTP and HTTPS ports in CARBON_HOME/repository/conf/axis2.xml as follows.

<transportReceiver class="org.apache.synapse.transport.nhttp.HttpCoreNIOListener" name="http">
<parameter locked="false" name="port">8281</parameter>

<transportReceiver class="org.apache.synapse.transport.nhttp.HttpCoreNIOSSLListener" name="https">
<parameter locked="false" name="port">8244</parameter>

Add the following registry mounts through CARBON_HOME/repository/conf/registry.xml of ESB READ-ONLY node.
<dbConfig name="mysql-reg">
<validationQuery>SELECT 1</validationQuery></dbConfig>

<remoteInstance url="https://localhost:9443/registry">

<!-- Governance data will be stored in /_system/governance collection of central registry instance -->
<mount overwrite="true" path="/_system/governance">

<!-- Configuration data will be stored in /_system/esbnodes collection of central registry instance -->
<mount overwrite="true" path="/_system/config">

Copy mysql jdbc driver (mysql-connector-java-5.1.7-bin.jar or later) to CARBON_HOME/repository/component/lib directory of WSO2 ESB read-only node and start the server.

sh wso2server.sh

Now we are done with the clustering setup but we have not done any configurations related to the deployment synchronizer yet.
We will look in to the registry based deployment synchronization first.

Step 6 - Registry based deployment synchronizer

In this mode, once you deploy an artifact from the READ-WRITE node, the artifacts will be stored in the relevant collection in the configuration registry. The other nodes of the cluster will use the registry checkin-checkout client and checkout the deployment artifacts to their file system. Then with the hot deployment functionality, the artifacts will get deployed in the cluster nodes.

Lets look at how we can use the registry based deployment synchronizer.

  • Log in to management console of ESB READ-WRITE node (https://localhost:9444/carbon)
  • Navigate to Configure --> Deployment Synchronizer UI

  • Select Auto Commit option and click on Enable
  • Log in to management console of ESB READ-ONLY node (https://localhost:5/carbon)
  • Access Configure --> Deployment Synchronizer UI
  • Select Auto Checkout option and click on Enable
  • Create a proxy service (eg:-proxy1) in READ-WRITE node
  • After 60 seconds (the default synchronization period), log in to the management console of the READ-ONLY node of the cluster.
  • You will notice that the proxy1 is listed in the services list of READ-ONLY node

Step 7 - SVN based deployment synchronizer

Deployment synchronization can be achieved using a SVN repository as well. Lets look at SVN based deployment synchronizer.

In this mode, instead of a registry, we use a subversion repository as the deployment artifact store. As we check-in files to SVN, the synchronizer use a SVN client API and commit and update deployment artifacts periodically by using a SVN location.

  • First, revert the registry based deployment synchronizer settings which we did above. (Disable deployment synchronizer in Configure --> Deployment Synchronizer UI)

  • Have a proper SVN location. You can use an existing SVN location or create a new one.
  • Add the following configuration in CARBON_HOME/repository/conf/carbon.xml of both ESB nodes. Make sure to specify correct SVN credentials and location URL according to your environment.

  • Restart both ESB nodes.
  • Deploy a proxy service from either one of the nodes
  • Changes will get reflected into other nodes after 60 seconds (default synchronization period)

We looked at two different ways of sharing deployment artifacts among cluster nodes. If you come across any issues when configuring either one of the above approaches, please drop me a mail.

Tuesday, November 15, 2011

How to pass system properties when WSO2 Carbon server is running in daemon mode

When you start wso2 carbon server using a startup script such as wso2server.sh, you can simply pass system properties such as -DosgiConsole, just by passing system property in command line.
e.g:- sh wso2server.sh -DosgiConsole

However, if you start the server as a System process (daemon), how should you send those parameters?

Lets look at how we can start OsgiConsole, if we start server in daemon mode.

1. Open CARBON_HOME/repository/conf/wrapper.conf

2. Add the following parameter under Java additional properties section

3. Start the server as "sh wso2server.sh -start"

4. Open a new shell and "telnet localhost 1234"

Sunday, November 6, 2011

ApacheCon NA 2011 - a week full of Apache community gathering

ApacheCon is the official event of Apache Software Foundation which brings together the global Apache community in a week full of trainings, live demos, hands-on sessions and various other meet ups.
The event will be commenced tomorrow, 7th of November 2011 in the Westin Bayshore Vancouver Canada.
I'm witnessing a lot of technology enthusiasts from diverse parts of the world are checking-in at Westin Bayshore hotel at the moment predicting a successful function.

With the tightly-coupled relationship of WSO2 and various Apache projects, WSO2 is playing a key role in ApacheCon 2011 as well. WSO2 is presenting 3 talks and is an exhibitor Sponsor too.
He is also talking about an architecture for enabling multi-tenancy for Apache Axis2 on Thursday, November 10th.
Prabath Siriwardhana will also do a half-day training on web services security on Monday, November 7th.
Come and meet us at WSO2 booth at ApacheCon. You will find how WSO2 built world's first free and open source PaaS and renowned SOA platform using various Apache project components and how we adhered to the Apache process in project releases.

Friday, October 28, 2011

Why is testing taking so long?

When you are involved in software testing, you may have heard the following 3 common questions at the end, beginning or middle of any testing cycle.

1. Why did not you find that bug during testing cycle?
2. Why is testing taking so long?
3. Why did not you automate the damn thing?

I have answers for all three questions but Michael Boloton, the testing genius in our era, explained the answer for question 2 with some great examples. I would recommend this to be a must read for anyone work in software engineering.

Study the facts given by Michael. You will realize the truth of test estimation and traditional way of managing test processes.

Saturday, September 17, 2011

Was I able to do that?

For the last 2-3 months, the readers of my blog may have seen a small icon at the top of the pages giving you a hint that I'm presenting at WSO2Con 2011, the technology conference of the year. A few months back, Samisa, the engineering head of WSO2, forced me to submit a talk for this conference. Hesitating, I prepared an abstract of a talk about SOA and quality and sent my proposal. After a few weeks, I was informed that my proposal had been accepted and that I would speak at the third day of the conference. First, I was shocked. Why? I always thought negatively about presentations and public speaking. As my batch mates at university of Colombo knows, I myself and a lot of others had a bad experience w.r.t presentations during campus era :). Worst, I had not given a public speech for a time that I remember. In summary, it had been kind of a nightmare to me.
I did not have any other option, but do this speech at an level that I can watch it later without cursing me at least :) Therefore, I started preparation. The subject and topic is not a surprise for me. It is the subject that I live in and spent 8 years of my career, testing and QA. SOA is also not new to me, I have been working with SOA for 5 years out of 8 years. Therefore, I knew that I have a LOT to talk. But just talking non-sense was not required. SOA testing is a relatively new topic and there is not universally agreed methodology for testing service oriented solutions. I thought to put together my experience into a methodology and present.
I borrowed SOA design, concepts and principles book written by Thomas Erl from WSO2 library and read it completely. It was a great reference and I was able to clarify a lot of concerns that I had about some SOA concepts.
Then, I started to prepare slides. When the days of WSO2con were coming closer, the nervousness and stress going up straightly. One day I met with my close friend and the presentation and public speaking master blaster at WSO2, Prabath Siriwardhana. He nicely explained some of the tips that he followed during presentations and he mentioned that practicing a speak is the best way to do it nicely. So, I practiced. I practiced a lot everyday. I learned a lot. I explored everything related to SOA testing. I my self came into a position that I was able to answer very complicated questions related to SOA testing. The confidence started to build. BUT, nervousness and negative mind set still did not disappear.
As usual, I'm not a closed person. Hence, I openly talked with colleagues at WSO2 and my friends about my nervousness and possible failure cases. Fortunately, I live in a world where 90% of my friends, WSO2 colleagues are truely friends. They listened to me and helped me a lot. The senior wso2 folks such as Afkham Azeez, Selvarathnam Uthaiyashankar, Samisa, Prabath, reviewed my slide deck and helped me to correct some errors. Then, most of the folks listened to my boring rehearsal sessions at office premises :) Iranga, one of my closest friends, visited me all the way from his office and went through the slides and speech. His input was also invaluable because of his experience on QA.
Finally, on 15th of Thursday, September 2011, I delivered my first public speech at WSO2Con in front of a large gathering including a lot of folks from various companies all over the world.

As the presenter, I'm not sure whether it is a success or failure, but everyone came into me and congratulated that it was a nice talk. I still cannot believe! I'm very lucky to see the tweets as below as the response to my speech!

Monday, September 12, 2011

Must attend talk in WSO2Con2011 - Engineering to take over the world by Samisa Abeysinghe

WSO2Con 2011, a week full of tutorials, tech talks, networking events will be commenced shortly at WatersEdge, Colombo, Sri Lanka. If you are attending the conference, I'm sure you will look for a session which gives you an overall picture about how WSO2 has become a leading technology provider and what are the secret ingredients of the process that has been followed.

There is no other individual than Samisa Abeysinghe, VP of Engineering at WSO2, who knows top to bottom of WSO2 engineering process as well as lead the engineering team for the last five years to become where we are now. If you are bored with listening to traditional process sermons which are usually heard in conferences, this will be a totally different session for you. We are at WSO2, while redefining the middleware space, reinventing the software development process too. We have our own way of designing, developing and testing products. Wanna know more about how we do that? then do not forget to attend the session by Samisa which will be held on 15th of September 2011 at 15.30 PM!

Sunday, September 11, 2011

Must attend talk in WSO2Con2011 - Security in practice by Prabath Siriwardhana

Time flew in light speed and finally the days have arrived! WSO2Con 2011, a conference which you should not miss. Though you are a seasoned IT professional or a student keen in learning latest technologies, it is the event of the year.

Out of more than 25 sessions which are planned to be held, my attention goes to one particular session first. It is "Security in Practice" by Prabath Siriwardhana which will be held on 14th of September, 2011 at 3.30 PM. Why do I like it than any other session?

Prabath is a brilliant speaker. I like the way he presenting. He has his own style which attract anyone in the audience.

Prabath is the expert of the most complex and hard-to-understand aspect of Service Oriented Architecture, Security.

You can imagine the complexity of delivering a presentation in such a tough, complicated subject matter

But, Prabath has the ability to inject complex subject matters into your head in very nice manner. You should not miss this great session of Prabath if you are attending WSO2Con 2011!

Tuesday, August 23, 2011

WSO2 Stratos Application Server - Apache Tomcat As a Service

Few days back, one of my friends developed a new web application and he wanted to try it out by deploying on a J2EE server. I was at his place and was observing the steps he followed to deploy the web app.
  • Downloaded the latest version of Apache Tomcat Server
  • Extracted and installed it on a local server
  • Ran the startup script and started the server
This took considerable amount of time. In the world of everything offered as services, my friend was not aware of the free service of hosting web applications in cloud. In other words, my friend has not known of "Apache Tomcat As a Service".
If I was him, I would have tried out my web applications with Apache Tomcat by following the three simple steps given below.

Step 1 - Create a demo user account at WSO2 StratosLive

This is a simple registration procedure and follow the steps given in my previous blog post to create a demo account.

Step 2 - Access WSO2 Stratos Application Server
You can access Application Server from WSO2 Stratos Manager home page or just type https://appserver.stratoslive.wso2.com in your favorite browser.

Step 3 - Upload web application

Click on Manage --> Web Applications --> Add in the left navigation menu. You will be directed to the following screen.

Browse the webapp (*.war) in your local machine and click on upload. I downloaded a sample calendar.war webapp from here for demonstration purposes. Once the file upload is done, the deployed calendar.war will be listed in Running Web Applications page where you can do all web app management tasks such as session handling, reloading webapps, deleting etc..

Isn't this the simplest way to run your web applications? You will not be charged a cent for the steps which I demonstrated above. Once you are satisfied with testing your web apps on Stratos Application Server, you can upgrade your usage plan as needed and have an enterprise ready web application running on cloud!

Tuesday, August 16, 2011

WSO2 Data As a Service - Have your own data storage in cloud!

WSO2 StratosLive, the most complete open PaaS (platform-as-a-service), includes multiple types of SOA middleware services which can be used to build and host your SOA based solution. One of the newest additions to WSO2 middleware platform is, the functionality which enables you to have your own database in cloud for free!
This post guides you through the steps of creating your own data storage in cloud using WSO2 StratosLive.

Step 1
First, we need to create an account in stratoslive. You can have a demo account free of charge.
Open a browser and access WSO2 StratosLive landing page. Click on Get Started Now for Free
You will be directed to domain registration page.

Fill the form and follow the instructions in registration email to activate the account.
Once you logged into WSO2 StratosLive, you will be shown all of the cloud services which are available for use. Out of those services, click on Data Services Server link.

You will be directed to the home page of WSO2 Data Service Server.

Step 2

Now, we have accessed the home page WSO2 Data service server in cloud. You will notice Databases menu item in the left navigation pane as shown below.

Click on Databases --> Add option in the above screen. You will be directed to the New Database page. In this page, you will be asked to provide Database Server Instance Name and a name for the database which you are going to add. By default, WSO2 Data Services Server provides you with a pre-configured Amazon RDS instance, in which you can create your database. You also add your own database server instance and use that instead as you preferred. But in this example, we will use the default database server instance. Lets select the default DB server instance and enter "qa" as the name of the database.
Once you click on create, you will be prompted a confirmation dialog, "Database has been successfully created".

Your newly created database will be listed in database list as shown below. Note that, the name of the database is suffixed with tenant domain name.

Step 3

We have created our first database successfully in cloud using WSO2 Data Service Server. Lets, add a new user to the database and create table so that we can use the database for any DB related operation.

Click on Manage icon in the above table. Database User Management page will be shown.

Before adding any user, you should add a database user privilege group. Click on Add Privilege Group and add a new privilege group. Specify select, insert, update, delete and create privileges for the group as shown below.

The newly created privilege group will be shown in Database User Privilege Groups page as follows.

Now, go back to the previous page and click on add new database users icon to access Database Users screen where you can add DB users. Click on add new user.
Specify "test" as username and password and select "group1" as the privilege group.

Step 4

Once you create a new DB user, it will be listed in Database Users page with the options to explore database using the given user credentials, edit, drop and create carbon datasource.
Click on explore database. Database console screen will be displayed as follows. This console allows you to issue any SQL statement and act as an SQL editor for your database.
You may type any SQL statement and hit run button so that the query will be executed on the database which has been created in the default amazon RDS instance provided by WSO2.

Now, you can use this DB as your storage media in your SOA platform in cloud. For example, you may deploy a webapp in WSO2 Stratos AppServer which talks to this particular DB. Or you may simply create a data service using this database.

Friday, July 22, 2011

PaaS testing

PaaS (Platform as a Service) is an application delivery model which provides services to design, develop, test, deploy and host applications. It plays a key role in cloud computing space similar to the other services, IaaS (infrastructure as a service) and SaaS (software as a service). Google App engine, vmForce and WSO2 Stratos are some of the popular PaaS offerings.
This post is not about detailed concepts of PaaS or cloud computing. Rather, I'm going to look at the testing and quality assurance aspects of PaaS. PaaS testing is not a widely discussed topic. There is no pre-defined model for PaaS testing. We need to explore the features of PaaS and derive an approach for testing PaaS apps. I will go through some basic components based on my experience of testing WSO2 Stratos opensource cloud platform.

Hosted services testing - Minimized deployment overhead

In simplest terms, PaaS testing is about testing a hosted platform. So, there is no deployment and configuration overhead for the test team. Testers are expected to access the hosted services remotely using web browser and carry out functional and non-functional testing.

Lesser test platform combinations

When you test a standalone product or a combination of products, you should try out the possible platform combinations such as different types of clustering setups, multiple DBMSs, Application Servers or various operating systems. However, when you test the same product suite or platform on cloud (PaaS), you are bound to one optimum configuration stack. You have one or two choices. There is no requirement to try all possible platform setups.

Multi-tenancy aspects

Multi-tenancy allows a single application to emulate multiple application instances. With multi-tenancy, a single application can be shared across many organizations. Therefore, series of tests should be carried out carefully to verify the multi-tenant aspects. For example, if company A logs in to a web application hosting service in your PaaS offering and deploy a web app, then the users of company B should not be able to locate it unless company A made it public. In other words, the configurations done by one tenant should not be exposed to other tenants.
These mission-critical aspects are some of the key requirements which should be taken into account in any PaaS testing model.

Performance and scalability

Any hosted service should conform with the SLAs. Similar to a website, services included in a PaaS can be accessed by multiple users concurrently. Therefore, sufficient amount of performance testing must be carried out. The usual usage pattern of standalone technology platform can be dramaticallty changed when it is hosted as a service in cloud. For example, the general use cases of WSO2 Carbon SOA middleware platform are different from the standalone version when the platform is exposed as a service. WSO2 SOA middleware platform consists of 12 different products and all of them includes a management UI for administration purposes. In non-cloud based deployment, the multiple, concurrent users access to the management UIs are minimum since it is very rare that hundred of administrators accessing management console at once.
However, if this platform is hosted as a service in cloud (PaaS), then the concurrent user access to the management UIs are a highly desirable scenario.

It is a requirement that the hosted platform should be able to handle load seamlessly without affecting consumers. Auto-scaling is a key feature of WSO2 Stratos cloud platform. When new resources are needed, WSO2 Stratos transparently adds services and when load goes down, WSO2 Stratos automatically brings services down. Testers should ensure, with the load, the new EC2 instances spawning up and down correctly. The tools like hybridfox are very useful in these situations.


If your PaaS provides users with the ability to store their data in cloud and various database centric operations, then the testing will be much complicated. As a tester, you must ensure the availability of data sources, accessibility and provisioning aspects. You must also take extra care that, if the data are geographically distributed in cloud, then it adheres to the legal requirements of the users of your PaaS.


Security is the utmost important aspect of any cloud offering. It is recommended to have a separate group of security testers who do penetration testing and ethical hacking in order to ensure secure infrastructure of your PaaS. WSO2 Stratos cloud platform allows users to deploy their webapps, web services and various custom code. Therefore we must ensure that no vulnerable code are deployed and a code deployed by one tenant does not affect the operations of others or whole platform.


Public PaaS offerings are usually based on virtualized infrastructure such as Amazon EC2. Therefore, a full functional test cycle should be carried out on EC2 VMs inorder to make sure that there are no regressions due to virtual servers.

In addition to the above key areas, PaaS testing model should consists of the general web application testing aspects such as cross browser testing, accessibility testing, localization testing etc..

Monday, July 18, 2011

Data driven testing with Jmeter user parameters

This is a follow up to one of my previous posts which explained data driven web service testing using CSV config element in Jmeter. There, we used CSV file to read input data for SOAP/XML-RPC sampler.
In this post, we will look in to using User Parameters pre-processor element as the data source instead of a CSV file.

Step 1

We are going to use the same web service which we used in my previous post, temperature conversion service. Please add the SOA/XML-RPC sampler, the SOAP request and the necessary thread group as described in step 1 and 2 of that post

Step 2

Lets parameterize the payload of SOAP message so that different requests will be sent to the service with each test run. Instead of reading data from a CSV file, we can add a User Parameter pre processor element in Jmeter test plan.

Right click on the thread group of your jmeter test plan and select Add --> Pre Processors ---> User Parameters
Click on Add Variable and specify celcius as the name of variable. Add few users and enter celcius values for each user as shown below.

Step 3

Now, parameterize the payload of SOAP as follows.


Step 4

Increase the thread count corresponding to the user count in your user parameters pre-processor element and run the test. You will notice that the Celcius figure will be varied in each request.

Based on your requirements, you can select either CSV config element or User Parameter pre-processor element for data driven testing. If you have large number of variables to be parameterized, CSV config is the best option.

Wednesday, June 15, 2011

WSO2 Carbon-3.2.0 - The latest with many new features and a release to remember

On 12th of June 2011, the latest version of WSO2 SOA middleware platform, WSO2 Carbon-3.2.0 has been released. Since 2009 January, WSO2 SOA middleware platform has been grown and matured and now it consists of 12 different products.

In this latest version, two new members joined into WSO2 Carbon product family. Those are WSO2 Complex Event Processing Server and WSO2 Message Broker.

In this post, I'm not going to explain the new features and enhancements included in each of these products. You can find them out in the respective product pages in wso2.com.
But, I would like to share how we evolve in terms of features, process and release methodology during the last couple of years.

In late 2008, we started the implementation of WSO2 Carbon platform. Before that, we had four isolated products (WSO2 ESB, WSO2 WSAS, WSO2 IS and WSO2 Registry) but those had limitations to work as a platform. The major objective of WSO2 Carbon platform was to build a suite of products which can be integrated easily and run together with minimum configuration overhead. At the end of the first WSO2 Carbon platform release (version 1.5.0), we were able to achieve that objective to some extent. The first release was a VERY different experience for us since we used to do releases with one product at a time so that the test team had sufficient time and resources to try out various scenarios. Because of that, we had to move in to a totally different testing paradigm.
In early 2009, we had to do a patch release of WSO2 Carbon platform (version 1.5.1) because we uncovered some critical issues after the first Carbon release. With this, we badly felt the need of adjusting our testing methodology to align with WSO2 platform vision. We spent hours in each build to test management console UIs manually and our first objective was to cut down that time and effort. In order to do that, we started developing a selenium based test automation framework. In 2009, we did 3 WSO2 Carbon platform releases (version 2.0.0, 2.0.1, 2.0.2) and we were able to make use of selenium tests heavily in all those release cycles.

In 2010, we went through a major UI refactoring process across the whole WSO2 Carbon platform. Unfortunately, we could not keep maintaining selenium tests with the pace of UI changes. Therefore, we wanted to have a test framework with minimum maintenance overhead. We wanted to call methods of various features bypassing UI elements so that the frequent UI changes do not break the tests. We started admin services based test automation framework, which used to call the operations of administration services associated with each of the features programmatically (Java/Junit).
2010 was an extremely busy year for most of us since we did a lot of frequent releases, but there were very limited automated tests. At the end of 2010, WSO2 Carbon became a fully componentized, modular SOA middleware platform which consisted of multi-tenancy and a lot of usability enhancements. As the testing team, we kept on observing the growth of WSO2 Carbon platform more than anyone else since we deal with almost all aspects of each and every product. As I said at the beginning, one of the objectives of WSO2 product platform was to provide users with ease of integration. By end of 2010, we realized how far we achieved that. Installing a new feature on any of the WSO2 Carbon based product became a simple click'n'click process. Most features worked out-of-the-box.
Now, after 2.5 years of the first Carbon release, our product platform has been improved in various aspects with 12 different products. Our development and testing methodology has been changing with each release to accommodate the pace of growth and objectives in WSO2 SOA platform. In the latest 3.2.0 release, our focus shifted towards more platform level testing, which required us to learn and familiar with each product. With the platform aspect, we cannot do individual product releases. We do all 12 products at once, which demands huge work load on both development and testing teams. At WSO2, we do not trust or rely on the number of testers in the team but their individual capabilities. We realize the need of having 80% of test coverage through some kind of automation in order to sustain with the pace. For all these years, we do not adopt or follow any specific process blindly because that works for some other organization so that we do. Instead, we are following a context-driven testing approach. We learn and enhance our processes with our own mistakes and experiences. While WSO2 is revolutionizing the world of enterprise middleware, testing and development methodology of middleware will also be redefined.

Tuesday, May 17, 2011

WSO2 SOA middleware Deployment Tips - 1 - validationQuery to avoid broken DB connections

I thought to put together some best practices, guidelines on deploying WSO2 SOA middleware platform. This post will remind you a well-known best practice used in most of the product deployments.
When you are maintaining DB connections, it is always recommended to use a validationQuery to check the health of the TCP connection of the connections stay in DB connection pool.
Because the connection opening is an expensive and time consuming operation, after a connection is created, it will be kept open for a specific time in the pool. When re-using these connections from the pool, there can be situations that the TCP connection to the DB is interrupted and the connection consumer gets errors such as communication link failures etc..
In order to avoid that, a validationQuery, an SQL statement specific to DBMS type, can be used which runs before using the connection.

In WSO2 middleware platform, you usually use central governance/configuration registry to store SOA metadata and various governance tasks. Governance registry supports multiple DBs as underlying data stores. You establish the connection to DB in CARBON_HOME/repository/conf/registry.xml as follows.

<dbConfig name="wso2registry">

This is one of the places where you may experience DB connection issues as I explained above. Therefore, it is always a best practice to use a validationQuery.

If your DBMSs is MySQL or MSSQL, then use the following.

<validationQuery>SELECT 1</validationQuery>

In Oracle;

<validationQuery>SELECT 1 FROM DUAL</validationQuery>

In Postgres,

<validationQuery>SELECT version(); </validationQuery>

Similarly, when you make DB connections in WSO2 Data Services Server, make sure to use validationQuery in data source definition section as follows.

<property name="org.wso2.ws.dataservice.validationquery">SELECT 1</property>

Wednesday, April 20, 2011

Monitoring WSO2 ESB using JConsole

My latest tutorial on how to monitor WSO2 ESB using JConsole is now published on WSO2 Oxygen Tank

Tuesday, April 12, 2011

How to send JSON messages to web services deployed on WSO2 Application Server

JSON (JavaScript Object Notation) is an open and text-based data exchange format, that provides a standardized data exchange format better suited for Ajax style web applications. You can find more information about JSON from www.json.org
WSO2 SOA middleware platform supports sending and receiving JSON messages.
This post takes you through the steps to deploy a simple web service in WSO2 Application Server and write a client using Axis2 ServiceClient API to invoke the service by sending JSON message.

Download and install WSO2 Application Server 4.0.0 or later

Step 1
We are using a simple web service, which echo's user input as follows.

public class EchoService {

public OMElement echo(OMElement element) {
return element;

You can download the service archive (EchoService.aar) from here and deploy on WSO2 Application Server.

Step 2

Have a look at CARBON_HOME/repository/conf/axis2.xml (CARBON_HOME is the root directory of WSO2 Application Server). You will notice that the JSON specific message builders and formatters are enabled by default.

<!--JSON Message Formatters-->
<messageFormatter contentType="application/json"
<messageFormatter contentType="application/json/badgerfish"
<messageFormatter contentType="text/javascript"

<!--JSON Message Builders-->
<messageBuilder contentType="application/json"
<messageBuilder contentType="application/json/badgerfish"
<messageBuilder contentType="text/javascript"

WSO2 Application Server accepts any JSON message with the content type application/json, application/json/badgerfish or text/javascript. In this example, we will use a message with the content-type, application/json

Step 3

Now, If you invoke the service using the Tryit utility associated with EchoService and trace the message, you will notice the payload of the SOAP message as follows.

<p:echo xmlns:p="http://service.carbon.wso2.org">
<echo xmlns="http://service.carbon.wso2.org" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xs:string">charitha</echo>

Basically, the message payload will be similar to the following.


Step 4

Now, we are going to send the above payload as a JSON message. The JSON format of the above message payload will be as follows.


Add the following class to your java project. Compile it using the libraries included in CARBON_HOME/repository/component/plugins directory. You may also need to add CARBON_HOME/lib/endorsed into your class path.

package org.wso2.carbon.service;

import org.apache.commons.httpclient.HttpClient; import org.apache.commons.httpclient.methods.PostMethod; import org.apache.commons.httpclient.methods.RequestEntity; import org.apache.commons.httpclient.methods.StringRequestEntity;

public class JSONClient {

public static void main(String[] args) throws Exception {
String strURL = "http://localhost:8281/services/JSONProxy/"; PostMethod post = new PostMethod(strURL); RequestEntity entity = new StringRequestEntity("{\"echo\":{\"value\":\"charitha\"}}","application/json", "UTF-8"); post.setRequestEntity(entity); HttpClient httpclient = new HttpClient(); try { int result = httpclient.executeMethod(post); System.out.println("Response status code: " + result); System.out.println("Response body: "); System.out.println(post.getResponseBodyAsString()); } finally { post.releaseConnection(); } }

You could use the same axis2.xml located at CARBON_HOME/repository/conf as the client axis2.xml when creating ConfigurationContext

If you forward the message to tcpmon and trace it, you will see the request as follows.

POST /services/EchoService HTTP/1.1
Content-Type: application/json; charset=UTF-8
User-Agent: WSO2 WSAS-3.2.0
Transfer-Encoding: chunked


Sunday, February 27, 2011

Data driven web service testing with Apache Jmeter

You will not get enough coverage in your web service testing, if you repeatedly send the same request to the webservice under testing. You must read data from a data source and parameterize each request. If you are dealing with SOAP messages, you should parameterize SOAP message payload.
This post guides you how to do data driven web service testing using Apache Jmeter.

Install Apache Jmeter 2.3.4 or later

Step 1

We are going to invoke a publicly available web service, Temerature Conversion service . The WSDL of this web service can be accessible through http://webservices.daehosting.com/services/TemperatureConversions.wso?WSDL
You can invoke the CelciusToFahrenheit operation of this webservice using SOAPUI and capture the request message. It will be similar to the below.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tem="http://webservices.daehosting.com/temperature">
Step 2

Lets create a new JMeter test plan and add a thread group. Then, add SOAP/XML-RPC sampler into the thread group and copy and paste the above SOAP request into the SOAP/XML-RPC data section of the sampler.
Enter the URL of the service as http://webservices.daehosting.com/services/TemperatureConversions.wso (You can capture this from the location attribute of the address element in the WSDL)
Now add a listener to view the result.
Save your test plan. Your test plan will be similar to the below.

Run the test and check the results. You will get the Fahrenheit value of the provided Celsius figure.

Step 3

Now, you can increase the thread count and extend this test into a performance test. However, in that case you will be sending the same request again and again. It will not be a good simulation of a real-world scenario. You should be able to alter the payload (in our example, Celsius value) of the SOAP request with each thread.
In order to do so, you should read Celsius data from a data source. In Jmeter, you can easily read data from a csv file.
Lets create a csv file, temperature.csv and save it in the location where you saved the above JMeter test plan.
Enter a set of values row-by-row in the temperature.csv

Step 4

Next, we will add CSV Data Set Config element which will read data from the csv file.
Right click on Thread Group and select Add --> Config Element ---> CSV Data Set Config
Now you can configure your CSV data source as follows.

Filename: <Give the full path of temperature.csv>
Variable Names: celcius

Keep the other values intact.

Step 5

Now access the SOAP/XML-RPC Data section of the request and replace the hard-coded celcius figure with the variable name we have configured in CSV Data Set Config element (i.e:- ${celcius})
After parameterizing, your SOAP request will be as follows.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tem="http://webservices.daehosting.com/temperature">

Increase the thread count corresponding to the row count in your csv file and run the test. You will notice that the Celcius figure will be varied in each request by reading data from CSV data source.

In this way, you can easily do data driven web service testing using Jmeter.

Saturday, January 29, 2011

Test Reviews - An efficient way to validate test coverage

When the complexity of testing increases, deriving test scenarios becomes extremely hard. Individual testers are not always capable of thinking all aspects of the products and finding out test cases. In agile processes which demands frequent releases, this becomes much harder since individual testers do not get sufficient time to learn and explore products.
At WSO2, code reviews have been an integral part of engineering process. Code review groups get together and go through different code segments in weekly basis.
Similar to code reviews, WSO2 QA team conducts weekly test reviews. The primary objective of reviewing tests is to finding out gaps in a particular product's testing methodology.
Usually there are 2 main actors take part in a test review meeting.

- Tester(s) who test a particular feature
- Developer (s) who implemented the feature

Tester goes through the existing test scenarios and developer provides his or her feedback on the tests. Specially when a particular use case is not clear enough, tester can clarify it during reviews very effective manner.
It is the testers responsibility to try out the new test scenarios which have been captured during the review in next release cycles.
Test review will not take more than 1 hour and testers should be prepared properly to get the maximum out of this short time and capture more and more test scenarios.

Similar to the manual test scenarios, all automated tests can also be reviewed during test reviews.
In agile processes where you do not have detailed system specs to derive tests, test reviews are one of the most useful mechanisms to validate your test coverage.

Tuesday, January 25, 2011

How to enable child-first class loading in WSO2 Application Server or Axis2

By default, WSO2 Application Server (or Axis2) uses parent-first class loading mechanism.
If you deploy an AAR service, which can load classes from the following locations.

  • CARBON_HOME/lib (CARBON_HOME is the location where you installed WSO2 Application Server)
  • CARBON_HOME/repository/deployment/server/axis2services/lib
  • AAR service/lib (lib directory under your service archive)

If your service implementation class has a package import for a class, which is available in both CARBON_HOME/repository/deployment/server/axis2services/lib and the lib directory under service archive, the class placed under CARBON_HOME/repository/deployment/server/axis2services/lib will get loaded.
Which means, the parent class always get loaded first.
The class loading sequence is CARBON_HOME/lib ---> CARBON_HOME/repository/deployment/server/axis2services/lib -----> AAR service/lib

Sometimes, we want to load the class from service archive lib first without loading the class which is available either one of above parent locations. In other words, child-first class loading will be required for some instances.
child-first class loading can be enabled simply by adding the following parameter in to services.xml in your service archive.

<parameter name="EnableChildFirstClassLoading">true</parameter>

If you set this parameter in axis2.xml of your WSO2 Application Server (or Axis2) instance, all services will use child-first class loading mechanism.

Wednesday, January 19, 2011

WS-Addressing with SOAPUI

WS-Addressing is a standard way of including message routing data within SOAP headers without relying on transport specific routing properties. If you are talking to a web service which expects WS-Addressing information in request SOAP messages, you could either follow a programmatic approach to write a client and insert WS-A headers manually or use a commercial or free web service client tool. SOAPUI is the simplest tool which can be used to insert WS-Addressing headers in to request SOAP messages.
In this blog post, we will;
  • Talk to a WS-Addressing enabled service which is hosted in WSO2 Application Server using SOAPUI
  • Assert response SOAP message to validate WS-A headers

1. Download and install the free version of SOAPUI 3.0.1 or later
2. Download and install WSO2 Application Server 4.0

Step 1

We are going to invoke the default HelloService which is shipped with WSO2 Application Server. Go to WSO2_APPSERVER/home/bin and start the server by running wso2server.sh{bat}
Access management console with https://localhost:9443/carbon and log in using the default administrator credentials (username=admin, password=admin)
Navigate to Manage --> Services --> List in the left menu and select "HelloService"
You will be directed to the dashboard of HelloService. Click on Modules link.

You will notice that "addressing-3.1.0" is listed under "Globel Level". By default WS-Addressing is enabled for all the services hosted in WSO2 Application Server. Therefore, you do not have to configure anything at the server side to enable WS-Addressing.

Step 2

Now, we will create a new SOAPUI project referring to http://localhost:9763/services/HelloService?wsdl as follows. Make sure to select "Create Test Suite" check box as well.

After creating the project, select greet request under HelloServiceSoap11Binding TestSuite

If you click on run just by providing input string, you will notice that WS-A headers will not be included in the request SOAP message which can be observed if you toggle "Raw" option.

Step 3

Lets add WS-A headers in to the request SOAP message. Select greet request and click on WS-A option and select the following properties.

Enable WS-A addressing

Add default wsa:Action

Add default wsa:To

Randomly generated message ID

Keep the rest of the default values and click on run. You will notice the following WS-A headers in the request message if you look at the raw message in SOAPUI.

<soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">

Also, the response headers will be similar to the following.

<soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">

Simple.. isn't it? we just invoked a service with WS-Addressing headers using SOAPUI.

Here, we manually verified that the response message includes the correct WS-A headers. We looked at the response message and verified each header. If you maintain an automated test suite to verify your web service interactions, the above procedure cannot not be considered as over until you add an assertion so that the test automatically reports the correctness or failures of response/request messages.

Step 4

Now, we will add an assertion to validate response headers. In this example, we are going to add an XPath match assertion to check whether the wsa:Action header of the response equals to urn:greetResponse

Click on Assertions option and select "Add an assertion to this item" icon. Then select, XPAth Match as the assertion.

Next, you should specify the XPath expression and expected result as follows. First, click on Declare button which will automatically define the namespaces. Specify //wsa:Action as the xpath. Also, specify urn:greetResponse as the expected result.

Run the test again. You will notice that the test will be marked in green denoting that the test is successful.

Saturday, January 15, 2011

We are honored - Hats off to WSO2 QA team!!

When I joined WSO2 back in 2006 October, I always had a doubt whether I could survive in a world full of various technologies. Before joining WSO2, I used to work under well defined processes, managed by various policies, tests were driven by comprehensive requirements specs, Spent time with the QA teams writing detailed test cases etc.. But I was challenged to test Apache Axis2 at WSO2 as my first duty without having any of those materials. Apache Axis2 is a web service engine, middleware used by developers. So, ideally, I had to look at it through middleware user's in other words web service developer's perspective. It was a great challenge for a person who was new to web services and it was indeed a different experience for a QA engineer.
At the same time, Evanthika, who joined WSO2 one month before me, and myself started testing the first WSO2 middleware product, WSO2 WSAS. First, we started to write comprehensive test cases as we do in traditional QA processes. Later on, we understood that our methodology did not go along with WSO2's vision. We thought to adopt to a completely different agile testing mechanism and started to act fast without being an overhead for a fast-moving organization. We learned everyday, talked to developers and clarified doubts, lived with the product and uncovered a lot of critical bugs. We tried every attempt to report the real bugs and improve quality of the products with each releases.
In early 2007, Evanthika and myself were allocated to test Apache Synapse and WSO2 ESB which demanded us to explore different methods for testing because it was a completely different experience for us. Eventually, we defined our own testing methodology which nicely moved along with WSO2's release processes.

After 4 years, our team grew up and became highly effective with a lot of hard work done by our great team, Krishantha, Yumani, Chamara, Ishani, Pavithra, Nirodha, Thilini, Chamara Anuradha. Now I'm confident that we are in a position to accept testing of any product with very short ramp-up period. Our lightweight process helps everyone to focus on uncovering bugs fast as well as move forward with various different technologies used by WSO2. It is not an easy task to test complex middleware products as I explained in a previous blog post. We have built the foundation for the new testers to join our team and move along with the world of complex middleware testing.

With all these efforts during past few years, we have been awarded as the outstanding team in 2010 at the WSO2 awards ceremony which was held yesterday at Cinnamon lakeside, Colombo. As Paul Fremantle mentioned during the ceremony, QA teams are kept aside and do not appreciate their work much in most organizations. But WSO2 believes the values of each team and always respect the hard work performed by our QA team.

I agree that we are not yet perfect and there are situations some bugs slipped through our test cycles. But we always try our best to maintain an acceptable quality with all releases. It is evident by the usage of our products in large organizations such as eBay, Deutche Bank etc..