I have been testing SOA middleware applications for nearly 3 years. I also tested traditional CRM, Telco billing applications for more than 4 years. Quality assurance of middle ware is completely different from the traditional business application testing. It requires a lot of effort. I have observed following facts/concerns which related to middleware testing.
1. Target audience/end users of middleware apps
Most middleware products are developed for the use of highly technical tasks and they are used by programmers. When it comes to SOA middleware, product end-users are technically savy engineers. Therefore when testing them, QA/test teams should understand the purpose of it, target audience and act accordingly.
2. Complexity of applications
The developers understand the use cases of middleware applications when implementing them. Obviously without knowing the requirements, they can't be implemented. Therefore the developers of middleware applications gets the knowledge of the application under development while it is being developed regardless of the complexity of it.
This is not applicable for QA/testers. In agile like processes, due to the release crunch, QA gets no time to learn the use of middleware. They do not even get time to derive test cases/scenarios. So, QA teams are forced to test applications with these limitations.
3. Test Automation
Majority of the middle ware application tests can be automated. However, it cannot be done by overnight. Without having a better understanding of the applications, no body can automate them. Automation is highly effective for middleware testing.
Having said that, I must emphasize that, automation is not the solution for everything. Specially in middleware testing, human intervention is critical. If you are a middleware tester, you must live with the application and its surrounding technologies. You must learn the use cases of middle ware app and apply them to derive more and more test scenarios. In my career, 80% of highly severe bugs were uncovered by exploratory testing.
It takes times to learn the technologies behind middleware apps. It takes time to understand the use of them. A content management system or a telco billing application can be learned and understand with considerably less time and effort. However, middleware applications are different and deriving use cases are not simple.
If you have any doubts about the complexity of middleware testing, the best way to clarify them is to involve in middleware testing for few days. Work with middleware test teams. Then you can understand the real situation and issues behind it.