Popular Posts

Sunday, December 21, 2014

Myth of 100% test automation

Can ALL tests be automated? Needless to say that putting effort on automating "some % of" scenarios is not worthwhile. I have identified some obvious examples;


  • Scenarios which involve time consuming work-flow executions
  • Unstable features
  • Some end-to-end scenarios where assessing multiple outputs by humans are must
  • Usability assessments
  • Tricky timing related/inconsistent bugs


Why do some engineering teams praise on 100% automated coverage even knowing these well-known facts?
My understanding is, they do not know what QA is! When quality is just an afterthought and presence of critical bugs in products is not an important matter, praising 100% automation and living in that dream is the best way forward just for the sake of temporary satisfaction. First, understanding the difference between Testing and Checking is very important since that can prevent setting the unrealistic goals and expecting something not possible from test automation.
Michael Bolton once defined these two terms concisely in here and then refined the same by James Bach in here. I would recommend these two articles for anyone who needs more understanding about what professional testing is.

--Open for further discussion

No comments: