Low Cost Test Automation with Open Source

I want to start with two quotes from Gartner:

“Companies are on survival path”.

“Open source will quietly take over”.

Well besides the fact that Gartner says is already something to mention, I see a vivid connection between these two statements. Basically one is a partial solution for the other. We all see the cut budgets for development and testing yet we are required to deliver quality products and make sure that we meet competition. During these times, when every penny counts, we seek for cheaper solutions that will provide the same results. Well seek no more; open source solutions and specially in the test automation area provide exactly what we were looking for.

Before we dig into the automation using open source tools, I think it’s only fair to give some background on how I see open source tools. So what is an open source tool? What are the advantages and what are the disadvantages?

“Open source is a development method for software that harnesses the power of distributed peer review and transparency of process. The promise of open source is better quality, higher reliability, more flexibility, lower cost, and an end to predatory vendor lock-in.”

The above definition kind of speaks for itself so I will try to list the advantages of open source tools as I see them:

Price – you don’t need to pay money for the actual open source tool. To be honest in some cases you will require professional help from someone that is familiar with the product or maybe even developed it, but the actual tool is free of charge.

Quality – surprisingly (to some), open source projects maintain very high level of testing, quality of the releases and of course a  large number of dedicated tester that report anything that they don’t like. Usually these projects are being developed by a devoted team of individuals that participate purely due to their passion for the project. Not everyone can start contributing to the project and standards are very high. Don’t forget that open source means true transparency of the process, what you do is being evaluated by many others and this is the best motivator for delivering of high quality results.

Up to Date – open source projects greatly influenced from what people need. Basically once a project is released it’s being evaluated for usability, relevancy and quality all the time. If there are features that are missing or not usable, be sure that someone there will point it out. These inputs usually are taken into consideration by project developers not mentioning the fact that if something bothers you, one of the option is just adding it to the project.

Community – open source projects are all about the people involved in the project. You don’t have to be the developer to be involved. People can contribute by testing, documenting, helping in the technical forums, etc. Having a large community just reduces the risk of using the project – you can always find someone that already solved your problem.

So let’s say that you have decided to use an open source automation tool, but you don’t know how to decide upon which tool to use. Here are some tips that might help you.

Professional services – make sure that there is someone that provides professional services for this product. It’s true that in most cases you will find help in the community but when you are talking about tight deadlines, specific customization for you product you might need experts.

Companies behind the project – in many cases the initiator or the supporter of an open source project is a big commercial company. This sometimes means that the product is mature and stable or at least had a good basis for future development.

Community – I have talked already about this before. Just make sure that the community is active and large enough to get answers when you will need them.

Roadmap – it’s important to see that the tool that you are choosing going to directions that you might benefit from. If there is no roadmap whatsoever it’s not a good indication.

Needless to say that considerations that are valid for test automation open source tool are also true for any other open source tool that you are considering using.

In case you were wondering which test automation open source tools are available, here are a couple of names that you should know:

xUnit – one of the most popular tools for unit testing. Such frameworks are based on a design by Kent Beck and are part of the extreme programming methodology. We can find implementation of this approach for almost every language: JUnit (http://www.junit.org), NUnit (http://www.nunit.org), etc.

JMeter – long running project that allows testers and developers creation of load tests. This tool support variety of protocol and is easy to use (http://jakarta.apache.org/jmeter).

Selenium – one of the most popular tools for Web testing. This tool supports most of the currently used browsers, runs on different platforms and allows integration with almost any framework that you have (http://seleniumhq.org).

JSystem – this is a complete testing environment that allows use development, execution and management of tests. The framework is Java based and contains many advanced feature and integration with tools like Selenium, Soap, SNMP and many others (http://www.jsystemtest.org).

SoapUI – this is a very common tool that allows invoking, inspecting and developing web services. This tool can be easily integrated into load and functional testing framework (http://www.soapui.org).

I hope that after this short review you will be more open to adopting open source test automation tools in your organizations.