”Integration is a ‘pay now or pay more later’ kind of activity. That is, if you integrate throughout the project in small amounts you will not find yourself trying to integrate the system for weeks at the project’s end while the deadline slips by” (Extreme Programming, Don Wells)
The Integration Hell
An automation project takes a lot of integration. By Nature, the automation process integrates different platforms and MUST run perfectlyin an unattended manner where no human logic can be applied.
Running the test automation scenario on your desktop set up is not enough. Apart from developing a quality code, system implications must be considered. In simple words, you want the whole thing to run smoothly, in an unattended manner, with 100% reliability on the main set up.
The main set up (A.K.A “the BIG set up” or “the Regression set up”) is the closest to real life simulation the SUT (System Under Test) gets. The main set up is where all the devices and platforms are working together. It may combine different Operation systems, Testing Equipment, Simulators, and of course the SUT/s.
Leaving integration only to the final project stages of our test automation project, might lead us to what Xtreme programmers call “The Integration Hell” where everything brakes and no one understands why…
So, what can be done to avoid this integration hell?
When developing code integrate it frequently. This can be done by releasing the code (Check IN) into the code repository whenever possible. If everyone will work with the latest version, the required code integration will be minimal.
Re-Run your test until it runs smoothly
You should frequently run the automatic test on your dedicated development setup, until it runs smoothly. Don’t leave it with errors or warnings.
Assemble tests into scenarios
After you’ve completed a single test, integrate it with other tests. That is, add it to its relevant scenario. This way you’ll be able to inspect Inter-test relationship and minimize the surprises.
Assemble multiple scenarios
When you have more than a single scenario (which includes 1 or more tests) combine it with another scenario. That will give you the opportunity to check their inter-scenario relationship.
Start Overnight Looping ASAP
You should start “looping” overnight scenarios ASAP. Start even with a single test, than add the second one and so on… That will save you a lot of time when you’ll reach the final project stage. It can also produce usable results from early project stages.
Dedicate integration time to each phase
While planning ahead, always keep integration in mind. Each and every phase should have a dedicated integration time slot.
Remember the complexity of the final Integration
Toward the end of the project there is the final integration stage.It is a special stage, all systems must work together, and anything less than perfect is not acceptable. Things that we have over looked or postponed must have an answer now. Everything, even the smallest details that we’ve dismissed few weeks ago, must be resolved. All scenarios must run smoothly on the main set up, unpredictable things will happen and “Murphy” will come and visit. Keep that in mind and plan your Gantt so you’ll have enough time and resources for this stage.
Burying in mind that test automation projects are very sensitive to the quality of Integration, and following the above ‘golden rules’ of continuous Integration will minimize the risk of not meeting your project deadline, it will also help you avoid the end-of- project integration hell where everything might brake.