Sunday, July 06, 2008

Best time to do performance testing in an Agile environment

In the traditional model of software development major release/milestone dates are fixed, and performance testing is done just before such milestones, and it occurs mostly after the completion of coding, integration and system testing for a chunk of requirements. In an Agile environment too, release and milestones dates are planned, however, between the start date and the major release date, multiple iterations keeps happening and developers keep churning the code.

So, the next logical question to ask is,

In an Agile environment, is it still advisable to do
performance/scalability kind of testing before the major release date Or is there a better way to handle this ?

My personal view is, the non-functional requirements(the -ilities, scalability, availability, performance, etc) cannot be separated from the implementation of functional requirements. It is not a good idea to say that we would implement the functional requirements first and then look at non-functional requirements. According to me, one cannot write a piece of code implementing functional requirement without affecting the non-functional requirements. Even the simplest If-Else statements, for() loops we write affect the -ilities in a big way. It is always advisable to have the non-functional requirement related SLAs in mind all the time
and, after writing each piece of code, ask yourself does this line of code
meet the SLAs ?

Moral of the story is, in an Agile environment, performance, scalability, Availability testing is done all the time right from Iteration 1.

Best Practice: It is better to write unit tests to check the conformance of your code against the SLAs, and automate them. Creation of such unit tests and, automating them is the key to success in implementing non-functional requirements in an Agile environment.


Unknown said...

Bravo! Some days I feel like I'm the only one preaching things like Performance Unit Tests and the fact that Performance Testing is, by it's very nature, agile.


Scott Barber
President & Chief Technologist, PerfTestPlus, Inc.
Executive Director, Association for Software Testing
Co-Author, Performance Testing Guidance for Web Applications

"If you can see it in your mind...
you will find it in your life."

Brad Johnson said...

I wholeheartedly agree that early/often needs to apply to performance as in all other test areas. A primary issue of iterative performance testing, performance regression testing or any other early/ofter perf testing is that test environments and staff usually don't support the practice. What's needed to change this?

Anonymous said...

Great advice and we adopt a similar stance. However, in practice it is not so easy to write automated tests that verify some of your SLA's. For example, unless you have a testing environment that mirrors production than you can't test against your performance, through put or availability SLA's. You can try to do some "rough" extrapolation but it's not as comforting as testing against functional requirements. Some things are just hard to automate...