Inventing your own Light weight methodology - A Case study

In the past I had implemented a project by inventing my own light weight methodology and I am sharing my experience in this post.

Even though the inspiration came Agile values and principles, but didn't follow any specific methods like XP or Scrum as a whole.


Context:
1. "<5 " member team
2. Collocated team (dev team in India and technical SME in US)
3. Web application development
4. Team composed of members who are fresh out of college and in equal proportion experienced members
5. Team was well aware of the technology
6. The team members were passionate about learning the new technology and hard workers too.


I was an architect and also played a role somewhat like a Scrum Master.

1. Every day in the morning when the team had arrived, we used to sit in a circle and ask individuals plan for the day and if they need any help in resolving any issues. This could be compared to a scrum meeting except that we didn't ask the 3 questions nor we stood in a circle.

My observation is, it is not that the ceremony which matters but the values behind the ceremony that makes the difference.

2. When each team member used to speak out about his plan, I used to make a list of action items on a white sheet of paper. Let us call it as Action List.

If your team sits closely in cubicles and does not have access to huge team rooms, white boards or post-its, then the above method of writing on a white sheet of paper could help.

3. Many a times the onsite team would have done some review of the application and would have suggested changes through email. I would add them to the Action list

4. Once every one has completed sharing their action items and impediment list, they used to take a quick 10-15 minutes break and return back to work.

5. The action List was kept at a visible location so that everyone can see.

6. The team members used to glance through the Action list, depending on the priority (discussed verbally) they pick up the task and start working on the same.

7. After completing each task, they would come back to the Action list sheet, take a look at it, pick up one of the pending tasks voluntarily. As soon as they complete any task, they used to put a check mark on the same.

8. There was a progress review in the afternoon and at the end of the day another review on where we stand. It was more like a retrospective but without much rituals like +/Delta analysis or so.

9. At the end of the day before the team used to leave work, we used to quickly talk about the tomorrow's plan.

10. Informally the team used to discuss about the design, coding and other issues during the cafe breaks too.

11. As an architect I used to sit and code, and at the same time if any of the team member's are stuck then I used to jump in to help them out. If I can't help them, then I used to reach out to other experts through my contacts and get them the necessary help.

12. Estimation is done by the team. Since the team had experience with the domain and technology, based on past experiences, they used to come out with the estimates.

13. Couple of other good engineering practices include,

• Daily code reviews
• peer reviews
• Continuous integration
• Daily builds
• Regression testing using Selenium
• Unit testing using JUnits and EasyMock
  

14. Interestingly we didn't have a separate testing team, it was the developers responsibility to test it thoroughly and release it to the customer for UAT.

Above method that I had applied is a very simple one and does not have too many ceremonies attached to them.

This project was successful, team and the customer was happy.

The Semco Way

I had read Ricardo Semler's book, Maverick a few years back. Today I came across Semco's survival Manual on their site and is very inspiring.

As you read their list of values and practices, you could imagine the self organization and self management going on in Semco.

I highly recommend any one to read Maverick which inspires you to think differently.

Need for the Right Person to Drive Agile Adoption

It is very important to identify the right person to drive Agile adoption in the organization rather than randomly choosing some senior person to drive it.




The reason being, Agile is all about

• People
  
• Being Flexible
• Being Patient
• Open for new ideas
• Learning
  

Recently in one of the stories I heard, the CEO of a company who had been hearing a lot about Agile to adopt Agile in his organization. So, he decided to form an Agile Research Group to drive Agile adoption in the organization. He choose one of the senior most Vice Presidents(VP) to drive this, who in turn choose some senior and mid-level managers to start gathering info.

Just like any other company, the Agile core team started meeting weekly, they started the Agile discussion forum and started gathering as much information as possible.

After a month of starting this Agile group, the core team submitted their findings to this Vice president (VP). After few days the VP called the core team for a meeting and the team didn't believe what they heard from the VP.

Cutting a long story short, the VP didn't believe that there exists concepts like Iteration, Daily Stand up meetings, self organizing team, Iterative and Incremental development approaches. He didn't believe the reports submitted by the core team. In this case, the VP himself became hindrance to the Agile adoption.

This may not be the case in all the organizations, and it could be one "off" the case, however it is very important to choose the right person, especially when it comes to Agile adoption.

In another case study, the CEO of the company appointed a senior person from Quality group to drive Agile adoption. This Quality group head was coming from a strong CMM and Waterfall background. The team assigned to try Agile on a sample project used to read books, attend trainings and try the practices on their projects.
Many a times this Agile group failed in implementing the practices, but each time they failed the senior person driving Agile, used to tell them I told you Agile won't work kind of statements.
Such statements discourage the team. One has to remember that, It is normal that the team might fail while implementing new practices, and in such situations Patience is the key.

I believe that the following key attributes are needed in the person driving Agile Adoption

1. Good understanding and knowledge of Agile
2. Be open to new ideas
3. If the team fails in between, analyze the situation and encourage them rather than pushing them down
4. Trust the people around
5. Patience
6. Persistence

Remember that the wrong person in the right place not only could become bottleneck for the Agile adoption but also could lead to failure.

You could read this article also on Project Perfect blog

Have you come across such stories, you can share them here....

Interviewing candidates for Agile Projects

If you are running an Agile project and looking at hiring a suitable candidate, here is an excellent article which describes the step by step process of extreme interviewing.

Introduction To Lean

Recently I was watching the web cast on Lean Concepts for IT Professionals by Durnall and Parkinson. They have provided a very good introduction about lean covering its history and principles. It is an hour video and worth watching. While watching the video, I made some notes and here are some of them :

1. Mass manufacturing addresses the challenge of volume, where as the Lean addresses the challenge of variation

2. When volume doubles, economies of scale reduce cost by 15 - 20% per unit but costs go up 15% to 35% every time variety doubles.

3. Jidoka is all about automation. Humans are flexible but not good at repetitive tasks, however Machines are good at repetitive tasks but are not flexible. Jidoka talks about using both Humans and Machines to create a good system


4. In Ford the machines in the assembly lines are stopped around 8 – 10 times a day but in Toyota, they stop the machines nearly 27000 times a day. This is mainly because, each time any employee of the company finds a fault with the product, they stop and do root cause analysis before allowing the machines to run. The quality gets built in by the time the product is rolled out.

5. There are 8 different kinds of wastes that can happen in a system

• Overproduction
• Waiting
• Unnecessary Transportation
• Over-processing
• Excess Inventory
• Unnecessary Movement
• Defects
• Unused employee creativity

6. Toyota CFO expects the entire financial year report on an A3 size format, so that only needed information has been added rather than waste

7. There are two kinds of Kaizen, Point and Flow

8. There are two sets of people, the bike thinker and the frog thinker. Basically, a bike can be disassembled easily, individual components can be optimized and put back. However a frog, once disassembled cannot be optimized in isolation without affecting the other parts. Software development needs Frog thinking.

Tips for effective offshore software development

A decade of experience while working with distributed development environment has taught me some lessons and techniques to effectively work in this mode. In the following paragraphs, I have shared my observations and also, way to tackle common issues in distributed environment.

Image Source: http://www.benscheele.com/ray_imgs/abstract/abstract1.jpg

It is very important for onsite teams(read it as non-Indian outsourcing partner) to understand the culture of offshore teams (read it mostly as Indian teams). Specifically the cultures are extremely different in countries like India, china, Brazil, Manila as compared to the cultures in US, UK, or any other European countries.

Many a times the business people from onsite companies visit offshore, finalize the software vendor, sign the contract and return back. By this time, they would have already signalled setting up onsite and offshore software development teams. The onsite/offshore development teams start working on the software project even before they can properly spell each others names .

I have also observed the following differences in the way of working between onsite and offshore teams

Saying "No" : Offshore teams don't say "no" easily. They won't come up with issues easily as they don't want to hurt the other person. This is more of a cultural issues I think. I have come across many instances where offshore developers were fine getting the blame for not doing something even though the onsite teams have not provided them the sufficient information to proceed. Many a times in small and medium sized firms vendors give more than necessary due importance to the customers to keep them happy and are never blamed. This in turn adds significant fuel to fire in ensuring that the offshore developers keep quiet and nod their heads.

It is necessary for offshore teams to stand up and say "no" when necessary and at the same time , the onsite team leads should keep a tab on "bullying" developers.

Networking & Socializing : Many offshore developers when they come to work tend to socialize and would like to get the work done more informally rather than through formal means. Interestingly western culture is different. Since I used to live abroad, I have observed that the western developers don't want to be disturbed and clearly separate the personal from professional lives. They are very strict about the timings and whom they speak with during work. One of my relatives have been living and working for a fortune company since last 10 years in silicon valley. He says he has never spoken to the coworker who sits next to his cubicle, as the other person too does the same. Both of them come to work at 8 in the morning, eat meals while working, drink coffee while working and go home early in the evening. However in countries like India, having lunch together, cafe together is socially acceptable way of living .

I have observed that many onsite teams who are new to Indian culture don't consider this socializing aspect as cool and they get furious about "wasting productive hours".

It is extremely important to exchange key team members of both the teams for a few weeks so that they understand each others team. Knowing each others name is not sufficient.

Technology and Communication: Somehow western programmers come with inherent ability to communicate effectively as compared to the offshore programmers. So, offshore vendors always spend extra effort to correct this weakness by pushing the senior and more experienced programmers to the front line to manage projects. So, it is difficult to find "programmers" with 10-15 years of experience in offshore teams, however you can easily find "managers". So, don't expect people at 10+ years sit and code as compared to the onsite counterparts. The subject of whether experienced offshore developers should code or not itself is a debatable subject, but let me keep it for another day :-)

Completed 2 Speaking Events

Recently completed 2 speaking events, one in Bangalore and the other in Hyderabad.

The event was sponsored by ITGrids in alliance with Agile Alliance and was attended by nearly 50 people from various software companies in Bangalore and Hyderabad.

Agile adoption in offshore companies

I have been observing an Agile adoption trend/pattern in offshore companies (read it as India). The offshore companies are mostly into softservices with a small chunk into product development.

Offshore Software Services companies seem to be adopting Agile methods mainly due to the request/push coming from onshore customers. Many of these services companies still have projects running applying waterfall model too. My experiences while working with many of these services companies has made me to realize that, they want to retain both Waterfall/CMMI, ISO kind of certifications along with Agile. These companies are neither oriented towards Agile nor Waterfall/CMMI, and they are only customer and Profit centric.

P.S: In the above paragraph, I have used Waterfall and CMMI together. Even though I am not an expert on CMMI but I have observed that the CMMI recommendations are mostly applied with waterfall process by many companies.

Typically this is what happens when a new customer walks into one of the offshore services company. The offshore company show them the menu with various process options. Based on customers choice, the process would be customized and applied on that particular project.

Above observation is leading me to come up with some new questions and I have also attempted answering them...

Questions include

1. Can CMMI and Agile coexist ?
2. Can Agile methods sustain in such environments ?

Answering Q. 1 above, I do believe that CMMI and Agile can coexist. I have observed many companies bringing these two giants together.

Answering Question 2 above needs bit more explanation. I feel that, Agile methods cannot sustain in such environments. Reason being, the CMMI concepts coupled with Waterfall model needs a mindset which is totally opposite to the one needed by Agilists.

Some of the conflicting practices/mindsets include

1. Self organizing/Self Managing teams in Agile as opposed to "one man show" in Waterfall
2. Iterative Vs BigBang approach
3. Daily estimation Vs One time estimation
4. Customer collaboration Vs "Fire and Forget" customers policy
5. Regular Iteration Retrospectives Vs Postmortem sessions after the project

If the CMMI/Waterfall company wants to set up an Agile practice group, it is recommend to create two different groups to manage these two different processes rather than allowing only one person to manage both of them. I have interacted with many Quality/CEPG/CMMI Auditors, and they are so obsessed with waterfall concepts that they don't believe in Agile. If the stakeholders of the companies make such people as the sole group heads to run both Agile and CMMI processes, then it is pretty clear that, Agile would get slow death.

Another draw back I have observed in "Client driven Agile" projects include, that the offshore companies will never be able to appreciate the real value of Agile. Since Clients would be driving the Agile practices, the offshore teams just follow them blindly without knowing the Whys behind them. This leads to misconception and creation of Agile myths.
Successful implementation of Agile methods in offshore projects depends on couple of key points.

1. It needs a good support from both onsite and offshore team
2. The onshore team also has to ensure that the offshore team and the stakeholders running the company are well aware of Agile in addition to understanding the values and benefits of Agile methods.

Even though Agile is picking very fast in offshore projects, benefits of the offshore companies taking lead in propagating Agile is more than the onshore teams driving it !

Impact of "Individual Performance rating" on projects

Many thought leaders have recommended organizations to follow "team performance ratings" rather than "Individual performance" ratings. Many books and blogs have been written about the topic. Even though I have read the cons of individual performance ratings in the past, recent story that I heard from a friend reinforced my belief about the negative impact of "individual performance ratings" on morale of the employees.





Here is the story, John (Say, not real name), is working for a mid sized company and the company is following waterfall model of software development. John is working on a project which has tremendous pressure and tight deadlines. His project has fixed schedule and fixed budget(cannot add more people to the project too), no room for any flexibility in the project plan. The release date of the project is carved on the stone already. Just imagine what would be the fate of the developers working on this project !!

So, John went ahead and escalated this issue to his superior about the team loosing morale, loosing patience and getting stressed out on this project. Superior refused to provide any support because this project is a high profile project and if the superior tries to bail the team out or try to do anything which affects the project deadline, the Superior's "promotion" which is due this year would get affected !!

After listening to the above story, I felt that, as long as an organization has "individual performance ratings" in place, people tend to keep their individual priorities/preferences above "the team's" or "organization's" goal. In order to get good rating, Every one in the team works hard to achieve their personal goals and in the process forgetting the organization/team goals.

Taking the above story a bit further, what might happen to the project now ? May be the team would put all efforts to meet the deadline, might even don't express their concerns around stress with any one (avoiding getting beaten up during the performance appraisal). However this stress is definitely going to affect the quality of the code that the team is developing. isn't it ?

Do you have any such stories ? feel free to share them here.

Identifying a Scrum Master for a new Scrum Project

Recently my article on "Identifying a Scrum Master for a new Scrum Project" got published on Project Perfect. I have pasted the article below with a bit modification.

We know that the Scrum Masters should posses the following skills to be effective in their roles:

• Be a sheep dog, ensuring that the team follows the Scrum Practices properly
• Be a servant leader
• Impediment remover
• Be like a parent - Take charge if the kids are going in the wrong direction.
• Avoid command and control mentality.
• Good facilitation skills
• Communication and Negotiation skills: Even though these two skills are mandatory for any team member, I thought of explicitly bringing this out here.

I feel that above "skills" cannot be acquired through trainings or bootcamps and not even by reading books. However one could acquire the above "knowledge" through various sources like books, trainings and attending seminars. In order to really implement them and make it a part and parcel of life, would require dedicated practice over a period of time.

If we come across a person with the mastery in above skills, then he/she would have had:

• A lot of experience in dealing with various situations
• Experience working with in a multi cultured environment
• Experience in making tough decisions under various circumstances and specifically the ones related to management

Even though imagining a person with above skills might create a picture of a serious person, I don’t mind adding another point to the above skill list as “The person should be Light Hearted”.

I have also observed that the most of the above skills are found in senior(experienced) people in the project as they would have gained experience by implementing projects in various capacities.

Now coming back to the actual point, if your project is transitioning from waterfall to Agile (Scrum), can you pick any team members from the existing waterfall team to be a Scrum Master? It is a tough decision, isn't ? . What criteria would you to pick that special person for the team ?

Even though there is no rule that only "senior" person should be picked as Scrum Master, I have observed in most of the projects that the senior most person in the team typically possess “most” of the above skills as compared to the rest in the team, and that senior most person typically is the "Project Manager".

Many “waterfall to Scrum” projects typically would pick the “Project Manager” to be a “Scrum Master” but where they fail is in identifying the missing gaps in the skills of the Project Manager that stops him becoming a true Scrum Master. The “Project Manager-Scrum Masters” continues with their old PM skills and never become a true Scrum Master and thus the project never achieves the desired results of a true scrum project.

Since, the Scrum Master need not posses deep software and technical skills, I don’t mind recommending someone from a non-software background for this job. I have seen that many great leaders come from a non-software background.

Metrics in Agile development

Metrics seems to be the part and parcel of any software development project. In typical waterfall development projects, different types of pre-defined metrics would be used across various projects even though each project is different. However when a team migrates to Agile methodology it becomes bit complicated. They get confused about the metrics to choose. This is because as such Agile methods are not prescriptive and freedom is given to the team to invent the ones which adds value to project. This instant freedom makes the team to come up with more questions than answers.

If you are new to Agile and would like to know the metrics to use in your projects, here are some tips

1. Don't try to invent new metrics as soon as you move to Agile, instead use some of the existing CMM and other matrices and use it within an iteration.

For ex: In typical non-Agile companies the metrics commonly used include,

• Total number of defects,
  
• new change requests,
  
• resolved defects,
  
• total major,minor defects, etc.

Convert them into Agile by using them in iterations. For ex:

• Total number of new defects found in each iteration
• New features added to backlog in each iteration
• New tasks discovered in each iteration
• Number of resolved defects in iteration, etc

2. Above discussed point could be used as a starting point, however going forward it is critical to identify some of the pain points in the project and create metrics around them. Pain points can be identified by looking at what the customers/stakeholders are unhappy about most of the time.

3. It is also important to display the up to date metrics in a visible area in an Agile development environment.

4. There should be some action associated with the numbers captured as part of metrics,whether it is good or bad. If the numbers are as per the target/plan, then one should create actions to increase the bar. If the numbers are not good, then one should take actions to reach the bar.

5. Metrics can drive the behavior of the team. It is very important to choose the right metrics.
Recently I heard from some one that a management of a start up company imposed a metric to measure the productivity of individuals based on the number of times a developer checks the code into the repository. This metric obviously resulted the behavioral change in developers, as they were more worried about how frequently they check the code in to meet the target rather than any other deliverable.


Here are couple of good websites with information on Metrics in Agile environment

• Appropriate Agile Measurement
• Principles of Agile Metrics
• AgileMetrics.com
  

Article published on Project Perfect

Project perfect team recently published my article on pros and cons of open office space on their blog. You can find the article here

Open plan offices Vs Cubicle culture

Recently I witnessed a good deal of discussion about the "best" way to arrange seats in software companies to improve productivity, reduce cost, etc.

Even though many Agilists can give a blank cheque in support of "open plan offices", we should be careful about implementing them without careful consideration. I have worked both in open plan and in cubicle offices. Each style has its own pros and cons.   

My personal opinion is, seat arrangements should be "context dependent".  One of the contexts I suggest to look at is the "type" of work. If the work involves more communication and interaction, then it is better to arrange that particular team in open space. However if the work involves solo, less communication/interaction work, then cubicle or closed room kind of design is recommended.

The office space could have separate designs within the same company while accommodating people with various work types. The R&D team ,specifically the researchers could be made to sit in cubicles to concentrate on individual researches. However the development team which need to communicate more often can be made to sit in an open space environment. 

If the development team is made to sit in open space plan, then I strongly suggest applying "Caves and Commons" type.  My personal experience is that, in the absence of "commons" people tend to receive and make phone calls all over the place distracting the people around them. 

Even though I have been discussing about the infrastructure setup for Software development teams, it aids my belief that Agile methods should not be implemented in isolation trying to think only from the software process point of view, however the process should be applied by including various parties in a software company, like infrastructure, HR, finance, etc.  The process should be applied end to end.  

Local sub optimization won' t improve productivity !!

Another dimension to this discussion is, many a times we discuss about  improving the "productivity" of the teams through various soft skills training, usage of expensive collaboration tools. However  many companies over look the office space design. If the office space is dimly lit, crowded with ventilation issues, then no matter of what tools, techniques and process we use, it won't make much difference.   

Kanban tidbits

Here are some tidbits

• "Kanban", the Japanese word translates to "signboard"
• Taiichi Ohno developed Kanban
• Toyota management were looking at ways to reduce/eliminate waste. One of the ways that they discovered to eliminate waste is by reducing the inventory, and to build the product "Just in Time". They were inspired by the supermarket system because supermarkets stocks only what is needed for their customers and this is based on buying patterns.Customers can walk in with confidence that the product is available, and choose whatever they want. The items are replenished as and when the stock goes down and everything happens "Just in Time"
• JIT manufacturing was the brain child of Kiichero Toyoda, founder of the Toyota Motor Company
• Kanban is a means through which JIT is achieved
• Kanban is more of an implementation process rather than a planning process

Agile means no documentation - common misconeption

Before I begin my Agile training programs, I provide an opportunity for the participants to share their understanding about Agile. Most commonly heard definition include

"Agile = No Documentation".

This is the most commonly misused and mis-communicated definition. I don't blame the newbies however it is the value

Working software over comprehensive documentation

that has been defined at a higher level of abstraction by Agile Manifesto that creates such a misunderstanding coupled with inexperienced Agile evangelists.


For most of us, as soon as we hear the word "documentation", the first thing that comes to our mind is the "Microsoft Word" document, and the ISO-CMM level prescribed requirement/design documents that we might have written in the past, isn't it ?
So, when a newbie reads the documentation related value from Agile manifesto, they generally assume that one needs to dump documents, and rely purely on running code(working software). However, Agile manifesto is trying to say that "prefer" working software over (not abandon) comprehensive documentation. What it means is, try to create working software, because this is the only thing that adds value to the customer's business and not the extensive documentation.
Next question I generally get is "if we don't do extensive documentation, then how do we retain the knowledge ?', "what is the contingency plan for attrition as this would result in loss of precious knowledge ?"
My take on this is, "do document" in whatever way you can to protect the customer/yourself/project to retain knowledge. Agile value is trying to say that, don't do documentation for the sake of doing it, however document information from the context of adding value to the customer.

In one of my past experiences working on an Airline and defense related projects, we had applied Agile methodology, and found that extensive documentation was indeed a must to conform to various standards. We wrote necessary documents as it added value to the customer, and customer was willing to pay for that.

Documentation not necessarily mean capturing information on "Microsoft word" ! one could write something on the white board or flip chart and take pictures using a digital camera. The images stored could be considered as a project document. Similarly, recorded videos, good java docs(for example), screen captured images, etc could well be considered as project documents.

Jerry Weinberg's take on Agile

I am a big fan of Gerald Weinberg,and recently in an interview on Citerus, he has made some encouraging remarks about Agile. Thought would share the portion of the interview here

Q: You must have seen a whole bunch of ideas, about how to best do software development, grow and die over all those years. Do you see the agile movement as a pendulum swing or is it a move in a new direction?

A: How about a pendulum swing in a new direction? It's a pendulum swing because approximately every decade, there's a fresh movement to "solve the programming problem." High-level languages, structured programming, object-oriented programming, ...

But it's a new direction because it's the first movement to focus largely on social processes rather than purely intellectual ones. For that reason, I believe, it has more hope for success than the earlier movements, each of which made a little progress, then largely ran out of steam before achieving its grand promises.

Of course, agile won't achieve all its grandest promises either, given the conservative nature of human beings, but that's all right. After another dozen decades or so of incremental improvement, we'll begin to see some really fine software development. Well, I shouldn't say "we," because none of us will see them, but at least our great-great-grandchildren will be able to look back at us and laugh at our crude methods.

Gerry is also famous for using analogies to explain things and, making comments on lighter side. Here is an example...

Q: If you're the J.K Rowling of software development, who's Harry P then?

A: Well, first of all, I'm not a billionaire, so it's probably not correct to say I'm the J.K. Rowling of software development. But if I were, I suspect my Harry Potter would be a test manager, expected to do magic but discounted by software developers because "he's only a tester." As for Voldemort, I think he's any project manager who can't say "no" or hear what Harry is telling him.

Applicability of Decision Market technique in projects

Decision Market a.k.a prediction Market  is one of the techniques being used by many universities, governments, research institutes, finanacial companies to predict the future. These predictions have helped many organizations in avoiding obstacles, making better decisions, and in turn becoming  more profitable.  I am of the opinion that this concept can be used in various aspects of software development too.  For ex:  Iteration Planning, Release Planning, Product Planning, etc

This is how I am envisioning applying prediction market concept for release planning

1. Invite all the stakeholders (from the management team), key people from the project team who have good experience on the project

2. Allow them to go to a hall/room and cast vote on a preselected key concept like "the probability of having the release X on time".

3. Ensure that no one discusses their answers with each other until they come out of the room.  This would ensure that people don't get influenced by others decision.

4. Let each person also give the reason while casting the vote on why there could be a delay in the release. This would provide an opportunity for the stakeholders to correct the mistakes and avoid any unplanned obstacles/impediments.

5. Announce an incentive for the people who are making the accurate predictions. 

At the end of the release, identify all the issues faced during the release, and also the actual release status. Whoever has predicted the right things should be given the promised incentives.

More info about image can be found here

Even though the original prediction market talks about punishing the wrong predictors, I think it may not be a good idea in the software development senario. If  one is betting in an open market, one would not really care if the other person is winning or loosing. However in a closed environment like  a project team, punishing one person over the other, leads to animosities and things like that.  

In fact, Planning Poker based estimation technique could be considered as an avtar of prediction market.  During planning poker, the team is given an opportunity to provide their view on the effort needed in completing the task. View from the majority is taken and a consensus on the estimation is drawn. 

Agile 2008 conference had a session on estimation technique using decision markets. Since I didn't attend the conference, am not sure about the content covered during the conference. 

Even Google uses the decision market technique to make key organizational decisions. 

Going back to basics - Incremental, Iterative, Agile, Spiral

Words like Incremental, Iterative are quite often used in an Agile environment. In most cases Incremental and Iterative are used interchangeably. However it is not true that they are one and the same.

Incremental is anything that is built in small chunks, however Iterative is much more planned even though it is also built in chunks. Most of the Agile methods we see are incremental and Iterative.

There is also a debate that the current definition of "Iteration" which specifically talks about building something incrementally but in a time boxed environment is the definition introduced by Agilists, and not the true definition.

Brad Appleton has done a lot of research about the definitions and history around usage of Incremental and iterative words. Check this link out for more details

Agile, Scrum and CSM

Nowadays a lot of companies/software developers wants to get Certified Scrum Master(CSM) certifications. I don't understand why they are in such a hurry. Mostly CSM courses cost anywhere between 1000 - 2000 USD.

I have heard, The Scrum Alliance and the CSTs(Certified Scrum Trainers) are making a lot of money out of it. After looking at the commercial aspect emerging out of Agile methods, and on a lighter note,
I have come up with an analogy.

Agile could be compared to any open source software developed by a bunch of geeks to solve a specific set of problem(s).

Scrum could be compared to the companies who make use of open source softwares, and build a product around it, sell it and make money.

Importance of Scrum Meeting and Kaizen Line stoppages

Many a times Scrum meetings starts becoming boring and if you are one of them, who feel Scrum meetings look like micro management or merely a status update meeting, then this article might give you an insight into why you have such bad feelings about scrum meetings.

Scrum meetings are the backbone of any Scrum lifecycle. It is conducted on a daily basis, and the team members/product owners(as needed) and Scrum masters are expected to be part of this meeting.

I am not going to explain all the details about how to conduct the Scrum meeting but at a high level, 3 questions are answered,

• what I did yesterday,
• what I would be doing today and
• the roadblocks in achieving ones goal.
  

image copy right

Many a times Scrum meetings are converted to a status update meeting. However the intention of adding Scrum meeting to the Scrum Lifecycle seems to be to
1. Make the people to commit their goals in public. Psychologically, this would ensure that people would try to keep their word. This is done with the first and second question used in the Scrum meeting.
2. Impediments/roadblocks are brought to the forefront on a regular basis. This is brought out by answering the 3rd question in Scrum meeting.

The 2nd intention(impediments/roadblocks) mentioned above seems to be taken from the Lean/Toyota Production systems's Line stoppage concept. The idea is that a problem should be addressed and discussed as it occurs, rather than sweeping it under the rug to be forgotten. During these line stoppages all the employees and senior management would take part to understand the problem, brainstorm on the solution and, team effort is used to address the problem.

The problems that Kaizen line stoppages seek to eliminate are extremely costly to the company and ultimately grow worse as they move down the line to your final customers. By being proactive and seeking to eliminate the problem at the source, you're avoiding a lot of costs and headaches in the future. In addition, your employees will appreciate the process, as their input is being taken into account and their opinion is being listened to.

In Scrum, the question around "roadblocks" provides an opportunity to bring the issues to the forefront providing an opportunity to tackle it proactively.

If the values behind Scrum meetings are not understood properly, it merely reduces to a status update meeting, making it boring and people resisting to attend the meetings.

"World is Flat" and Distributed Agile development

I have been reading the book World is flat, since the last few weeks. In this book Thomas L. Friedman, argues the importance of outsourcing. I came across this wiki, in which the benefits of outsourcing is summarized as

Friedman argues that outsourcing has allowed companies to split service and manufacturing activities into components, with each component performed in most efficient, cost-effective way.

Many software and BPO companies globally have realized the above advantage of outsourcing. At the same time the global companies wants to improve the efficiency and productivity by applying efficient software development and industrial practices. One of the ways to reap the above benefits in software industry is by applying Agile methods. Currently there is a big hype in the software industry around Agile, and specifically in India, I am seeing a big wave of Agile hitting the software industries.

Even though I don't have statistics around outsourcing/distributed development, I strongly believe more than 60% of the software development happening in India is mostly in a distributed mode. Many of these developments are at different phases of implementing Agile practices.

Nowadays I have been hearing many Agile thought leaders arguing that "distributed" team is not a team and anything the distributed team calls as "Agile" is really not "Agile", like this one. Obviously if this is not called "Agile" then what are we practicing in this distributed mode ? So far the above mentioned argument is mainly coming from the developer community, thought leaders and not from business people. Business people seem to not care whether it is Agile or not, and at the end of the day, they want the application to be developed and want to make profit out of it. At the same time, developers don't care whether it is Agile or not and at the end of the day, they want to develop applications satisfying the needs of the stakeholder.

I agree that if some thing should be called as Agile, it needs to follow the values and principles mentioned in Agile Manifesto. I also agree that there is a value in following those principles. However if tomorrow, somebody comes and makes a rule saying that "Agile" word should not be used for projects in a distributed mode, then definitely one should start thinking about inventing a new methodology that can be tailored to distributed development.

I have started thinking, may be this is the right time to invent something new and that is more like "Agile" and somewhat like "lean" , that is flexible, efficient (like the "collocated Agile") and also mode free(collocated or distributed) !!! I don't know who else in the world is thinking about this, atleast I have started to do so.

Transitioning from Waterfall to Agile - Some tips

The day comes when suddenly somebody in the organization starts talking about Agile, and decides to implement Agile . This senior person who made this decision, would have heard somebody saying

Agile improves productivity !! and saves money Or the customer would have said, if you don't practice Agile I will not give the project to you(in an outsourced scenario)

Whatever the reason be, now the development team has to start practicing Agile. Note that moving from waterfall to Agile is not only needs the change in software development process but also the entire thought process surrounding the software development. Changing the thought process is not easy, as people practicing Waterfall would have reached some kind of comfort zone and immune to the pain and suffering from the consequences of practicing waterfall. So, it would take a lot more effort for them to unlearn the old way and learn the new way of doing software development.

In order to make the transition smooth here are some tips which could act as pain killers

1. First and foremost thing is to identify a good Agile coach.

2. Get all the team members to undergo training on Agile (XP, Scrum, etc)

3. Don't force all the Agile practices at once. Take one or two practices at a time and give sufficient time for the team to learn and practice.

- For ex: One can start with shorter iterations and scrum/daily stand up meetings.

4. Don't wait for the right day to start Agile. Start today. Go run to a shop and get hold of a Good Agile book(s).

Some good books for starters include,

• Agile and Iterative development a manager's guide by Craig Larman,
• Ken Schwaber's book on "Agile project management with Scrum",
• or any Extreme Programming book.

5. Start with 4-6 weeks iteration rather than 1 week iteration. Many new comers to Agile feel suffocated with 1 week iterations. Earlier the waterfall teams would have delivered softwares once in 6 months, and suddenly asking them to deliver in a short period makes them resist to Agile.

6. Take a simple and low risk project to try Agile. Also, with the help of Agile coach ensure that this project succeeds

7. Try to make use of visible tools like Flip Charts, Burn Down charts, Post-It notes, spreadsheet and encourage more interactions among developers. Make the developers to come out of their cubicles and start designing things on boards.

8. Ensure that management provides complete support to the Agile coach.

9. Even though having indepth understanding of Agile values and principles are key to succeed in an Agile environment, the team may not be able to grasp all of them at once. As and when each practice is introduced to the team, show them the relationship to the values and principles.

10. Don't bother if you would like to start with Scrum or XP. Instead, put all the Scrum and XP practices in a big basket, take the simplest one at a time and start practicing it. Over a period of time you would understand what is best for your team.

11. Scrum and XP advocates specific team structure like having Product Owner, Team, cross functional teams, no hierarchy, a team coach, scrum master, etc. This is a very sensitive issue. Suddenly informing the team that all of you are same, might heart the ego of senior people. I have heard stories where the seniors have quit teams as they were equated to the junior members by calling all of them as team and by dismantling the hierarchies . So, don't worry about dismantling the current team structure. Let the team learn slowly the importance of the values and decide what is best for them.

12. Let the project manager's slowly transition from their current responsibilities to become scrum masters.

Agile on a fixed bid project and Dr.Agile

Here is an article by Scott Ambler who explores the ethical implications of "fixed iron triangle". The article explores why many of business users insist on defining the price, scope, and schedule early in an IT project. It then overviews the overwhelming evidence for why this is an incredibly bad decision, and then describes an ethical approach to addressing this questionable desire.
Dr.Agile, a very good Agile readiness tool. I have not tested this personally, however one of my friend who tested has given very good reviews. Dr. Agile uses over 300 different indicators to determine which agile practices an organization is ready to adopt.

How to fail with Agile 20 Tips

Most of the time we talk about how to succeed in anything and give a lot of tips and best practices to others. How many of us do really remember those tips ?

What if somebody lists all the bad things one should do to fail ? Do you read it ?

Does this interests you ?

If so, then read this new article, How to fail with Agile, 20 Tips to avoid success by Client Keith and Mike Cohn

We don't have any resources available in resource pool

I am sure you would agree that we use the above statement atleast couple of times a day, especially if you are a project manager in one of the software companies. A few years ago, while I was interacting with Craig Larman, he mentioned that "human beings" are not fungible resources, they cannot be called as "resources". That was the first time I learnt that ugly side of calling programmers/software engineers as "resources".

The word resource, in the past was widely used to mean things that are "replaceable"/fungible. For ex: A broken chair can be replaced by another new chair. Another interesting feature of a resource is, you can start using it immediately. Ex: Once you buy a new chair, you can start using it from the next minute.

I don't know from where it all started. However I have been hearing the usage of the word "resources" implying mostly "people/programmers" since several years in the software industry. When we hire a new programmer, we say that the new resource is being hired in place of the old one. We are aware that two human beings cannot be same, one "developer" cannot be replaced by another cloned "developer" with exact specifications. They are two different people coming from wide range of experiences, even though they have "similar" work experiences and designation.

So, let us consciously stop calling "human beings" as "resources".

Here are some more people who think on above lines

• Mark Gregory


• Aaron


• Bill Ryan

Is it a sin to use tools in Agile software projects ?

The first value of Agile Manifesto clearly says, "Individuals and Interactions over Process and tools". Does this mean that usage of processes and tools needs to be reduced ?

Many Agile evangelists prefer using just flipcharts to write their design diagrams rather than using rational rose/any other diagramming tools. These Agile designers are proud to say that they don't use any software tools. However Kent Beck one of the Agile signatories shares in his latest white paper on "Tools for Agility", that when he and others created the manifesto they never wanted to disregard the usage of tools. However they felt that tools are ideal "If" it helps them in certain areas of work and specifically in some of the transitioning activities. Kent is not negating the usage of tools, however the tools like flipcharts, post-its and white boards have their own limitations. They cannot be shared if used in a distributed environment, it might get lost, and things like that.

In the white paper mentioned above, Kent makes some powerful statements. Some of them are as follows

• Every quantitative change of an order of magnitude creates a qualitative change


• A transparent team can more cheaply and effectively coordinate their efforts towards shared goals


• It may be hard to unlearn habits and beliefs, but in a world of wide and
  free flowing information, keeping secrets is a position of weakness


• You never know when you are going to be found out. Transparency is the new strength

I personally beleive that tools are very much needed in any software development, however tools should consciously be choosen and introduced into the project environment. Also a constant inspection and upgradation is essential to ensure that the tools are not becoming bottlenecks and hampering productivity & creativity.

Retrospective smells

Retrospective is the heart of Scrum life cycle. Many Scrum Masters who are new in facilitating Retrospectives find it hard to handle it. Keeping the sensitive nature of this exercise and keeping the emotional aspect in mind, one needs to be really creative, courageous while managing the team doing retrospective.


More info. about the image

George Dinwiddie shares the following retrospective smells.

1. Retrospectives that limit themselves to the three questions, "What worked? What didn't work so well? What are we going to change?"
2. Retrospectives that don't ensure that all the participants are represented. It takes care to get the thoughts and feelings of the introverts.
3. Retrospectives that don't establish safety, or at least acknowledge the level of safety felt by the participants.
4. Retrospectives that are designed to lead to a particular conclusion.
5. It's not a retrospective if you've got "the answer" before you start.
6. It's not a retrospective if the answers come from the leader instead of the participants.

It is very clear that retrospective as a tool can fix things or break things if not done properly. Knowledge and experience of the scrum master plays a key role in facilitating retrospective. Here is a good web resource that has been created by Agilists to share their retrospective experience and in turn help others.

Best time to do performance testing in an Agile environment

In the traditional model of software development major release/milestone dates are fixed, and performance testing is done just before such milestones, and it occurs mostly after the completion of coding, integration and system testing for a chunk of requirements. In an Agile environment too, release and milestones dates are planned, however, between the start date and the major release date, multiple iterations keeps happening and developers keep churning the code.

So, the next logical question to ask is,

In an Agile environment, is it still advisable to do
performance/scalability kind of testing before the major release date Or is there a better way to handle this ?

My personal view is, the non-functional requirements(the -ilities, scalability, availability, performance, etc) cannot be separated from the implementation of functional requirements. It is not a good idea to say that we would implement the functional requirements first and then look at non-functional requirements. According to me, one cannot write a piece of code implementing functional requirement without affecting the non-functional requirements. Even the simplest If-Else statements, for() loops we write affect the -ilities in a big way. It is always advisable to have the non-functional requirement related SLAs in mind all the time
and, after writing each piece of code, ask yourself does this line of code
meet the SLAs ?

Moral of the story is, in an Agile environment, performance, scalability, Availability testing is done all the time right from Iteration 1.

Best Practice: It is better to write unit tests to check the conformance of your code against the SLAs, and automate them. Creation of such unit tests and, automating them is the key to success in implementing non-functional requirements in an Agile environment.

Bahá'í Consultation model and decision making

We are making decisions all the time, either by ourselves or by collaborating with others. There are times where it is easier to make decisions and sometimes tough. One of the classic example is performance moderation session that all supervisors and their teams have to go through in their career, one time or the other. Have you ever attended such meetings in larger organizations ?, if not, here is a preview.
Each supervisor/manager brings the data about their team members and each of these supervisors would have come to the meeting with their own agenda i.e. to get promotions/good hikes to their team members. Most of the time, there are more candidates than the allocated budget and this is a tough situation as only a few, can be chosen for hike/promotions. In situations like this where each one is pursuing their own goal and not a common one, decision making becomes even more difficult.

Above example is an extreme one which may not happen daily, however we commonly come across group meetings where collective decision has to be taken.

How do you make decisions in such situations ?
In such situations one of the best suited techniques would be to apply the Baha'is consultation model". As you might be aware, Baha's is a religion. The goal of consultation the Bahá'í way is to discover the best course of action to take for the well-being of all.

Here are the 4 steps in Bahá'ís consultation technique:

1. Establish the full facts;
2. Decide on the principles to be applied;
3. Discuss the matter;
4. Make a decision.

In this technique, each participant would be given an opportunity to express their opinions, and everybody has to vote for all the ideas shared by the participants. This in turn results in ideas becoming "group's" property rather than individual's. People who would have come with malafide intentions cannot push their ideas as the rest of the crowd has to be in agreement. Respect for people and ideas are given highest priority in Baha'is consultation model.

Here are some good resources throwing more light on this model

http://www.planetbahai.org/cgi-bin/articles.pl?article=167

http://www.infonet.st-johns.nf.ca/bahai/consult.html

http://www.bcca.org/services/lists/noble-creation/bic-5.html

This technique could be applied during Estimation and retrospective sessions in Scrum.

Self Organizing team and its limit

There is a limit to everything and including the limit, a self organizing team can reach.

Self Organizing teams are characterized by the following features

• The team members share a common goal
• They collaborate to accomplish their goals
• Each team member shares the responsibility while managing the tasks
• They make their own decisions to achieve the necessary goal
  
• They take responsibility of both success and failure

Copyright

Self organizing teams are considered to be powerful and productive as compared to teams managed by a manager. However the question arises, can the team make decisions which could possible go against the company's goals ? if so, is it all right for self organizing teams to make such decisions ?

For example, a company might want the team to attend the CEO's biweekly presentation, however the team does not want to do so. Can such decisions be allowed ?

Even though self organizing teams can make their own decision to achieve the goal, the team cannot do whatever they want to do. There are certain limitations and framework within which they have to operate.

For example, the decisions made by self organizing teams should

• be in line with organizational goals
• Customers goals
• Goals of the project
  
• be Socially acceptable
  

If the team does not want to follow one or more of the above framework parameters, the stakeholders of the projects have all the rights to take necessary action. It is also important for the stakeholders to understand the root cause of the problem by applying Five Whys or any other techniques.

When would you not apply Agile methods ?

I have been applying Agile methods from so many years and been interacting with many Agile experts. So far I have not seen/heard stories about the projects where it had been felt that Agile cannot be applied. Agile methods should be looked more like a risk mitigation strategy on software projects rather than a "software development process". Agile promotes continuous improvement through learn and adapt cycles.


Following are the possible characteristics of a project where Agile need not be applied:

• The project carries no risks
  
• The stakeholders are not interested in continuous improvement of project quality.
• The stakeholders are confident that the project would go as per the project plan without any deviation. Even if it deviates, stakeholders/teams are fine with it.

Here is Ken Schwaber's take on "not" applying Scrum on projects:

I wouldn’t use Scrum if I were embarking on work in which there was absolutely no possibility of change or the unexpected. I would then confidently plan the project and await the predicted results on the day when it was due for the cost that was estimated. Wouldn’t this be a wonderful world ?

Stakeholders and management team keep coming across situations to make decisions around complex and debatable subjects, similar to the one we are discussing now(Agile - Not Agile). In such situations Ralph Stacey's Agreement & Certainty Matrix might come handy.