Agile Software Movement - New Synergy Group

Agile Software Development Movement:
Momentum and Challenges

This section first describes why and how agile software development is gaining momentum in industry, and the challenges presented by the momentum.

A. Why and How Agile Software Development is Gaining Momentum

Challenges for software development projects are well known in industry for the last few decades: Cost and schedule overruns, failure to meet the real requirements of their intended users, poor quality, failure to meet performance and scalability goals, etc. These challenges are further exacerbated by increasing pressure to develop more ambitious and complex software in shorter cycle times, at reduced cost, and in an increasingly volatile global business environment.

Quickly responding to rapidly changing and often volatile business environment (such as rapid market changes, global competitive pressures, and regulatory changes) is critical to enterprise success. Such rapid changes and volatile environment create an urgent need for agile development processes for software products and services. Agile development and project management methods are proving that they can deliver rapid results needed by the enterprise to compete in today's fast-paced marketplaces.

While agile methods have their roots in small, co-located teams, according to a survey by Forrester Research, Global-2000 companies (20,000-plus employees) have the highest adoption rate at 34%, vs. companies with 1,000 to 4,999 employees at 21% and companies with 5,000 to 19,999 employees at 28%.

Recognizing growing momentum of agile methods, Microsoft Visual Studio Team System (VSTM) product includes Microsoft Solutions Framework (MSF) for Agile Software Development. In addition, partner organizations offer support for the SCRUM framework (the industry-leading agile practice).

Agile development has now a strong following in Silicon Valley at both large companies (such as Google and Yahoo!) and also at entrepreneurial start-ups. There are now venture capital firms that fund only those companies with software development teams that have proven track records in agile development.

The Agile Alliance is a nonprofit organization that promotes the concepts of agile software development as outlined in the Agile Manifesto. The Alliance reports a membership of nearly 6,000 around the world. The industry demands for engineers and managers with knowledge and experience in agile development, project management and specific agile methods (such as SCRUM) are well reflected in the job postings statistics collected by New Synergy Group from Dice and Monster job boards as of September 15, 2009 – which is in the midst of the Great Recession (see Table 3).

Table 3: Jobs advertised with expertise in agile methods and SCRUM

Jobs that need knowledge of “Agile” methods

Jobs that need SCRUM knowledge

DICE

2,792

873

Monster

2,033

572

B. Challenges Posed by Agile Software Development Projects

Most university software engineering and software process courses offered today are heavily oriented towards conventional water fall method of software development. Reading about agile methods, practices, processes from books helps, but it still does not address the challenging issue of how to develop and apply customized agile methods (such as customized SCRUM) that will fit well in your enterprise’ culture and ultimately will really serve you. Remember, the process is a means to an end (successful completion of software development project on time and under budget that meets the real needs of intended users); the process is not an end unto itself. Customization of agile methods require careful attention to several factors, such as:

Nature and challenges of the software project at hand (complexity, uncertainty, volatility, etc.)
Skills, strengths and profiles of members of the software project team (management, development, quality assurance, release, delivery, customer support, etc.)
Budget, schedule and resources
Technology infrastructure
Involvement and team work of all stakeholders, especially customers
Enterprise culture

Moreover, although agile project management, process management and product management are very helpful, they are often not adequate or even the most appropriate responses in some situations. They need to be supplemented with application of lean methods, and also systems thinking and system dynamics to address systemic problems and scalability challenges. However, we come across another formidable challenge: Not many software project managers, product managers and engineers have much familiarity with lean methods, systems thinking and system dynamics discipline.

Many ideas of agile software movement of 2000’s arose from lean manufacturing movement in 1980’s and lean design in 1990’s originated in Japan. Agile software movement has incorporated many of those ideas (such as elimination or reduction of waste, increasing the flow, pull-based work, etc.) in agile software development and project management.

Deep systems thinking requires a very good grasp of the total picture (components of a system, and more importantly, their interconnections) and unusual ability to develop solutions holistically and systemically. This requires deep knowledge of the concerns and often conflicting interests and viewpoints of multiple stake holders of a software project (executive sponsor, product manager, paying customers, actual end users, project manager, software engineers, quality assurance engineers, product release engineers, product delivery engineers, post-delivery customer support team members, etc.), and very good understanding of the enterprise culture and enterprise business dynamics.

Services: Training | Workshops | Consulting | Management | IP Development