Featured Article: October 2010
.
Building Organizational Behavior to Support Agile Development
by ARZIKA guest writer, Dr. Charles A. Suscheck, D.CS.
In the last few years there has been a trend away from command and control development processes toward agile processes [1]. Most organizations are not well positioned to adopt an organizational behavior that supports agile methodologies, particularly when examining the role of people, teamwork, and group behavior. Such a change in organizational behavior is difficult to understand and even more difficult to execute. In order to understand the complexity of the post-industrial organization, scholars [2,4,6] have suggested that the metaphor of a jazz improvisation band can be used as a way to conceive the group dynamics required to support post-industrial organizations. By illuminating the nature of decentralized and lateral structure, team-based and mission centered behavior, reciprocal power relations, and the emphasis on creative thinking, metaphors, particularly the jazz improvisation metaphor, help to understand the key characteristics of a well formed agile team [2,3].
There are many similarities between participants in agile development and jazz improvisation. Novice improvisational musicians and agile participants must have a broad base of personal and shared experiences that enables them to create as an ensemble. In order to improvise together, jazz musicians must have skills to draw upon (both personal knowledge of the genre and technical virtuosity), be open to ambiguity, and be willing to support the ensemble’s effort to create music. They must support others’ new ideas, and consciously experiment with those ideas, deliberately focusing on interrupting habitual patterns [6], all of which relate directly to agile development: an environment of so-called controlled chaos.
Participants in the agile development process must be skilled, experienced, and work together in a collaborative and supportive manner. Agile team members must be motivated, able and have a high mixture of skills (particularly in Scrum). Formal documentation and design are kept to a minimum – only documentation which directly supports product creation- which calls for members to understand software development and engineering principles and be able to infer designs within a framework of minimal documentation [5]. Agile developers, in particular those participating in an emergent process, need strong communication, interpersonal, and technical skills.
Seasoned musicians know how to use errors as a source of learning [6]. They work as teams with minimal structure for maximum flexibility, quickly distributing tasks, and constantly self-reflecting to understand what’s happened. They quickly take turns in leading and supporting the team. Mistakes are a way of exploring new creative expressions and learning.
Traditionally, the business world sees mistakes as something to avoid (as in reducing scrap and rework). In agile development, much like jazz improvisation, people in the project must be committed and open to using mistakes as a form of learning, not necessarily as something to be avoided or punished. In fact practitioners, such as Cockburn [7], make the point that incremental and iterative software development was specifically developed so people can learn from their inevitable mistakes. Incremental development is a critical success factor so teams can learn about their own development process. For example, daily stand up meetings, in Scrum and Extreme Programming, provide an area for members to ‘hear’ (in jazz terms) what each member is doing so as to better adapt their combined efforts.
Successful jazz improvisation requires strong group participation and an enabling leadership style. Jazz music is created through the combined improvised effort of an ensemble and not through any one isolated individual – the outcome of the whole is much more than the sum of the pieces. Jazz emerges as collaborative group interaction within a shared orientation, expertise and technical mastery, as well as the ability to work within and contribute to a group. Organizational leaders must view agile development as group activities rather than any one individual’s work.
Group efforts, whether playing jazz or developing software in an agile environment, call for insightful leadership. Improvisational leadership must be open to new ideas and allow mutual collaboration and creativity to emerge. The leadership in jazz improvisation is more aligned with the notion that work can be self-motivating . It requires an increase in creative freedom rather than explicit direction that emphasizes too much control, inhibiting the creative flow of ideas. Thus leadership seeks to foster teamwork and egalitarian relationships, encouraging every member to remain open to experimentation [4].
Leadership, in both agile development and jazz improvisation, must promote acceptance of ambiguity, facilitate collaborative team work, encourage emergent behavior, and promote exploration. Cockburn goes so far as to state that “legislating how a person is to solve problems invites trouble” [7]. Since requirements and design are frequently in a state of flux, agile development teams must be open to uncertainty; they must embrace change and the freedom that comes with ambiguity and not be tied to a planned driven environment where the emphasis is on policies and procedures. Both jazz improvisation and agile organizations favor rapid response and immediate value over predictability, stability, and high assurance [6]. In order to foster team confidence to the point of embracing ambiguity, the agile manager must avoid dictating what the team must do and rather act as a coach, allowing teams to perform via emergent behavior.
As with jazz, agile development relies on social aspects of the team dynamic. The team is collectively responsible for the success of iterations and the project as a whole leading to trust, a sense of shared responsibility and commitment, which are critical to the team’s success. Team members maximize each other’s strengths to minimize any one member’s weakness through self-organization. As a case in point, the agile manifesto suggests “Build projects around motivated individuals. Give them the environment and support their needs, and trust them to get the job done . . . The best architectures, requirements, and designs emerge from self-organizing teams” [9]. Larman [8] concurs: “the primacy of people and interactions [is] promoted in agile methods by the emphasis on communication.” Taking their cue from the jazz improvisation metaphor, organizational leaders must foster the social aspects of software development; extreme programming, for example, co-locates developers in a bullpen environment to induce such social interaction.
By applying the jazz improvisation metaphor to agile development, we can learn:
• to view agile development from the level of group behavior rather than individuals
• that participants must have skills and experience in software engineering
• that individuals must be open to experimentation, supportive, and comfortable with ambiguity
• that a mixture of skills must be spread among the members of the team
• that leadership must allow for and be open to ambiguity, act as a coach, and
foster the social aspects of software development
• that mistakes should be used as a form of learning
The jazz improvisation metaphor is not perfect – it has limitations. For example, jazz musicians practice individually and rehearse as a group before a performance; it would be stretch to apply rehearsal and practice to software development. Not all projects developed using agile development operate correspondingly to a jazz ensemble practicing improvisation; certain projects are necessarily routine and do not require thinking out of the box. In fact not all jazz improvisation is successful; at times the musical exploration can lead to a failure – as can be heard in any number of live examples.
Employing the jazz improvisation metaphor as a way to think of the organizational behavior necessary to support agile development can lead to a better understanding of the cultural and behavioral shift needed to induce more creative group interaction. Despite areas where the metaphor fails, there are a number of aspects of improvisation that can be applied to group dynamics in agile software project management.
.
About the Author
Dr. Charles Suscheck is an agile thought leader at Pillar Technology who specializes in agile software development adoption at the enterprise level. With over 25 years of professional experience, Dr. Suscheck has held positions of Process Architect, Director of Research, Principle Consultant, Professor, and Professional Trainer at some of the most recognized companies in America. He has spoken at national and international conferences such as Agile 200X, OOPSLA, and ECOOP on topics related to agile project management and is a frequent author in industry and academia.
.
.
References
1. Ambler S. Agile Adoption Rates Survey. (2006) Retrieved 10/4/07 from http://www.ambysoft.com/downloads/surveys/AgileAdoptionRatesSummary.pdf
2. Hatch MJ. Jazz as a Metaphor for Organizing in the 21st Century. Organizational Science Special Issue Jazz Improvisation and Organizing, 1998; 9(5), pp. 556-557.
3. Suscheck C, Ford R. Jazz Improvisation as a Learning Metaphor for the Scrum Software Development Methodology, to appear in Software: Process and Improvement, 2008
4. Barrett FJ. Coda Creativity and Improvisation in Jazz and Organizations: Implications for Organizational Learning Organizational Science, 1998; 9(5), pp. 605-622.
5. Boehm B, Turner R. Balancing Agility and Discipline: a Guide for the Perplexed. Addison-Wesley: Boston; 2003.
6. Crossan MM. Improvisation in Action Organization Science, Special Issue: Jazz Improvisation and Organizing 1998; 9(5), pp. 593-599.
7. Cockburn A. Agile Software Development. Addison-Wesley: Boston; 2002.
8. Larman C. Agile and Iterative Development: a Manager’s Guide. Addison-Wesley: Boston; 2004.
9. Beedle M et al. The Agile Manifesto. 2001 Retrieved 10/2/2010 from www.agilemanifesto.org/principles.