Dave West gave the opening keynote speech today at RE09, titled “Delivering Business Value with Agile Approaches to Requirements”. The keynote description had definitely caught my attention, and I confirmed in a quick chat right before his talk that Dave was fresh from the Agile 2009 conference (he was sporting his Agile Alliance/Rally logo’d lanyard).
I took extensive notes on my laptop, but wasn’t able to ‘live blog’ – we’ve got free wireless, thanks to the conference organizers, but power connections are scarce. My battery’s not as good as it once was, and having the wireless on during the day today would have killed it. So I apologize for the delay in getting this post online, but hope you find it worth the wait!
In a nutshell, Dave delivered – he was entertaining and provided some hot-off-the-press stats on agile adoption. The only ‘promised’ topic which I didn’t feel was well addressed was how ‘formality and discipline play just as important role with Agile methods as with traditional approaches’, and it was a bit under the bar re ‘provide concrete recommendations on organizations can resolve the conflict and build a better requirements discipline’. Everything else he covered was up to, or exceeded, my expectations. There were also some thoughtful questions from the RE09 audience in the limited time for Q&A.
Below are my detailed notes. Be advised that most of the percentages cited below are either subject to transcription errors, or were my approximations from reading column heights on a chart. Also, the bulleting is still a little rough/confusing, and I know I’ve used some personal abbreviations in here – with your permission and understanding, I’ll clean those up later.
Brief mention of his background and prior work on RUP.
(joke: what’s the difference between a terrorist and a methodologist? you can reason with a terrorist)
Mentioned IIBA.We’re always on the brink of technological change. But Forrester sees a wave of significant change coming.
On one of his early projects, they pointed him to the “requirements room’ – full of shelves of requirements docs. He walked into system analysts’ room, asked how they are managing requirements; they said they don’t, they talk to the Navy procurement people. He asked the developers how they work with the systems analysts; and they said ‘what system analysts?’ (EDS project)”Reqts can be a little dull, and they’re only useful if people read them”Dave has written some books, including Head First OOA/D (joke: we don’t have to read them, but he does recommend we buy them)
Today is Ivar Jacobson’s birthday – known for OOA/D, components, UML, UP; best known for use cases. Ivar was working for Ericsson, involved with MIT, went back to Sweden to work on ‘soft switch’ – project was a disaster, chaos, … “tell me what the system does” – couldn’t get an answer, just came out with a stack of phone-book sized requirement docs. Features were described in infinite detail, but no overall view of what it does. He went through those docs, built a thread through the docs to figure out what it did. The reason he built them because it was necessary to understand what the system did. … this was the origin of Objectory -> UP
Dave just attended Agile 2009 conference – 3500 ppl, 1000+ sessions – last week.
State of agile adoption
o Photo of dog jumping obstacle course (from web search of agility) – it’s all about dogs
o What is being adopted?
? Early adopters are scaling
? Later adopters piloting and trialing
? (have ‘crossed the chasm’)
o Simple Scrum like team process being adopted – fills a void in how people work today. “Software development is a team sport”. High performance teams are fundamental. eg iPhone core team. Analogy to, if you could control how a contractor builds your house, you would use a similar approach, want daily updates
? Use of backlog
? regular meetings
? (work only in one place – eg emails, documents, project plan, …)
? visibility & communication
o Agile engr practices are being used (‘the art of software engineering is improving, as the body of knowledge is improving’ … build requirements that you can test. Very common to read requirements which cannot be tested)
? Test early
? Continuous integration and build
? Refactoring (“there is no perfect architecture”)
o Mixed processes are a reality – combination of agile and non-agile
? Moving from “big A” to “little a” – paraphrasing Shakespeare, “Agile is dead; long live agile”
? Forrester-Dr. Dobb’s July 2009 Developer Technographics Survey:
• Asked senior mgrs what development method they primarily follow – pie chart, 30% agile, 32% waterfall, 38% iterative (UP)
• Asked 1298 developers – 36% agile, 31% none, 21% iterative, 13% waterfall … mismatch!
• Example: developer said he had worked in 4 different companies with ‘different processes’, but did the same thing at each one.
? Agile has got developers on a team actually doing the same thing, for a change, because they *want* to work this way.
? Beyond tire-kicking: 38.5% ‘mature’ implementation of agile methods; 30.8% mid-way in adoption; 25% just started adopting agile; 5.8% still evaluating.
? Small teams of early adopters – requirements discipline is very different than when scaling up
o Motives for going agile: (52 ppl; study still in progress) – many corporate objectives, not just development efficiency
? 60% more frequent releases
? 55% greater predictability of releases
? 58% better biz-IT alignment
? 65% more opportunities for mid-course corrections
? Increased maintainability 13%
? Increased team motivation/morale 44%
? Improved product quality 60%
? Always evaluate promising new approaches 23%
? Peer or other recommendation 18%
? Other 8%
o What are we really seeing:
? Frequent delivery: software as well as other artifacts, e.g. requirements
? Experimentation
? Customer/biz involvement increased
• Different working relationship with the business
• Decision making faster and based on evidence
• Collaboration rather than contract (trust)
? Teams organized in a different way
• Small (10 or less)
• Very team focused
• Different mgmt approaches – more fluid, more coaching oriented
? Projects with > 10 people had almost no chance of success
? SEI study on large projects in mid 2000’s – any project > $10MUSD had almost 0 chance of success – delivery differed from expectations significantly
? Cited Ralph D. Stacey book, strategic mgmt and org dynamics: challenge of complexity – diagram adapted from them – technology on X (close to certainty, far from certainty), requirements on Y (close to alignment, far from alignment) – simple, complicated, complex, anarchy – C&C best suited for agile
o Heavy adoption of scrum
? Showed diagram adapted from Mike Cohn – product backlog -> sprint backlog -> 1 day cycle inside 2-4 week cycles -> potentially shippable product increment
? Prioritize backlog by risk or importance
? “lightweight approach focused on delivery”
o Contrasted Sommerville ‘software engineering’ vs Schwaber ‘Agile project mgmt with Scrum’ – docs replaced with collaboration, iterative and continuous process.
o Project managers and business analysts give him the most push-back on agile adoption, limited time for planning. -> friction points
o Cartoon – “I need a horse” – but I don’t need to ride it, and it needs horns, and I want to milk it. Business analyst: I’ll see what I can do.
o Agile viewpoint:
? The business – has a problem
? The development team – provides a solution
? Business analyst – in the middle, with waste in between the analyst and each team. ? eliminate, connect them directly, reduce the waste, reduce/eliminate the documentation (“nobody reads it”)
? While he was at Rational they created SoDA to generate mounds of requirement docs to keep the PM’s happy and to keep the developers from having to write it.
Issues w/ agile and requirements management
o “Truths” about requirements: (photo of geek in white shirt ,tie, glasses, wearing a bright yellow hard hat, brandishing a rolled-up document)
? Tend to be out of date
? Long and hard to read
? Describe a solution, not a problem (??)
? Take way too long
o English ‘needs a lot of words to explain something very badly’
o Requirements tend to be written for consensus-building, trying to address lots of audiences -> why they get big.
o BUT:
? Customer has no time
? Many customers
? Development team members are not good communicators
? Business in many locations
? Problem needs analysis
? Solution requires thought
The role
o Scrum and the product owner
? Empowered voice of the customer/management team
? Owner of ROI and product backlog
? Responsibilities:
• Define features
• Provide VOC
• Decide on release date & content
• Prioritize according to market value
• …
? Product owner = business analyst? (title, responsibility, location, skills)
o Need for a separate BA discipline
? Diagram with # of stakeholders on X, complexity on Y
• Simple – business driven
• complicated
• complex
? what is important:
? work still needs to be done – smaller problem, simpler solution
? dev team can help, but might not have the right skills
? business does not either –
• asking the right questions builds the right solutions
? agile increases the value of requirements
• just focus on solution not documentation
o implications for requirements role
? BA needs to be empowered to make decisions
• no waiting around for signoffs
? Need to know business (not just how existing systems work) – “real problem today: nobody knows how systems work” – ex: bank, Cobol project
? Must be able to work with dev (team player, not doc creator)
? Expecting a lot of people … seeing in Dr. Dobbs survey an increase in salaries of requirement-related roles.
? Vendors also investing effort
o Documentation: BC cartoon on monorails (same as Ivy used yesterday)
o “get on the ship or you will be swimming”
o Discussed airbus crash … UK uses mix of metric and imperial units … some subsystems were talking in mixed units.
o Working with the business – Q1 2009 Global Software Development Process Online Survey: – what mechanisms do you use to work with your customer?
? 72% functional specifications
? 62% use cases
? 57% prototypes
? 46% Nonfunctional specs
? 37% JAD sessions
? 35% storyboards
? 17% wikis
? 10% other
o What tools? (same survey)
? 99% office other than visio (word docs)
? 77% Visio
? 41% BI, data mining, …
Documentation
o Agile and requirements management:
? Only specify things that get implemented
? Specifying things at the appropriate level of detail
? Empower stakeholders and users to take ownership and control of the project scope
? Integrating estimation and progress tracking into the requirements space
? Iteratively evolving the requirement spec alongside the implementation
? Continually adapting the techniques to reduce project risk
o Example: asking his fiancée what things needed to be done at the cape house – she wrote a long list, emailed to him and herself … “if people think they only get one shot at asking for what they want, then you will get a very long and complex list”
o Ref Mary Poppendieck on lean software – waste
o Seeing a movement back to function points, more formal sizing mechanism creeping in, even with agile. Process of sizing illustrates that eg building a deck is a really big task.
o Artifacts:
? Stories (not ‘user stories’ necessarily)
? Use cases (bank customer -> withdraw cash)
? “a use case describes a sequence of actions a system performs that yields an observable result of value to a particular actor”
? UML 2 – stick man’s arms have moved down a bit
o Stories
? Conversation
? Communication among all parties
? Not too small (> 1 p-day)
? Not too large (…)
? …
o Relationship between stories and other requirements:
? Product backlog -> defect, stories – project life span, drive development, support reporting
? System spec -> use case, traditional reqts: system life span, are development work, used to maintain system
o Diagram from the essential unified process, adapted from Use Case Modeling, Bittner & Spence, A-W 2002
o Importance of visualization:
? Relationship between visual and need
? Vendors, eg IBM and Microsoft ‘blend’? – sketch pro product – design + development
o Role of data and information:
? Semantic technologies (mashups)
? Simple data models
? Simple vocabularies/taxonomies – integrate w/ corporate definitions
o Process and rules:
? For complex processes and rules bases – use of CEP platforms
? Simulate and review with business – best way to find issues with logic and flow
? Describe in most appropriate forms
Summary
Requirements are key for agile projects
o Responsibilities:
? product ownership,
? product direction & value,
? business quality
o Artifacts –
? backlog,
? stories, visual protos, info models
o Working practices –
? iterative, with feedback
? collaborative (discussion, wiki updates) – apply “crowdsourcing” paradigm – “really interesting topic”
? decision making, driving product direction.
Audience Questions –
(question is in italics; rest is a paraphrase of Dave’s answer)
Daniel Berry, University of Waterloo – on role – but how do you get developers who are not good communicators to do a good jobs in the biz analyst roles? – specialist role still needed on larger projects. on smaller ones, the PMs or others can mentor.
unknown audience member, Univ of Saskatchewan – what about at the end of the project, when there are no docs to support maintenance? Use/need these docs, but do not write/use them to drive the project – create them by thinking about how to maintain, test, deploy, … consider the audience. Don’t write long docs for developers, because they won’t read them! Use video, youtube segments, record architecture in PPT slides discussed in Webex, annotate a wiki to illustrate key architectural decisions. Learn from open source community. Look at Eclipse for a good example. Can also add a few of these doc items as backlog items, then explicitly make decisions about them. Lots of orgs thinking about product line architecture, eg Deutsche Bank – now have to think about them.
(keynoting tomorrow) Alistair Sutcliffe, Univ of Manchester – at what level does agility start to not pay off, complexity starts to snow the process?
“nothing works at those levels” even though we pretend that it does. End up with a large-scale macro process, then break into chunks and try to apply agile to each of those chunks. Ex: 700 stakeholders – how many communication channels is that? … break it up into smaller problems for smaller teams, consider using crowdsourcing to get input.
‘feature crews’ = agile teams
Mention of Agile growing and CMMI shrinking (?) – Dave questioned if CMMI is becoming less relevant.
(unknown audience member) – Any data on applicability of agile methods for safety-critical systems?
No, he has focused on IT systems. But some of the things, eg scrum practices, can make sense on any team activity.
Agile makes explicit decisions about (architecture, things you do on any system) and continuously deliver *some* working software, enables early testing.
On one real time embedded system – first 6 months were on key architectural decisions – could still run this part using Scrum. Put agile into the context of the lifecycle.
[…] http://www.agileteams.com/blog/2009/09/02/re09-keynote-on-agile-and-requirements/ a few seconds ago from web in context […]
A blog that is about as useful as Mr West's presentation
A blog that is about as useful as Mr West's presentation