Agile Methods W04

 Agile Methods

• Heavy vs Light Methods
• What is an agile process?
• Values/tradeoffs of the agile manifesto.
• Principles of the agile manifesto.
• Reasons to go agile.
• Problems with agility.
• How to make agile modeling work.
• Scrum
• Agile and Plan-based Factors

Rapid software development

Rapid development and delivery is now often the most important requirement for software systems
– Businesses operate in a fast-changing environment and it is practically impossible to produce a set of stable software requirements
– Software has to evolve quickly to reflect changing business needs.
Plan-driven development is essential for some types of the system but does not meet these business needs.
Agile development methods emerged in the late 1990s whose aim was to radically reduce the delivery time for working software systems

Agile Development

• Program specification, design and implementation are interleaved. The requirements document defines the most important characteristics of the system.
• The system is developed as a series of versions or increments with stakeholders involved in version specification and evaluation.
• Frequent delivery of new versions for evaluation, every 2-3 weeks.
• Extensive tool support (e.g. automated testing tools) used to support development.
• Minimal documentation – focus on working code.

Heavy vs Light Methods

Heavy Process

• Predictive making it difficult to respond to change.
• Elaborate, long-term, detailed planning.
• Disciplined, detailed process.
• Lots of documentation produced in a bureaucratic environment.
• Process oriented.

Light(Agile) Process

• Adaptive – embrace change.
• Iterative and incremental.
• Fast cycle/frequent delivery.
• Tacit knowledge – project knowledge in the participants’ heads rather than in documents.
• People oriented.

Predictive vs Adaptive

• A Predictive process attempts to plan and predict the activities and resource (people) allocations in detail over a long period of time -the waterfall model.
• This works well until things change so their nature is to resist change.
• An adaptive process accepts change as an inevitable driver and encourages flexible adaptation,

Agile Methods

• In the ‘80s and ‘90s there was a widespread view that the best way to achieve better software was:
  – Through careful project planning.
  – Formalised quality assurance.
  – Use of analysis and design methods supported by case.
  – Controlled rigorous software process
• Necessary because systems were large, long-lived systems e.g. government or aerospace:
  – Large teams working for different companies and geographically dispersed.
  – Critical systems.
  – Long life i.e. maintenance.
• Not suited to small/medium business applications.
• Dissatisfaction with the overheads involved in software design methods of the 1980s and 1990s led to the creation of agile methods. These methods:
  – Focus on the code rather than the design.
  – Are based on an iterative approach to software development.
  – Are intended to deliver working software quickly and evolve this quickly to meet changing requirements.
• The aim of agile methods is to reduce overheads in the software process (e.g. by limiting documentation) and to be able to respond quickly to changing requirements without excessive rework.

The Agile Manifesto

• The cost of change grows through the software lifecycle, the question now is not how to stop change early in a project but how to better handle the inevitable changes throughout the lifecycle.
• Software engineers cannot eliminate change, driving down the cost is the only viable strategy.
• While embracing change quality must be retained.
• Group of industry experts met in 2001 to discuss ways of improving current software processes that were document driven and process heavy.
  Goal- speed up development and respond to change. The Agile Alliance.

Agile Methodologies emphasise the following key values:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

That is while there is value in the items on the right, we value the items on the left more.

Pros & Cons of the Manifesto

Individuals and interactions over processes and tools.
– Pros: capable people can figure out how to accomplish tasks without a need of process.
– Cons: without process productivity gains cannot easily be repeated on future projects.
– Working software over comprehensive documentation.
– Pros: without the need to document extensively software can be developed more quickly.
– Cons: harder for people who did not develop the system to maintain it.

Customer collaboration over contract negotiation.
– Pros: can adapt more easily to changing requirements.
– Cons: customer representative may not have adequate knowledge of what is needed by their organisation.
– Responding to change over following a plan.
– Pros: adaptability leads to producing what the customer really wants.
– Cons: customer may keep changing the requirements leading to longer delivery time

The Agile Manifesto Principles

• Our highest priority is to satisfy the customer through the early and continuous delivery of valuable software.
• Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
• Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
• Business people and developers work together daily throughout the project.
• Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
• The most efficient and effective method of conveying information to and within a development team is a face-to-face conversation.
• Working software is the primary measure of progress.
• Agile processes promote sustainable development. The sponsors, developers and users should be able to maintain a constant pace indefinitely.
• Continuous attention to technical excellence and good design enhances agility.
• Simplicity—the art of maximizing the amount of work not done—is essential.
• The best architectures, requirements and designs emerge from self-organizing teams.
• At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.

Agile Method Applicability

• Product development where a software company is developing a small or medium-sized product for sale.
  – Virtually all software products and apps are now developed using an agile approach
• Custom system development within an organisation, where there is a clear commitment from the customer to become involved in the development process and where there are few external rules and regulations that affect the software.

MoSCoW

• The practice of critically reviewing the requirements and prioritising them in the
  context of what it would take to produce a successful system.
• We MUST have these features, the system is of no use without all of this functionality.
• We SHOULD have these features, these features will lead to an excellent system but we could succeed without them.
• We COULD have these features, if we have more time this functionality would be very useful.
• We WON’T have these features, they are not needed and have been dropped.

Reasons to go Agile

1. Revenue
   The approach is iterative and incremental enabling some benefits to be realised early as the product continues to develop.
2. Speed-to-market
   Research suggests about 80% of all market leaders were first to market. Agile development supports early and regular releases.
3. Quality
   A key principle of agile development is that testing is integrated throughout the lifecycle.
4. Visibility
   Active user involvement provides visibility of both the project’s progress and the product itself, which in turn helps to ensure that expectations are effectively managed.
5. Risk Management 
   Small incremental releases help to identify problems early on and respond to change
6. Flexibility / Agility
   Change is accepted and excepted. The timescale is fixed and requirements emerge and evolve as the product is developed. The user must understand this concept and make the necessary trade-off decisions, trading existing scope for new.
7. Cost Control
   The scope of the product and its features are variable, rather than the cost.
8. Business Engagement/Customer Satisfaction
   Due to active user involvement, high product and progress visibility and flexibility.
9. Right Product
   By embracing change (with the appropriate trade-offs), the team build the right product.
10. More Enjoyable!
    The active involvement, cooperation and collaboration make agile development teams a much more enjoyable place for most people. Less documentation and meetings.

Problems

Active user involvement and collaboration
These principles are very demanding on the user representative’s time and require a big commitment for the duration of the project.
Requirements emerge and evolve – flexibility.
Creates potential for scope creep, which can create the risk of ever-lasting projects. Also,
there is much less predictability, at the start of the project and during, about what the project is actually going to deliver. This can make it harder to define a business case for the project and to negotiate fixed price projects. Contracts may be a problem as with other approaches to iterative development
Testing integrated throughout the lifecycle.
Testers are needed throughout the project and this effectively increases the cost of resources on the project.
Frequent delivery and feature sign off.
User acceptance testing is continuous. The users need to be ready and available for prompt testing of the features as they are delivered and throughout the entire duration of the project, time-consuming.
Intensity
Team members may be unsuited to the intense involvement that characterises agile methods.

Practical Problems With Agile Methods

• The informality of agile development is incompatible with the legal approach to contract definition that is commonly used in large companies.
• Agile methods are most appropriate for new software development rather than software maintenance. Yet the majority of software costs in large companies come from maintaining their existing software systems.
• Agile methods are designed for small co-located teams yet much software development now involves worldwide distributed teams.

Agile Methods and Software Maintenance

• Most organisations spend more on maintaining existing software than they do on new software development. So, if agile methods are to be successful, they have to support maintenance as well as original development.
• Two key issues:
  – Are systems that are developed using an agile approach maintainable, given the emphasis in the development process of minimising formal documentation?
  – Can agile methods be used effectively for evolving a system in response to customer change requests?
• Problems may arise if the original development team cannot be maintained.

When to go Agile

• Uncertain or volatile requirements.
  – If you don’t have stable requirements, then you aren’t in the position to have a stable design and follow a planned process
• Responsible and motivated developers.
  – Adaptive processes rely on you trusting your developers, so if you consider your developers to be of low quality and motivation then you should use a predictive approach.
• Customer who understands and will get involved.

When not to go Agile

• Large team – over a hundred located in different places.
  – New approaches have been used on a small scale and are difficult to scale up. They have also been created with an emphasis on small teams.
• Critical systems development where a detailed analysis of all of the system requirements is necessary to understand their safety or security implications.
• Fixed price, or more correctly a fixed scope, contract.

Factors which ensure Agile Modeling will Work.

• You are taking an agile approach to software development.
• You are working iteratively and incrementally.
• Uncertain or volatile requirements.
• The primary goal is software development.
• Stakeholder support and involvement.
• An AM champion exists.
• Developers are responsible and motivated.
• Adequate resources are available.

Agile Project Management

• The principal responsibility of software project managers is to manage the project so that the software is delivered on time and within the planned budget for the project.
• The standard approach to project management is plan-driven. Managers draw up a plan for the project showing what should be delivered when it should be delivered and who will work on the development of the project deliverables.
• Agile project management requires a different approach, which is adapted to incremental development and the particular strengths of agile methods.

Agile Planning

• Agile methods of software development are iterative approaches where the software is developed and delivered to customers in increments.
• Unlike plan-driven approaches, the functionality of these increments is not planned in advance but is decided during the development.
  – The decision on what to include in an increment depends on progress and on the customer’s priorities.
• The customer’s priorities and requirements change so it makes sense to have a flexible plan that can accommodate these changes.

Agile Planning Stages

• Release planning, which looks ahead for several months and decides on the
  features that should be included in a release of a system.
• Iteration planning, which has a shorter term outlook, and focuses on planning the
  next increment of a system. This is typically 2-4 weeks of work for the team.

Story-based planning

• The planning game is based on a set of user stories that cover the functionality of the final system.
• The project team read and discuss the stories and rank them in order of the amount of time they think it will take to implement the story.
• Stories are assigned ‘effort points’ reflecting their size and difficulty of implementation.
• The number of effort points implemented per day is measured giving an estimate of the team’s ‘velocity’.
• This allows the total effort required to implement the system to be estimated.

The Planning Game

A ‘Prescribing Medication’ Story

Examples of Task Cards for Prescribing Medication

Release and Iteration Planning

• Release planning involves selecting and refining the stories that will reflect the features to be implemented in a release of a system and the order in which the stories should be implemented.
• Stories to be implemented in each iteration are chosen, with the number of stories reflecting the time to deliver an iteration (usually 2 or 3 weeks).
• The team’s velocity is used to guide the choice of stories so that they can be delivered within an iteration.

Task Allocation

• During the task planning stage, the developers break down stories into development tasks.
  – A development task should take 4–16 hours.
  – All of the tasks that must be completed to implement all of the stories in that iteration are listed.
  – The individual developers then sign up for the specific tasks that they will implement.
• Benefits of this approach:
  – The whole team gets an overview of the tasks to be completed in an iteration.
  – Developers have a sense of ownership in these tasks and this is likely to motivate them to complete the task.

Software Delivery

• A software increment is always delivered at the end of each project iteration.
• If the features to be included in the increment cannot be completed in the time allowed, the scope of the work is reduced.
• The delivery schedule is never extended.

Agile Planning Difficulties

• Agile planning is reliant on customer involvement and availability.
• This can be difficult to arrange, as customer representatives sometimes have
  to prioritise other work and are not available for the planning game.
• Furthermore, some customers may be more familiar with traditional project plans and may find it difficult to engage in an agile planning process.

Agile Planning Applicability

• Agile planning works well with small, stable development teams that can get together and discuss the stories to be implemented.
• However, where teams are large and/or geographically distributed, or when team membership changes frequently, it is practically impossible for everyone to be involved in the collaborative planning that is essential for agile project management.

“In preparing for battle I have always found that plans are useless, but planning is indispensable.”

President Eisenhower

 

Scrum

From rugby, a scrum is used to restart a game. The creators of Scrum believe no matter how much you plan as soon as the software begins to be developed chaos breaks out and plans go out the window. The best you can do is react to where the rugby ball squirts out, then sprint until the next scrum.

• Scrum is an agile method that focuses on managing iterative development rather than specific agile practices.
• There are three phases in Scrum.
  – The initial phase is an outline planning phase where you establish the general objectives for the project and design the software architecture.
  – This is followed by a series of sprint cycles, where each cycle develops an increment of the system.
  – The project closure phase wraps up the project, completes required documentation such as system help frames and user manuals and assesses the lessons learned from the project.

Intro to SCRUM in under 10 Minutes

Scrum Process

Scrum: Backlogs

Product Backlog
Collection of requirements (user stories) for the product – at project start
– a few, detailed user stories; collection evolves over time and requirements will be refined over time
Managed by the Product Owner

Sprint Backlog
Collection of requirements (user stories) that are selected for implementation during next sprint Managed by the Team

Scrum: Roles

Product Owner
Decides which requirements are implemented for a product version.
Decides about when product increments will be shipped.

Team
Implements requirements.
Decides how many requirements are implemented in a Sprint.
Organizes its activities (-> tasks) independently.

Scrum Master
Takes care of the proper implementation of Scrum.
Supports the team in process-related issues.

Scrum: Sprints

Sprint
Period (2-4 weeks) in which a shippable product
increment (executable, tested, and documented) is
created by the Team

Time-boxed
i.e. ends exactly at the scheduled time

At end of Sprint: Product Owner accepts/rejects the final results (i.e., the software)
– Partially completed or incorrect results will not be shipped (no compromise on quality) and go back to the Product Backlog for inclusion in the next Sprint (Backlog)

The Scrum Sprint Cycle

• The starting point for planning is the product backlog, which is the list of work
  to be done on the project. During the assessment phase of the sprint, this is reviewed and priorities and risks assigned. The Product owner is closely involved in this process and can introduce new requirements or tasks at the beginning of each sprint.
• The selection phase involves all of the project team who work with the Product owner to select the features and functionality from the product backlog to be developed during the sprint.
• Once these are agreed, the team organise themselves to develop the software. Short daily meetings (scrums) involving all team members are held to review progress and if necessary reprioritise work
• During this stage, the team is isolated from external interference, with all communications channelled through the Scrum master.
• The role of the Scrum Master is to protect the development team from external distractions.
• At the end of the sprint, the work done is reviewed and presented to the Product Owner. The next sprint cycle then begins.

Scrum Team

Small team size:
– Typically 5 to 9 team members.
Cross-functional:
– Design, coding, testing, etc.
– Members must have a broad range of skills.
– Every team member is an expert in his/her field but can also take over responsibilities of other team members.
Teams are independent/empowered
– Decides which requirements to include in next Sprint (i.e., the team has the power to reject too many requirements).
– Decides independently which tasks to perform to implement the requirements.
Members should work in close distance (ideally in the same room).
Members should be full-time.
Membership should change only between sprints.

Teamwork in Scrum

• The ‘Scrum master’ is a facilitator who arranges daily meetings, tracks the backlog of work to be done, records decisions, measures progress against the backlog and communicates with customers and management outside of the team.
• The whole team attends short daily meetings (Scrums) where all team members share information, describe their progress since the last meeting, problems that have arisen and what is planned for the following day.
  – This means that everyone on the team knows what is going on and, if problems arise, can re-plan short-term work to cope with them.

Key Practices

• Teams are self-organised and self-directed. There is no team leader. Teams organise in a symbiotic manner and set their own goals for each sprint (iteration).
• Once a sprint has begun, Scrum teams do not consider any additional requirements, any new requirements are placed on a backlog of requirements that still need to be addressed.
• Members should work in close distance (ideally in the same room).
• Members should be full-time.
• Membership should change only between sprints.
• At the start of every day, a Scrum meeting takes place.
• At the end of each sprint, the software is demonstrated to the Product Owner, the goal is to assess the work results and obtain approval.
• Directly after the review, a retrospective is held:
  – What went wrong?
  – What went well
  – What could be improved and how?
• The goal is to improve team collaboration and the application of Scrum.

Scrum Benefits

• The product is broken down into a set of manageable and understandable chunks.
• Unstable requirements do not hold up progress.
• The whole team have visibility of everything and consequently, team communication is improved.
• Customers see the on-time delivery of increments and gain feedback on how the product works.
• Trust between customers and developers is established and a positive culture is created in which everyone expects the project to succeed.

Agile and Plan-based Factors

Most projects include elements of plan-driven and agile processes. Deciding on the balance depends on:

System Issues

1. What type of system is being developed? Plan-driven approaches may be required for systems that require a lot of analysis before implementation (e.g. real-time system
with complex timing requirements).
2. How large is the system that is being developed? Agile methods are most effective when the system can be developed with a small co-located team who can communicate informally.
3. Is the system subject to external regulation? If a system has to be approved by an external regulator (e.g. the FAA approve software that is critical to the operation of an aircraft) then you will probably be required to produce detailed documentation as part of the system safety case.
4. What is the expected system lifetime? Long-lifetime systems may require more design documentation to communicate the original intentions of the system developers to the support team.

Team Issues

5. What technologies are available to support system development? Agile methods rely on good tools to keep track of an evolving design
6. How is the development team organised? If the development team is distributed or if part of the development is being outsourced, then you may need to develop design documents to communicate across the development teams.
7. Is an incremental delivery strategy, where you deliver the software to customers and get rapid feedback from them, realistic? If so, consider using agile methods.
8. How good are the designers and programmers in the development team? It is sometimes argued that agile methods require higher skill levels than plan-based approaches in which programmers simply translate a detailed design into code.

Organisational Issues

9. Is it important to have a very detailed specification and design before moving to implementation? If so, you probably need to use a plan-driven approach.
10. Are there cultural or organisational issues that may affect the system development? Traditional engineering organisations have a culture of plan-based development, as this is the norm in engineering.

Sources

• Sommerville Software Engineering
• Braude Software Engineering
• www.agilealliance.org 
• www.agilemodeling.com