Scrum

The post provides a high-level overview of the challenges to be considered when applying the Scrum delivery framework to larger scale projects. Scale in this context relates to the size of the product being delivered; one product only. Scale here does not relate to multi-product deliveries, distributed team considerations, technical complexity or user concurrency. The term product could relate to a software product, a service or indeed any tangible outcome. The Scrum framework is used worldwide in a wide variety of contexts including fighter aircraft development, farming and education in the classroom.

In essence the Scrum agile framework is founded on the idea that self-empowered, cross-functional teams delivering working product increments in business priority order produce a better outcome in a quicker timeframe than would otherwise be achievable. There’s considerably more to Scrum than this statement affords, however for brevity this context sufficiently sets the scene.

In practice many implementations dilute the value of the Scrum framework by introducing elements of traditional Waterfall style linear process; such hybrid projects are often a disaster as aligning two process model takes discipline and care, in most hybrid cases the very opposite is true. As such the real value of Scrum is achieved through adherence to the framework in its entirety; subjective preference and selective pick-and-mix type implementations are fraught with risk and equally inefficient. Scrum has key advantages over many frameworks in this respect; the principles and practices are clear, short and intuitive and therefore the learning curve is short and initial adoption straightforward.

Before moving to discuss the main topic of this post i.e. Scaling Scrum, it is worth considering the basic mechanics of the Scrum flow.

1. A Product Owner manages a product backlog of user stories prioritised by business value; the higher priority the more detailed the story may be. Lower priority stories may be very high-level and are referred to as epics.
2. A ScrumMaster supports an empowered team to deliver a set of stories within a set timeframe, i.e. sprint cycle. The definition of delivered (or done) is agreed up front but typically equates to production quality working software that satisfies user defined acceptance criteria. The ScrumMaster ensures the team is focused on a defined sprint goal and resolves obstacles.
3. During a sprint the team synchronises on a daily basis during a stand-up meeting; team members make commitments not to the ScrumMaster but to their peers.
4. After each sprint cycle a retrospective meeting is held to inspect and adapt the process.
5. On a continual basis the Product Owner inspects and adapts the user stories on the backlog – adding emerging detail.
6. One sprint leads to the next on iterative basis within the context of a release.

The main sprint artefacts are the Product Backlog (whole-product) and Sprint Backlog (agreed stories for a sprint cycle). Velocity reports are also significant in terms of understanding how accurate estimation has been and how productive the team are.

The typical Scrum team size should be in the 4-8 region a larger team can detract from effective communication and add risk to adoption of the Scrum values of Focus, Courage, Openness, Commitment and Respect; all of which work better in a small team setting. Additionally the Scrum events (ceremonies) start to become less effective the bigger the audience. This is true for any two-way collaboration; effectiveness is inversely proportionate to audience size. And so, if the optimal Scrum team size is less than 10 yet the framework is applicable to large, complex project delivery how does Scrum scale?

Scaling Scrum – Challenges

Before considering approaches to Scaling Scrum it is worthwhile first to consider some of the key challenges likely to be encountered.

1. Focus. The level of difficulty in maintaining an effective product backlog is linear to the scale of the product. Scrum requires a whole-product mindset; the bigger the product the more challenging this can become.

2. Communication. Sub-dividing a larger project across multiple Scrum teams introduces communication boundaries. Face to face communication, a key Agile principle, can be difficult to maintain. More teams typically means more time spent on synchronisation (i.e. communication), this overhead is unavoidable.

3. Interdependency. Product backlog items are seldom standalone concerns with no interdependencies. Interdependencies can be managed effectively within a single team environment, for a multi-team environment interdependencies introduce process overhead.

4. Empowerment. Perhaps a more human factor than the preceding points but important nonetheless. Multiple team environments reduce the empowerment of the single team. This can impact upon the motivation and courage of the team as consideration (respect and so on) to other teams becomes a factor. Empowerment in this context is therefore limited in a manner not encountered by a single team.

5. Scrum thinking. Scaling challenges can often result in non-Scrum practices or adaptations creeping in. This can be perception related “Is Scrum still appropriate?” or a lack of confidence in respect to how to approach the problem in the face of proven linear alternatives.

Scaling Scrum – Typical Approach

A typical approach to Scaling Scrum is to sub-divide the product backlog into themes (or features) and to introduce a Scrum team per feature, i.e. a Feature Team.

1. Focus. Ideally each Feature Team would have a dedicated Product Owner perhaps operating within a Product Owner hierarchy with a Chief Product Owner at the top. The whole-product mindset requires a single product backlog, multiple product backlogs would require cross-prioritisation and as such the approach offers no value. To ensure manageability each Product Owner will have feature specific views on the product backlog. Feature rollups will also be added to provide Product Owners at higher levels of the hierarchy with a summarised view. Also in relation to manageability epic user stories play a big part in keeping the size of the product backlog within the 100-150 region.

2. Communication. A new Scrum event is introduced; the Scrum of Scrums. In short this meeting is focused on the discussion of dependencies and related problem solving. The regularity, duration and attendees for the Scrum of Scrums is not prescribed, instead need should drive all such factors. As with many aspects of Scrum, the Scrum of Scrum meeting can be hierarchical; for example groups of teams of related features may have a Scrum of Scrum meeting, selective attendees of which may attend a higher level Scrum of Scrum meeting at the product level. Release kick-off meetings and shared team members (perhaps on a cycle) are other effective means to make cross-team communication integral to the process. On very large, or very complex projects dedicated integration teams can be introduced to ensure communication and dependency challenges are de-risked.

3. Interdependency. In addition to the communication and resolution of dependency activity occurring within the Scrum of Scrum meetings, look ahead planning is often implemented. This approach requires that a rough idea of the backlog items for the subsequent 2 sprints are tentatively identified during each sprint planning meeting. Historical average velocity can help in terms of sizing. With an indicative view of the sprint backlog well in advance dependencies can be identified and discussed well in advance.

Large Scale Scrum with LeSS

LeSS (Large-scale Scrum) is comprised of 2 scaling frameworks that extend Scrum; basic LeSS supports up to 8 teams, LeSS Huge supports scale into the thousands of participants on a single product delivery. In both cases LeSS provides an approach to scaling that is Scrum in both principle and practice.

LeSS
The LeSS framework is founded on the principle of whole-product focus; one overall Product Owner manages a single product backlog that is delivered by 2-8 teams each with a ScrumMaster. In this context, the scaled Product Owner is supported by multiple teams working directly with customers, business users and stakeholders. On larger scale products the Product Owner role will be focused more on big-picture prioritisation rather than detail clarification; supporting teams will perform the latter. LeSS Sprint cycles are aligned across all teams and whilst each team manages their own sprint backlog one and only one potentially shippable product increment is delivered. The Sprint planning event in LeSS is split into 2 defined events; Sprint Planning One (2 hours) where all teams collectively determine the backlog items for the sprint and Sprint Planning Two (2 hours) where an individual team (or multiple teams working on a common feature) define how the stories will get to done. The Sprint review event is the same in LeSS as single-team Scrum, but an additional Overall Retrospective event is added to provide 2 cycles of process inspection and adaptation; team-level and then product. The approach to communication and interdependency management is left at the discretion of the teams involved however guidance is provided in respect to observer attendance at team Daily Scrum events, multi-team meetings, Scrum of Scrum meetings and the use of Open Space and Continuous Integration techniques.

LeSS is not only a collection of principles and practices but also a process framework with a concrete set of rules (updated February 2020) to guide application.

LeSS Huge
LeSS Huge provides framework support for product deliveries beyond the 8-team limit. The LeSS Huge approach starts with categorisation of the product backlog into requirement or development areas, each with a dedicated Area Product Owner who has an area specific view onto the overall product backlog. Sub-division by requirement areas is customer-focused, sub-division by development areas focuses on the architecture of the product. In either case each individual area can be considered a scaled-up feature team which will be delivered by a (basic) LeSS implementation with up to 8 teams. With LeSS huge sprint cycles are aligned across all areas and a single product increment delivered, not one per-area. Sprint review and retrospectives are held at the area and product levels.

LeSS Huge is a significant undertaking and will have a long adoption cycle.

In conclusion the LeSS frameworks provide a structured, empirically validated extension to Scrum to support product delivery at scale. Given the increased risk LeSS implementations should be guided by expertise and experience with the LeSS principles, framework and rules. The Certified LeSS Practitioner certification and related course should be considered as part of any process risk mitigation strategy.

Scrum@Scale

Scrum@Scale is a framework for multiple Scrum Teams, authored by Jeff Sutherland, which is designed to be lightweight and based on Scrum principles and values.

Scrum@Scale contains two cycles which separate accountability for how work is done from what work is done; the ScrumMaster cycle and the Product Owner cycle. This clear separation reduces organisational conflict and provides the basis for optimal productivity and the coordination of combined effort toward a single goal.

The ScrumMaster cycle (the how) focuses on continuous improvement, impediment removal, cross-team coordination and deployment. The Scrum@Scale framework uses Scrum of Scrum (SoS) principles to organise multiple Scrum teams working together to deliver an integrated set of product increments at the end of every Sprint. A new role, the Scrum of Scrums Master is accountable for the delivery and communication aspects of the combined Sprint, an impediment backlog is introduced to prioritise impediments across coordinating teams. A Scaled Daily Scrum Event is attended by representatives from each team to ensure synchronisation on impediments, dependencies and improvements.

The Product Owner cycle (the what) focuses on strategic vision, backlog prioritisation, decomposition and refinement and release planning. The Scrum Product Owner role is expanded to a Product Owner Team which is responsible for communicating the strategic vision for the product, business alignment, prioritisation of a single product backlog and release planning. A new role, the Chief Product Owner coordinates priorities with the Product Owners that work with each Scrum team within the Scrum of Scrums.

Scrum@Scale introduces two new executive teams. The Executive Action Team serves as the organisation level ScrumMaster with overall responsibility for impediment removal and process quality and continuous improvement. The Executive MetaScrum Team fulfills the organisation level Product Owner role with overall responsibility for aligment to strategic priorities. The Executive MetaScrum Team holds a stakeholder alignment meeting every Sprint; the MetaScrum Event. This event provides an opportunity to make strategic decisions that affect the entire organisation. The Chief Product Owner presents the Product Backlog to the Executive MetaScrum Team in order to support decision making in respect to strategy, funding, resource planning and customer, market or regulatory commitments.

The ScrumMaster cycle and Product Owner cycle have two touchpoints; team level process and product/release feedback. The former simply relates to the non-scaled Scrum process where the team self-organises to improve the velocity by which product backlog items are delivered in a releasable state. The latter relates to continuous improvement in two areas; product feedback impact on the product backlog and release feedback impact on the deployment process. For both touchpoints metrics (improving velocity, quality, team happiness etc.) and transparency (absolute honesty) are key to success.

In conclusion Scrum@Scale follows a scale-free architecture where there are no artificial constraints applied and scaling can follow an organic path, based on actual requirements and time-scales. Whilst simple in concept, the practical application of Scrum@Scale will require significant commitment at all levels of the organisation.

References

Scrum Alliance
Scrum Guide
The LeSS framework
Scrum@Scale

More often than not quality assurance is considered to be an implicit benefit of agile process with no specific techniques or practices applied. Integrating effective quality assurance can be one of the most challenging aspects of applying the Scrum framework.

Before considering how quality assurance factors can be applied to the Scrum framework, let’s start with a clear definition.

Quality assurance (QA) is a set of practices which collectively ensure that standards of quality are met (i.e. technical standards) and specified requirements are delivered (customer or user needs or expectations). Standards of quality may include for example measures of correctness, integrity, maintainability or usability. Quality assurance does not mean software testing; this is one element of a quality assurance process. Effective QA means there are limited defects in the first place. The core elements of a QA process include standards, audits, solution governance, testing, defect management (analysis, tracking and resolution) and communication.

Scrum Concepts

The following points describe how QA should be considered in relation to some of the core Scrum concepts.

Developer. In Scrum terms this means any project contributor such as Software Testers, UI Designers in addition to Software Developers. This term is obviously confusing and a source of misunderstanding.

Scrum Ceremonies. QA should attend all meetings (Sprint Planning, Product Backlog Grooming, Stand-ups, Reviews and Retrospectives). A team-wide, test-first philosophy is often the driving factor behind high performance Scrum teams. This concept is also referred to as acceptance-test driven development.

Working Software. This means fully tested quality software – not code complete.

Definition of Done. The DoD must include strict criteria for testing and quality.

Customer Satisfaction. Conditions of Satisfaction (or Acceptance Criteria) defined for each User Story drive the quality assurance process.

Scrum Artefacts

The following points describe how QA should be considered in relation to some of the core Scrum artefacts.

User Stories. The User Story is a concise description of a feature told from the perspective of the person who requires it. User Stories are best considered a reminder of a conversation and should be limited to the detail required for effective estimation and prioritisation.

The following (INVEST) factors should be considered in respect to the quality characteristics of a User Story;
Independent – can it be delivered in a single sprint, or do dependencies exist?
Negotiable – the user story is not a specification (reminder of a conversation)
Valuable – is there obvious intrinsic value to the user?
Estimable – can it be estimated (sufficient detail, available knowledge)?
Small – is it sized appropriately (to fit in a sprint)?
Testable – are there appropriate Conditions of Satisfaction – or Acceptance Criteria?

The Conditions of Satisfaction (CoS) are most significant to QA. Each condition is a specific, user-focused requirement for acceptance and should be written using the Given-When-Then format (or heuristic technique) as below.

Given [context] – e.g. I have created a Customer Record and Sales Invoice.
When [event] – e.g. I attempt to enter a Sales Payment above the Sales Invoice Total.
Then [outcome] – e.g. I am warned that the payment exceeds the outstanding amount.

The high-level CoS will drive the team discussion of the User Story (in a test-first approach), a “Design and Test” whiteboard approach can be useful in this respect where the team consider both the design and testing approach at the same time. Each high-level CoS will be translated into lower-level (detailed) test cases (technical; unit and integration or functional).

User Story Test Cases. The Conditions of Satisfaction (CoS) for a given User Story provide a high-level framework for the definition of specific test cases. Such test cases should be structured to address a small, focused end-to-end workflow (or scenario) to avoid interdependencies and should enable collaborative and automated testing. Effective test cases provide a simple set of step-by-step test instructions (or steps). Note, the CoS covers the happy path scenarios only, test cases must be developed also that cover critical path, alternate path, negative path, technical, validation and edge case scenarios.

Product Backlog. The Product Backlog is a single, ordered list of requirements (typically but not exclusively User Stories) used by Scrum teams. The list is constantly evolving, business value ordered and estimated by the team – including QA.

The following factors should be considered in respect to the quality characteristics of the Product Backlog;
Prioritisation – are the Product Backlog Items (PBI) accurately prioritised to a consistent scale based on business value, ROI, interdependencies and risk?
Estimation – are the PBI consistently estimated (and benchmarked) by the team?
Well Groomed – is the backlog frequently groomed by the team?

Product Backlog grooming entails User Story discussion by the whole team (lead by the Product Owner) with a view to revising and refining estimates and adding details such as new Conditions of Satisfaction. Regular sessions should be scheduled for this purpose; ideally 10% of the Sprint time should be dedicated to this task. In practice this is rarely the case, however given the direct relationship between the completeness of the CoS to the efficiency of the quality assurance process it is important that sufficient time is allowed for effective grooming.

Scrum Practices

The following points describe how QA should be considered in relation to some of the core Scrum practices.

In Sprint Testing. User Story testing must be completed within the sprint – full-stop. The Definition of Done must include this criterion. Testing a sprint behind the development is common bad practice and directly violates the working software principle; additionally this approach builds up technical debt and defect debt to be resolved at a later time. Instead, the sprint backlog must be shaped to enable collaborative design, development and testing of each user story within the sprint. High performing Scrum teams will also automate some of the tests within the sprint. The following simplistic process describes how collaborative sprint development can be applied.

Developers and Testers working collaboratively – Team Swarming:
Write the test case
Write the code
Test together (developers and testers)
Fix the bugs (developers) / Automate the test case (testers)
>> next user story

Test Automation. The ability to automate the execution of test cases (i.e. unit tests and acceptance tests) is critical at scale. In order to quickly identify cases where the current sprint development breaks the code or functionality developed in a previous sprint it is necessary to execute the related tests efficiently. Test automation also supports evaluation of the impact of changes (What-if analysis) that can help with design decisions. Further QA related benefits of test automation include the ability to apply load testing (e.g. multiple concurrent users) and performance testing types. Unlike manual testing which must be conducted in-sprint, test automation can be added outside of the sprint and be developed a separate team. High performance Scrum teams will automate critical tests within the sprint.

Test data quality has a significant impact upon the effectiveness of testing. Random data simply doesn’t work and prefabricated test data is often written to expectations and is not representative. Using production data (real data) for test purposes can violate data protection regulations and comes with the risk of inadvertent communication to the customer, supplier etc. or PII or financial information leakage. To mitigate this an approach that combines real data with obfuscation (scrambling or scrubbing) techniques can produce large scale, representative test data sets without the risk. This can take significant time to perfect however the return on this investment will be high.

Note, in addition to automated tests, a quality test plan still includes manual testing to cover exploratory testing and usability testing types. Automated tests are systematic and unable to gauge the quality of the user experience for example.

Regression Testing. The ability to run test cases developed in previous sprints (and perhaps by different teams) enables the impact of a current development to be understood immediately. This testing practice is referred to as regression testing. Regression testing effort gets bigger over time, as with User Story test cases, regression testing should occur in-sprint and automation should be used wherever possible. It is good practice to reserve time at the end of the sprint to allow the team to complete regression testing, this should also include performance and security test cases. At scale, it is often necessary to be selective in terms of which test cases to focus upon. An effective but contentious practice is to reserve the final sprint for end-to-end regression testing – this sprint is referred to as a hardening sprint; the key risk with this approach is that defect resolution from earlier sprints is deferred to this sprint. The hardening sprint should be driven from the regression test plan not the product backlog and be managed by the QA lead.

Post Release Support. Once the product is released – and despite the QA rigour applied – it is inevitable that production defects will arise. Such defects should be reviewed first by the Product Owner to determine priority and relevancy. Where defects are to be addressed immediately (show-stopper bugs etc.) the resolution work should replace similar sized work in the current sprint. In every respect the defect should be treated in the same manner as a User Story. Defects can also be placed on the Product Backlog for grooming. Where the Scrum team is providing defect resolution it is good practice to retain velocity for production defects; this proactive approach reduces the requirement to pull User Stories from the current sprint that may be in progress.

There are typically two interpretations of the term “development process” – one being the tools, practices and methods applied in software development (i.e. methodology, plus build automation, standards etc.) the other being the process applied to get from requirements to working software (i.e. iterative or waterfall, plus how the analysis-design-build-test-release disciplines are executed). This post outlines one high-level approach to the latter in the context of Salesforce developments. The intent of this isn’t to be overly prescriptive, generally speaking each project requires its own defined process that factors in resources available (and their skills and experience critically) plus the nature of the work and the timescales. That said it is a truism that failed projects fail for a variety of reasons but successful projects are typically successful for the same reasons. A fundamental success factor being the adoption of a clearly defined and simple process – others being team empowerment and shared commitment.

The process above assumes an iterative process and focuses the initial iteration on the foundation of a robust data model, set of user profiles and permission sets, role hierarchy, record access model and statement of the reporting requirements for the project. Subsequent iterations improve the quality of the foundation over time, as new functional areas are developed. The data model in this context will include a statement of the org-wide defaults for each object and the specifics of each relationship (master-detail, lookup, mandatory lookup etc.). The record access model is critical – this shows how each user population maps to a user profile and role and how they gain access to the data required, i.e. sharing rule, Apex managed sharing etc. In my experience defining an approximate access model upfront and then refining during the feature build-out helps to avoid expensive refactoring later in the process and sets out a clear understanding for all contributors to the declarative and technical build. A piecemeal approach to defining a sharing model is commonplace – this rarely provides a clear and cohesive result. For the similar reasons defining a list of permission sets upfront ensures that user profiles are kept clean and focused, avoiding proliferation of profiles down the line. It may be surprising to see analytics such as reports and dashboards being considered during the foundation stage, this however is one of the primary inputs to the definition of a fit-for-purpose data model. I’ve worked on countless projects where reporting has been overlooked until a late stage, at which point it has become apparent that standard reporting features can’t produce the reports given the structure of the data. Ideally the data model should be designed from the outset to work well for both transaction processing and analytics.

A final point for consideration is the by-exception approach to identifying technical components. When breaking out the solution components required for a certain feature – expertise must be applied to ensure that standard product functionality or declarative options (workflows, reports etc.) are considered fully before bringing expensive technical options such Visualforce or Apex to bear.