In the traditional infrastructure working model, the infrastructure team was in charge of the operational tasks, which were all manual. These tasks were repetitive and followed the same steps for their resolution every time. Environment building or decommissioning activities were considered as projects that came as requirements in a form of templates. Multiple meetings were required with stakeholders to finalize the requirements. It took weeks to build the servers with all the required configuration because different skills from individuals were required to complete their tasks and there was not much coordination because people were part of different teams and each team had its own set of priorities. Moreover, the requirements kept changing when a lot of rework was needed. All kinds of planned activities were considered to be projects. There was a need for faster delivery along with changing customer expectations. In the new working model, infrastructure operations teams not only are caretakers of the systems and environments but are also responsible for introducing automation and streamlining processes for development teams. The requirements are called epics, which are further broken down into user stories and tasks. There are planned, streamlined meetings and set delivery patterns. Teams communicate more and are closely connected. Moreover, entire teams work toward a common goal. Scrum is a methodology that is well suited for teams that are working on “change the business” (CTB) activities like infrastructure as code. The concept of infrastructure as code (IaC) has been accepted in most organizations for more than a decade now, and it has helped teams to do the following:
The implementation of the Scrum process is similar to the way it is implemented in a development project. Infrastructure teams that are accountable for automation need to plan for its implementation from every perspective. A well-defined strategy that encapsulates people, process, tools, and automation will ensure that the teams learn and scale up quickly.
| People |
|
| Process |
|
| Tools and automation |
|
The Scrum methodology is a generic framework that can be implemented easily for product development teams as well as operations projects that are deploying infrastructure as code or are automating standard operating procedures through runbooks. The framework has defined roles, ceremonies, and responsibilities that foster a culture of iterative development, trust, and transparency between team members. Sometimes the infrastructure team designates a small team as the DevOps team whose core goal is to strengthen automation and set up infrastructure as code that repeatedly delivers functionalities in small sprints. The team works closely with the business and important stakeholders who share requirements in the backlog that are assigned a priority and that follow the regular agile product lifecycle.
The framework runs in sprints and continuously delivers value.
Product planning: This is the product backlog creation stage. The product backlog is a queue of requirements that are shared by the stakeholders. In this stage, the customer and stakeholders interact with the product owner and share the requirements. In the agile world, these are high-level requirements called themes or epics. The product owner understands the asks and states the priority and importance of these needs.
Product backlog grooming: The sprint planning session is where the team picks up the high-priority requirements, provides estimates, and gets started on the sprint. The product owner reviews the requirements with the Scrum team, including the Scrum master. Also, they further details the high-level requirements or epics into smaller units called stories.
Sprint backlog: A sprint backlog is another requirements queue that is a subset of the product backlog. At the end of the sprint planning session, the stories are moved to the sprint backlog from the product backlog. The team is assigned stories and meets daily to address the progress.
Task execution: Stories are requirements that are the smallest unit that has to be implemented, and each story has one or more tasks that state the activity or work to be done. Teams update their stories on a regular basis.
Daily meets: This is a daily meeting known as a standup meeting. The Scrum master drives the sprint cycle and embraces changes and addresses any team issues. Every day the team meets to share their work status and discuss any risks or showstoppers.
Sprint review: This is known as a sprint demo or review. At the end of the sprint cycle, the work is demonstrated to the product owner, and all the stakeholders’ feedback is collected and tracked. The outcome of a sprint review meeting is a “go” or a “no-go” decision of the MVP that is produced. These MVPs could be runbooks, scripts, SOPs, etc.
Sprint retrospective: This is the last meeting of the sprint cycle where the team meets again to study the cycle flow, the stories that were completed and approved, the stories that could not be completed, or the feedback that was shared by the relevant stakeholders. All the lessons that were learned act as inputs for improvisation for the next sprint cycle.
Let’s further detail the roles, artifacts, meetings, and practices that are important in the Scrum model.
To implement the Scrum methodology in the infrastructure IT ops world, the following roles are needed.
Product owner:
Stakeholder:
Scrum master:
Scrum team:
A work item can be visualized as a deliverable. Unlike in traditional development where a requirement was analyzed, designed, architected, developed, tested, and deployed in one go and followed the work breakdown structure (WBS), in the Scrum world the work items are iteratively delivered. So, an epic is a big requirement that must be delivered. This is broken down to features and then further to actual requirements called user stories, which are linked with tasks. While epics and features can be spread over multiple sprints, stories and tasks are tied to a specific sprint.
All epics are saved in a backlog. These epics and user stories are prioritized by product owner. There are three backlogs: product backlog, release backlog, and sprint backlog. A prioritized epic is pulled from the product backlog and is put into a release backlog, which in turn will have features and user stories. The sprint backlog is the lowest level, which is a backlog for the current sprint. Epics and user stories stay in the backlog until they are prioritized and moved into the release and sprint backlog as agreed on by the product owner and the team. Once we have them in the release backlog, we start estimating on the delivery timelines. Story points are assigned to the user stories, which are a measure of the amount of work that needs to be done to accomplish the story. There are many ways of doing story point estimation such as planning poker, T-shirt sizing, etc.
The first phase to start with in the Scrum project is the discovery phase. This is the phase where project requirements get discussed and user stories are being written in parallel to get an initial confirmation on them. After the discovery phase, the sprints get started.
When we talk of sprints, it is a 3-4-3-week fixed iteration that has a sprint goal to be achieved by the end of the sprint. You can also understand them as an iteration. In a development team, each sprint could be delivering application functionalities like in sprint 1 the goal may be to launch the basic cloud platform features and in sprint 2 the goal may be to add new functionality like security and compliance on the cloud. So, with subsequent sprints, the platform grows and offers more features and capabilities. The question on which functionality goes first is answered by the product owner and the stakeholders who are in constant touch with the customers. Compared to a development team sprint delivery for infrastructure as code, the team will involve delivering the infrastructure setup in iterations/sprints, like in sprint 1 the goal may be to autoprovision test environments with the minimum specifications on a particular cloud. For sprint 2 the goal may be expanded to cover the autoprovisioning of test environments with custom specifications on multiple clouds and so on. Thus, the idea is to deliver working features that are well tested and deployed. Since infrastructure projects are different than application projects, the size of the Scrum team and the duration of a sprint are areas that may differ from the way application development teams are organized in Scrum. Because of a lack of empirical data, it is prudent for organizations to take the recommendations and tweak them to suit their needs based on the kind of projects getting implemented. In general, the infrastructure as code type of projects will align more or less to software development project sprint cycles and team sizes; however, the more complex infrastructure projects may require tweaking of the team size, skills, external expertise, and sprint cycle.
Each sprint has four ceremonies that it follows.
The first ceremony is the sprint planning meeting. This is the meeting where the sprint scope gets finalized. It defines which stories will get picked up based on the total story points that the team can accomplish based on its available bandwidth. The artifact that gets defined in the meeting is a backlog, which lists the user stories for the sprint. The product owner must be present along with the Scrum master and team.
The second ceremony is the daily standup meeting. This is a daily 15-minute meeting where entire team participates and answers three powerful questions: What did they accomplish yesterday? What do they plan to do today? Do they have any blockers? This helps in bringing transparency in the team and gives a sense of ownership and motivation to achieve the sprint goal. The product owner need not attend every day but can join in when needed. The Scrum master and team participate in the meeting.
The third ceremony is the sprint demo or sprint review. The sprint product or increment is presented to the product owner by the team. This is a time to get a feedback if the team could not get it earlier.
The fourth and last ceremony is the sprint retrospective. This is a time where the team reflects on what went well, what did not go well, what could have been done better, and action items. This helps the team to improve in the next sprint.
The way we track and control in other SDLC models, we do the same in agile as well. Information radiators are used for tracking the status of the release and sprints. Let’s see some of the useful information radiators:
Here are some best practices: