We're DevOps, now what? – Part 2

“Company A” was large and the IT division in which I was responsible for transforming had more than 600 staff and in excess of 500 applications. The operations team, whilst they maintained the production environment and performed the software releases, consisted mainly of level 1 and some level 2 support. They didn’t possess the skill set required to script or automate the deployments, which was left for the development teams to define and create. There was little involvement between the two teams other than to walk through the run book prior to a release or in identifying and resolving production incidents.

The Type 6: DevOps Evangelists Team structure was adopted to help bridge the gap between the development and operations teams by helping to define some good practices, establish a reference model with one of the more forward thinking teams within the division and provide the appropriate training to upskill the staff.

The structure employed in my current role is more of a Type 2: Fully Shared Ops Responsibilities model.

Whilst the deployment scripts and the operational running of the production environment is done by the “operations” team, they also develop the application code by fixing bugs and other small enhancements. Both teams can perform production releases which is governed through the change and release management process. In my previous company this approach wasn’t permitted due to a segregation of duties policy, however with an automated deployment process and proper change and release management controls, there’s no reason why the development team should not be able to push code to production.

As indicated in my introductory article, I wanted to look at different aspects of DevOps and using experiences from my previous roles look to see how we can improve what we do in my new role. Credited to Jez Humble the co-author of the DevOps Handbook, C.A.L.M.S is an acronym that defines a conceptual framework for the integration of the development and operations teams and is often used as a maturity framework to assess what needs to change in an organisation to adopt a more DevOps approach.

C — Culture
A — Automation
L — Lean
M — Measurement
S — Sharing

I’m going to address each of these in a subsequent article, starting with the more difficult one — Culture.

DevOps is all about the culture of your organisation. When embarking on a DevOps transformation, most organisations tend to start by looking at tooling and how to automate the deployment pipeline. This probably occurs because it’s the easiest and most tangible thing for teams to grasp. And whilst there are some obvious benefits of doing this, this in itself is not DevOps. Indeed, creating a pipeline to speed up the delivery before the organisation is ready is likely to lead to more problems.

“Company A” had a minimal tolerance to failure with a corresponding change management approval process in place which often required 50+ individual approvals globally but how effective was it? Changes would rarely be rejected by these approvers and it was more of a box ticking exercise that took considerable effort from the development teams to chase these approvers in each of the regions. And if all of the approvers weren’t received before the CAB then the change would require additional approvals from senior management! Did all of these approvers improve the quality of the releases or reduce the number of change related incidents? Of course not. A risk based approach to change management should be adopted to reduce this burdensome process and pave the way for continuous delivery.

It’s important that when a production incident does occur, the appropriate level of monitoring is in place to quickly identify the issue and that all parties work together to resolve the problem and learn from the experience; continually improving. Having an attitude were people are willing to take responsibility for the issue rather than finger pointing and blaming others is part of the required culture.

Development teams typically do not often think about their application code running in a production environment and the effort required to keep it running smoothly. This could be from a logging or monitoring perspective, the creation of temporary files on disk or temporary database table usage, system integration failure or other general housekeeping. Having the operations team involved earlier in the development process can lead to these types of concerns being raised and addressed prior to the production deployment. The operations team should be working with the development team on establishing the operational environment and the necessary scripting to configure and deploy the application as it is being developed. Both teams working as one.

Experimentation and innovation should be encouraged to drive improvements, with all team members having a shared responsibility and a common goal for delivery of the best possible solution.

The topologies shown above describe the two different organisational structures and how they apply to DevOps and I believe the closeness of the 2 teams represent the maturity of the culture found within them.

What type of organisation are you and how do you think this relates to your team’s culture? I’d be interested on hearing your views.

This news article was originally published in full on the Medium news website.