In recent years, DevOps has become incredibly popular, from CIO to IT engineers. There are various industrial pain points that foster its popularity:
- The cycle of traditional software development model (such as waterfall) is too long
- It is difficult for the development team and operation team to cooperate as they don’t share the same responsibility
- The operation team pursues stability and fully controls the production environment via the “bulky bureaucratic process” (usually with ITIL process)
Many evangelists even claim that DevOps + Cloud is the future of IT operations management, and ITIL will die. However, this argument is like the last concluding sentence in a fairy tale, with a happy ever after.
But this is far from reality. After experiencing various groping, bumping and crawling pits, many companies despondently discover that DevOps is not the silver bullet for IT operations and maintenance, and does not provide a truly systematic solution! It is far from the happy ever after, and thus raises the question of whether DevOps actually possess the capability to replace ITIL.
Why DevOps can’t replace ITIL in operations management and services support?
(1) DevOps doesn’t provide methodology for operation management
In theory, DevOps' major recommendation for operation is to integrate the development and operation roles into one team to eliminate organizational barriers, and unify roles and responsibilities. However, DevOps does not provide any advice on how the integrated DevOps team manages its operations.
For any crucial business application operation, if only the redefinition of the personnel role, or rotate developers to operation task, but lack of corresponding strategy, procedures and management control, operation management will return to the state of “chaos”, to which ITIL was born to react. No one is responsible for the long-term quality of service, and no management framework is established to organize internal collaboration and improve the quality of operations.
When adopting DevOps, organizations recruit "DevOps engineers" and "full stack engineers", who are fromthose software development engineers in the past, the skills have not changed, the learning ability has not changed, the requirements for domain expertise have not changed, is the word of DevOps is magic wand? Humanity has not changed, the mode of division of labor has not changed, and the quality and stability requirements have not changed. Can DevOps eliminate the basic management methods of operation specifications, control procedures and management processes?
Operationally, DevOps advocates implementing CI (Continuous Integration) and CD (Continuous Delivery). Even though many people interpret the CD as Continuous Deployment, it is the first step for a new software to enter the production environment and start to run. But there is no concept like CO (Continuous Operation) in DevOps to cover managing operation phase.
Fig1: Comparison between DevOps and traditional mode
Therefore, DevOps framework does not involve much of operation management and service support.
(2) The main founders and evangelists of DevOps suggests the integration of DevOps and ITIL
Fig2: Cover of book The Phoenix Project
In the widely circulated DevOps evangelist novel The Phoenix Project, the author Gene Kim specifically explores the relationship between DevOps and ITIL. In the book, he argues that ITIL and ITSM are still the best codifications of the business processes that underpin IT Operations, and actually describe many of the capabilities needed in order for IT Operations to support a DevOps-style work stream.
(3) Many well-known Internet companies use the combination of ITIL and DevOps to achieve orderly and efficient operation and maintenance
On Quora, the world leading knowledge-sharing website, many people have raised similar questions like “Should ITIL Die?”, which fall under popular posts. One of the questions is as follows (see note 1) ：
Fig3: Discussion about ITIL on Quora
(4) The DevOps Master certification program launched by EXIN integrates ITIL for the operation domain guide
EXIN, the international standard training and certification institution has launched the DevOps master certification in recent years. ITIL/ITSM is one of the core modules in its knowledge map (see note 2) .
Fig4: EXIN DevOps Master Certification Domain Framework
Traditional ITIL is facing the five "sins"
Although many emerging concepts and models, including DevOps, are difficult to replace ITIL so far, traditional ITIL practice is facing more and more challenges:
(1) ITIL is not agile and faces difficulty to adapt to the fast-paced software development iterations and business changes
In the current enterprise practice, ITIL processes require more amount of approvals and supported documents. It is inevitable to give the impression of “bureaucratic style” and “heavy process” to the business departments and development departments. Of course, this is not a problem of ITIL, but because the process has been enforced on control points for a long time due to various reasons. It is always a one-way enhancement, and no one dares to reverse for simplification from time to time. It is needed to revisit all ITIL processes and related procedures for optimization and simplification.
(2) Poor user experience makes ITIL tools difficult to use
Many traditional ITIL platforms are old in architecture and technology, and the UI is outdated. It can't support mobile, location-based and social-based UX experiences and user-end devices. The process form needs to be manually filled in dozens of fields. Searching historical work order information is particularly difficult and time-consuming. The system reports are even worse than calculating the data manually. These are the ITIL phobias for the operations engineers and service users of the millennium generation.
(3) ITIL platforms are difficult to modify and customize for maintenance
Any changes to many traditional ITIL platform, even adjustments of color or font to forms require code changes, not to mention to process modifications, field creation, and KPI calculations. Today more and more non-ITIL standard processes and service process of HR, Finance, Facility function departments require quick implementation on ITSM platforms.
(4) Configuration management relies on manual input and manual auditing
As the quantity of IT systems and devices increases rapidly and the complexity increases significantly, the operation team must rely on CMDB to record and manage configuration information, so as to support scenarios such as fault impact analysis, fault root-cause analysis and location, and change risk assessment. However, it has proven to be impossible to manually maintain the accuracy of CMDB in time!
(5) It is difficult to satisfy requirements from dynamic management
The application of technologies such as cloud and container makes the operations team face the instantaneous changes of objects and operation scenarios. It cannot rely on the manual creation of work orders, manual triggering actions, etc. It requires ITIL process tools to support dynamic code/scripts calls and real-time integrations between itself and third parties.
Agile ITIL: Make the operation process agile, automated, and friendly
As DevOps can't replace ITIL, and traditional ITIL itself can't meet DevOps requirements, why not make ITIL agile like DevOps?
What are the characteristics of agile ITIL?
Let us use the core concept of DevOps to deduct and transform ITIL to be agile.
Fig5: Deducting Agile ITIL from DevOps
Key points of agile ITIL:
1. Change process automation and integration
As shown in the following figure, implement process integration with change and release automation as the core:
Fig6: process integration and automation
- The change & release process is key to agile ITIL:
- Process simplification: Evaluate existing change management process, and remove unnecessary requirements for approvals.
- Change category (standard, normal, major, emergent, etc.) downgrade: Evaluate each sub-category of the existing change process, whether its implementation can be automated, whether the risk is controllable, and degrade them into standard change or lower categories.
- Refinement of the standard change process: Evaluate the secondary and tertiary sub-category of standard changes, mapping the implementation activities into automated scripts or a flow of orchestrated scripts. This way, more and more standard change will be automated in execution.
- Process integration with change and release process:
- Incident process integration: For those incidents with low risk, low impact and standard procedures, the standard change process is automatically triggered by an incident, and the script for repairing is automatically executed.
- Service request process: Assess how to automate the service fulfillment of each service request type, and implement the automation for those types of service request. Integrate the service request with change and release process for service logging and trace auditing, and then call automation for service delivery.
- Software release process: Make the deployment activity automated by supporting automation platforms, and integrate into change and release process.
- Job management process: Left-shift the operational responsibility of batch jobs with the development team, and use job as code API to generate job tasks and schedule. Through integration with change process, they are automatically delivered under central automation orchestration management.
2. Configuration management process automation and integration
Fig 7: Configuration Management Automation Scenarios
Turn CMDB manual management into automated management with the following steps:
- CMDB initialization: Through automated scanning of the managed environment, information collection, relationship discovery and application system modeling of all configuration items(CI) are completed without manual input. Non-technical information still requires external input; application model and relationships still need to be refined based on auto-discovery findings.
- Scan and audit daily: With CI automatic discovery at daily (at least) frequency and audit, ops can automatically compare the difference between the configuration baseline data and the current actual data, and find CI changes. Verified changes can be updated to the CMDB after manual confirmation.
- Integration with change process: Scan the zones where the change will take place and take snapshot as baseline before the change is implemented, and scanned again after change implementation, then imported new information into CMDB with manual confirmation.
The core capabilities of the auto-discovery include:
- The number of supported devices and software (whether it can achieve the fullest possible coverage of common types, models and versions in the industry)
- The speed of discovery (to ensure accurate data is received on time from CMDB)
- Capabilities of support multi-cloud service providers, etc.
3. Agile communication and support services for users
Use technology that supports mobile, location-based, social, and other popular user preference to provide IT service users with support through multi-channels (App, instant messaging, Web, SMS, etc.) to continuously improve the user's digital experience and satisfaction.
Fig8: agile communication & supporting service samples
Furthermore, intelligent customer service (chatbot) can be used to provide personalized service support for users.
4. Process customization based on a graphical configuration
The process platform must be agile and capable of supporting rapid optimization and iteration to continuously meet the needs of IT operations and service support. A process platform that provides process customization based on graphical configuration will reduce the cost, time, and risk of process creation and modification, and serve production operations faster. The platform customization relying on manual code change is becoming a historical legacy.
Fig9: Sample of process creation based on GUI
5. Ops as code
Ops as a code has become a trend of IT operation and maintenance tools. Considering the scale of the managed object, the complexity of the architecture, the dynamics of the scene, and the requirements of real-time management, all operation and maintenance management tools must provide an external API, and the third-party system or code may integrate and manage the logical arrangement to realize dynamic real-time operation and maintenance.
Fig10: Ops as code Framework
Reference Architecture of Agile ITIL
To achieve the above agile ITIL capabilities and better implement the DevOps-style agile operation and maintenance, an implementation architecture is shown in the following figure:
Fig11: Reference Architecture of Agile ITIL
The system can be divided into four layers which include:
- User layer: Agile support and communication channels, support a variety of service technology such as mobile, location, social and other technologies, introduce intelligent customer service supported by AI, and support services in a more efficient, faster, and user-friendly way.
- Presentation layer: Agile operation and maintenance dashboard, define KPI for agile operation, display KPI real-time and historical data in various ways, and demonstrate the efficiency, value, and control ability of agile operation and maintenance for stakeholders and management team.
- Process management: Focus on the change release management process, automation and integration with other processes, and connect with the automation layer. CMDB configuration management based on the automatic discovery of CI information.
- Automation layer: The automation layer automates the implementation activity of the change and release process including:
- Enterprise-level central automation orchestration and scheduler: Realize logical orchestration of change and release tasks, change window scheduling, resource allocation, process supervision, automation action result feedback to the requested process, etc.
- Automated deployment tools: Dynamically deliver automated scripts to target objects, providing a run-time environment for automated scripts, and as a communication channel for automated scripting and orchestration schedulers
- Automated script library: Provides a low-coupling, fine-grained script library for easy programming of complex automation tasks
- Customized scripts: Customized development for users with special requirements that are difficult to implement with existing automated script libraries
- API interface: REST API interface provided by other third-party tools or scripts, called by scripts or orchestrator
BMC Agile ITIL Solution
BMC Software, the leading company for ITIL solution, can provide a full stack to implement Agile ITIL with the following products and modules:
Fig12: BMC Agile ITIL Solution
To adapt to the rapid changes in the digital era and when integrated with the DevOps development model, the traditional ITIL can be transformed, and the agile ITIL can be realized through process optimization, integration, and change automation.
Fig13: Features and benefits of Agile ITIL
Acknowledgement to Datta, Satarupa;Chatterjee, Bishakha from COE on content refinement!