Chris Boos on Automation

For arago as a small vendor and as a company servicing larger providers it is always a good idea to keep track of the big players and add that extra innovation and quality that makes a difference to the customer. This post will share some of our insight on how the big four vendors of IT service management deal with “automation” in their product portfolio.

To keep you from thinking we are out of our minds going against vendors like HP or BMC I can safely say that we work together very well with e.g. IBM and that none of the big four has anything close to our IT autopilot. In fact the big vendors seem to aim more at building tools for technical staff. Following this sentiment most automation tools from these vendors take some tedious task and drastically reduce the number of keystrokes you need to perform it with their tools. This approach is commonly sold as automation. Rightfully so, because some manual effort is performed by the machine after the tools are properly implemented. Another main feature of such tools is guiding the actions performed by IT service management staff by enforcing policies or providing runbooks and thus reducing the margin of error. But with all these tools the brainwork is still done by the guys sitting in front of the screens. IT experts only get a park of instruments to play on, rather than something that will play the basic rhythm and the background music automatically, letting them focus on playing the lead instrument. The autopilot approach as used by us and as described before provides for the intelligence that plays the background music on all available instruments. The only larger vendor I have come across putting some of the brainwork into their tools is EMC with its intelligent root-cause analysis platform SMARTS (now EMC Ionix Operations Intelligence).

So let us take a look at the product portfolios of the big four vendors in IT service management – BMC, ca, HP and IBM – and how they deal with the “automation” buzzword:

Table 1: Big Four - Automation Tools

 The concept of automating a single “kind of task” at a time automatically leads to many facets of automation. So there are many different “kinds of automation” on the IT service management tool market right now and many of the ITSM experts keep talking about these different automation concepts as a kind of baseline. There are for example ff0000;">data center automation000000;">, runbook automation000000;">, process automation etc. 

 Table 1 shows what tools from the big four vendors support which kind of automation approach. I will not go into philosophic descriptions on the different kind of automation. You may follow the links and take a look at some of the blogs I read where intelligent guys have wrung out their brains to come up with a definition. You can however see that you will need quite a few tools when you are trying to automate everything possible. You can clearly see who is hunting which buzzword with their latest acquisition or newest product. Every vendor except CA has focused their efforts and put or is currently putting a lot of work towards integrating their solutions. ca has acquired a zoo of very good tools and thus has the ability to provide any kind of automation tool approach. It is viable to ask about the outstanding integration aspects however.

Table 2: Big Four - ITIL Support

 As looking at buzzwords usually makes my eyes hurt, let us take a look at the actual work that is done in IT operation from an ITIL point of view. The operational processes at the core of ITIL v3 (and V2) are Incident Management and Problem Management as reactive processes, Capacity Management and Availability Management as proactive processes and Change Management as the only way to modify the IT currently in service. As automation should focus on taking all or at least some of this operational workload we have looked at the same vendors and checked which tools you need to support each of these processes that make up the everyday life of IT service management staff – see table 2. You can see that in order to support all your operational processes with automation approaches you will need the whole park of instruments to play on. I have come across many companies trying to minimize the risk of vendor lock-in by supporting different parts of their operational processes with tools from different vendors. Well, this is replacing the risk of vendor lock-in b the risk of bad integration plus it is giving away all the thought some of the brightest engineers have put into integrating one vendor´s portfolio.

In my opinion, if you really just want the instruments to play your IT service management band all by hand, you should at least get the instruments that are delivered in tune. But if you want better results, you should only play the lead instruments and leave the background music to a machine – that itself plays the instruments available. If you have this machine (the autopilot and/or other more solutions that do some brainwork), integration becomes a 2nd tier problem and you can go along with a heterogenious toolset. As there is no legislation concerning the working conditions for machines, there is no problem in bothering such a machine with sub optimal inter-vendor integration. The reaction speed of the autopilot will make up for the few extra steps needed to bring tools from different vendors into tune.

I finally find the time to write to you on the integration of automated IT operating into today´s working environment. One should think that automation means that just some other tool will be installed into the void of the IT service management jungle and maybe some administrators use this tool and become a lot better and a lot faster. Actually that is exactly what I am NOT talking about. If you are interested in my opinion on the “Automation Market” you will read an article here soon. So what I am talking about is more of an auto pilot – a machine that actually looks at problems and chooses to take action.

The question I am always beeing asked is ‘does this integrate with an established working environment and established processes e.g. ITIL?’ Yes it does. Indeed it needs established processes to function properly. An automation engine like ours (ff0000;">aAE) basically replaces the initial contact to IT experts.

Fig 1 - classic ITIL incident management

Let us look at the ITIL V2 and V3 incident management process for example. As you probably know (see figure 1) a normal ITIL incident management process is either initiated by an alarm from some monitoring system or by a user contacting the helpdesk. The helpdesk handles all the bureaucracy and then passes the incident on to the IT experts who will perform further analysis – if required collect additional data and perform additional analysis – and then either take immediate action to produce a solution or initiate a change process to take this action. As you can see in figure 2 in an automated incident management process the automation engine takes the place of these IT experts. This also includes the engine communicating with the helpdesk, performing additional analysis, requesting additional data, documenting its actions and so forth. When the automation engine cannot find a solution it will contact the IT experts and ask them to step in. Only this time the experts will get a well analyzed incident with most of the boring work and analysis already done and well documented so they can actually work on something new and interesting.

fig. 2 - ITIL incident management with automation

This is how we introduced this auto pilot into our own ITIL compliant IT service management unit. We promised the real technical experts that they would never be bored to death by everyday tasks and tedious busywork. Instead the engine puts only these problems on their desk where an expert as such is actually required and can use his or her talent instead of just keeping mindlessly occupied. If you want to read some more on the human element and concerns connected with the introduction of automation you might want to look at the article “Plays Well with Others” written by Ellen Fussell Policastro last August. In this Article automation is looked upon not in an IT sense but in an industrial sense. This environment deals with change more practically than just IT and therefore it is probably an early adopter for the automation change on its way now.

So you can see that in an environment with well defined processes it is very easy to place an automation engine or an IT operating auto pilot. In an organization that does not have IT operating processes in place yet, just finding the proper interfaces for the automation engine and redefining the roles of the IT experts is probably a piece of work for Sisyphus.

fig. 3 - ITIL integration of automation

Incident management is just one example of how an automation engine that actually acts like an auto pilot can be integrated to dramatically reduce cost in IT operation while simultaneously increasing quality and making the jobs of IT experts much more interesting. As can be seen from figure 3 the automation engine places itself between CMDB with enterprise monitoring system and the process layer actually involving IT experts. This is not only valid for the reactive ITIL processes like incident or problem management but also for proactive processes such as availability or capacity management where our autopilot engine will itself invoke work load automation tools in order to up- or downscale an IT environment according to predicted usage and demand.

This level of integration into established processes and behavioral patterns of technical advanced staff is very rare for a tool that radically changes the workload of IT operating teams, service managers. So this approach is one of the few roads available to actually move one step ahead in an environment that produces ever more complex IT applications, interdependencies between IT services and speed of change within the environment. Just think about the kind of pressure a fully cloud computing based banking data center would put on administrators…

They would have to cope with a dynamically changing environment, changing dependencies and rapidly changing communication matrixes. Automation could handle the ordinary tasks in such an environment without being pressurized by speed and changing preconditions and contact IT experts with exact and well documented information when an unknown issue occurs – relieving them of the pressure generated by the dynamic IT environment and making much more use of their actual expertise. If that is not what we want (I could not really see a reason) it is certainly what we need to keep up with a changing world without further demolishing the image of the IT-Crowd.

I wish you all a happy new year. This may sound hollow as the upcoming year is starting out with immeasurable uncertainties. A recession is unavoidable as the economic mechanisms are working their way through the different economic sectors and into everyday life. Given the origin of this recession – the financial industry with capital being one of three pillars of our economic system– even systematic change may be in store. The greatest problem is decisions being held back due to these uncertainties thereby creating an even greater economical impact. Thus what we definitely are feeling as a crisis is a powerful well of change. This well will flood through economy, society and of course technology. We will need strong decision makers and innovators – real entrepreneurs – to embrace change and make use of its power to tackle some of the grand challenges built up during the last 50 years.

For those of us promoting new technologies the willingness to embrace change is often the biggest obstacle in putting these new technologies to use. Think about the argument of how cloud computing cannot be a good thing because it changes the relationship between our data and our computations we are so much used to. Or think about bringing the concept of automatic system operation to the administrators who will no longer be just operators but turn into system experts. All these high tech concepts require a dramatically changed way of approaching everyday problems and those of us implementing these new technologies know that inventing the technology is less than 50% of the way. The biggest challenge is attracting enough interest in all players the new technology touches, in order to make them embrace the required change to effectively make use of the new technology. The current situation may prove to be one of the most potent accelerators for technological change possible. So to all of you – those who invent, implement, decide upon or just make use of new technologies – make wise, well thought of and brave decisions embracing change. You will be the ones who will contribute towards a speedy way out of the current uncertain situation.

After giving you so much leeway ( ) by posting a few personal stories from the past summer to past autumn we are all back to business and I want to share some of the reading and thinking that I have done during the quiet time between Christmas and New Year´s Eve in the articles coming up this week. I will start out with a little catching up on the “clouds are bad discussion” started by Richard Stallman with an interview given to the Guardian in September 2008. I do believe there was a good deal of stubbornness and corporate mistrust behind condemning the cloud concept as you will read. I will then continue with a post on integrating the concept of automation – rather than just tools – into IT operation processes and tool infrastructure. After you have read Roland´s post on “Automating What?” in November you may be interested in how the concept of automation is integrated into everyday IT service management and how our concept of e.g. an automated incident management is incorporated into a working IT environment. Following this post I will try to show a landscape of technology and tools and the way the ongoing development is focusing in on automation as a concept. This process was started when tools were used to ease the manual process of maintaining system functionality (e.g. system management tools) and continued by the automation tools that enable complex changes to be performed by entering a simple command (e.g. change automation or run book automation tools). The process is now at a point where actually decisions are taken by the automation software (e.g. what hardware is used to do what tasks by which is decided by workload automation tools) and will finally come to tools that make use of all the experience of system administrators in order to automatically decide how to keep systems alive. Thus automating incident- problem- capacity- and availability management. This kind of tool is what we have been using and developing for quite some time now ( see the aAE) and the post will show how this kind of tool integrates with the whole landscape of tasks and tools involved in IT service management.

Some time ago I published an article on the future of IT operation after we are through with all the ITIL implementations (still) taking place. Assuming that all the nice failure handling, proactive failure avoiding and communication processes like Incident, Problem, Capacity and Availability Management are in place, implementing automation is the logical way to move ahead. Compared to implementing ITIL automation actually changes the things that are done and the way they are done. As you may have guessed this statement alone was fertile ground for interesting and heated debating.

Generally the article concluded that implementing the ITIL processes concentrates on the interfaces between IT experts, clients, business requirements and the like where automation concentrates on the way IT operation is actually “produced” (in an industrial meaning of the word). Even though these two may be viewed separately the article shows how an automation environment highly depends on monitoring and IT component data. An ITIL environment puts forth a valid definition of both data sources for a complete IT environment and is therefore a good foundation to start implementing automation.

An IT operations environment with implemented ITIL processes also has common interfaces to the acting staff members. This makes it very easy to “inject” a new entity – like an automation engine – into the whole system. In such an approach the automation engine wraps itself around the data sources of CMDB and monitoring systems. All communication that would today be directed towards human recipients is handled by the automation engine first. Only if the automation engine is not able to complete the task the IT experts are involved.

This short description reveals how well an ITIL implementation prepares an IT organization for implementing automation.  It also shows how automation is made completely transparent to the business using the IT – as the automation engine acts like any human entity taking part in the ITIL processes.

The article itself gives a short overview of the “operational” ITIL processes and how their implementation builds the foundation for automation. If you are interested you may read the whole text here.