Chris Boos on Automation

Not just for all of us who have to deal with day-to-day operation of IT the topic of automation seems to be of great interest. Naturally the interest of people maintaining systems and services becomes the interest of vendors. I had the pleasure of attending the HP BTO Talk in Frankfurt and was glad to find out, that automation itself is the main focus of HP´s system management efforts.

For the first time since HP acquired OpsWare in 2007 I was actually able to see the platform in a customer environment. Swisscom attended the event and demonstrated their efforts in network automation. More impressive was the presentation of Mr. Rossa from Wien IT, who was able to show how standard changes and standard procedures in provisioning were captured into the automation suite.

I have seen more complex provisioning environments but in the HP presentation on the OpsWare platform I could get a glimpse at the visualization and reporting offered behind the scene. Coming up from the network layer they really found a very intuitive way to show what is actually available and going on in an IT infrastructure.

The strategic presentation offered by Mr. Winkler from HP put forward automation as the key to the HP software strategy. I consider this corporate understanding to be a major advantage in market development – much more than all the thousand features us techies like to talk about every day. So in my opinion HP´s view of the future is absolutely correct:

Good IT operation is, when you see nothing of it

I was a little amazed to see that the actual automation of operational tasks as well as tasks dealing with incidents and problems are still in a fairly basic state. All the cute things we have been talking about in this blog are still in the vision only. Simple rules and actions can be applied but that is all. Compared to the field of automated deployment, standard changes and predefined tasks the automated reaction to upcoming problems is not in an advanced stage. Even though there obviously is a really fancy interface for cross platform command execution. This interface could actually be hooked up to an automation engine like aAE and voila, commands would go out to the world. I actually think we will give this a try.

All in all I have to say that the visualization is impressive and the strategic alignment of the softwarestack is convincing. I will keep a close eye on the things happening around there – even though integration all the new acquisitions may still take some time.

Well, what a very basic question… Should we be discussing automation engines, when we should not have trust in them automatically taking action? Surely not, and obviously we are discussing automation engines.

So why do I hear so much about the lack of trust towards automated actions? It may be a stunning change in the field of system administration, that some entity takes automatic action where normally a system administrator would have typed in a couple of commands up to now. And change always induces fear and prejudice. Questions like “do you really trust the engine to restart this business critical service?” are not really uncommon. Well why should the machine not do that? After all the only action a system administrator would have taken is to restart the whole machine instead of just the service?

This simple every day example shows the real problem: Trust

We seem to have a problem when faced with the necessity to trust a machine or some lower level of reactive “intelligence”. Maybe this is just due to the many science fiction books we have read on robots and machines gone mad. In the end we are the ones who gave the engine the rule set by which it acts.

Actually we trust in automation every day we step into a lift. Much more than that, we rely on hard wired automation when we breathe or when our heart beats. I think none of us would be too happy about the idea of having to think and act out every breath and heartbeat consciously and willingly. Not much difference in automated actions in IT administration - and just like you can hold your breath automated actions can be overridden at any time.

This sounds very logical, doesn´t it? But logic is not the drink for “unsinkable rubber ducks” (the term true believer nowadays it too closely connected to politics - and besides much less enjoyable). So a good argument usually does not help much. In order to get on with automation either management uses force or try to employ man´s oldest habit - laziness (maybe we could get entangled in a discussion on greed or laziness being around first). And do not get me wrong, great things like the wheel were invented because of laziness. And on the way, we build trust towards automation in a non intrusive way - i.e. everyone involved can discover for himself that automation helps and is not evil. So this is how it is done:

1. Setup the automation engine in full
2. Disable all automated commands and redirect them to a trouble ticket or service management tool.
3. Have administrators use this tool and hence make them see what the engine would have done.
4. After a while people will start to copy and past the commands form the trouble ticket or service management tool into the various command lines.
5. This is the time to enable automatic command execution. The connection to the service management or trouble ticket system stays as it is. So the commands executed are not in any way “block boxed”.
6. There will not be mistrust and all the discussions, bad feelings and politics attached to it.