Chris Boos on Automation

The term IT Automation has become a real buzzword over the last couple of years. With the economic meltdown still hanging over our heads, previously used potentials such as outsourcing and off shoring being a disillusionment , many IT managers have come to the conclusion that it is about time to deal with improving the “Run the Business” part of their IT themselves if sustainable results are to be expected. This little article will take you through the usage of the new “Automation buzzword” and will eventually show a solution on how to really automate IT operations.

The more IT turns away from being voodoo to being a business the more IT follows the idol of “the old economy´s producing industries” and this brings the term automation into focus. Normally this means giving a good tool to the IT operators (administrators) to make their job easier or to control their work. In other words the IT guys are buying a great shiny new hammer that will drive a nail into wood with two blows as compared to the ten the old hammer required. Or they install an assembly line that take an IT issue from one expert to the next until all of them have contributed their 10 cents to the final result. Of course IT cannot use such old style terms and this is why terms like Data Center Automation or Automated Provisioning describe nothing more than the shiny new tool and terms like IT Process Management or Operation Orchestration represent the optimized factory processes or assembly line approach.

But is this really IT automation?

Well if IT was the delivery of completely standardized products (service catalogue) and their different varieties this claim would be true. But most of the effort in IT operations today is consumed by dealing with non standard issues. This is because a long time before the shiny new tools were available, smart administrators were automating repetitive jobs in scripts, jobs and by using other neat methods – much less transparent, rarely reusable, but still automated. The unpredictable part of the administrative workload is generated when a “working system” (a fully functional productive IT environment) is changed following a client’s instructions or when a previously unknown challenge pops out of the blue and has to be dissected by several experts before it can be resolved. If we shift our focus from the workload created from such unpredictable tasks to the impact their time consuming resolution has on business, we can see that exactly these unpredictable tasks are the major causes for quality problems and can sustainably influence availability and performance over time.

These unpredictable efforts and the impact they have on business performance are not addressed by implementing standard IT automation approaches (installing a new tool or improving process controls). These tools are certainly worth their while and are responsible for ensuring today’s administrators and their managers can just about handle the massive growth of issues they have to deal with each year, but they do not address the basic problem of applying expert knowledge to unpredictable issues.

Looking at the IT industry in general and IT operations in particular, one must be reminded of the gold rush. Not only is the number of IT workers constantly increasing – despite ongoing crisises – the suppliers for this industry and their organic growth are in full swing. And this is why the established suppliers like to sell new shovels to the IT gold diggers – so long as this is possible. Only when the administrators realize that it is impossible to deal with their workload – not to speak of their personal ambition of doing something interesting and creative in IT – will big suppliers wake up and try to change their products. So why is the basic problem not being resolved? Well, quite simply because selling shovels (new tools for IT operation) is still big business and actual change is a big challenge.

Comparing IT automation with industrial automation is for the main part invalid. One should look at automation of IT operations from a different angle. Industrialization in the IT sector is moving forward every day, but since the major part of work delivered in this sector is virtual or at least intellectual, this new angle of looking at IT operations will have to focus on mental abilities like knowledge. Each and every one of us knows a machine that was built do deal with exactly such a challenge. This would be the autopilot in an airplane or other vehicles. This device (by the way first designed in 1914) has to create a result (keep a specific heading, etc.) even when unpredictable and unknown events and issues occur. It is the goal of this machine to keep the result intact as long as possible and react to change in its environment dynamically. Every one of us trusts these machines and these machines do nothing but take and execute decisions that would normally be taken and executed by a human. Only when the autopilot no longer knows what to do or is facing a problem that can only be resolved by innovation, creativity or even emotion, the real (well educated and experienced) creative human expert is contacted.

And this is exactly what IT operations should be like!

We have been working on and with such a machine for more than ten years now to solve workload overflow in our IT service delivery. With an 80% level of automation when dealing with standard environments (OS, network, etc.) and a 30% level of automation when dealing with individual applications, this approach has reached a satisfactory level of maturity. On top of actually eliminating manual intervention and giving our experts the time to focus their knowledge and creativity on interesting and completely new challenges, the faster reaction time of an IT autopilot are noteworthy because they reduce the business impact of upcoming issues. This results in a win-win-win situation for our customers, our staff and our company. Our customers get a better service at a better price, our employees have more interesting tasks to deal with and our company has a better margin.

By looking at the autopilot concept for IT operations it becomes obvious that a new angle of looking at a problem brings new and innovative solutions. Because in an overcrowded gold digging area you can have the best shovels on earth and still have no competitive advantage. An advantage would only be created by finding a new area to dig in or by creating a completely new method to mine the gold. This kind of a new approach is exactly what IT Autopilot for operations is. Looking at other industries and experiences from other sectors has created the difference between IT automation and autopilot for IT operations.

• Matthew – http://www.flickr.com/photos/bb_matt/
• Tony Oliver – http://www.flickr.com/photos/tooliver/
• Nicki Dugan – http://www.flickr.com/photos/thenickster/
• Andres Rueda - http://www.flickr.com/photos/andresrueda/

for providing their images under Creative Commons license.

For arago as a small vendor and as a company servicing larger providers it is always a good idea to keep track of the big players and add that extra innovation and quality that makes a difference to the customer. This post will share some of our insight on how the big four vendors of IT service management deal with “automation” in their product portfolio.

To keep you from thinking we are out of our minds going against vendors like HP or BMC I can safely say that we work together very well with e.g. IBM and that none of the big four has anything close to our IT autopilot. In fact the big vendors seem to aim more at building tools for technical staff. Following this sentiment most automation tools from these vendors take some tedious task and drastically reduce the number of keystrokes you need to perform it with their tools. This approach is commonly sold as automation. Rightfully so, because some manual effort is performed by the machine after the tools are properly implemented. Another main feature of such tools is guiding the actions performed by IT service management staff by enforcing policies or providing runbooks and thus reducing the margin of error. But with all these tools the brainwork is still done by the guys sitting in front of the screens. IT experts only get a park of instruments to play on, rather than something that will play the basic rhythm and the background music automatically, letting them focus on playing the lead instrument. The autopilot approach as used by us and as described before provides for the intelligence that plays the background music on all available instruments. The only larger vendor I have come across putting some of the brainwork into their tools is EMC with its intelligent root-cause analysis platform SMARTS (now EMC Ionix Operations Intelligence).

So let us take a look at the product portfolios of the big four vendors in IT service management – BMC, ca, HP and IBM – and how they deal with the “automation” buzzword:

Table 1: Big Four - Automation Tools

 The concept of automating a single “kind of task” at a time automatically leads to many facets of automation. So there are many different “kinds of automation” on the IT service management tool market right now and many of the ITSM experts keep talking about these different automation concepts as a kind of baseline. There are for example ff0000;">data center automation000000;">, runbook automation000000;">, process automation etc. 

 Table 1 shows what tools from the big four vendors support which kind of automation approach. I will not go into philosophic descriptions on the different kind of automation. You may follow the links and take a look at some of the blogs I read where intelligent guys have wrung out their brains to come up with a definition. You can however see that you will need quite a few tools when you are trying to automate everything possible. You can clearly see who is hunting which buzzword with their latest acquisition or newest product. Every vendor except CA has focused their efforts and put or is currently putting a lot of work towards integrating their solutions. ca has acquired a zoo of very good tools and thus has the ability to provide any kind of automation tool approach. It is viable to ask about the outstanding integration aspects however.

Table 2: Big Four - ITIL Support

 As looking at buzzwords usually makes my eyes hurt, let us take a look at the actual work that is done in IT operation from an ITIL point of view. The operational processes at the core of ITIL v3 (and V2) are Incident Management and Problem Management as reactive processes, Capacity Management and Availability Management as proactive processes and Change Management as the only way to modify the IT currently in service. As automation should focus on taking all or at least some of this operational workload we have looked at the same vendors and checked which tools you need to support each of these processes that make up the everyday life of IT service management staff – see table 2. You can see that in order to support all your operational processes with automation approaches you will need the whole park of instruments to play on. I have come across many companies trying to minimize the risk of vendor lock-in by supporting different parts of their operational processes with tools from different vendors. Well, this is replacing the risk of vendor lock-in b the risk of bad integration plus it is giving away all the thought some of the brightest engineers have put into integrating one vendor´s portfolio.

In my opinion, if you really just want the instruments to play your IT service management band all by hand, you should at least get the instruments that are delivered in tune. But if you want better results, you should only play the lead instruments and leave the background music to a machine – that itself plays the instruments available. If you have this machine (the autopilot and/or other more solutions that do some brainwork), integration becomes a 2nd tier problem and you can go along with a heterogenious toolset. As there is no legislation concerning the working conditions for machines, there is no problem in bothering such a machine with sub optimal inter-vendor integration. The reaction speed of the autopilot will make up for the few extra steps needed to bring tools from different vendors into tune.

UK-based start-up Phorm probably considered its software the hottest stuff in online advertisements since Google adwords. Now, Phorm finds itself at the center of a blamestorm where it gets bashed by EU regulators and web platforms such as Amazon or Wikipedia for violating privacy rights.

So what’s it all about? Phorm intends to cooperate with ISPs to monitor customer web usage. A test trial with BT has already been implemented. Based on the user profiles created, Phorm will display matching advertisements.

A short side note on the technology used: Phorm effectively creates the “mother of all cookies” by placing their cookie containing a unique user identification (UID) on every website you visit. Normally, a website can set cookies only for itself, not for other websites. Here’s the trick: The ISPs reroutes all incoming requests to a Phorm server that pretends to be the desired domain (i.e. www.amazon.com). Using this fake, Phorm is able to set its cookie and reroute again to the real desired web server.

So Phorm manipulates existing web protocols for cookie handling in a fashion that is more than dubious. “Big brother is watching You surfing” comes to mind immediately. Let’s keep bashing them, everyone.

On the other hand we should be aware that tacking users and adapting content to their profiles is nothing new. In fact, it’s a major trend in web 2.0 to create adaptive web sites for unique user experiences. We at syngenio have created EBIT 2.0, which adds user adaptive advertisement to online banking applications (without using cookies and without web tracking users).

Amazon itself constantly profiles its customers. Just the same thing as Phorm does, only limited to the www.amazon.com domain. The data is used to create the famous recommendations. But Amazon has also used it to exclude existing customers from price reductions targeted at attracting new customers. Not so nice.

So Amazon protesting against Phorm has little to do with user privacy. It’s about who owns a customer and the value hidden in customer data. Right now, it’s up to the individual web sites to make the most of it. (And many still have a long way to go … I can tell from my own discussions with the financial industry about their own web sites…) Phorm tries to bring in the ISPs. It’s as simple as that.

Which brings back an old question: Should cookies be considered harmful? It’s been so long since I last discussed that, I can’t even remember what happened to all the cookie paranoia from the early days of the web. I guess we just traded it in for nice features based on cookies. Well, what do you think? Responses welcome …

After reading about it on TechCrunch I just integrated Google Friend Connect onto 800080; font-family: Calibri;">www.hcboos.net. It might seem a little strange to add additional social web features to a blog – that is a social site itself – but I want to make it as easy as possible to give feedback, add comments and get together.

Google Socla Bar on this Site

After I spent this week entirely getting people together in actual meetings, I think all of us high tech “evangelists” can get great use out of these social features and we should really get down to using them.

If you want to register, just use the social bar at the bottom of the page.