Chris Boos on Automation

Time and again people ask me, what they see, after implementing an automation engine. My answer usually was “well nothing really, you will see that your applications have a better uptime…”, but obviously that is not what people want to hear. The whole idea of an automation engine is, that things happen in the background and no one has to sit in front of some console watching lights turn red.

Still people want to see what is going on. And as automation is a matter of trust – the trust of system administrators and managers, that such an engine will improve IT service instead of messing it up – it probably is a good idea to enable a peek under the hood of the machine. Actually as we are using a graph algorithm approach to finding the automatic steps to be taken in order to resolve a problem, it sounds like we should show a graph of the whole thing.

So that is just what we have done. In the screenshot attached you see the prototype of “aAE Visualizer”. This JAVA Application actually displays the IT model and the issues and events travelling the model. On the model Graph it is possible to see where issues are created and how they travel the engine in order to find actions to take. But this visualization application is not just a pretty way to let interested people “look at what is happening” in our automation engine; it also allows to locate hotspots in an automated IT landscapes easily. Hotspots always indicate a challenge. Either a hot spot is an error in the model – a place where problems travel in circles without finding any resolution – or a hot spot is an actual bottle neck in the IT infrastructure that is not visible from a capacity management point of view.

So I am very happy to announce, that this visualization application will not only make my job of explaining how “automation works” easier, but will also allow our administrators to locate model problems or IT landscape problems with much less effort than before.

Well, today we finished PULSE 2008 with the last two sessions and after the party last night the sessions were so small, that we really did enjoy some interesting face to face discussions about strategic development in service management, automation and clouds.

BUT, after that, we moved out to Universal Parks Orlando and did a lot of “dragon dueling” on the roller coaster. A really great ride (you can see it in the background). The inverted coaster makes you feel like you are shooting into the landscape…. It IS worthwhile to stand in line just a little longer to ride the front row – I did on both coasters more than once…

I’m working on a simple formula to describe clouds and cloud computing. After working through numerous blogs and posts, reading vendors press releases and visiting their websites, I feel confident enough to give a first try:

What do you think about this:

Cloud = Computers*x + Virtualization + Automation + Service

where x depends on the cpu power of the utilized machines.

I’m begging for comments.

Roland

Ok, I may not have been too enthusiastic in writing about our Pulse visit up until now, but as we just enjoyed a really great session about virtualized datacenters I thought this would be a good reason to start doing so. Ok, it’s quite a different kind of automation compared to what we deal with usually, but it was quite impressive to see a reporting service being installed “on demand” on a virtualization cluster during the demo part of the session by means of just a few clicks in Tivoli Provisioning Manager. Ok, so you may wonder what is so exciting about the automated installation of a virtual system? Well, aside from the fact that you felt that the speaker (Vanja Gorazi) was really deep into the technical stuff behind it, it was probably that the way this happened was simply done right. Instead of specifying you need an xOS with ySQL to support the zReporting system, you simply choose the type of service needed and do not need to worry about all packaging and stuff behind the scenes. Oh, and did I mention that automation didn’t stop with the installation but continued with automatic addition of required or removal of unused resources by means of a component named Intelligent Orchestrator? Of course, the rules engine behind it sounded quite basic, so I guess there may will be some more interesting perspectives around here… guess I’ll need some time to think about that when this is over…

Considering the fact, that only 30-40% of the energy consumed by a data center is used by the actual computational equipment and considering that another 30%-40% of the energy consumed by the IT equipment is converted to heat, only 21%-24% of the energy eaten up in data center is actually converted into computing power. Looking at these numbers from the other side means for each Euro spent on “computational energy” 4,76 Euro are spent on “overhead”. This ration can be improved by optimizing air conditioning, getting rid of heat hot spots or generally using energy efficient and modern equipment. Still it seems unlikely that this will help to get anywhere below 3 Euros of “overhead” for each Euro spent on “computational energy”. These numbers are taken out of the keynote presentation given by Steven Sams at IBM PULSE 2008 (Also check out the “Raised Floor Blog”, where Steven Sams is one of the authors)

On the other hand this means that reducing the energy needed for computational equipment will in absolute numbers decrease the excess energy consumption by a factor of more than 4. So improving the facility is a good start, but reducing the energy actually needed by computational equipment is the real price. The way to reduce energy needed is a direct result from capacity management. Generally speaking this means – in the best case – turning off as many components as possible – or if that is not possible, at least cutting their energy usage by slowing the CPUs or putting virtual instances into suspend mode until their service is really required. Does this sound easy? Well it does, but how does it work? Virtualization certainly is the key technology, but what good would virtual machines be, if their resources could not be allocated automatically depending o their actual use or – if you want to be cautious – by their predicted load and therefore by their predicted usage. A specialized set of rules is put behind process and operational automation, to perform the scale-down and scale-up of the virtual machine resources. This automation can even decide to turn off hardware, that is currently not needed or at least to slow the CPUs of hardware that cannot be turned off, but is in little use.

Modern “or very green systems” come along with special agent to detect energy consumption and usage deriving possible executions. But how about all the legacy applications – the applications that are running on more than 95% of all the components, using up energy in our data centers today? An automation engine that actually acts like an operator (someone who could manually cut down on power use) could examine the equipment in the IT landscape it is acting upon and execute general rules to reduce energy usage. By combining both technologies – the more effective combination of modern hardware and specialized software for new applications and a general automation engine for all the legacy applications – the power of virtualized components can actually be converted to green power. This is not just a fabulous business case, but it also is a good thing for the environment and hence for all of us.

Yesterday I had the chance to get a feeling for one of the hottest topics in IT infrastructure. A panel session at IBM PULSE 2008 was dedicated to the topic of Cloud Computing (even though IBM marketing people don´t seem to like the term and have come up with quite some innovative words – words no one uses, so let us stick with the cloud). The panel was buzzing with intelligence, unfortunately we as the audience could not really match up. So we listened to a pretty much directed discussion on how cloud computing would replace today´s approach to hardware and infrastructure in general. Well I do agree, no one needs dedicated servers when resources can be allocated dynamically and come preconfigured and interconnected. Kristin Hansen stripped the key features of a cloud down to simplicity (users do not care how their resources are set up, they just use them), mobility (obviously use is possible from anywhere and even a large computing cluster could be controlled from a phone like device) and elasticity (you only setup or pay what you really need). Sounds fine to everyone and Google and Amazon have definitely shown to the world that this concept works in a closed shop environment. According to Dave Lindquist IBM is working on a methodology and technology to make most applications “cloudable”. The most interesting remark I heard during the discussion was the “Cloud Computing is the combination of technology (virtualization and automation) and discipline (a stringent way of breaking down the offered services into small blocks in order to recombine them quickly and automatically upon the user´s request as well as defining standards or service catalogues to be offered)”. I guess the discipline part will put forth a great deal of discussions between process consultants and methodology consultants and in the end there will certainly be a couple of good ways to set things up. Just as certainly there will be the need to standardize these processes and methodologies in the end, so clouds are not proprietary but keep mobile even between cloud providers.

Naturally I am more interested in the technology part, that is needed behind cloud computing. Technology - in this case - not referring to the cloud management servers and agents themselves, but the technology surrounding them. The first technology that comes to mind is virtualization as without this core there will be no cloud, at least no cloud that can integrate legacy applications rather than working in a very tightly closed universe like Google does. There are quite some good approaches to virtualization – commercially as well as open source – and the approach taken should really depend on the needs of the applications to be run on a specific part of the cloud. It does probably make sense to even merge the available virtualization technologies within one cloud. It might make sense to use containers build into the operating system or complete hardware virtualization depending on the kind of application to be run and therefore a cloud manager will have to deal with all kinds of virtualization technology.

More on my focus is the service management side of cloud computing and I strongly believe that automated operating is a key component of a good cloud infrastructure. Definitely the cloud infrastructure and management components will take care auf automatic provisioning and resource management, but as soon as legacy applications – that do not really know that they are running on a beautifully scalable environment – are involved manual administration of these applications would mean chasing an ever changing rabbit across a chameleon planet – an image most amusing to bystanders but neither funny to administrators nor to the ones paying them. So in my opinion an automation engine could be fed IT model data and monitoring feeds directly from the cloud manager and could thus deal with the ever changing environment and keep the application automation rules up to date with the cloud components currently in use. This automation engine cannot use a drill down approach, because the infrastructure might not even support drill downs and can change ever so often. The automation engine assuring a good foundation for quality service a professional service management will have to use a more human “circle in” or divide and conquer approach.

Does this sound familiar? By the way, check out the articles on the “Blue Cloud”; technical pioneers at work (other bloggers also think about the blue cloud)…. Also interesting is the cooperation between Google and IBM on producing cloud standards

I know it. So does does Dave Bartlett @ IBM. <SHORT_PEEK>

I wish Dave had given more information about their showcase.

But wait, doesn’t this showcase sound familiar to me? Yes, I knew it. The “Microsoft Home of the future” <READ_HERE>. Interesting. These ‘home’ thing is a good idea to create a personal relationship between the customers/partners and the vendors technology and research.

Roland

One should think that Florida in spring is pure sunshine, but actually it looks quite rainy these days. So Cy and I have the time and concentration to follow the IBM PULSE 2008 Conference, which we are attending this year. Just for a short introduction, I have been listening to the opening keynote moderated by Al Zollar (really good show). The best speech in the opening block was given by Steve Mills who introduced the challenges the IT service industry is facing today. He talked about the introduction of “smart devices” or intelligence into most devices that are making up our assets (not just IT but all infrastructure) and the possibilities these new interfaces offer. He also talked about the growing amount and growth of data processed and stored today. This was (of course) followed by a presentation of the challenges the energy hunger of IT components makes us all face in terms of environmental, social and economic consequences. All this wrapped up in the expectation, that with IT becoming ever more part of every day life and every day business the expectations towards quality of service of IT systems are not just rising but are at roughly 100% - he made a very impressive connection to telecommunication (the dial tone is there when you pick up the phone) and the power grid (power is simply available, when you need it). In order for IT to meet these expectations Mr. Mills proposed that the IT professionals are facing their final challenge in industrialization of IT services.

Does that sound familiar? Yes, it does (at least if you have read quite a bit of what we have been writing here)… Mr. Mills went about industrialization on a rather abstract level, but one of his key points wars process automation and automating manual labor. The latter being exactly where the automation technology I am focused on comes into play. Especially when all the smart devices keep on generating more and more monitoring data it is not just laborious to handle all the events the attention of administrative staff is supposed to be pointed to, but at some level just becomes impossible. That is when we need an “engine” or an “industrial robot” to take care of processing the events and taking the right action.

I have seen great approaches for visualizing events and monitoring data here as well as very interesting techniques and technologies to get in control of automated processes (I should point out, that these concepts were obviously inspired by practices from the production industry), but I have not seen much on tools that actually perform the work. I think we will have some very interesting conversations with colleagues from all branches of the IT industry on our ideas concerning actually automating the operating of IT systems. At least my first impression at a “meet the experts” event after the key note is, that IBM PULSE 2008 is the right place for an open minded discussion on how far we can possibly take this approach of automating IT operating and what technology is needed or even available to do that. I am very excited about that, especially since the idea of automation – an idea techies have been discussing for quite some time - seems to have penetrated strategic management and people like Mr. Mills show how far someone with a vision can take the concept of automation.

As we have learned in a previous post of my valued collegue Chris, automation is a very easy thing -

It is the execution of actions based on conditions.

From the technical standpoint this simple explanation sums it up. But for me, working on productmanagement and marketing issues, I can tell you, that there are loads of other aspects of our featured subject Automation.

Today it’s possible to automate almost everything, starting from doors to production plants and - to switch over to the IT world - from simple applications to system run books even whole datacenters. All in common is the goal to make life easier and to prevent the “user” from doing unnecessary, repeating tasks, like opening doors, pressing knobs, typing commands into a shell window or even executing restart scripts.

I’m sure the potentials of Automation becoming the “next big thing” seems to be huge. So is the number of vendors offering offering products. The wheels are turning and the M&A guys have already started earning money. HP bought Opsware in summer last year for $1.6 billion and BMC Software by that time acquired RealOps, the so called “Run Book Automation pioneer” for $52.5 million and in March this year BladeLogic for an impressing price of more than $800 million. The latest news is CA signing an OEM agreement with Opalis, which leaves plenty of room for rumors. Happy merging.

The answer to the question “why companies are spending these huge amounts of money for Automation technology?” gives Bob Beauchamp, president and CEO of BMC, who said

“Organizations around the world will spend more than $140 billion this year running data centers, Automation is the only way IT can bring this spending under control and still meet the requirements of their businesses.”

Just a last word to think about: How much are companies spending for all the other stuff outside of datacenters?