Chris Boos on Automation

As you may have noticed – by me being silent = not part of my personality – the organization of the Frankfurt CloudCamp is putting all of us to the test. But today I have some great news for you.

We have published a first glimpse at the agenda of CloudCampFRA. I can honestly say that I am much looking forward to a fabulous event. Thanks to our sponsors we are able to organize a unique event in the successful history of creative CloudCamp unconferences and this is attracting great speakers.

You will be enticed into cloud philosophy presented by Simon Wardly (a great show is guaranteed) in the keynote. Visionaries like Tim Cole will give you security insights for the cloud and market makers like Cedric Hüsler will speak about business oriented cloud development.

With CloudCamp Frankfurt focusing on the business possible in clouds today, we have put together an agenda structured in four tracks:

1. Security and legal issues in the cloud
2. Business with the cloud
3. Developing for the cloud
4. Building and maintaining cloud infrastructures

No – there is no track for putting down yet another 25 descriptions of what a cloud is and why one needs one´s own definition….

I am also happy to announce (before this is even out on the official CloudCamp site) that we were able to increase the number of seats available at CloudCamp Frankfurt, so more of you can register and join in on interesting discussions, networking and game changing eco system building for the cloud environment of your choice. Thanks again to a great crew of corporate and media sponsors.

Look out for more news on CloudCamp Frankfurt at t3n and O´Reilly or follow @CloudCampFRA or our modern PR partner @psmw.

See you at CloudCamp Frankfurt

   Chris

PS: If for some highly improbable reason you do not have time to join us on the evening of Monday 28^th September in the Museum of the Moving Images in Frankfurt you have another chance to join a CloudCamp on the 20^th October in Munich

As Organizers of Cloudcamp Frankfurt, we are proud to present the new event homepage for Cloudcamp Frankfurt: At

www.cloudcamp-frankfurt.de

you’ll find latest information about this upcoming, unique event. Beside the agenda, news and photos the site integrates all our media channels and will also show the press coverage. Following the event, we will have the slides and Audio and/or Video recordings of the talks, so please stay tuned.

Remember: Cloudcamp Frankfurt will be the place, where business meets Cloudcomputing, so be sure to register.

See you there.

As you may have read on the blog of Mark Masterson , we are one of the primary sponsors of CloudCamp Frankfurt. This CloudCamp will take place at the Museum of the Moving Images in Frankfurt am Main, Germany on September 28th (Monday) evening. Of course you can register at www.cloudcamp.com/frankfurt but much more important you can send me your ideas for lightning talks and panel discussions to my contact address.

Museum of the Moving Images Frankfurt

I think many CloudCamps have dealt with many good concepts on solving the technical challenges put forward by a parallel on demand computing environment. We want to address these issues in a track called “cloud programming” in the un-conference of Frankfurt´s CloudCamp. But we also want to put a stronger emphasis on commercial issues and what is actually possible today in a commercial environment.

So if you have anything interesting to contribute (not to sell) to the conference or if you are willing to share insights you have gathered at your own company or with clients, we are very happy to register your ideas and consider them for a talk, panel or even a track moderation.

I am looking forward to many interesting comments and e-mails.

For arago as a small vendor and as a company servicing larger providers it is always a good idea to keep track of the big players and add that extra innovation and quality that makes a difference to the customer. This post will share some of our insight on how the big four vendors of IT service management deal with “automation” in their product portfolio.

To keep you from thinking we are out of our minds going against vendors like HP or BMC I can safely say that we work together very well with e.g. IBM and that none of the big four has anything close to our IT autopilot. In fact the big vendors seem to aim more at building tools for technical staff. Following this sentiment most automation tools from these vendors take some tedious task and drastically reduce the number of keystrokes you need to perform it with their tools. This approach is commonly sold as automation. Rightfully so, because some manual effort is performed by the machine after the tools are properly implemented. Another main feature of such tools is guiding the actions performed by IT service management staff by enforcing policies or providing runbooks and thus reducing the margin of error. But with all these tools the brainwork is still done by the guys sitting in front of the screens. IT experts only get a park of instruments to play on, rather than something that will play the basic rhythm and the background music automatically, letting them focus on playing the lead instrument. The autopilot approach as used by us and as described before provides for the intelligence that plays the background music on all available instruments. The only larger vendor I have come across putting some of the brainwork into their tools is EMC with its intelligent root-cause analysis platform SMARTS (now EMC Ionix Operations Intelligence).

So let us take a look at the product portfolios of the big four vendors in IT service management – BMC, ca, HP and IBM – and how they deal with the “automation” buzzword:

Table 1: Big Four - Automation Tools

 The concept of automating a single “kind of task” at a time automatically leads to many facets of automation. So there are many different “kinds of automation” on the IT service management tool market right now and many of the ITSM experts keep talking about these different automation concepts as a kind of baseline. There are for example data center automation, runbook automation, process automation etc. 

 Table 1 shows what tools from the big four vendors support which kind of automation approach. I will not go into philosophic descriptions on the different kind of automation. You may follow the links and take a look at some of the blogs I read where intelligent guys have wrung out their brains to come up with a definition. You can however see that you will need quite a few tools when you are trying to automate everything possible. You can clearly see who is hunting which buzzword with their latest acquisition or newest product. Every vendor except CA has focused their efforts and put or is currently putting a lot of work towards integrating their solutions. ca has acquired a zoo of very good tools and thus has the ability to provide any kind of automation tool approach. It is viable to ask about the outstanding integration aspects however.

Table 2: Big Four - ITIL Support

 As looking at buzzwords usually makes my eyes hurt, let us take a look at the actual work that is done in IT operation from an ITIL point of view. The operational processes at the core of ITIL v3 (and V2) are Incident Management and Problem Management as reactive processes, Capacity Management and Availability Management as proactive processes and Change Management as the only way to modify the IT currently in service. As automation should focus on taking all or at least some of this operational workload we have looked at the same vendors and checked which tools you need to support each of these processes that make up the everyday life of IT service management staff – see table 2. You can see that in order to support all your operational processes with automation approaches you will need the whole park of instruments to play on. I have come across many companies trying to minimize the risk of vendor lock-in by supporting different parts of their operational processes with tools from different vendors. Well, this is replacing the risk of vendor lock-in b the risk of bad integration plus it is giving away all the thought some of the brightest engineers have put into integrating one vendor´s portfolio.

In my opinion, if you really just want the instruments to play your IT service management band all by hand, you should at least get the instruments that are delivered in tune. But if you want better results, you should only play the lead instruments and leave the background music to a machine – that itself plays the instruments available. If you have this machine (the autopilot and/or other more solutions that do some brainwork), integration becomes a 2nd tier problem and you can go along with a heterogenious toolset. As there is no legislation concerning the working conditions for machines, there is no problem in bothering such a machine with sub optimal inter-vendor integration. The reaction speed of the autopilot will make up for the few extra steps needed to bring tools from different vendors into tune.

Last Friday night a difficult decision had to be made. Unfortunately a company BBQ in honor of five of my coworkers who have been with arago for 10 years now was in competition with Bruce Springsteen´s concert in Frankfurt. This may be considered a lack of internal planning or an opportunity for everyone. The solution was to push the party forward a bit. give everyone a chance to have a longer BBQ in the garden of our old offices and to go to the Bruce Springsteen Concert (and for those of us who do not need so much sleet, even to return to the party after the concert).  So a bunch of us went to the Commerzbank Arena in Frankfurt and arrived exactly when Bruce stepped onto the stage after being stuck in a major traffic jam for half an hour.

I think not too many words should be spoken about the concert, because GREAT is about the only word that describes it adequately. So I will leave it at that and just say that a lot of new songs, some old classics and many wonderful new versions of Bruce´s songs were played in a three hour non-stop show. Just look at the “playlist” as published on the “Bruce Blog” by Stan Goldstein.

I absolutely love these “old” rock-stars. People like Bruce Springsteen go onto the stage andactually have something to say. They come up there and play great music and have a good time while doing it. A special note on Bruce: If any of us are fit enough to give a stunning 3 hour stage performance, tell 40.000 people that George W. Bush is a devil and look about 40 at the age of almost 60, we should have a BIG party! Congratulations!

I never understood the bad press “Working on a Dream” had when it came out. After this concert I would recommend an ear-checkup to anyone who has anything less than “fabulous” to say about this artist, this band, the new and old songs and the stunning performance of the current tour. All of us who went just loved it! Even the local press agrees with this opinion

Talking and thinking about automation so much can be like sitting in a forest and not seeing the wood from the trees. Only recently I have discovered that some of the ideas we are already applying at arago are a step beyond what is generally called IT automation. Therefore I want to give a clear picture of what is possible compared to what is widely known to be top of the pops automation technology.

Automation Tools for Stressed out Admins

As you have seen in previous posts, there are many buzzwords describing automation technology and they are all more or less cool and more or less useful approaches towards making the process of maintaining ever more dynamic IT and application landscapes. But this is not where it stops. All of these tools just take a part of the work process, wrap it into a nice user interface and hopefully standardized configuration. None of these tools actually does the maintenance work, takes the necessary decisions or finds solutions to upcoming problems or adequate answers to imminent questions. But that is what we want, isn´t it? So what we want is something to automatically use all these automation tools to “just do the job”, something to automate the automation.
The best way to explain these different automation tools and their application in IT is comparing IT maintenance to flying an aircraft. In order to keep the plane up and running there are many different tools and technologies that automate the actual flying process. There are also many systems that automate the task of executing all the commands that originate from the flight support systems. These commands are transported to the actual aircraft mechanics where changes in the wing positions, thrust, flaps etc are executed.  All these systems themselves automate manual tasks. A pilot flying a modern aircraft no longer has to manually move parts but uses automation tools to do the job for him. Still he is flying the plane. For tedious standard situations our pilot has another tool, called the autopilot. This system does the job of the pilot in such standard situations. The autopilot uses all the other automation tools aboard the aircraft to perform the task of keeping the plane up in the skies. Theoretically the autopilot would only call for assistance when it cannot cope with the situation at hand and that is when the pilot has to step in.

Autopilot Approach to IT Maintenance

It is exactly the same in IT. With all the automation tools around, you should use an auto pilot that can handle all kinds of standard situations when managing incidents, problems, changes, IT capacity and overall availability. At the core of this auto pilot for IT operations is an autopilot engine. A large set of possible actions is stored within this engine. The job of the engine is to combine and recombine these possible actions to resolve any upcoming issues automatically. Only when it encounters a situation it cannot resolve after applying possible actions should it contact the IT experts and ask for their assistance.
This approach changes an IT expert from someone who has all kinds of good automation tools at his fingertips but is constantly battered and chased by important and urgent issues to an expert who is contacted only when his expertise is required.

This auto pilot approach minimizes the probability of human error (which is constantly high in IT operations, as there is always more than one task that needs attention in a normal environment), guarantees short reaction time, relieves the IT experts of tedious standard tasks and give them time to concentrate on important and interesting issues. An autopilot in IT operations pushes the job of an IT expert up the value chain and improves service quality at the same time.

Kevin Timmons

A friend of mine – Kevin Timmons - won the Bintan Triathlon last weekend in his category 12 (!!!) minutes ahead of everyone else. Look at these results and bow your head. Congratulations Kevin and keep up the fun.

Cloud Computing was a big topic at this year´s EMC World in Orlando. I think it is a given that virtualization is a pre requirement for any kind of Cloud concept that can be implemented today and thus EMC is playing a vital role in the Cloud space with VMware. Following the “keynote” on Cloud Computing and virtualization on Tuesday showed quite well what EMC expects. First of all I want to mention, that I thought it a pretty neat idea to turn a keynote into a panel discussion, because that demonstrates the impact Cloud Computing has on IT – it touches every part of IT and thus every part of a major vendor like EMC gets involved.

EMC World - Cloud Keynote Discussion

The discussion clearly showed that EMC is thinking of what can be done with the Cloud today rather than proposing the overall concept and waiting for it to be technically possible. For EMC Clouds have to tackle the space of legacy applications rather than requiring the users to rewrite all their software. In my opinion this is the absolutely logical step and therefore I liked the content showing how different concepts at EMC support making today´s applications “Cloud ready”. The biggest step into this direction is VMware´s latest release of vSphere that enables outtasking of compute resources on demand while turning the hardware available internally into a resource pool that can be allocated flexibly and automatically. This is supported by resource and system management software as well as storage. It is all done by adding management capacities and predefined behavior to the virtualization capabilities already available and by bringing other components closer to the virtual world by adding direct control over hardware though interfaces to the management program driving and allocating the virtual resources. This concept creates a resource pool out of all involved components (storage, network, compute) that can be allocated dynamically. The feature that tops up the concept is the ability to externalize such services if peak loads require additional resources.

By simulating the environments we are using today and bringing this simulation into such an dynamic space a pre Cloud becomes reality very quickly. This is what can realistically be done today and this is what makes Cloud concepts available to legacy application, short term project requirements as well as test and integration environments.

For my taste the fact that the Cloud concept would require the reprogramming of all software was a little overdone. Yes, I too believe that there is no way dynamic parallelization of computation cannot be reached unless you write programs for such a kind of super dynamic scheduler (like Google does).But this is where computing is headed in the long run. To try and reimplement everything on the spot is absolutely unrealistic and therefore the concepts of bringing at least some of the benefits of Cloud concepts to today´s applications and architects is great. But to say that reimplementation can be avoided in a very long term perspective is just incorrect. I think we should have learned something from the immense cost generated by maintaining the big monolithic legacy apps we do rely on today (If you want something, you make something new because changing the old think to look new will create more cost through maintenance in the long run).

Last but not least the EMC team emphasized many times, that the VMware approach created much less dependencies for customers than giving their applications into the proprietary domains of Google App Engine (where your program only runs with the Google API) or Amazon EC2 (where the virtual machine itself is hard to retrieve once deployed). This is a valid point. And despite the hype created around Amazon EC2 or Google App Engine this addresses the fear of many business users. On the other hand one should state too that EMC as well is building features into their “Cloud OS” that make a customer “want” to use EMC hardware and other EMC preferred services. All in all EMC is doing a good job of opening up the specs and standards for these kind of dependencies enabling other providers to step into the world of VMware and be just as well integrated.

In the long run I am sure standards for Cloud machine images, templates and Cloud programming interfaces will evolve. I think this will be an evolutionary process rather than the job of a standardization committee, because the Cloud idea spreads so quickly and many many different concepts are being tried out every day. Survival of the fittest is not the worst thing to happen here.

As one should save the best for last I can say now that EMC management and engineers obviously understand the need for more effective automation technologies. The discussion returned to the point that such very dynamic infrastructures and environments simply can no longer be managed manually and that the current toolsets available in system and resource management will have to take major steps towards actually automating the maintenance process fully. So in the eyes of EMC and VMWare management and engineering the operational auto pilot discussed in this blog many times and actually implemented in the aAE (arago Automation Engine) is not just a good way for cost cutting or freeing up resources for innovation and change, but becomes an absolute necessity in a dynamic environment where the speed of change is too high to be reflected in human experience. Thus I conclude that the idea of preserving these experiences within an automation engine as described before is the best way to protect investment into these experiences.

You will definitely find more information on Chuck´s Blog and for a more day to day recap you can look at Len´s Blog

I am sitting in the blogger´s lounge at EMCWorld 2009 – a really cool idea from ZDNet. After half a day of the conference I must say I am really impressed. In his keynote Joe Tucci (CEO EMC) talked about the challenges presented by the current economic downturn and EMC´s reaction to them. Technologically they concentrate on the areas of:

1. Storage
   storage virtualization and the trend towards SSD
2. information management
   where they are moving from a content management platform or system towards an information management framework with multiple repositories
3. security
   though virtualization and HA solutions on the one side and a group of solutions around identity management and security on the other side
4. clouds
   strategy for bringing the dynamic and flexible aspects of a cloud infrastructure to legacy applications while keeping security, reliability and control at the level they are today and promoting automation as a key point in making dynamic infrastructures possible on a large scale

Paul Maritz (CEO vmware) elaborated on the latter point by giving an actual demonstration of vSphere – vmware´s brand new “cloud operating system”. Even thou I think the term cloud OS is used a little prematurely, the concepts of delivering a dynamic management solution with the virtualization solution is obviously well designed and a great next step. This “cloud OS” will automatically manage resources from a service perspective – including automated provisioning and SLA tracking. This resource management does not only include computing power as before, but now also extends towards storage (dynamically moving storage, WOW) as well as automated HA, user based environment templates and the possibility to externalize resources on demand. To me this approach and the actual availability of the solution shows how a technology driven company can harvest the fruits of a clean and diligent design process even in turbulent times while at the same time making a big contribution to its customers cost reduction scenarios. 

Joe Tucci also made a very strong commitment towards EMC remaining a technology company and their strategy within the economic downturn. To him this means

• getting closer to the customers,
• securing talent,
•  no cuts in R&D budgets,
• increase in cash reserves,
• opportunistic  M & A as was as
• strategic investment.

To me this sounds like a viable strategy of a well positioned company.
Client virtualization and automated operating were put out as the next “hot things” they will be dealing with.

And I can tell you that I found some people to talk to about operating auto pilots and automation beyond dynamic provisioning really quickly. I will be going to an engineering round table this afternoon and I will surely keep you posted.

EMC World 2009

Not just because I am a roller coaster fan (especially the Dueling Dragons in Universal Park), but mainly because I am interested in technology and innovation, you will find me at EMCWorld 2009 in Orlando (18. May – 21 May 2009). If you are around and want to strike up a chat on Clouds or automated IT operation please contact me or send me a tweet.