Chris Boos on Automation

Is it just me? Or do you also hear clouds everywhere you go but still you do not really know what to do with them. So even though I try to focus on the technical aspects of operating a dynamic environment  – which basically means putting an autopilot into the driver´s seat, since neither me nor you can “administrate a moving target” – I feel that the topic of cloud computing needs some real business focus. With everybody getting involved on the technical side and all the lawyers talking about all kinds of legal issues and risks a question I hear over and over again is “when can I use the cloud?”.

Well, here´s the good news: you can use “the cloud” (no such thing by the way) today. You just have to be satisfied with a limited business angle so far.

While the concept of cloud computing looks to change the way we buy, operate and use IT, many players present their services as an early part of the cloud movement. There are infrastructure services (no I am not going to mention the same names over again) or standards for outsourcing peak infrastructure needs as included e.g. in vSphere. You can also see a lot of the “Software as a Service’ guys positioning themselves to offer “cloud services”. Well to my judgement a lot of that is about marketing (SaaS works without clouds just as well (just as badly) as other IT does), but service providers like SaaS-providers are the first wave of businesses to benefit from the cloud idea because it is easy for them to embrace it.

Still, if you are involved in building, planning, operating or controlling a normal business in an IT sense, you probably want to know “When is the right time to go to the cloud?”, ”How can I start?” and “What professional services are out there?”. Well, you can attend many good events created by individual companies to promote their ideas (IBM Pulse, EMC World, …). You can join the evolutionary discussion on clouds at a CloudCamp and in a LinkedIn group. You can even go into the design of a completely open sourced concept and framework for cloud building and join Thomas Uhl and LiSoG in their efforts. Or you could join one of the fathers of cloud discussions – John WIllis in his efforts to get the world to focus on cloud computing. All these actions will definitely get you involved into the process of getting cloud computing into the real world.

But your involvement into any of these much needed and very interesting activities will not really answer your questions as stated above because all these activities are very much focused on either a single approach, a very wide range or an evolutionary discussion. This is why A. Fossen from A-Server has come up with the idea of “a CloudStorm”.

CloudStorm 2010 is a series of events in Europe and the US where you do not only hear from one provider or vendor, but from many and where the focus is very much on today and interaction between the vendors rather than convincing you to follow one vendor into deep dependency (avoiding dependencies is exactly what  cloud computing is all about in the end). So if you want to get a good overview, see many examples of cloud usage that is taking place today and get into a discussion with vendors or have them run against each other (you will find out there is more cooperation than competition, clouds could actually become the first real driver for business eco systems) you should attend one or more CloudStorms. When Arvid presented the idea of a CloudStorm to us, we chose to get involved into the whole series because I strongly believe that using cloud computing is not a question of the future, it is just a question of where to begin – and this is exactly where CloudStorm is headed.

So join us all in the next event in London on Feb. 22nd . Free for attendees and just before the DataCenter Expo (so you may already be there anyway). That is, if you are not at IBM PULSE 2010 at the time (which I will be). If you cannot attend, the next event in London will be on March 15th connected to Cloud Computing Congress. After that, we are especially happy to meet you in Germany on May, 4th. So you see, there is not really an excuse not to get involved and take home lots of information where you can start to actually use cloud computing and in what legal and technical context you can use it for your business.

Become part of the CloudStorm, register today at www.cloudstorm.org.

Last week we had an interesting lunch break with an eloquent cloud. We used this environment and especially the good atmosphere to discuss the current situation of cloud computing from a business angle. There is no recording of the actual discussion, but I have translated the slides I used as a guideline and want to share them with you.

The conclusion was very clear: Clouds have a fabulous business case, but internal restrictions (psychologically, regulatory and in company procedures) currently restrict using Cloud computing.

It was also very clear that these restrictions are likely to disintegrate over time.

At this year´s JAVA development conference (JAX 09) I had the honor and pleasure of giving the keynote presentation for the track on IT in the financial industry. As this touches many of the concepts we deal with in this blog, I want to share this presentation with you and maybe have a good discussion about:

1. Why clouds are cool, but average administrators hate them.
2. Why SOA can be a heaven or hell.
3. Why no one should think that writing code is a creative process.

Enjoy the presentation.

  Chris

The history of delivering IT Services is certainly an evolutionary process. This is not even considering the huge evolution that has taken place in the technology available to deliver such services. The evolution in IT delivery or IT operation is more or less an evolution of tools. It began with the host operating systems where much of the software that came with the computer was only used to manage the machine itself. Skipping many steps, these tools went through the various stages of network and system management to business service management or business transaction management tools. The latter’s claim to fame is actually achieving what business service management set out to do – making IT manageable from a business point of view.

Automation Auto Pilot

Speaking abstractly all these tools are automation tools. They automate several steps of work that an IT operator, administrator or delivery manager previously had to perform manually. But they are still just tools. They make life easier for the one who is doing the job, but would you call an industrial hammer an automation tool? Therefore I think it is time to take a look into the fish tank of (IT-)tools and approaches available today and show how evolution points towards engines (not so much the tools) that actually decide what to do and then take the action autonomously – only asking for permission, reassurance or assistance if required by process or if no solution is available to them. Such an engine could be called an automation auto pilot and is sitting on top of all the tools available to IT experts today.

We have been developing and using such an engine for more than ten years now and have achieved very good results in quality improvement, availability of documentation as part of compliance and cost cutting. But why do I most strongly believe that this is not an exotic idea, but the logical next step?

If we focus on the two dimensions IT management tool that can takes actions automatically or facilitate taking complex actions on a complex IT and application landscape, we end up with a trigger axis and an approach axis. The trigger axis describes under what conditions an action or tool invocation is triggered. The approach axis describes what kind of action will be taken and how flexible these actions can be taking the trigger conditions into account.

At the left of the trigger axis (x) we place “scheduled”, in the middle “event triggered” and at the right automated. This means that a tool positioned to the far left of the trigger axis will take action at a predefined time. Tools placed in the middle will take action if certain events occur and tools to the far right will take action as they become necessary. On the approach axis we placed “standardized” at the bottom, “rationalized” in the middle and “dynamic” at the top. This means that tools that perform predefined actions without reacting to any information gathered while executing (e.g. cron scripts), would be placed on the bottom, tools following a predefined process but building branches into the process that take current conditions into account would be placed in the middle and tools that combine the best process to be taken for the given situation out of a pool of possible actions are placed on top.

Tool Classification Dimensions

Placing the tools and concepts currently on the market onto these axes will show a clear evolutionary development from a scheduled standardized batch process to an engine that combines possible actions to a solution as the situation requires. The auto pilot function that I was talking about earlier is such a tool that would be placed up and to the right on our chart of automation evolution.

In the chart presented below, the placement of “hot” topics such as data center automation, work load automation and even run book automation are much more “old school” in their approaches and are therefore placed accordingly. Our auto pilot engine clearly takes up the “new approach” position – with a very notable difference – we have been running a successful business on this model for a long time. Thus this is not a fancy idea, but a valid approach and current trends in management software are pointing to exactly this approach.

Automation Auto Pilot as Trend

Maybe this “sorting of the tools” article has helped a little to place other thoughts on automation published here. It will certainly be necessary when we look at why dynamic automation becomes more and more unavoidable as complexity and change rate increase. E.g. following the current discussions on cloud computing from the Atlanta cloud camp organized by John Willis or even the dynamically evolving enterprise clouds as described by Mark Masterson, an automation auto pilot is the only way to keep track of an IT landscape that is fully distributed and dynamic. Just solving the problem of distributed computing and dynamic resources from an OS point of view by creating good cloud managers or VMs does not solve the problem of keeping business applications alive and available with proper execution quality and correct business results. If any of you have ever configured e.g. the Tivoli Correlation Engine in an Enterprise console successfully you know how much work that is. Putting your environment in a cloud would essentially mean you would have to review all correlation rues every time the cloud manager changes your environment. Not possible you say – well that was only the correlation engine. No other system management, IT service management or business service management tool or visualization was even touched. So you see, something will have to be done in order to keep the actual delivery of business services up and running when moving to a fully dynamic environment – this something is an autonomous automation engine or an automation auto pilot.

When discussion comes to clouds, there are some arguments, that are often repeated by skeptics: data privacy and security but also the availability of data.

These points are aiming the spine of IT, touching the most valuable IT-assets, data. From private photo collections to enterprise data warehouses, loosing data is usually worst that could happen and is often not an option. This is true, independent which computing paradigm you follow: if you keep your hardware under your table, in the closet, in a (On-Premise)-Data Center or if you store it in the clouds, that means in data centers of the computing-service provider of your choice somewhere in the world.

And this is the point: Why is it, that with any new technological invention these discussions are brought up? I remember these 25 years ago, when RAID-systems came up, starting to conquer the later on so called SLED (Single Large Expensive Disks). Rember that Fujitsu Super Eagle, 19″, 8 Units, 150 pounds and 600MB Storage? How is it possible to securely store a single file by splitting it up and store it on 5 disperate, cheap SCSI-Disks? And years later, you might say, skeptics where right – I save raids dying for some reasons and we had to restore data from logically corrupted tape backup – **it happens.

Human Nature?

I guess it is human nature to bring up these arguments. Security and steadiness are human needs that follow directly after basic needs like food, shelter or clothing.  Because I understand that this is an important issue, I’m willing to have a candid discussion on this topic.

Living in Germany I’m not only looking human needs, but also have to keep Law and order in mind. Laws that influence not only public life, but we also have certain laws aiming at handling, storing, processing of data, trying to protect any indivual person from harm by loss or misuse of their personal data and also putting regulations on any institution that handles any kind of person-related data. Other countries have similar laws or regulations, eg. HIPAA in the healthcare sector.

Holiday Photographs

I assume, that anybody in the past, storing his/her personal holiday photographs, didn’t waste a thought nor a dime on keeping redundant copies. Possibly the most important images of the first day in school were duplicated. Nobody asked about redundancy or data security. Ok. Privacy was not the issue, keeping them in the locked drawer. But when it comes to personal photo cloud-storage, eg. Smugmug, based on Amazon S3, people start asking questions. My personal opinion is, that Smugmug’s business model is based on the fact that you pay money for a secure and reliable data storage (“All that was left after a twister struck my house are my holiday photographs” <LINK>), so the will take every measure, that this won’t happen.

Talking about Risks

If I still feel uncomfortable with the situation, it’s up to me to develop my private data protection strategy and keep files stored on my local harddrive. So I still can use the cool, new community and sharing features and still have a local copy of my photographs. Same point applies for enterprise computing. Just bigger databases and more users. Repeat after me: Security is just a matter of personal needs and money.

I don’t want to start a case in favor or against Cloud Computing like others, but instead I suggest to openly discuss risks and their management <See Bernard Golden>. But this discussion is not related or limited to cloud computing. Any data processing or management, regardless if paper/pencil based, server-based or cloud-based imposes risks that have to be assessed and hopefully mitigated or maybe not, so you have to bear or share them.

So What?

The cloud-vendors have to face the enterprise grade security discussion and need to offer concepts and architectures that provide the personally or even statutory relvant level of security. Maybe in the past the they didn’t do enought to help cloud computing to come out of beta state and enter the enterprises. In the mean time we should be looking for new ideas like RAIC described be Storage Architect Chris M. Evans <HERE> and Enterprise Architect Mark Masterson <HERE>.

I’m sure there are more people and companies working on concepts and products addressing these issues, so let’s find solutions and look forward to new opportunities, rather than only whining about dangers and risks. Trying to keep the status quo also bears risks. Especially in these times.

Roland

Richard Stallman has started a big philosophical discussion about cloud computing by giving an interview to the Guardian in September 2008. Many participants in the Web community have taken this interview as a starting point to declare clouds and cloud computing as the evil itself. Basically Richard says that clouds are not a good idea, because you lose control over who processes your data and where it is processed. Stallman declares this dangerous, because only few players will offer these services thus putting all of us subdued to the will of these few corporations. Well, this is certainly a legitimate view of the world. But is this not fear of being controlled or at least controllable by someone else (especially a corporation) standing in the way of obvious technological progress? As you may have read in my late article “Heads in the Clouds” the concept of cloud computing is a logical step in technical evolution – although (and this is where I come quite close to Richard) is badly being misused by marketing.

Personally I think we are in many parts of our life more dependent on few people and corporations making decisions than we think. If you do not believe that I suggest reading “State of Fear” by the late Michael Crichton – you will find how much we are influenced and controlled by governments, media and corporations in our beliefs that even I believe if giving up control was an issue, we lost it long ago.

The concept of cloud computing makes great sense. Technologically because it takes one of the biggest challenges (parallelization) out of the programmer´s hands and allows him/her to focus on innovation instead. In Business because it saves a great deal of money as you are no longer obliged  to have all the computing power in store you might need at some stage. I found a more detailed well written reply “Cloud Competition, Lock-In, and Why Richard Stallman is All Wrong” going into Richard´s argument in detail at the SmoothSpan Blog written by Bob Warfield you might enjoy. If you are interested in the evolutionary aspects of cloud computing the current state and its off spring was described by James Urquhart in the “Wisdom of the Clouds Blog” article on “The Two Faces of Cloud Computing” which neatly fits into my current view on clouds and the future technical development awaiting us as described by “Heads in the Clouds”.

I would recommend facing the topic of cloud computing with less fear and more enthusiasm because this may actually be a step towards better software and services while at the same time cutting cost and “being green”.

I wish you all a happy new year. This may sound hollow as the upcoming year is starting out with immeasurable uncertainties. A recession is unavoidable as the economic mechanisms are working their way through the different economic sectors and into everyday life. Given the origin of this recession – the financial industry with capital being one of three pillars of our economic system– even systematic change may be in store. The greatest problem is decisions being held back due to these uncertainties thereby creating an even greater economical impact. Thus what we definitely are feeling as a crisis is a powerful well of change. This well will flood through economy, society and of course technology. We will need strong decision makers and innovators – real entrepreneurs – to embrace change and make use of its power to tackle some of the grand challenges built up during the last 50 years.

For those of us promoting new technologies the willingness to embrace change is often the biggest obstacle in putting these new technologies to use. Think about the argument of how cloud computing cannot be a good thing because it changes the relationship between our data and our computations we are so much used to. Or think about bringing the concept of automatic system operation to the administrators who will no longer be just operators but turn into system experts. All these high tech concepts require a dramatically changed way of approaching everyday problems and those of us implementing these new technologies know that inventing the technology is less than 50% of the way. The biggest challenge is attracting enough interest in all players the new technology touches, in order to make them embrace the required change to effectively make use of the new technology. The current situation may prove to be one of the most potent accelerators for technological change possible. So to all of you – those who invent, implement, decide upon or just make use of new technologies – make wise, well thought of and brave decisions embracing change. You will be the ones who will contribute towards a speedy way out of the current uncertain situation.

After giving you so much leeway ( ) by posting a few personal stories from the past summer to past autumn we are all back to business and I want to share some of the reading and thinking that I have done during the quiet time between Christmas and New Year´s Eve in the articles coming up this week. I will start out with a little catching up on the “clouds are bad discussion” started by Richard Stallman with an interview given to the Guardian in September 2008. I do believe there was a good deal of stubbornness and corporate mistrust behind condemning the cloud concept as you will read. I will then continue with a post on integrating the concept of automation – rather than just tools – into IT operation processes and tool infrastructure. After you have read Roland´s post on “Automating What?” in November you may be interested in how the concept of automation is integrated into everyday IT service management and how our concept of e.g. an automated incident management is incorporated into a working IT environment. Following this post I will try to show a landscape of technology and tools and the way the ongoing development is focusing in on automation as a concept. This process was started when tools were used to ease the manual process of maintaining system functionality (e.g. system management tools) and continued by the automation tools that enable complex changes to be performed by entering a simple command (e.g. change automation or run book automation tools). The process is now at a point where actually decisions are taken by the automation software (e.g. what hardware is used to do what tasks by which is decided by workload automation tools) and will finally come to tools that make use of all the experience of system administrators in order to automatically decide how to keep systems alive. Thus automating incident- problem- capacity- and availability management. This kind of tool is what we have been using and developing for quite some time now ( see the aAE) and the post will show how this kind of tool integrates with the whole landscape of tasks and tools involved in IT service management.

There has been so much talk about cloud computing that at least my head is spinning in the clouds right now. The great idea of the “Do What I Mean Program” (to me the original idea of a cloud) is the logical next step from the “Do What I Mean Network” (Tim Berners-Lee´s original idea of the web). Unfortunately too many buzz word hunters have jumped onto the term too quickly, therefore this topic feels a little more like the emulsion resulting from !$%&/ hitting the fan than anything else. Still the idea is worthwhile, and this is why I have decided to put some time in structuring the cloudy talk out there.

In the long version of this article article you will read about the actual cloud – a distributed computation power that solves computer science problems and enables “us” programmers to focus on the actual value added we expect from our software. You will also read about the common cloud – the marketing talk out in many computer papers and on so many consultancy slides. I will try to show some examples of actual cloud implementation of both “kinds of clouds” and I will take you to the problem with the cloud approach – the problem being that only software either written explicitly for a cloud or really well modeled can actually make use of a cloud.
So, if this is your cup of tea, keep on reading and enjoy. Then again, if you want to read about how this great new technology will change your life and how IT drives business and such – do me a favor and consult either your local astrologist or read whatever IT tabloid you are addicted to and fancy.

Read the complete article here…

The worlds-famous retiree Bill Gates annouced some weeks ago, that in the future Microsoft will have many millions of servers in their datacenters, which first of all is a very impressive proposition. Nicolas carr has some thoughts on Bills speech <HERE>.

If you break down the numbers – Microsoft claims to add between 10.000 to 20.000 servers to their infrastructure each month – and read datacenter newsletters like <THIS> and <THAT>, you soon will realize, that this definitely will become true rather sooner than later. Sure, MS has to because we might be right before a huge shift of paradigm in the software industy – moving every type of application and data from your destop straight to some datacenter somewhere (hi Google) . Thus making Microsofts main business modell looking kind of old-fashioned.

If you bring things together and look at “Live Mesh” and “Office Live” you will understand, where the world will be in 3 to 5 years from now.

And, I’ll promise you: Microsoft is paying attention to the enterprise market much better than Google and Apple together.

Roland

I’m working on a simple formula to describe clouds and cloud computing. After working through numerous blogs and posts, reading vendors press releases and visiting their websites, I feel confident enough to give a first try:

What do you think about this:

Cloud = Computers*x + Virtualization + Automation + Service

where x depends on the cpu power of the utilized machines.

I’m begging for comments.

Roland