Private Cloud is the new paradigm of IT, it’s not a sea-change, or a bolt from the blue, but I believe the next evolution of enterprise IT. Mike does a great job listing out several key steps specific to his realization of an automated data center that help enable the Private Cloud. His are very focused on the Infrastructure component of the transformation required. I think that there are two other key components in the transformation to Private Cloud: Applications, what is my right-sized Application Portfolio, what is my cloud sourcing strategy for those rationalized Applications, and how can I develop new Applications that benefit from the new paradigm; and Governance, what are the policies and processes required to manage the new paradigm, what do I automate, how do I secure the environment, what is the fewest number of IT controls I can implement to be compliant and what is the unified console that provides be the transparent insight into my environment from resource management, risk and compliance perspectives. It’s important to make progress against the Application, Infrastructure and Governance components in a relatively lock step fashion, getting too far out ahead in the maturation and implementation of one of the components leads to poor benefits realization efficiency and can actually cause the other areas to regress.

How has the web changed your life?
How has the web changed business and society?
What will the web look like in 20 years?

How has the web changed my life?
I was a research assistant in the Physics Department at the University of Notre Dame during the summer of 1991 working with Prof. Carol Tanner’s team researching Optical Atom Traps. The lab I was working in was not too far away from the computer lab with the recently acquired NeXT workstations. These things were exceedingly cool as up to that point I’d only been exposed to Apple II’s in my rudimentary programming classes and rather clunky IBMs that my Dad got through work. There was a team at CERN that was doing very similar work to the ND team and they were publishing their notes and results to an internal system utilizing the CERN httpd server, which funnily enough ran very well on NeXT. At this point I’d never heard of Tim Berners-Lee or his grand vision, it was simply regarded as the next wave of Physics documentation management. I remember being a little dismissive of it at that point, mostly because I was just in love with lab notebooks and couldn’t see how a computer would be better than that.

I forgot about httpd for two years until I was a software engineering student at the Illinois Institute of Technology and got reacquainted with the very nascent Web. I discovered Yahoo and all this new content that was coming online and played around with the W3C httpd server more, learning about HTML and UNIX administration in the process. The Web changed my life because it was the gateway drug to Solaris and Irix I must be honest. I became a UNIX snob, thrilled by the power of the Sun Sparcs and SGI Indys running their server daemons and databases. The Web and the openness of its communication lured me away from the closed systems that I had been programming for, after seeing the power of the Web there was no way I was going to sit in a cube and code 1 function or class for some humongous software package for three years. I became a Web administrator at Chicago Kent College of Law supporting the Circuit Court and the paperless law school and from there I went into consulting for first the Web, then intranets, then Data Centers until finally I was running operations for MyPoints.com, one of the top 10 web properties in 1999 and 2000. I learned a lot along the way about connecting people and ideas and have been able to develop a much more expanded vision of the power and purpose of systems and I am very grateful.

How has the web changed business and society?
Let me count the ways, they are legion. The power of the web to bring people and information together is exactly what drew me to it. It’s changed the way that I do just about everything, from shopping and learning about new products to finding information, teaching my daughters, watching movies and TV and interacting with my friends and peers. I think it is great the way that many companies are expanding their use of the Web, engaging their customers, learning what they expect of the products and services, how they’re being used, how they could be improved and allowing customers to get together to share even more about themselves around a shared passion or interest facilitated by that company. The Web has even changed product design, more and more companies crowdsourcing their designs or creating contests via the Web to develop new products. Nike has done a great job of this allowing everyone to design their own custom shoes and hosting design competitions online. Awesome stuff. Web-enabled customer forums help people get answers, best practices, unvarnished opinions and new contacts all in one place. EMC has done a ton of work in this area and I’m proud of the communities we’ve built for our customers. Gina Minks blogs often about our communities and has done a ton of work setting them up and managing them.

What will the Web look like in 20 years?
Well there certainly has been a lot written about the future of the Web and Technology but I’ll add my 2 cents. I am very much in the school of Neal Stephenson and his views presented in Snow Crash and The Diamond Age. The Web will become more immersive and more pervasive, if that’s even possible. I’m not sure if Virtual Reality will really take hold, but there certainly is a lot of potential there. I think the biggest differences we’ll see is around search and the ability to federate searches and be able to more quickly integrate and analyze the results a la Wolphram|Alpha on steroids. We’ll also see a lot more integration of location aware and other context based integration into search and content presentation. I’m especially excited by the possibilities of more integration of open-source and crowd-source design like that at Local Motors and in the Makers by Cory Doctorow. I guess I’d sum it up by saying ubiquitous access, high bandwidth, context aware natural language search and analytics with data privacy and even more by the way of integration of social networks and academics etc. Needless to say I am excited to be a part of the continued transformation.

At this point I’d like to tag that font of information Christopher Kusek aka CXI and Kathrin Winkler who I hope will talk about the Web and sustainability in 20 years!

The virtual infrastructure is a key enabler of the cloud, but it’s not the cloud. At EMC we’ve developed a product and services portfolio that enables the Private Cloud vision of any device, anywhere accessing your information and your applications regardless of the infrastructure it happens to live on. Our Virtual Computing Environment coalition extends that enablement by including the components of unified internetworking and compute with the cloud operating system. Private Cloud is more expansive than VCE and the first technology solution offered by it in the form of the VBlock. The real differentiator between the virtual infrastructure and the Private Cloud is any device, anywhere is able to access your applications and information with your governance controlling it regardless of the underlying infrastructure, be it internal assets or those provided through the public clouds.

It’s the integration of GRC into the environment that delivers on the Private Cloud promise of all the agility, flexibility, scalability, multi-tenancy and automation associated with cloud computing tempered with the security, availability, resiliency, and control of the data center. This means that getting to a Private Cloud has to be about a lot more than deploying new technologies, it’s a wholesale transformation of IT and a new way of interfacing with the Business and your customers. A lot of what has been promised and demanded by frameworks like ITIL, SOA, MOF, COBIT, etc. is now able to be delivered through the infrastructure and toolsets supporting it. It’s possible to implement the Service Catalog and things like automatically approved changes into the resource management infrastructure to begin to provide real self service of IT where appropriate. The appeal of many existing public cloud solutions are the ease with which users can consume them: a credit card; a few clicks; and bam you have storage, or a server, or a CRM system. An integrated approach for GRC can provide this same user experience, plus the enterprise necessities like Service Levels, Business Continuity, Data Protection and the like for enterprise IT. This is the stuff that gets traction with the people I talk with about cloud and to me is the real promise of Private Cloud, a promise that is actually deliverable today.

I’ve spent the last few months talking with customers all over the world about Cloud Computing in general and what EMC means by Private Cloud in particular.  I’ve been fortunate enough to get a lot of feedback from the CXO level down to the managers and administrators that will be tasked with running these clouds.  A few common themes have emerged in these conversations.  Rarely does the question, “Why Cloud Computing?” come up, it’s almost as if Cloud is a foregone conclusion, hyped into the mainstream.  I am almost consistently asked by people at every level, “So now what?”.  EMC and our partners, and the market in general, has done a good job of laying out the groundwork and vision for Cloud Computing and its benefits and a hardware and software portfolio to enable it.  The question becomes how do I actually execute against the vision with the products to make it reality, as it does with most paradigm shifts.

It seems to me that a lot of IT organizations are positioning themselves for Private Cloud, knowingly or unknowingly.  The virtualization of the data center, not just of servers, but real enterprise virtualization is a key milestone on the path to Private Cloud.  Not only does it provide the framework to build a Private Cloud on, it brings real benefits to the organization in terms of reduced Capital Expenses, Operating Expenses, time to provision, mean time to repair and improved customer satisfaction for internal and external customers.  These benefits are core to the allure of Private Cloud and IT is keen to realize them as quickly as possible.

I’ve often seen, and industry analysts seem to weekly report, that virtualization efforts seem to hit a wall when around 20-30% of the workloads in the data center have been virtualized.  There are many reasons for this, ranging from applicability of previous virtualization solutions to enterprise workloads, and insufficient application owner and line of business buy-in to the transformation leading to lack of approved downtimes and applications not being approved for P2V.  We’ve helped a number of customers push through this wall and drive towards their goals of 80-90% of workloads being virtualized through the development of enterprise virtualization programs, acceleration services, documenting the activities and processes surrounding the virtualization of servers and applications, training and comprehensive communication and marketing plans to get the buy-in of the stakeholders and application owners.

It’s not just driving enterprise virtualization that will help IT realize the benefits of Private Cloud, however.  A lot of outsourcing companies operated for years on the concept of “Your mess for less”.  For this to be a real transformation it can’t just be the same old problems running on a shiny new architecture.  A key component of the journey to Private Cloud has to be the rationalization of the application portfolio.  We are constantly adding new applications and features and functionality into the environment, and for every “server hugger” out there I’d argue there’s an “application hugger”, we all have our babies and we’re certainly not going to let them be torn from our arms.

A systematic review of the existing application portfolio to identify opportunities for retirement, feature\functionality consolidation, replatforming and virtualization on proprietary unix systems provides the roadmap for how many of the promised savings can be realized.  If you want to embrace x86 as the chosen platform you have to figure out how to get as much of your application portfolio as possible onto it.  Coupling this portfolio rationalization with a comprehensive business case for Private Cloud provides the framework for driving line of business and application team compliance and for a realistic timeline of how quickly you can actually realize Private Cloud.

So that accounts for the infrastructure and the applications, now for the trifecta, governance!  A new model of computing requires a new model of governance and the associated tools and processes.  Thousands of virtual machines crammed into a small number of cabinets dynamically allocating and deallocating resources is a daunting environment if your key governance tool is Microsoft Excel.  The identification of appropriate services to provide, service levels to achieve, and a chargeback model to allocate costs are required, absolutely required, to have any chance of successfully building and operating a Private Cloud successfully.  This requires transparency into what you have, what you’re using, where it is, who owns it, what it requires, how it is to be measured and monitored, backed up, replicated, encrypted, allowed to grow or shrink, &c.  Sounds scary, I’m sure.

The service catalog, an integrated management tool framework and automated processes allow you to monitor, maintain, provision and recover the costs of such an environment.  Your administrators, engineers and operations teams need to be trained on the technologies, service levels, communications plan and have their roles and responsibilities well documented to empower them in this kind of model.  New tools and proactive methods for communicating with your clients have to be developed and integrated to ensure they understand what services you are providing them, how they are being charged for them and what service levels you guarantee.  I personally think that self-service plays a key role in the development of a Private Cloud, or most cloud models for that matter, and integration of Change, Release and Capacity Management into a self-service portal can make the difference in your client’s adoption of this new paradigm.

We’ve packaged these services up under the umbrella of Accelerating the Journey to Private Cloud and have integrated our Technology Implementation Services, and several new EMC Proven Solutions into a holistic stack to enable our customers. It’s not a light switch or a silver bullet, it still is a journey, but we’ve worked hard to take the lessons learned from many years of data center consolidation and migrations, process automation, custom reporting and dashboards, building innovative solutions and architectures, product training and managing transformative programs and integrate them into an effective services and solutions stack to accelerate the journey to Private Cloud and realize real benefits today.

In many ways the transition wasn’t as difficult as I thought it might be, while EMC is a very large company the organization I joined was only a couple hundred back then.  We’ve both grown a lot in the last five years, I went from being a project manager to global CTO for my practice and the organization has gone from a few hundred people to a 2700 person global organization.  It’s been exciting and back breaking and fun.  While the consulting side has been growing the overall make up of the company has changed dramatically too, more than 50% of our revenue coming from software and services, a real sea change from where we were five years ago.  I’ve been lucky enough to watch our product portfolio morph into what I do truly believe is the most comprehensive in the industry for information management, we’ve accomplished much more than I would’ve hoped for in 2004.

This has been a real milestone for me, for ten years I hopped between jobs just about every year seeking new opportunities and challenges, constantly looking for the next organization I could help grow, not really wanting to grow stale in my role.  I can honestly say that hasn’t been a worry for me the last five years, constant challenges and new opportunities.  For a long time I never thought I’d find a place to settle down, but I can easily see myself working for EMC in 2014, and I couldn’t be happier about it.  So as I look back on the last five years I am grateful: for the work, the opportunities and mostly the incredible people that I have the privilege of working with.

Here’s to another five great years!

If you scratch beneath the surface a bit the magic and mystery of the Cloud becomes a little more apparent: you have a high-performance, omnipresent network; a flexible delivery engine that is highly scalable and efficient; and a management framework that provides the appropriate Service Levels, security, compliance and communications the customer is seeking.  To truly deliver a cloud service you first have to identify and define a service that can be readily doled out to customers clamoring for it.  I can think of tons of services internal to an enterprise that would qualify for this designation, so I think the concept of a private cloud is a cogent one.  Take for example File Sharing, or Email, or Market Data, or Order Processing.

So why now?  The emergence of good allocation and resource management tools certainly makes the management of the service a lot easier, add adaptive authentication, identity management and role based access, couple that with the virtualization capabilities and infrastructure components geared to hypervirtualization and you have the recipe for easy to deploy private and public crowds.  The market adoption of frameworks like ITIL and ISO 20000 and their focus on Service Level Management provides the appropriate mindset for the IT organization looking to become service oriented.  Now ride all of that on a ubiquitous, converged, highly available fabric and you can provide these services to pretty much any client, via any platform, any where.

Suddenly Clouds aren’t so amorphous but really the next logical progression of virtualized infrastructure, Service-Oriented Architecture, and IT Service Management.

If the tools performed as expected, they would be much more widely used and there wouldn’t be a need for a group of consultants (like me) to go around developing reporting dashboards and returning a year or two down the road because the next round of people to fill the admin posts don’t understand it either.  But, then, if the tools worked I’d be less valuable as a consultant, so I guess we should just keep the tools the way they are;  I like being able to pay my mortgage.

The customer, like most, was approaching the development, testing and management of their infrastructure through a number of siloes: people thinking about the long term strategy; another group concerned with the implementation of systems; a group that tested the integrated infrastructure; a group responsible for the daily management of the environment; and an organization dedicated to interfacing with the customer to understand their requirements (and on occasion their satisfaction).  Strategy, architecture, engineering and operations were divided across the organization with several silos within each knowledge area.  No one was incented to work together, no one had a vision of the entire infrastructure as a “system” and finger pointing was the order of the day during any outage.  Walking around the several floors the IT department was spread over there was an air of discontent, people bolted for the door at 5pm, at the latest, were largely disengaged and took pride in the walls they put up around their particular part of the organization.  Worst of all the business, their customer, was unhappy and questioning why they were spending so much on that black box called IT.

How do you solve this sort of issue through the wonders of Product Management you ask?  One of the great things about Product Management in my opinion is the sense of ownership that people working on a product have.  Whether your product is toothpaste, armchairs, or IT Infrastructure it is your product and you work hard to ensure that it is the best it can be, you think about it a lot, you constantly try to improve it, you become an evangelist for it and if you want it to be successful you try to see it as your customer does.  Sounds like great traits for an IT organization, right?  We worked together to instill the idea that the infrastructure was their product, identified their customer, actually spoke with them to understand their requirements and the perception of the current environment and defined roles and responsibilities for everyone in IT, mapped them to the PML and communicated it to the entire organization.  Pretty straightforward steps.

The entire infrastructure itself was seen as the brand, with product lines for Compute, Database, Storage, and Network.  Each product line had several products.  Within each product there were stacks, otherwise known as Tiers of Service.  Those stacks were made up of assemblies, or types of technologies, for example Platform (array), Interconnect (switches), Process (ITIL) and Resource Management in the case of the Storage stacks.  And the lowest level of granularity was components, or a particular instance of an assembly, again using the Storage example some components in the Platform assembly were Symmetrix, CLARiiON and Disk Library.  Certain assemblies were managed cross-product line like Process and Resource Management to ensure there was one coherent approach and system.

I’ll continue to share more about this particular experience in transforming an organization to use Product Management for IT and talk about tactical steps for implementing this yourself in future blog posts.

Now I know Newman is heavily biased on the “Infrastructure” part of his world, but I come from a little town where you didn’t buy stuff you couldn’t use effectively.  You call it infrastructure, we call it “stuff”.  We drive pickup trucks not because they’re cool but because they have utility: as a snow plow, hauling trash, parades, hay rides and general gittin’ ’round.  All of that while laughing at people who buy fully loaded dualies for commuting.  It happens.  I could never see myself writing an outsourcing contract that lost money just on this experience alone.

Making the connection yet, James Burke?  In my experience, these outsourcers get into trouble because they forget three things…

1. What you’re supposed to do with all that infrastructure
2. HOW to do it, and
3. How to write a contract to support it

Newman touched on the third part there, which I’ll get to in a moment but the first two just demand more air time immediately.

I’ve seen too many outsourcers allow themselves to get bogged down trying to use every single feature on some gear without any focus on why they’re doing it (they get pushed around a lot, I have noticed).  Perhaps the inclusion of some buzz words in the proposal closed the deal, but at what cost?  Some manage to stay afloat or get ahead but most don’t have the skilled talent on hand to pull it off.  It is tough staffing the top 10% of industry folks on every job.  90% of the time you have lesser talent.  The solution here is to frame up the services and support you will deliver in light of your available talent, ability to standardize complex activities and the strength of your plan-B should things go wrong.  The customer should NEVER see the outsourcer fall down on service levels.  It is death to your business.

On the other hand, too many of them ask “what’s a service level?”  That’s the contract conundrum.  If they could teach themselves to write better service level agreements tied to real world metrics these problems would be trivial to solve.  I’ve seen several contracts where the language for change orders and “extra” services conditions is longer than the actual service definitions for what WILL be performed!  What the heck are you people thinking?  Put a box around what you’ll do and with what equipment you’ll do it based on a real need from the client.  Write that need down.  Put equitable performance penalties around both sides of the deal and manage to the service level, not the contract.

I know that last part sounds contradictory but if you can maintain your availability, ticket turnover and performance metrics fewer people are going to ask you to do things that “uh, sorry, we don’t support”.  Know what your limits are, use the equipment the best way you know how (don’t stretch it) and let your people find their way to meeting the requirements.  The bottom 90% are still some pretty damned smart folks.  Left alone to establish repeatable methods and standards they perform exceptionally with little waste effort.  Yes, Average Joe can pull this off with some basic controls.

All that, and what Newman was saying.

On the other hand you people can continue to write contracts that demand 75 hours a week from your best resources just to break even.  Knock yourselves out.  Meanwhile I need to take a look at a new truck with my father.  Should be fun.

Stephen Foskett has a great break down of the problem in his post on “Storage Utilization Remains at 2001 Levels: Low!“, but I’d like to expand on his breakdown to include database utilization at the bottom of his storage waterfall.  I often use the “waterfall” to explain utilization to our customers.  In this case knowledge truly is power and like Chris Evan’s mentions in his post on “Beating the Credit Crunch” there is free money to be had in reclaiming storage in your environment.

It’s not just knowing about stale snapshots sitting out in the SAN, knowing how many copies of the data that exist is imperative.  One customer had a multi-terabyte database that was replicated to a second site, with two full exports on disk and replicated, a BCV at each location and backups to tape at each site.  That’s 8 copies of the data on their most expensive disk.  Now I’m all for safety, but that’s belt, suspenders and a flying buttress holding up those trousers.  A full analysis of utilization needs to take these sorts of outdated/outmoded management practices into account for a full understanding of what is really on the floor.

Old paradigms regarding the amount of overhead at each layer of the utilization cake need to be updated, the concept of 15% – 20% overhead for the environment is a great concept, until that environment gets to be mutli-petabyte, then you’re talking about hundreds of terabytes of storage sucking up your power and cooling.  Of course storage virtualization is supposed to solve problems like this, but proper capacity planning and a transparent method of moving data between arrays and/or systems with realistic service levels in place can address it just as effectively.