So why is this interesting? I spent all of last week at SNWin Phoenix, AZ, running the SNIA Data Deduplication Hands-On Lab. Not surprisingly, data deduplication was once again a very popular topic throughout the conference. In the concurrent sessions, many of the vendors were busy arguing about technicalities, and this can definitely make for interesting debates, as well as the occasional farce. While some lab participants were getting their first exposure to the technology, many of them were now well versed in the particulars of the technology and their interest and questions were surprisingly singular in nature. They wanted to know how deduplication can save them money in the face of rapid data growth averaging 50% a year or more.

Well, there are la lot of ways to achieve cost savings with inline deduplication technology, ranging from automation of labor intensive DR processes to the whole tape replacement story. But, curiously, I instead found myself telling these users the same story I've been telling customers since I first joined Data Domain in 2006 - namely the "continuum" of performance and capacity increases demonstrated by each successive generation of our systems. How specifically does this drive down costs? Ultimately, it's all due to our SISL scaling architecture, which leverages the upgraded multi-core CPUs and memory in these new appliances - but, for purposes of this blog post, I'd prefer to address the question at the system and real-world implementation level.

Consider the new mid-range and entry-level deduplication storage systems Data Domain announcedearlier this week (the DD610, DD630 and the DD140 entry-level system). Each of these new systems replaces a previous generation that was similarly targeted for either mid-range enterprise applications/regional data centers or remote office locations. In the case of the new mid-range systems you get 60% more capacity and 100% more throughput performance. For the new entry level remote office system, you get 100% more capacity and 50% better performance. So, cost per Gb goes down (yet again) and price/to performance improves. Users can protect 60%-100% larger datasets and they can write their data 2x faster, enabling them to meet their nightly backup windows.

Based on what these users at the SNW SNIA labs were asking me, I remain convinced that Data Domain's proven track record of ongoing significant performance and capacity increases (while relative costs decrease), may actually be the most important element of the value proposition inline data deduplication offers.

So, I guess we should "continue the continuum?"

The story underscores the point that the replication needs of large organizations can be virtually boundless and can require tremendous configuration flexibility. Data Domain recently expanded the capabilities of its replicator software in two significant dimensions.

First, replication fan-in for many-to-one topology now supports up to 180 remote sites all replicating into a single system at a central hub site. One might wonder who would ever need that much, but the aforementioned customer is one of many who I have spoken to who have such an environment.

Second, Data Domain systems now support cascaded replication. In other words, you can now replicate data from one location to a secondary location, and then from the secondary location to a third site. While this might be more than some customers require, it's a requested feature that is more commonly asked for than you might think.

Recently I was discussing Data Domain systems and technology at an energy company. Since the bulk of the conversation was about their two data center locations, I assumed that cascaded replication would be of little interest. Turns out I was way off target. The company in question recently experienced a security related 'incident' that exposed a weakness in their dual data center model. The company realized then that they needed to have a stronger segregation of duties and are moving towards adding a third data center with strong isolation from either the first or the second site. In their new model, sites A and B will selectively cross-replicate with each other, and then both replicated data sets will also replicate into a secured and hardened site C. Administrators at sites A and B will not have a physical access or administrative access to equipment at site C.

However, I believe the main adopters of cascaded replication will be organizations that want to create a replication topology that maps precisely to their distributed site model. I see cascading fitting well into organizations with a combination of small remote sites, medium-sized regional hubs, and large global data centers. In this model you will see any number of small, remote sites replicating into regional hubs. These regional hubs will then replicate the remote site data plus their own local data to the larger, global data centers for longer term retention and a minimal amount of tape creation as may be required.

Let me backtrack a little and point out that Data Domain has already benefited from an industry-leading replication capability before the recent enhancements. In fact, some of the competition have been shipping a very limited and inflexible replication capability for a few short months, while others are still promising 'Replication 1.0' in the near future.

With Data Domain, IT architects can now even more easily design and implement whatever replication topology their businesses require. With a flexible replication technology, customers can mold their replication strategy around the environment with respect to WAN topology, regional affinities and the desire to massively centralize or eliminate tape automation.

VMware has a great website for planning your time at the show. If typing “vmworld” into your browser and pressing ctrl+enter is too much effort, I’ve provided this convenient link:

http://www.vmworld.com/community/conferences/2009/

VMworld always has an extensive schedule of events and activities, not to mention a list of sponsors and exhibitors a mile long. This year is no exception. So how do you not get overwhelmed by all the possibilities? No worries – just sit back and let me do the planning for you.

Item 1:  Make a short list of your one or two biggest virtual wishes in advance. Touring the floor can be a marathon. If you have one or two problems to solve it can really help keep you focused on finding the right vendors to talk to.

Item 2:  Select two or three speaking sessions to attend.  See item 1 for suggestions on topics.  It’s hard to pick speaking sessions while the show is in progress.  Reading the list with a tote under one arm and a Caesar salad on a paper plate balanced on the hand opposite while standing in a moving sea of humanity takes more talent than most of us have. Also, since you’re here at dedupematters.com, I’ll go ahead and assume you’ve got some level of interest in the Data Domain story. Therefore, I’d like to be so bold as to suggest a couple of sessions. “Planning for Optimized and Cost-effective Storage Utilizing Deduplication and Virtualization” (Session BC3223) featuring Data Domain customer Jules Thomas, and “The Real Deal on Dedupe and Virtualization” featuring our own Enterprise Application Technologist, Daniel Budiansky.

Item 3:  Make extensive use of the hands-on labs. There’s no substitute for the experience of doing it yourself. There are labs covering a vast array of subjects, so there’s sure to be more than one that gives you the opportunity to experience something new.

Item 4:  Spend some time with a big vendor.  I know it sounds obvious, but it’s easy to overlook the big companies because you’re already familiar with what they have to offer.  Cisco is certainly on my short list.

Item 5:  Stop by the Data Domain booth. We’re going to be wearing EMC shirts for the first time. I think a bunch of us are planning on sneaking over to the regular EMC booth and trying to blend in.  Who knows, if we play our cards right we might even get invited back to EMC World in Boston next May. Yeah, we love that show!

Most importantly, have a little fun while you’re in town. Maybe it’s time to put on your black leather baseball cap, rent a red Trans-Am, and ride in style to the Foreigner show on Wednesday night. I wonder if they still have that big, inflatable jukebox?

However you pack your bags, and whatever your plans, I look forward to seeing you at the show.

Oh, and if you’re curious about how Data Domain can help protect your virtual infrastructure, check out our solutions page here . Or better yet, just stop by our booth. Once we’re done with our shenanigans, we’ll be happy to talk shop.

There are three generally accepted techniques for backing up VMs:

1. Running backup agents, installed inside the guest

2. Running proxy-based VMDK backups using VCB, with or without 3rd -party backup application integration

3. Running host-based VMDK backups, either from the service console or using a VMware-specific backup application, like Vizioncore vRanger Pro

Best practices documentation typically references using some combination of these approaches. Unlike most other deduplication solutions, Data Domain systems support all three methods and will eliminate the redundancy across all data stored. This support allows organizations to customize their data protection policies, and to best leverage existing infrastructure.

Because Data Domain systems deduplicate data inline, image backups can be reduced 40-60x or more before being stored, and entire virtual environments can be efficiently replicated using a minimum amount of bandwidth. We support VCB backups natively, and can be a target for most enterprise backup applications that provide VCB integration as well as traditional agents installed in the guest OS. Specialized applications, scripts, and even native VMware features like cloning, storage VMotion and templates can all use Data Domain storage.

In today's world, this simplicity and flexibility are critical forces which balance the underlying complexity of new technologies. Data Domain and VMware are like peanut butter and chocolate - both are great on their own, but together, they become greater than the sum of the parts.

Come visit us in booth #1302 at VMworld next week - the RPBC is on me.

On the other hand, vendors with post-process deduplication solutions might be less enthusiastic.  After all, the fundamental premise of their marketing has been that post-process systems are faster and bigger.  In reality, the inline deduplication throughput of a single DD880 exceeds the speed at which they can ingest data in native form to a 2-node post-process system. And the higher physical capacities they claim as a benefit?  They are but a side effect of the disk i/o and caching requirements of post-process deduplication – you can’t actually protect a larger dataset with all that space.  It’s kind of like a car salesperson pointing out the massive fuel tank in a gas guzzler as a competitive advantage, while disregarding the mileage it gets. 

A single DD880 can deduplicate 60TB inline, while concurrently replicating the deduplicated data, all within a 12 hour window.  As with all of our previous new product releases, performance and capacity increases are derived from upgrades to the processors and associated hardware in our systems.  This is the power of Data Domain SISL, our CPU-centric approach which enables fast inline deduplication using a minimum amount of disk.  In comparison to the DD880, for the same 60TB backup a post-process deduplication system would require at least 2-4x more physical disk and significantly more compute power in the form of complex multi-node configurations.  It just doesn’t make sense why our competitors would expect customers to do this.

In the end, this is about one thing – Data Domain has enabled a paradigm shift in disk storage for data protection and disaster recovery because once our technology is deployed, customers can save money.  The bottom line value of post-process deduplication, with its need for more of everything, will never match us.  Good luck to those vendors that decide to keep trying.   

Protection storage (i.e. backup disk, tape, or other media) performance is typically measured in data transfer rates.  These rates are commonly expressed in MB/sec or TB/hr.  Both aggregate speed and single stream performance are important to the backup workload.  Aggregate speed is indicative of the big picture; given x amount of data to backup, what is the minimum possible time that the storage system will require to complete all backups?  Single stream performance is an important consideration as well, especially for high-speed backup clients such as large, powerful database servers.  Ideally, both aggregate speed and individual stream speed will be very fast.  But how fast is fast enough, especially for the enterprise?

The answer as it turns out is not all that simple.  Storage vendors who compete on feeds and speeds have a tendency to focus the customer’s attention strictly on the characteristics of their own device, ignoring the fact that protection storage is a downstream element from what is often a very large and complex technology ecosystem.  Here are a few somewhat imperfect analogies that may help simplify and put things in perspective.

Pitching a Baseball (single-stream):  When a pitcher throws a baseball it reaches its maximum velocity upon release.  The baseball never accelerates on the way to home plate.  The catcher’s mitt doesn’t exert pull on the baseball.  Therefore, if you equate the storage device with the catcher’s mitt, you clearly need a mitt that can handle a 100mph fastball, but is there a material difference between mitts that can handle 150mph versus 200mph?  Conversely, if the catcher’s mitt can only handle a 70mph pitch then your catcher will be calling for knuckleballs and changeups all night long.  In a similar fashion, backup data never speeds up after it leaves the client, yet the backup storage device must be able to handle the fastest streams possible.  Backup storage devices do not exert pull.

Terminal Velocity or Backup to Null (aggregate):  Any collection of data residing in a data center has what I think of as a “terminal velocity.”  That’s the hypothetical transfer rate that would exist if all of the backup clients sent their data directly to a null device.  In other words, how fast can the existing backup clients (or other protocols and techniques) lift the data off of their provisioned storage and send it into an infinitely fast receptacle?  The goal here is to hypothetically remove network or fiber channel (i.e. transports), any other intermediary devices (backup servers), and of course the backup storage device itself.

Given these two ideas, it should be fairly clear that a backup storage device cannot be deployed as an accelerator.  It need only be thought of as a potential bottleneck.  The goal then is to avoid having the backup storage device slow down the backups, which coincidentally is also the goal of the transports (network or SAN), and the intermediaries (backup servers.)  The further upstream you push the backup bottleneck, the closer you come to achieving terminal velocity in your backup environment, which is exactly what you want to do.  The more successful enterprise shops are the ones who intuitively understand these principles and architect complete solutions instead of comparing storage device speeds and feeds in a vacuum.

So I still haven’t answered the question ‘how fast is fast enough for the enterprise?’  The answer lies in a compound question:  How fast must a storage solution be to complete backups within the time allotted *and* if you have such a solution, is the rest of the infrastructure up to the task?  Simply put, your backup storage should be fast enough to not be the bottleneck *unless* you are able to complete your backups comfortably within your backup window anyway, in which case you may not care.  At the highest level you can say that faster is always better, but only up to a certain point.
Now here’s how the performance question starts to get interesting with Data Domain and deduplication.  For years many vendors have been using speeds and feeds arguments, warranted or not, to position their solutions against Data Domain.  However, Data Domain Operating System 4.6 broke through an important inflection point in the evolution of deduplication.  As a result, the DD690 attained an amazing 750 MB/sec of aggregate throughput utilizing a very minimal number of disks.  This meant that for the first time it was almost as fast to backup straight to Data Domain’s deduplicated storage than it was to an “enterprise” post-process deduplication controller, or even to one of the many, non-deduplicating VTL systems. 

With the introduction of Data Domain’s new flagship product, the DD880, the ante has doubled.  The DD880 sports an outstanding 1,500 MB/sec, or 5.4 TB/hr of aggregate throughput, placing it clearly on top of the leader board across all backup storage targets, dedupe and non-dedupe alike.  By comparison, a single Data Domain DD880 controller ingests data faster than a 2-node NetApp 1400 (post-process dedupe), 2-node HP 9000 (post-process dedupe), or 2-node IBM 7650G (inline dedupe) active-active cluster.

Read this if you read nothing else:  Data Domain’s single-controller inline ingest speed is now faster than a 2-node active-active clustered post-process ingest speed.

With the introduction of the DD880, speeds and feeds are no longer an inline versus post-process argument.  In fact, the sole argument that supported post-process deduplication has just been obliterated.  Speeds and feeds are no longer a dedupe versus non-dedupe argument either.  The DD880 is the new standard in backup storage performance spanning all categories.  Now we can get back to the task at hand and select a solution based on what is really important – the business requirements and the other technological realities of the enterprise.  We can put the speeds and feeds arguments to rest.  The DD880 wins in an overwhelming landslide.

- There was one deployment of FCoE and everyone else was highly interested in FCoE going forward. I mentioned the political challenges of FCoE between the LAN networking guys and the SAN networking guys -- and the guy who actually deployed it said that he found no issue. The lines are still there and remain - the SAN guys still manage FCoE. The others in the room were still unsure how this was going to all play out. Again, no one in this group deployed iSCSI. I am a big proponent of iSCSI but FC still rules the SAN world in larger organizations.

- Everyone was using VMWare and no one had deployed or was overly interested in Hyper-V. This is consistent with other forums I have been involved in recently.

- There was some interest in SSD or EFD for Enterprise storage, but only one person has deployed. His take was that it was a waste of time and money. It did not give him the performance he was looking for. This is an important point. Storage performance is complicated and depending on the I/O profile, SSD/EFD may not be the answer you are looking for - it is not a panacea by any means. It would be great to remove a mechanical device from the data center - after all disk drives do three things: they read, write and break. But for the most part disk drives still provide the best price/performance and even with the advances in SSD/EFD, it still is only right for certain corner cases.

- There was no implementation of active archive storage with this group. There was a level of interest but they agreed that the challenges were identifying what to move, how to move it transparently, what to move it to and ensuring response times if that data is ever needed again. One person piped in saying that there was no single solution that allowed them to do database, email and file archiving without putting agents on their hosts. I suggested to go after the biggest problem first - which is unstructured data. That could save them a ton of money. There were nods in the room but you could tell that they weren't running out the door to make it happen.

- There were no plans for any of these folks to use a public cloud service within the next twelve months. And by the looks of it there wasn't any real interest beyond that timeframe at this point. There was only one company that was building their own private cloud. I wanted to pick their brains more about what they specifically were doing but they had to leave before the meet and greet session (I hope it wasn't my presentation that made them leave).

- Finally, it was good to see that close to half of the IT professionals had Green IT initiatives. Naturally IT folks have been focused on power, cooling and floor space for years and now a number of them were going green. While this is related it is different - having a green mandate goes beyond these three things (although it is a good place to start).

The next challenge with storage tiering is identifying what solution you are going to use to move the dormant data from primary to archival storage. This process needs to be transparent to the users and applications. And it would be nice if it was easy for IT professionals to implement and there was a high level of performance. In the NAS world you can use file data movers or virtualization solutions. Data Domain partners with F5 and its Acopia solution.

The third and final macro issue is what platform should be used for the active archival storage. Data Domain is typically seen as a disk-to-disk backup solution but it should also be considered as an active archive as well. Remember that the Data Domain solution is a NAS system and therefore a mount point can be easily created and it can be used to store files via NFS and CIFS protocols. One of the main reasons to do this is to take advantage of its dedupe capabilities. Since this is not a backup data - the dedupe rates will not be as high but you should easily get a 2-to-1 and maybe as high as a 5-to-1 ratio. What makes this even more compelling and valuable is using Data Domain for both D2D backup and as an active archive. It is this combination that presents a powerful value proposition. Consolidating these functions into a single Data Domain appliance further reduces cost and complexity. In addition to the direct CAPEX savings there is also the reduction of power, cooling and floor space.

There are companies that have saved significant capital costs by implemented tiered storage environments - even without dedupe. Imagine adding dedupe to the equation. Further, leveraging the same applicance that is being used for D2D backup and giving you 20-to-1 data dedupe. Not many people talk about providing a consolidated D2D backup and active archive solution, but it really does make a ton of sense if that single solution can support both with dedupe.

There is some discussion around the notion of data dedupe as a feature or a technology and that it will some day be pervasive. People having been saying this for more than five years and it is still not a reality. There are actually only a few vendors that actually have successful solutions in the market supporting this valuable but complex technology. The fact of the matter is that dedupe is complicated - creating dedupe algorithms are only a part of the challenge. Data integrity, performance and scalability are key considerations. If dedupe was easy then it would be pervasive and it is not.