In Oscar Wilde’s lighthearted play “The Importance of Being Earnest” the main character, Jack Worthing, decides to create a fictional brother named Ernest.  According to Jack, it seems that Ernest has lead an adventurous life but needs to be continually rescued by Jack.  In reality this allows Jack to escape into an alter-ego role and assume the life of his audacious brother without detection.  Through a series of ironic twists, Jack later discovers that his real name actually is Ernest and that his younger brother Algernon in fact has strayed from the straight and narrow, requiring Jack’s help.   Jack concludes by saying “I’ve now realized for the first time in my life the vital Importance of Being Earnest.”

The life of a storage architect is a little like that of Jack Worthing.  On one hand, they have to attend to the sometimes dreary work of provisioning storage capacity, searching for performance bottlenecks, resizing capacity of LUNs and volumes, and generally trying to keep everyone in the organization happy despite incessant data growth, new applications, and new demands from the business.

Secretly, I think most architects would rather be swashbuckling heroes like Ernest - responding with a confident “yes, I can do that!” when asked to help solve a tricky data growth problem with that critical app, or helping develop a way to replace an entire storage array with zero downtime.  In other words - to become a champion of their organization rather than an obstacle to its success.

This is where the importance of agility comes in.  Automate mundane tasks, detect bottlenecks faster, store data more efficiently and you’ll soon move ahead of the pack.  In other words, you’ll become more agile - which in turn helps your organization innovate.  What if you could provision an entire SAN in a few hours?  Upgrade entire storage arrays while applications and users merrily keep working? 

Becoming an agile storage architect is not a fictional play, it is a reality.  In this era of monumental data growth – agility will soon become a necessity.  There is no better time to start than now.  But how do you get started?  Allow the Dr to make a recommendation - register for our countdown to agility here.  On June 21st I’ll be moderating this webcast featuring NetApp founder and agile supporter Dave Hitz, along with several storage architects who will discuss their first-hand experiences and take your questions live.

Look forward to seeing you on the 21st!

DrDedupe

If the VIX = Stock Market Volatility Index, then I declare that FIX = Flash (Industry Volatility) Index. And judging by the latest flurry of activity, the FIX is very much in.  Expect much more noise and activity in the Enterprise Flash Market throughout 2012 as business leaders in our industry digest the scope of the NAND Flash disruption.  Hint – it won’t be over this year and when we’re done a few years from now we won’t recognize the current Enterprise Storage hierarchy.

When I served as Vice-Chair of SNIA’s Solid-State Storage Initiative almost 4 years ago, our cross-vendor team of storage specialists realized the disruption upon us would last at least a decade.  The early focus was on SSD’s and related FTL firmware, but rapidly evolved to new form factors like PCI, new densities incorporating MLC and sustainable performance specifications.

As the industry rapidly evolved, DRAM Storage Arrays quickly gave way to NAND Flash-based successors and two distinctive camps ultimately emerged: (Updated lists below May 18th thanks to my commenters!)

• Performance Camp – Focusing on the consistently low microsecond response times and multiple GB/sec of throughput – Avere, CacheIQ, Dataram, GridIron, Kaminario, Texas Memory Systems, Violin & WhipTail deliver hard-core performance in a Storage Array form factor, while Fusion-io and Virident do the same in a PCI Card form factor inside the server.  Infiniband-attached all-SSD NetApp E-Series arrays also fall into this category, although that is a relatively recent capability of our HPC product line.

• Value Camp – This group of vendors offers a rich software layer on top of NAND Flash which sacrifices ultimate performance in favor of very high performance combined with advanced storage efficiency and data management features – sometimes including hybrid configs of SSD’s alongside spinning HDD’s.  Astute Networks, NexGen & Nimble Storage, Nimbus Data Systems, Pure Storage, SolidFire, StarBoard, Tintri and the formerly independent XtremIO fall into this camp.  So do FlashCache-enabled NetApp FAS Storage Arrays.

(BTW – These are not meant to be exhaustive lists above, just Flash-focused companies that I’m aware of. Feel free to comment below with others I haven’t listed but should investigate)

Fad or Trend?

In fact, these two camps are indicative of where we at NetApp believe the storage industry as a whole is going.  We predict the emergence of a:

• Performance (aka IOPS) Layer– emerging in the storage stack storing an increasingly large working set of hot data, often very close to the application itself running on the server.  This will be complemented by a:
• Capacity Layer– which will seamlessly integrate with the Performance Layer above, while directly addressing the economics of storing, protecting and managing the orders of magnitude more (colder) Big Data than can be cost-effectively stored on solid-state media.

VST is Future Ready

NetApp’s Virtual Storage Tier architecture was born out of our unique position in the middle ground between these two camps.  Leveraging Data ONTAP’s well-established lead in primary storage efficiency, NetApp has been able to drastically reduce the effective cost of NAND Flash-accelerated storage via deduplicated and thinly-cloned FlashCache blocks delivering high performance FAS Arrays to our customers.  After tens of thousands of deployments, NetApp boasts the highest Flash attach rates in the industry.

But we’ve long ago recognized the Flash disruption doesn’t stop at the array.  Application performance demands will continue to pull Flash up the storage stack and into the server.  Over time, even storage semantics themselves will give way to persistent (non-volatile) memory semantics enabling simpler and faster high performance, real-time applications.  But Storage Class Memory [PDF] and NetApp’s future as a memory vendor will have to wait for another blog

In the meantime, Big Data capacity trends and Enterprise-level data management, availability, protection and efficiency requirements will further entrench non-Flash spinning disk (and resurgent tape) media into a scalable Capacity Layer which will serve as the foundation of this new storage stack, managing an organization’s “single source of truth”, regardless of performance.

Playing Offense or Defense?

EMC will doubtless cast their acquisition of XtremIO as a bold move ushering in yet another new tier of storage (0.5?) and primary storage option amongst a crowded portfolio.  However there is widespread speculation EMC felt the pressure to bolster that fragmented portfolio against NetApp’s highly anticipated Goldilocks Scale-Out Virtual Storage Tier.

Collateral Damage

As I alluded in my introduction above, when the Flash disruption is complete over the next few years (before the Storage Class Memory reverberation is expected) the once lucrative Tier1 Frame Array market will be literally disintegrated.  ESCON/FICON attached arrays will continue to leech off the relatively moribund mainframe market.  Fault tolerant data persistence functionality will move from Tier1 Frame Arrays up the stack all the way to the application layer.  As we’ve already established, performance capability will move to the server / host layer, leaving data management, protection and efficiency to a shared storage architecture.  A Capacity-optimized layer will dominate here, leaving precious little room for archaic Tier1 Frame Arrays.

Playing the Long Game by Looking Ahead

Given all the existing and anticipated Flash-based Storage Industry disruption, it’s somewhat shortsighted to map these upcoming new Solid-State Storage layers and form factors into existing categories derived from the HDD era.  Despite any shiny new razzle-dazzle, don’t expect the storage arrays of the future to be purchased for performance or availability characteristics.  And don’t expect high performance solid-state storage (Flash or otherwise) to provide cost-effective end-end data management, protection or efficiency capabilities. (Object / Archive Storage anyone?)  Being pulled apart from both the Performance and Capacity ends, the era of the all-encompassing Tier1 Storage Frame Array is rapidly sunsetting.

Only comprehensive Unified and Extensible Storage architectures like NetApp’s VST provide a future-ready framework for planning, building and running an infrastructure flexible enough to cost-effectively incorporate new storage media technical trends in a consistent, predictable and sustainable manner.

Chris Gebhardt

Follow me on twitter - http://twitter.com/@chrisgeb

Today VMware announced the release of VMware View 5.1.  View 5.1 has many new and improved advanced storage features that will help increase storage performance and scale.  In this blog post I will discuss some of the new and noteworthy storage related features.

http://www.vmware.com/company/news/releases/vmw-euc-portfolio-05-02-12.html

One of the new technologies in VMware View 5.1 is the View Storage Accelerator (formally known as Content Based Read Cache or CBRC).  This technology helps reduce the amount of data needing to be read from a storage controller during simultaneous read activities such as a boot storm after maintenance or HA event.  With View Storage Accelerator, up to 2GB of memory is allocated from the ESXi host to be used as a host based read cache. Ultimately this helps VMware View environments perform better and scale during boot storm events.  Now you may ask, well what about NetApp and the use of Virtual Storage Tiering, doesn't that solve the boot storm problem. Well yea it absolutely does, but using View Storage Accelerator in conjunction with NetApp’s Virtual Storage Tiering allows customers to scale their View deployments even further, driving down the cost, boosting the performance and ultimately improving the End User Computing experience. 

Here is how it is configured:

1. First log into the VMware View Administrator console.
2. Select View Configuration on the left hand side.
3. Select Servers under View Configuration
4. If you already have a vCenter Server configure click the vCenter Server and click Edit…
5. Here the Edit vCenter Server dialogue box will appear.  Click on the Host Caching tab.
6. In the Host Cache Settings you can enable and disable host caching, change the default host caching size, and override individual host cache sizes.
   
   
7. After you enable host caching, you can then create your desktop pool.  When you create an automated pool the Advanced Storage Options portion of the Add Pool wizard will appear.
   
8. Here you will be able to enable host caching on a per pool basis.
   
   
9. You can also choose if you want to cache just OS disks or OS and persistent disks.
   
10. And also choose what days not to regenerate the cache.  For example, you may not want to regenerate the cache during normal business hours.
    
    

The second technology, which I have been waiting for a really long time and am incredibly excited about, is the View Composer API Integration. This VCAI primitive is currently in a Tech Preview status and it allows VMware View to automate the creation of NetApp Zero Cost clones (FlexClone) within the VMware View interface.  In the past, customers had to choose between NetApp clones with VSC provisioning and cloning in manual pools or VMware View Clones in Automated Pools.  There are benefits and drawbacks to both methods.  On one hand the NetApp clones are created at the hardware level (not using copy on write snapshots) and are pre-deduplicated. This method creates desktops that have superb scalability but to create NetApp clones requires two tools: the NetApp Virtual Storage Console and VMware View Administrator.  On the other hand, VMware View Clones in Automated Pools provide excellent management workflows and automation but rely on hypervisor snapshots as underlying the cloning mechanism, which if improperly designed can result in scaling issues. (Note: Proper design refers to the use of technologies that reduce the amount of write IO to the delta file.  Technologies like profile/persona management (User Data Disks) and end user home directories are examples of such technologies that remove IO and allow VMware Linked cones to scale.)

VCAI leverages NetApp’s native cloning capability – without requiring VSC - to improve desktop scalability and provisioning times in View Composer.

Here is a brief demo of how this cloning works. 

Today the View Composer API Integrations are available as a Technology Preview when using VMware View 5.1, vSphere 5.0, the ESXi VIB for VCAI and NFS on NetApp Data ONTAP 8.1 Cluster Mode.

Some other storage related features worth noting: Customizable View Composer disposable disk driver letter and support up to 32 (increased from 8) host in a cluster when using Network Attached Storage (NFS).

Also a shameless plug for one of my favorite works…

If you are interested in reading more on VMware View workloads, my colleague, Chad Morganstern, and I have done a considerable amount of work to try to better understand the lifecycle of a Virtual Desktop and how Virtual Storage Tiering can boost performance, allowing customers to size for steady state.   https://communities.netapp.com/community/netapp-blogs/virtualization/blog/2011/09/06/netapp-and-vmware-view-5000-seat-performance-report

I've had to repeat myself several times today about the magnitude of the Big Savings by NetApp customers that I blogged about yesterday, so let me spell it out as clearly as I know how

Twenty-Five Billion Dollars

$25 Billion

I don’t believe in coincidences.  Yesterday we released a fact-filled press release on the vital importance and benefits of Storage Efficiency in aggregate (as the Foundation for Enterprise, Cloud & Big Data Infrastructure) on the same date as Greenpeace decided to focus high-profile attention on the environmental impact of large-scale IT (Cloud) infrastructures.  True to our theme, this is a Big Blog post, but read on below to uncover the story behind these Big Savings!

Big Transformation

In my role as part of NetApp’s office of the CTO, I have the great privilege of regularly talking to the architects and operators the largest IT infrastructures in the world. After years of doing this, I’ve observed a fascinating dichotomy.

These app architects are boldly forging new ground in technology areas such as parallel processing, distributed infrastructure, in-memory computing, continuous event processing and eventual consistency, amongst others.  For businesses that depend entirely on their data, I’ve noticed revenue recognition and expense accounting rules permit them to adopt a very top line CapEx-focused infrastructure investment profile, deferring bottom-line OpEx realities until the growth of the business stabilizes.

The Big Story is Big Efficiency

OTOH the unsung operators who maintain said massive infrastructures universally lament that the TCO of these exciting new app architectures “paves a road to bankruptcy” to directly quote a few of them from companies you all know and use every day.

While I’m no tree-hugger, I have a huge amount of respect for the work Greenpeace does and the worthy causes they bravely pursue.  And although the fine print of their latest report remains in question, there’s no doubt in my mind the Big Infrastructure unsustainability Greenpeace has highlighted proves they’re on to something …. you guessed it – BIG

Big Problem

I’ve long contended on this blog and during many of my industry keynotes, breakout sessions and panel discussions that the disturbing trends powering the build-out of MORE data centers featuring novel PUE is akin to treating the symptom of this Big Problem of our time.  Conversely – designing, retrofitting where possible and managing LESS data centers via more efficient *infrastructures* treats the root cause and is the foundation of …

The Big Solution

A little over 5 years ago I began to blog via a highly popular guest post on Robin Harris’ StorageMojo blog about some (not coincidentally) big installed base data about the analysis of drive failures.  Since then NetApp has embarked on one of the most ambitious Big Analytics projects outside of the Web / Social Media universe.  In fact, we just implemented phase one of our Hadoop-based next-gen solution to efficiently process One Trillion (not a typo) records – and counting!   You could say, at NetApp we do more than talk the current trendy Big Data talk. We also walk …

The Big Data Walk

This public disclosure of our ASUP.Next (slideshare link) project summarizes how we process real-world telemetry from our giant installed base to deliver cool services to customers while also sharing insights into the Macrocosm of Enterprise, Cloud and now Big Data IT overall.  Here are some important data points to consider.

Big Capacity

This simple bar graph above is an excellent proxy for NetApp’s growth over the past few years. Although I’m proud to highlight that our capacity shipped grew faster than revenue – proof positive that we deliver more value to customers with every passing quarter; there are other insights to discuss:

• The dominance of ATA drives.  FlashCache technology has enabled us to drive customer storage costs lower by not only Virtually Tiering data simply and automatically, but also by literally shifting customer purchases to the densest and implicitly most green and cost-effective drive types of any given period.
• The rapid progression of Exabytes shipped.  It took us a whole year to ship that gargantuan capacity in FY09.  Only half that amount of time by FY11 and starting last quarter (Q3FY12) only 90 days to do it!
• Our own capacity disclosures confirm geometric growth of storage capacity, trending towards exponential.

Big Storage Efficiency

Drilling down a bit further on the 5 Exabyte capacity of big installed base data we’ve accumulated, we see the overwhelming adoption of mature Storage Efficiency technologies such as Snapshots and RAID-DP, but also strong adoption of relatively newer Storage Efficiency technologies such as Dedupe and Thin Provisioning.  Thin replication via SnapMirror (the #1 storage replication technology by market share) continues its march towards universal adoption.  If anything, adoption of our Thin (Virtual) Cloning technology shows below my expectation here, but I suspect that # will grow commensurate with our growth in hypervisor-dominated compute Clouds.

How can you take advantage of this Big Information?

If you currently own a deployed (non depreciated) storage solution, check out our V-series payback guarantee program (where we translate the 5 EB into other terms):

http://www.netapp.com/us/products/storage-systems/v-series/pgp.html

If you want to learn how much you can save as a new NetApp customer, then the tried and true Storage Efficiency Calculator (SECalc) is available online:

http://www.netapp.com/us/campaigns/it-efficiency/?REF_SOURCE=bnritfhplm2

As an existing NetApp customer, you should definitely check out your custom MyAutoSupport dashboard:

http://www.netapp.com/us/support/autosupport.html

Big Coincidence?

Once again, I believe everything happens for a reason. There’s a revolutionary new release of Data ONTAP 8 coming that underpins the future of Big IT Efficiency and Big IT Savings. While the Big Marketing launch for that release will happen later this summer, as of *today* NetApp customers and partners can now download the GA version of this monumental Scale-out Data ONTAP milestone here.  Enjoy!

http://support.netapp.com/NOW/cgi-bin/software/

And as always please feel free to share your Big feedback via a comment below.