As I mentioned from my previous short post on Gazopa this post contains an interview from Mr. Hideki Kobayashi at Hitachi Limited in Japan.  Mr. Kobayashi is the Gazopa project leader running a team that spans both the United States and Japan.   With that said, let’s jump right in.

1. From your perspective what is the most exciting thing about Gazopa?
   
   The most exciting thing about GazoPa is that it enables users to search for  images by uploading images and through online drawings. With a conventional image search, only a key word could become a search query. So I view the approach of the technology within GazoPa.com as extremely, powerful, natural and highly intuitive.

2. What kinds of applications developers or companies are interested in the technology behind Gazopa?
   
   E-commerce sites and stock photo sellers have an interest in our technology. Game companies are also quite interested in our technology. We’ve also released a second Facebook application called GazoPa Drawing. It’s a kind of game, where seers compete with others to create the most similar drawings.

3. Is there anything that is coming to the Gazopa application which you would like to hint at?
   
   We’re planning to release a new application that uses our image search technology to solve e-commerce problems.

4. What kinds of applications do you believe that Gazopa technologies will be used in?
   
   E-commerce and stock photo sites and community sites that have lots of photos and images. However we are open to anything and anyone who has a thought an idea on how to use the technologies behind Gazopa.

5. Can you provide us some history of Gazopa from initial idea to its current state?
   
   We started development of the core engine more than 10 years. However at that time there was no iPhone and YouTube so we could not really express the capabilities of the core engine until now. We started developing GazoPa.com in the spring of 2008, and we were selected as one of finalists of TechCrunch50.  In September, 2008 we started our private beta, and later in 2008 we released our first Facebook application.  Following the Facebook  application, we also released the iPhone application to further illustrate how our technology can be used.  I’m also proud to say that GazoPa.com has entered open beta this month, so everyone should head on over to Gazopa or download the iPhone application and give it a try. (Note that to get the iPhone application to work you will have to sign-up for an account at Gazopa first.  We soon hope to remove this requirement in a future release of the iPhone for Gazopa application.)

6. There have been recent articles and releases by Google as well as others who suggest that they are in the image search business.  Google has even stated that this is a feature and not a product.  What are your response to the industry reports, analysts and competitors?
   
   Basically I view Google’s and other’s comments on this capability as business domain specific. If you want to become a general image search engine provider, keyword is still a main search query and similar image is a feature. However, if your focus is on similar image searching, one can focus on being a similar image search engine provider with specialized services and technologies suited to that business and technical problem domains.  Further one can also provide the core technologies in an ASP model to other online applications, applications on the iPhone, applications in Facebook, etc.  Hitachi is not out to compete with Google in terms of generalized search, we are looking to bring our technologies to market through other providers and we view Gazopa as an excellent demonstration of what is possible for others to make use of with our technologies.

7. A bit of a technical question.  What are the components and languages that the Gazopa application and services are written in?
   
   The Gazopa front end is developed in Ruby, and the core engine is developed in JAVA and C.

8. Finally if there are established companies, start-ups, open source developers, etc. who are interested in Gazopa who should they contact?
   
   Please contact me at: hkoba@gazopa.com

The act of innovating; the introduction of something new, in customs, rites, etc; A change effected by innovating; a change in customs; something new, and contrary to established customs, manners, or rites; A newly formed shoot, or the annually produced addition to the stems of many mosses. (source: http://en.wiktionary.org/wiki/innovation)

I think that too often in our modern world we are tricked into thinking that innovation can only occur in the domains of sophisticated R&D labs, an the innovations themselves can only be scientific in nature. I believe that Alfred is a great illustration that something simple but new can be extraordinarily innovative.  For the longest time one of my personal driving forces has been to think of one thing that no one has thought of before; I envy Alfred who has thought of this new thing.

Note that the above Youtube video shows the iPhone application in action.  In a future post I’ll be doing an email interview with one of the members of the Gazopa team at Hitachi America.

Hitachi (Hitachi Data Systems and Hitachi Consulting) is at the MS Sharepoint conference displaying our competencies for Sharepoint including enabling products and deployment approaches.  I’ve talked about the Hitachi Data Discovery for Microsoft Sharepoint (HDD-MS) in the past here and here.  Here’s a couple of really great pictures taken by my colleague Keiko Harada with respect to HDD-MS. I think that the first one is pretty cool and includes several of the Hitachi booth swag as the audience. The second photo is of our entire booth, with HDD-MS being shown integrated to the Hitachi Content Platform (HCP).  Again as I pointed out in my previous post, HDD-MS is a light weight plugin that enables direct from Sharepoint to HCP content storage.  Unlike our competitors’ products which require Sharepoint, content management offerings, potentially data archiving software, and then long term cloud storage — oops wait as far as I know ATMOS(T) isn’t integrated to any applications yet — we’re pretty skinny in terms of our implementation.  The combination of HDD-MS and a durable backing store assists both with making it easier to backup Sharepoint as well as being able to allow for simple eDiscovery across multiple Sharepoint sites, site collections, farms, etc.  Also with the capacity optimization features within HCP, if end users make a habit of using the sneaker-net consisting of email and USB drives to distribute content outside of Sharepoint leading to duplicates, then HCP can both perform the single instancing as well as compression allowing IT professionals to cope better with sneaker-net content distribution.  With all of this users continue to use their document libraries, Sharepoint permissions, versioning, etc.  We are just moving content under the covers, using REST, without disturbing their behaviors. Anyway kudos to Keiko and the team at the Sharepoint conference.

If you had not guessed already I’m sticking with the Kermit the Frog theme here.  And just for grins here is the Rainbow Connection with Kermit and Debby Harry singing.

So on my previous post I talked about files being “the objects” stored out there in the cloud.  I know, I know that is a really obvious point, but as I mentioned even truly novel ideas are obvious when you look at them on paper.  One of the greatest quotes of learning a horrible yet obvious point is “Soylent Green is people!” which is from the 1973 film Soylent Green.  In this the main character through various adventures shows that the reality behind the food people are eating is well people.  I feel like this was my point at files being objects.

There’s a pot of gold at the end of the rainbow?

Okay sorry for the digressions, in this post what I want to touch on is scale and infrastructure in the cloud.  The first point that I want to make: for new efforts which have not been done before that do not have commercial software already available of course new technologies emerge.  However there is always a hype cycle which everyone is familiar with and is rather like a gold rush in some sense.  However, hopefully the consequences are no where near the dotcom crash. Speaking of hype, I’m sure that during the early part of the RDBMS hype cycle databases were going to replace everything in the world including file systems.  Oops that didn’t happen.  The same with the hype surrounding distributed systems versus the mainframe where the distributed system was going to replace the mainframe and the mainframe was a dinosaur — yeah a T-Rex!  Well that did not happen either.  So cloud is not the end all be all for everything that the world is hyping it up to be.  Cloud is about a new set of problems that cannot be solved using traditional COTS approaches.  That is why Google, Microsoft, Amazon, Apple and others are using a lot of manpower and a mixture of various commercial and open source stacks to scratch their itch for their businesses.  This is largely my point behind citing good old Kermit’s song where the lyrics talk about the rainbows being visionary but merely an illusion.  So what I’m saying is that I agree with Tony about the current point in the hype cycle and that the promise of cloud solving everything sounds like a great vision but when you finally get there, like a pot of gold at the end of the rainbow it was a myth.

Hammer in a nail not a screw

I think back to several years ago when I talked to a budding SNS company.  They were looking for specialized file systems which were application specific and not general purpose.  A file system that was flat and tuned for this company’s particular application which required billions of small files and really really random I/O.  Essentially, two of NetApp’s highest end systems were not cutting the muster with their CPU busy rate pegged to as close to 100% as possible.  The bulk of the company’s image I/O was handled by their squid web cache, which responded to 95% of the I/O requests from their application and users.  While the NetApp systems were on their knees with only 5% of the remaining workload.  Basically the WAFL file system is not well suited to really small file random I/O, in the long term, without running what is effectively a defrag on the WAFL file system.  (In fact if it was read optimized we would call it the Read Anywhere File Layout or RAFL for short.  Wanna buy a ticket?)  Because the company in question could never take one of the systems down — the CPUs were pegged at 100% — they could never run the defrag and therefore this was causing them significant performance availability problems.  I know that in the intervening time NetApp has surely worked to remedy this, but I want to use this as an illustration of the fact that you have to use the right tool for the job.  If there was a specific image repository or store which was optimized for the I/O workload and capacity then I think that the company in question would have been okay.  They were effectively trying to hammer in a screw which is fatal for the screw, NetApp was the wrong tool for their application.  Hopefully you can take the leap here and see that the SNS domain is an emerging effort and therefore requires new technologies to solve related problems.

Modern Internet geology

One of the most famous communities on the Internet is Slashdot who largely predates many of today’s web communities as well as architectures which are now famous — like Google.  If you take a look at their architecture you can find that they dedicate special purpose systems to storing static content, a common practice for many web applications.  What I see as going on in the cloud segment today, with respect to emerging applications and businesses, is the big infrastructure crunch.  That is to say offerings like the Hitachi Content Platform which serves up static content via HTTP/REST allows these emerging businesses to not worry about the mundane parts of their infrastructure and instead focus on providing value added features to their end users.  The image and model that I have in my mind is of sedimentary rock which starts out as living vibrant things with a lot of action and motion, and eventually falls into a lake or another body of water becoming brown peat moss or middleware.  Give things a few million years and eventually this vibrant stuff turns into well rock that you can build new structures on top of.  The infrastructure crunch is really just like that.  It is about taking things that need scale and performance and sticking them in a layer that can provide these attributes and then well allowing the upper application layers to forget about this stuff.  In the case of the Slashdot architecture HCP could crunch down their architecture and hold their static content and then they could forget about having to manage these systems moving on to supplying more value added functions/features.

Mature applications

With all of this talk about the emerging applications what about the mature applications like ERP systems, email systems, groupware systems, RDBMS systems, etc.?  In the race to deploy things in the cloud, these systems need some approach to get them into a cloud like infrastructure which seems to me like a utility approach to IT infrastructure.  In my opinion Hyper-V, XEN, VMWare, Oracle-VM and others can do just that.  While the current driver for virtual machine deployments is really about cost savings — note that even Green gets people’s attention because it saves money — in the long term I believe it will be the way that mature applications make it into the cloud.  The switching cost of changing these applications is super high and the cost of engineering new applications which are functionally equivalent is super high too.  So in my humble opinion I think that we will see architectures much like what Amazon has deployed, but inside of a company as the private cloud.  My colleague Hu has just put up a great post on what he views as an in effective architecture, VMAX for these workloads.  I think Hu is on to something here and I want to augment his point: people who have already made an investment in their SANs and existing infrastructures want to reuse their investments and rightly so.  Throwing these SAN architectures out which have served companies for years to create strong infrastructures is a short sighted approach.

Conclusion

So just to summarize a bit.  The objects that everyone is talking about are files.  As we pass through this cloud hype I think we will find that a lot of what we thought it would be was basically the illusion that there is a pot of gold at the end of the rainbow.  You need the right tool for the right job and COTS solutions today don’t completely solve the infrastructure problems for emerging applications.  Currently we are in a phase I call the infrastructure crunch which seeks to commercialize the portions of these emerging technology architectures for added scale, reliability and performance.  Finally we cannot forget about the emerging applications, heck even Amazon did not do that, and we have to help people and companies maximize the return on their investment they’ve already made.

Storage Fusion - StoraFgUeSION - SfTuOsRiAoGnE

“HybridStorage” refers to the mix of flash and magnetic storage medias blended together to increase performance and efficiency. This particular effort is part of the HybridStorage Alliance to promote and educate the industry on the benefits and advantages of this approach. While I’m not personally involved in this organization’s efforts, Hitachi is. This bring back memories of the floptical disk and the NeXt Machine. However, I’m not going to discuss these devices here in detail. What I am going to describe is Storage Fusion which is a term I just made up because Hybrid storage was taken. I love thesauruses. Also, this is different than Controlled Instancing which places data at the right place, at the right time, for the right reason.

Specifically, what I see Hybridstorage proposing is that certain types of data can use or benefit from different types of storage media, not that we storage aficionados didn’t know this. So Hybridstorage is tiered storage media within a HDD form factor. Similarly, but at a higher level, Storage Fusion proposes a similar premise, certain types a data needs different types of storage or file systems. For example, with the proliferation of internet data and various services, it was noticed a while back that most of the bits on the web are static. While the storage industry calls this stuff “fixed content”, reference data, or write once-read many (WORM), these terms tend to be use to describe compliance or archiving types of storage uses and have a less active nature. In reality, there is a lot of bits patterns out there that never change, are highly active (read a lot) and are usually just replaced instead of being modified. I just used the term “bit patterns” but could also use ‘byte patterns” or binaries because most people use the term “data” to describe “their data” or information. What is an application, or operating system programs, devices drivers or binaries when stored on disk? This too is data, but is rarely referred to as “data”. An interesting difference with this type of data is that there are millions of copies in some cases of this data. Think of your Windows Vista or XP, or a Linux distro. For every installed version there is at least one CD/DVD copy associated with it, site licenses not withstanding. With many Linux distros there’s the online copy and usually a plethora of mirror sites, your downloaded ISO images and possibly the CD/DVD version you’ve burned. So protection of this data is usually not an issue, but the inconvenience of rebuilding your systems is always annoying but doable. Useless you make a Ghost image like I do.

So let’s scale this scenario up to say 500 TB and climbing. There’s data and there’s “data”. For some reason, when we get to this level of complexity, we end up saying operational data and user generated data. Operational data is the programs, applications, binaries, templates and configuration information that constitute the “state” of the system, and user generated data, with “user” including another machine, is the data that is stored by users or the results of applications.

Take an online picture storage and sharing web site for example. The web servers, database servers and application servers running the applications of this web service could store and provision the storage centrally. This would allow the compute nodes to essentially be stateless and easily replaceable during failures or upgrades. Likewise, the user generated data would be stored centrally in a content repository like the newly announce Hitachi Content Platform. The operational data is relatively small compared to the user generated data, but the activity of this data is highly active with much of it pinned in system memory. Booting nodes, loading application programs, serving up static web content and the assorted graphics that are locally stored are candidates for Storage Fusion where the file system is comprised of different storage types and media for the different types of data being stored. Hitachi Tiered Storage Manager does just this under the file systems. The operational data would end up on SSD or high speed SAS/FC drives using RAID-5 or RAID-10 protection levels depending on the databases. The bulk of the storage would be magnetic using SATA drive and probably a RAID-6 protection level and would store the user generated content, in this case pictures, that have a much lower activity level and that continues to diminish in activity over time. However, unlike Enterprise data management, this type of data cannot be deleted by the service provider so long as the service is available and the subscriber is paying the fees.

Actually when you think about this scenario and other similar use cases, only a small percentage of any of the data stored is modified. The databases are probably the most active with regards to updating data. The most activity is reading both operational data and user generated data, uploading data and storing it, maybe moving it somewhere after initial load and removing data. In these cases you could craft ways of using optimized file systems and object storage with different media types and RAID levels to obtain the most optimal storage infrastructure to meet performance demands.

• User permissions - objects must contain enough state such that when the objects are ported to another system the user permission state is persisted.  Well, guess what files can preserve those attributes and that is why things like robo-copy are used to move files from one system to another because ACLs and permissions are preserved
• Self describing format - the object should contain enough information to tell the surrounding systems what can and cannot be done with it. Well, guess what DRMed files — like what is on my AppleTV — are stored with security information included such as expiration time, permission to execute, etc.  Further systems like Microsoft Office include IRM that allows content authors to specify what users can and cannot do with the content.  Finally there are many applications out there that keep or store files in an XML format so that in the event a company goes out of business then the meaning can be regenerated by another program or even by a human — although I have hand edited/hacked Postscript files before
• Explicit/Implicit metadata - objects should have the ability to associate arbitrary metadata with them and also include a set of inherent metadata.  Well, extended attributes, the ability to create files associated to primary files, or even use directory structures to encode metadata are all relevant ways to add arbitrary metadata for files.  Further every file has implicit metadata like the size of the file, the inode number, etc.

I’m sure that there are more, but the point I’m trying to get at is that cloud storage should obviously be about files ladies and gentlemen.  We aren’t storing something new or different, we might be fully making use of the technologies and approaches which were already in place, but underutilized.    So when you next hear the latest cloud guru talking about exotic object storage systems ask a simple question: what are those objects you are talking about?  I suggest that you also follow-up with a healthy dose of: here is my business problem can you solve it at the right, cost, with investment protection (hey this is a nice way of saying that there is a roadmap and you can grow with your vendor), with the right performance and reliability, etc.  After all being a student of the games of billiards one of the key sayings is that a good billiard player can make use of a broomstick to cream someone at the selected game.  This implies that if you use block storage and Hyper-V to solve your problem and you did it in less time than your competition then you’ve done well.

Finally as to the title, well I started out an introduction with a reference to Kermit the Frog and now I will end one as well.

“Time’s fun when you are having files, oops flies.”

P.S.  From my somewhat regular discussions with Intellectual Property attorneys they often point out that patents are obvious and often simple once someone puts their idea on paper.  However before that point many people are baffled.  I think that while this file + cloud concept might be obvious to everyone now that I’m writing about, even my competitors, it needed to be said.

1. Chapter 1 - talked about the emotional hang-ups associated to getting data in the cloud.  So in essence while we can understand intellectually that our data can be protected, until we get it at a visceral level, we don’t really “get it.”
2. Chapter 2 - essentially points out that some of the emotional hang-ups are alleviated by the next generation of workers coming into the US over the next decade, nearly 80 million. It’s interesting to note the rough match in China is nearly 300 million people by some estimates. There is not an estimated number for this population in India, but we could guesstimate that the number is in the 100s of millions of people.
3. Chapter 3 - asked the question about TCO with the all of the rage being the cheap storage and Backblaze becoming the poster child du jour.  (Note the folks over at Object Matrix objected to my personal assurances of Hitachi’s continued survival.  Nick and I did have a little bit of a “loud” debate but his point of storing content in an open manner is spot on.  This is something that Hitachi has been practicing for sometime even on the block storage side of the house.)

So how does all this come together? Am I making good on my promise that Hitachi can and will help with various cloud activities?

Today we announced our intentions for cloud and the new Hitachi Content Platform (HCP) – which is slated for release later this calendar year.  We are augmenting HCP, which is already a venerable cloud storage contender, to include more features making it even more compelling for private cloud deployments, and we aren’t just stopping there. In the past I’ve mentioned that when EMC announced their intentions with ATMOS(T) they were really joining a crowd of folks who had already done this before — not to mention silently replacing Centera (but shhhh, that’s a secret). What EMC did not do very well, and we are pushing the ball forward on, is direct and indirect application integrations to the cloud.

In the HDS solution, the first integration is via our Sharepoint Plugin, Hitachi Data Discovery for Microsoft Sharepoint.  Since we first GA’ed this product almost two years ago, it used ultra-modern REST access to push files into HCP. We’ve continued pushing the ball forward since then to include persistence of Sharepoint metadata, scheduled retention, etc.  However, if we also look at the Hitachi NAS Platform (HNAS) models’ 3200, 3100, 3090 and 3080 file content into the cloud, we have what is sometimes referred to as a “cloud on-ramp” which allows applications that have not made the leap to REST to continue using CIFS/NFS, while also taking advantage of getting their data into a private cloud(note that these are topics which I’ve pointed out with more detail in the past here, here and here).  Our systems can be combined so that if customers have made a previous investment in consolidated SANs they can just add another application.  This is not like our competitors who can only offer stove pipes, which has little to no integration.

If you want, you can also use Hitachi Dynamic Provisioning (HDP) to work with your cloud on-ramp attached to an AMS to support an application that uses NFS.  The on-ramp can then send data over REST to a centralized private cloud on HCP that is attached to a USP-VM and make use of Tiered Storage Manager.  Further, this large centralized cloud is replicating to a bunker site which is made up of HCP 300 all running with embedded disk storage. And, we can do all of this today! So for us, cloud is not necessarily about a single product per se,  it is about making sure that applications which are tuned for the cloud can take advantage of it. It is about deploying a private cloud on top of existing assets, and enabling many capacity optimization features including single instancing, compression, HDP, WAN optimized replication, etc.  This is a key tenant of Hitachi’s strength and where we leave our competitors in the dust!

When I look at Henry’s article I see one point that I want to call attention.

Clearly, moving 100PB on even 1TB enterprise drives can potentially cause significant loss of data, especially as many clouds I am familiar with do not use RAID and maintain data protection via mirroring.

This does point out something that I think everyone should be paying attention to: multi-layer data protection!  Combining RAID, data replication, object cloning and policing, checksumming file systems, etc. should all lead to a more reliable system overall.  To prove my point I’m going to go into the realm of security for a second where multi-layered security is a norm.  Here is a really easy way to understand this concept, because it is more than just associated to the link above.  I know that there are many companies that for their use of firewall deployments intentionally select two vendors.  Now usually this would be for keeping each vendor honest with the entire customer experience however, in this case it has to do with having two different protection paradigms.  You see with one vendor there is one implementation including all features, bugs and vulnerabilities, but when two vendors are added there is another set of features, bugs, and vulnerabilities.  This implies that if a crack for one vendor’s system is pushed into the wild the other system has a higher chance of not impacted.  So it is the diversity of vendors, when combined, that actually increases the security and reliability of the aggregate system.  Hopefully, you are following me so far as I’m going to switch back to storage and the cloud now.

Henry’s point above really hints at this point: namely multiple data protection schemes are required at the media level to ensure that objects and blocks avoid the evilness of corruption.  Combining RAID, read after write on S-ATA, object cloning, remote replication, usage of alternative media types, etc. are all ways to improve the aggregate reliability of the system.  So I agree with Henry’s statement that we need to be looking into this as a key part of moving into the cloud.  Perhaps it is because of my hangups that I’m worried about these kinds of things.

I ‘ve been reviewing Symantec’s new FileStore and their soon to be released S4 offerings. There are quite a few resources available on FileStore from their slides on the Internet to their product page.  Firstly I want to welcome them to the clustered NAS business much like HP is there already with EXDS, their Polyserve appliance.  However it seems to me that Symantec like Oracle.SUN and Hitachi has joint the greater than 200TB file system club.  HNAS with WFS2 can support a file system size of 256TB and Oracle.SUN.  (Note NetApp, EMC and HP do not offer products with filesystems of this size.  Although NetApp will most likely debate that point with their Clustered Namespace available with Ontap V8 in GX mode.)

Since there is not a lot of clarity on S4 I’ll skip it until we can see something with some “real meat on the bones” to analyze.  With respect to FileStore I do have some questions which I would like to find out the answer to:

1. In the slide materials from Symantec there is mention of NFS Spec performance (SFS2008) however the y-axis has no numbers on it, but Symantec claims near linear scalability of performance up to 16 nodes and that they will be faster than the fastest number from NetApp.   My question is where are the draft numbers and why did the presentation even include a performance graph if they weren’t ready yet?
2. FileStore is a soft appliance and as such runs on any backend storage including SAN attached.  To me this is a validation of the approach which Hitachi has taken, and in some cases we offer RAIN as well as SAIN modes of operation.  The question that I have is largely related to price here.  The quotes in the press are that for just under $7k USD you can purchase a node pair.  Does that include Storage, nodes, etc.?  Is there a per capacity license, etc.?
3. Embedded DLM/ILM or data migration policies are something else which are a part of FileStore (SSD to SAS/FC to SATA).  Just like HNAS these features are baked right into the NAS software.  One point of differentiation between the two products appears to be the fact that HNAS can target external file/object storage systems such as the Essential NAS Platform as well as HCAP.  So no question here just merely an observation.

So I warmly welcome Symantec to the clustered/scalable NAS business and cannot wait to see how we compete in the market.

]]> http://blogs.hds.com/michael/2009/10/welcome-symantec-to-the-scalable-nas-market.html/feed http://blogs.hds.com/michael/2009/10/welcome-symantec-to-the-scalable-nas-market.html