One thing was clear. HPC is no longer just about the big boys. Vendors are waking up to the vast potential digital simulation holds for smaller businesses. There’s a broad emphasis on delivering simple, production-ready cluster solutions that run popular applications out of the box. Given that emphasis, it’s not surprising that Intel Cluster Ready was visible almost everywhere.

We had two Intel® Cluster Ready partners, Altair and Fujitsu, presenting their solutions in the booth. Ten Intel Cluster Ready partners gave theater presentations, 25 were promoting Intel Cluster Ready with signage and collateral, and more than 150 participated in the Intel Cluster Ready Partner Celebration. Their enthusiasm says a lot. Their videos and presentations say even more. I’ve posted a few highlights below. Enjoy!

• transtec explains how Intel Cluster Ready can help you achieve higher productivity with lower cost and risk.
• E4 Computer Engineering discussed how Intel Cluster Ready helps to shrink deployment timeframes to just days and simplifies management and use.
• Fujitsu talks about its new line of ready-to-go HPC solutions and how Intel Cluster Ready helps them deliver higher value across the full range of cluster requirements.
• Intel Cluster Ready team members and partners discuss their solutions, Intel Cluster Ready, and the market inroads their making at the Intel Cluster Ready Partner Celebration.

Want a broader overview of ISC ‘12? Check out these cool simulation examples from Altair and these 6-minute “hot seat” vendor videos—including this one from Dr. Stephen Wheat, General Manager of High Performance Computing at Intel. He talks about Intel innovation in HPC, including an upcoming Intel processor that can deliver up to a teraflop of performance—all by itself.

That’s something new in the HPC world. It took a lot of behind the scenes effort for Intel and independent hardware and software vendors to synchronize their development cycles and launch dates. Dozens of systems and applications were available on day one. Customers had other options, too. They could upload their application to a test bed cluster from a leading cluster vendor. Or they could spin up a Intel Xeon processor E5 family cluster in a public cloud in a matter of minutes.

The upshot of all this is that new clusters based on the Intel Xeon processor E5-2600 are now available from a number of leading hardware vendors, along with cluster management software and a variety of optimized applications. Many of these products are Intel® Cluster Ready certified (or registered, in the case of applications). That means you can mix and match them with confidence. Any registered application will run reliably on any certified cluster right out of the box, which makes implementation a lot simpler and less risky. Pick an application. Pick an appropriately sized cluster. Load one on the other and you’re off and running—at significantly faster speeds than you would have experienced just a few weeks ago.

If you’re not familiar with the new Intel Xeon processor E5 family, you’ll find that it delivers more resources across all key parameters in comparison with the previous-generation Intel Xeon processor 5600 series. Whether your workloads are CPU-, memory-, or I/O-bound, you should see significant performance benefits. Highlights of the new architecture include:

• 2X more I/O bandwidth, with a 32% reduction in latency
• 2X more socket-to-socket bandwidth
• 2 more cores and 2.5 MB more cache per core
• Up to 2X faster floating point performance
• More and faster memory (up to 768GB/1600Mhz in a dual-socket system)

If you’re an HPC user looking for cluster solutions based on this new processor, any of the following Intel Cluster Ready members would be a good place to start: AMAX, Atipa, ClusterVision, Colfax, E4, Eurotech, Fujitsu, Silicon Mechanics, Viglen, and transtec. Each of these vendors had Intel Cluster Ready certified systems or registered software available at launch. The solutions they offer range from small clusters that could fit on your desk to multi-rack supercomputers that push the limits of HPC capability. (You can find additional lists of certified clusters and registered applications on the Intel Cluster Ready website.

If you’re looking to build your own cluster solutions, you might want to check out the following resources:

• Intel® Cluster Ready Reference Designs
• Intel® Cluster Checker 1.8, which includes full support for the Intel® Xeon® Processor E5 2600 processor family (and the associated Romley platform).
• Intel® Xeon® Processor E5-2600 Family Cluster Solutions Guide, which provides an overview of supported components for the new Intel Xeon processors. This guide provides information and recommendations about Intel server hardware, new processor features, management software, storage, interconnects, and other HPC cluster components. It can help you get an optimized cluster up and running faster. You can download the guide here.

Regardless of whether you’re a user or a vendor, I hope you’ll look into the new resources that are now available. Cluster computing is getting faster, simpler, and a lot more cost-effective, which is opening up new opportunities for everyone. The Intel Xeon processor E5-2600 family is just another step forward on the journey. Well, maybe a leap.

Like many of you, I attended SC11 in Seattle this year. I have to say, the world’s largest HPC convention has become downright addictive—especially if you happen to be an ex-gamer. I had to extract myself from the “Gaming through the Decades” exhibition before I could even get started with more serious pursuits. Fortunately my willpower was well rewarded. In case you weren’t there, the focus was on big data and visualization, and the strides being made are truly amazing.

The advances made by the Intel Cluster Ready partner community are also very impressive. We held our third annual Partner Appreciation Award ceremony at SC11, and it was gratifying to see the progress we’ve all made in just the past year. The Intel Cluster Ready architecture is now represented across an enormous range of HPC solutions, from SMB clusters to some of the world’s Top500 supercomputers. Hardware and software support has also grown substantially, as has market penetration. For a look at a few of the most impressive developments, check out the award-winning partners below, and follow the links for more information. I think you’ll be intrigued.

We gave awards in three categories:

• The Pathfinder Award honors the vendor who has done the most to forge new paths into a previously unexplored area.
• Explorer Awards distinguish those who have continued to explore and implement programs that significantly advance the portfolio of solutions based on the Intel Cluster Ready architecture.
• Voyager Awards are for those who have contributed to expanding the global reach and overall market growth of Intel Cluster Ready solutions.

Here’s the list of the 2011 award winners. Enjoy!

Pathfinder Award

• Altair continues to drive innovation by making HPC cluster computing simpler and more productive for its customers. Over the past year, Altair integrated Intel Cluster Checker into PBS Professional, its HPC workload management and job scheduling solution, and registered three key applications for design and analysis: RADIOSS, AcuSolve, and OptiStruct.

Explorer Awards

• science+computing took Intel Cluster Ready into the upper reaches of HPC, installing a 292 TFLOP, 1700 node supercomputer at RWTH Aachen University. It’s the largest Intel Cluster Ready certified system to date, and one of the 30 largest supercomputers in the world. A joint success story will be coming soon to highlight the design win and promote Intel Cluster Ready.
• NEC delivered a huge Intel Cluster Ready win, with a certified 500-node cluster for the University of Erlangen. NEC then helped drive visibility and credibility for Intel Cluster Ready solutions with a success story and Web promotions.
• E4 Computer Engineering launched its E-Cluster 7225 platform, certified Intel Cluster Ready, which scales from four to hundreds of nodes! E4 has also integrated Intel Cluster Ready throughout its marketing communications and updated its website based on each new sales quote.

• Fujitsu has demonstrated a strong commitment to Intel Cluster Ready, integrating support throughout its PRIMERGY server product lines, and have achieved the greatest number of certifications in 2011 for certified Intel Cluster Ready clusters shipped in Europe. Starting at Fujitsu Technology Solutions it became a well-integrated part of the technical production workflow—beginning to end. Intel Cluster Ready is now an integral part of the Fujitsu standard cluster software stack for Intel architecture, and allows them to address a multi-geo market with HPC systems that are 'ready to go'.

• Bright Computing has played an integral role in enabling the Intel Cluster Ready ecosystem by fully incorporating support into its workflows and products, including integration of Intel Cluster Checker into Bright Cluster Manager 5.2. The company was instrumental in helping ensure that the complete HPC solution chain will be ready to support the Intel Xeon Processor E5 family at launch.

• The HPC Advisory Council made broad contributions to Intel Cluster Ready by testing, benchmarking and documenting implementation best practices for open-source applications, co-authoring and publishing a technical brief, and partnering in workshops to extend the awareness and value of Intel Cluster Ready solutions to new audiences.

Voyager Awards

• Flow Science created a cluster solutions guide for Silicon Mechanics and other OEMs to simplify the deployment of Flow-3D, a leading computational fluid dynamics (CFD) solution, on Intel Cluster Ready certified systems. Through this and related efforts, Flow Science is helping to bring HPC—and Intel Cluster Ready—into the digital manufacturing market.
• Silicon Mechanics has integrated Intel Cluster Ready into its technical workflows and sales processes to simplify and accelerate cluster implementation. The company also played a pivotal role in supporting the launch of Sandy Bridge EP and partnered with SIMULIA and Flow Science to deliver optimized, turnkey solutions based on the Intel Cluster Ready architecture.
• SIASA (Soluciones Informaticas Argentinas S.A.) was the first Intel Cluster Ready member in Latin America and has delivered more certified clusters than any other vendor in the region. As one example, SIASA helped the University of Cordoba implement the largest cluster in Argentina.
• Platform Computing is a leading Intel Cluster Ready partner and made a major contribution this year by launching Platform Cluster Manager 3.0, which makes it easier for non-technical customers to build, deploy, and manage Intel Cluster Ready certified systems. The company also provided valuable development support for upcoming Intel server platforms.
• ClusterVision – their commitment to Intel Cluster Ready is so strong that they have edited all their marketing materials and website to promote Intel Cluster Ready. They also implemented Intel Cluster Ready training into their annual sales meeting and launched a sales incentive program. The strategy has delivered new cluster solutions, including a cluster deployment for Ceres Power that was documented in a success story.
• Atipa has shipped eight "Rapid Launch" Linux Clusters in the last two years, all Intel Cluster Ready certified. Atipa has played a valuable role in enabling the HPC ecosystem for the launch of Sandy Bridge EP.
• Eurotech launched its Intel Cluster Ready certified Aurora product line. Then, after collaborating with AuroraScience on a dense, green cluster that’s certified Intel Cluster Ready, Eurotech helped produce a success story about the unique 10 Teraflop Aurora cluster that serves a variety of Italian research organizations and provides a basis for scaling to petaflops of performance.
• Univa has been a long-time supporter of Intel Cluster Ready and recently launched its UniCloud middleware, which makes it easy to deploy Intel Cluster Ready certified HPC clusters in private, hybrid, and public clouds (including RightScale and Amazon EC2).

That’s it for this year’s winners. I’d like to add my personal thanks to all of them.

Stay tuned for more information. I’ll be focusing in on some of these technologies and solutions in more detail in future blogs.

Whether you're a cluster user that expects optimal performance and functionality every time you run a job, or a system administrator that needs to keep the cluster in perfect working order for your users, running checks on a regular basis is important!

A great way to ensure that your HPC system, certified Intel Cluster Ready, remains in the same great shape as when it was first built, is to run the Intel® Cluster Checker tool regularly.

While it’s easy enough to run Intel Cluster Checker on your cluster(s) once a week, I've found there are times when I was too busy, or simply forgot to run a check. For this reason, and because I have multiple clusters to check, I developed a method to automate Intel Cluster Checker runs. This automated solution runs Intel Cluster Checker and reports any errors directly to me and/or the system administrator. Not only does automating this process save the administrator a substantial amount of time each week, it will eliminate the chance of missing a run, and ensures that the cluster remains in optimal health.

A Cluster That Checks Itself

In order to automate the process, it is important that your Intel Cluster Checker passes a manual run first. To do this, make sure the configuration file is optimized to ensure the cluster is operating at its best. With the perfect configuration on hand, a script is then needed to setup the system, run Intel Cluster Checker, and report the results.

For the first phase of the process, I wrote a script that sets up the environment and runs the most in-depth wellness check in Intel Cluster Checker. I used cron to schedule my script to execute once a week. After the results are complete, the script updates the message of the day to show when the last check ran and the results. If the check fails, the log file created by Intel Cluster Checker will be copied to the specified directory where it can be accessed and analyzed.

The cluster is now setup to check itself, in the middle of the night, and report its status upon login!

Read Part 2 for more about scheduling test runs, and the different levels of wellness tests in Intel Cluster Checker.

References:

>> About Intel Cluster Checker

>> Intel Cluster Checker Knowledge Base

Check out this paper, Bringing Consistency to a Heterogeneous Research Cluster, related to upgrading the GreenPlanet cluster at UC Irvine to Intel Cluster Ready architecture.  GreenPlanet is an organically grown HPC cluster that went from seven nodes to over three hundred in just a couple years.  The system grew  as principal investigators pooled grants and added more nodes to the system in batches.  Adding nodes in this fashion means the system is comprised of many different types of compute nodes which can become a pretty tough administrative task.  In the paper, Dr. Nathan Crawford states, “We were trying to keep track of all the differences in spreadsheets or in our heads, and like any academic institution, we had the problem of graduate students moving on and taking their knowledge of system details with them.”

Enter Intel Cluster Ready.  UC Irvine combined an equipment move with an upgrade of their software stack using Clustercorp Rocks+ 5.3.  This version has built-in functionality to install the necessary components to meet the Intel Cluster Ready architecture specification including a Roll to install Intel® Cluster Checker.

The result: changes to over 100 nodes in the system.  “The process of getting certified cleaned up a lot of underlying issues, and immediately
made the system much more stable,” said Crawford. "We’ll have more jobs running to completion, and scientists can spend less time trying to
debug underlying hardware and software problems.”

Scientists doing more science.  Nice.

>> Download the full case study

As an industry, we need to continue to push HPC technologies into mainstream and make it easier to access for people like Marshall.  Computational simulations can drastically reduce research cost and speed time to results - not just in academia but in commercial environments as well.  HPC is no longer a niche technology but is viable for broad academic and commercial use.  There are a lot more discoveries and advancements out there that we can get to faster with bigger and better simulations.

In case you missed it, read Teen Discovers Promising Cystic Fibrosis Treatment

Seamless Transitions...

I'd add that the Intel Cluster Ready architecture applies in the cloud as well.  In fact, using cloud images based on Intel Cluster Ready architecture allows more seamless transitions between clusters running in the cloud to physical clusters running locally.  As the video points out, cloud provides an entry point into using HPC or even a try-before-buy approach to HPC clusters.  Cloud bursting benefits as well, since the application that runs on the architecture locally would see the same architecture in the cloud.

The common architecture enables the applications to better understand and execute on top of the computing environment without being tied to the exact details of the solution.  In other words, you can get better mileage with a common architecture in both the cloud and local clusters.

Univa Acquires Grid Engine Expertise

Addition of Founding Sun Grid Engine Team Advances Data Center Optimization Strategy

Lisle, IL, January 18, 2010 – Univa, the data center optimization company, today announced that the principal engineers from the Sun/Oracle Grid Engine team, including Grid Engine founder and original project owner Fritz Ferstl, are joining Univa. The company will immediately offer superior engineering support for Grid Engine versions already installed and will publish a Univa version of Grid Engine before the end of Q1 2011. Univa will concentrate on improving Grid Engine for technical computing and HPC use cases in addition to promoting the continuity of the Grid Engine open source community.

As part of this announcement, Mr. Ferstl has been appointed Chief Technology Officer and will direct Univa's technology strategy as well as lead the company's growing EMEA business.

Grid Engine, a policy-driven workload and resource management system that automates workload distribution across a network of servers, is the most widely used system of its kind with over 4 million CPUs under management. The addition of Grid Engine technology and expertise directly supports Univa's data center optimization strategy by bolstering its workload and resource optimization capabilities, which already include the leading desktop management product, Grid MP.

"The first step to data center optimization is effective sharing of systems and workload distribution, and now Univa provides the leading resource management system as a primary element in our optimization stack," said Gary Tyreman, CEO of Univa. "Grid Engine is an area in which Univa has invested significant time and energy over the past 3 years, and we believe we are now best positioned to lead the pursuit of top HPC use cases and requirements that serve our most important customer sets. With this move, Grid Engine users gain a predictable and sustained future for Grid Engine that might not have been available otherwise."

"Combining the Grid Engine and Univa technology offerings was a once-in-a-lifetime opportunity that the new Univa EMEA team and I just couldn't miss," said Ferstl. "Now we'll be able to interact with and serve users worldwide investigating and understanding their data center optimization needs and providing the most innovative and enterprise-strength solutions to meet those needs."

Univa now offers the broadest and most innovative set of product capabilities for optimizing data center efficiency. Univa software continuously and dynamically optimizes an organization's efficiency with respect to workload and resources, people and processes, and application and license usage. This unique approach helps organizations earn rapid ROI through faster time-to-results and by keeping larger costs under control.

"Univa can now claim the most comprehensive set of category-leading products," added Tyreman. "We offer Grid Engine, the leading workload management solution; Grid MP, the leading desktop solution; and UniCloud, the leading dynamic cloud management software stack."

"I'm very excited by Univa providing continuity and ongoing support for Grid Engine," said Wolfgang Gentzsch, former CEO and President of Genias Software which originally developed the Grid Engine software acquired by Sun. "This is clearly a win-win-win situation: GE's thousands of users now have access to enterprise-class support at Univa, Univa customers benefit from the extended capabilities now available for fully optimizing data center efficiency; and the GE open source community can rest assured that robust development will continue thanks to Univa's commitment to the project."

"Univa developing and supporting Grid Engine is welcome news for the GE community," said Ian Foster, Professor of Computer Science and Director of the Computation Institute at the University of Chicago and Argonne National Laboratory. "This move by Univa brings with it a new competitive spirit and sense of innovation to a market that continuously benefits from pushing the envelope."

"This move propels Univa into a lead position in both the data center optimization and workload management spaces," said Steve Tuecke, co-founder of Univa and Deputy Director of the Computation Institute at University of Chicago and Argonne National Laboratory. "We selected Grid Engine as a core part of Univa's software stack almost five years ago due to its market leading feature set, large user community and vibrant open source community, all backed by a fantastic engineering team. Now that key members of the Grid Engine team are part of Univa, it seems a no-brainer for Grid Engine users to go with Univa for ongoing GE technology and services, as well as the broader data center optimization stack Univa has built around GE."

For additional quotes/reactions from the user and technology community, visit: http://www.univa.com/about/quotes.php.

Grid Engine support is available immediately from Univa, and the Univa version of the product will be available by end of Q1 2011. For more information about Univa's Grid Engine offering, including options for current Grid Engine users, visit http://www.univa.com/products/grid-engine.php.

About Univa Corporation

Univa, the Data Center Optimization Company, is the leading provider of optimization and management software for traditional, dynamic and cloud data centers. Our award-winning products are used by Global 2500 companies to improve resource sharing, amplify the efficiency of people and processes, and increase application and license utilization. Univa offers the industry's broadest, most innovative and integrated product set for managing shared, high-demand data center resources. From workload management to policy-driven provisioning across physical, virtual and cloud resources, only Univa provides a proven combination of enterprise-class capabilities, industry expertise, and community sponsorship. Univa is headquartered in Lisle, Illinois with offices worldwide.

SC10 was fascinating…of course. The world's largest supercomputing convention is always a great barometer of the HPC industry. It gives you a look at what's happening now, and where the industry is going (in case you weren't there, the words "cloud" and "exascale" were in wide use.) This year, SC also provided a great barometer of the progress of the Intel Cluster Ready architecture and the ecosystem building around it. That’s because it was the venue for our second annual Partner Appreciation Awards ceremony.

One thing I found particularly gratifying about the presentations this year is that several of the key awards centered around real-world solutions that are having an impact in the marketplace. The awards also highlighted the behind-the-scenes contributions of some HPC vendors that are breaking new ground in the small-to-mid-range HPC market.  These guys are doing some very cool things and they don’t always get the recognition they deserve. Take a look at the award-winning vendors below, and click on the links if you want to know more. I think you’ll be intrigued by the scope and value of their offerings.

Just for the record, we gave awards in three different categories:

• Pioneer Awards, honoring early adopters and promoters of Intel Cluster Ready.
• Pathfinder Awards, to distinguish those who are working with Intel Cluster Ready to forge new paths into previously unexplored markets.
• Explorer Awards, for those who have continued to explore and implement new programs to advance solutions based on the Intel Cluster Ready architecture.

Here’s the list of the 2010 award winners. Enjoy!

Solution 1 - The Seismic Coverage Validation Engine from Landmark is a preconfigured supercomputing solution for oil exploration that is expanding the use of HPC methods into smaller shops. And the award winners are:

• Landmark (Pioneer) delivers the complete appliance, the Seismic Coverage Validation Engine
• Appro (Pathfinder) furnishes the hardware (32 nodes and up) with its Ready-To-Go Clusters

Solution 2 - A number of vendors helped Simpson Strong-Tie deploy a new cluster in just two days, enabling the company’s engineers to run more complex models and get results two to three times faster, while halving the data center footprint and reducing total costs. (Read the case study.)

• SIMULIA (Explorer) provided the simulation software (Abaqus/Explicit) and the pre-sales benchmark testing that closed the deal.
• Silicon Mechanics (Explorer) furnished the hardware with its Hyperform HPCi for Intel Cluster Ready.
• Platform Computing (Pathfinder) has also optimized its PCM middleware for the Intel Cluster Ready architecture, providing a simple cluster management environment for non-expert HPC users.

Solution 3 - Two vendors were particularly instrumental in helping the University of Plymouth deploy a cluster to support ground-breaking research into renewable energy through wave farming.

• Viglen (Explorer) delivered the hardware optimized for out-of-the-box simulations.
• Concurrent Thinking (Pathfinder) provided SmartCluster, designed for complete cluster software provisioning with one-click Intel Cluster Ready validation.

Driving the Market Forward (other behind-the-scenes contributors)

• ANSYS (Pathfinder) is accelerating the adoption of HPC in smaller design shops, by proving the value of its simulation solutions running on Intel Cluster Ready certified clusters through in-house benchmarking.
• Clustercorp (Pathfinder) has optimized its ROCKS+ middleware for the Intel Cluster Ready architecture. ROCKS+ now enables exceptionally simple provisioning and build of certified clusters.
• Mellanox Technologies (Pioneer) and the HPC Advisory Council, which was founded by Mellanox, are delivering a range of compliant fabric solutions, along with installation guides for open source applications on Intel Cluster Ready clusters.
• QLogic (Pathfinder) provides fabric solutions optimized for the Intel Cluster Ready architecture, and has been instrumental in early testing and benchmarking of Intel MPI 4.0 on its InfiniBand hardware.

Honorable Mention
A lot of additional vendors are focused on the Intel Cluster Ready architecture, and are delivering significant new technologies and products. A handful of them received honorable mention awards, including:

• Bright Computing: Bright Cluster Manager for the Intel Cluster Ready architecture
• ClusterVision: Responsible for an array of Intel Cluster Ready cluster deployments
• Fujitsu Technology Solutions: Now providing a range of certified cluster products.
• ICON: Driving new market penetration by promoting Intel Cluster Ready for digital simulation.
• Panasas: High-performance parallel storage solutions
• SGI: A family of Intel Cluster Ready certified solutions, including its innovative Octane III desk-side cluster.
• Transtec: Responsible for a significant number of critical, real-world implementations.

That's it for this year's winners. I'd like to add my personal thanks to all of them. Stay tuned for more information. I’ll be focusing in on some of these technologies and solutions in more detail in future blogs.

>> Learn more about Intel Cluster Ready HPC solutions

While manning the Intel Cluster Ready demonstration pod, I was able to listen in on Drew and Kirk talk about how real-time simulations are helping protect football players from serious injury.  They showed how helmets outfitted with sensors are being used to study and relay data about individual impacts to the head of players.  Simulations for those impacts can predict when dangerous levels of force have occurred and players are likely to have suffered a concussion.

This data can be used to help design better equipment, but more importantly may help prevent serious, long-term injury.  Players who have experienced a concussion may not feel the immediate effects or may not want to leave the game.  Additional impacts, however, can cause much more severe injury.  The simulation data can help identify when players need to be pulled out of a game for medical evaluation and/or to avoid serious injury.

It was a really cool demonstration to see - they even cracked two helmets together to show a live simulation of impact. One of many neat examples of high performance computing in action that I got to see at the show.

>> See related stories

Desktop Engineering: Intel to Safeguard Brains

Intel Newsroom: Intel Teams Up with Leading Researchers to Make Football Helmets Safer