Welcome!

IBM Cloud Authors: Pat Romanski, Elizabeth White, Liz McMillan, Yeshim Deniz, Stefan Bernbo

Blog Feed Post

EMC Enters The Market With “Me Too” Flash Products

Yesterday EMC finally revealed the details on their not-so-secret Lightning and Thunder flash projects.  Fortunately this launch event didn’t include cramming small people into minis or firing Chad Sakac out of a cannon, but was more focused on the market and products EMC are bringing to it.  There was also a large amount of Twitter activity; look back at the #vfvache hashtag – this being the final product name of the Lightning product.  So what exactly did EMC announce?

 

Lightning AKA VFCache

 

VFCache (Very Fast Cache) is the final product name for the project that was called Lightning.  It turns out that this offering is nothing more than a PCIe SSD card for servers (not all servers mind you, but at this stage quite a few).  The initial offering combines with software to act as a very fast read cache to the host.  In Windows terms, this is implemented as a filter driver that sits above the STORPORT driver in the I/O stack, with similar implementations on Open Systems platforms.  The software component of VFCache tracks I/O and caches reads in order to speed up future I/O requests without needing to go to external disk.  Writes to disk are not cached by VFCache and EMC tried to make a virtue of the fact their product acts as a “write-through” cache, meaning I/O writes have to be committed to physical disk before the cache acknowledges them to the host.  Rather than being a benefit, write through mode in this instance is more likely to make the cache less effective by polluting the cache with writes that can’t be released until confirmed externally.  When there’s a difference in I/O if microseconds versus milliseconds, then this difference really matters.  However, I don’t think this is a design flaw, merely a placeholder for the future, as I’ll discuss later.

Disappointingly for EMC, VFCache 1.0 really is a 1.0 version in terms of feature support.  Within VMware ESXi for instance, the card installs with a device driver that only allows the cache benefits to be used when the filter driver is deployed into each ESXi guest, so it’s not simply a case of insert the card and off you go.  Moreover, the VFCache appears as a DAS device within VMware and so can’t be used in conjunction with vMotion.  For many organisations this is a huge omission as there’s a big correlation between high performance and high availability; the lack of vMotion isn’t acceptable.

 

The Competition

 

We can’t go any further on the VFCache discussion without mentioning the competition and in one of the presented slides, EMC paid homage to the market leaders, Fusion-IO.  Their ioCache product already accelerates VMware ESXi and Windows 2008 environments, using a similar hypervisor plugin approach.  ioCache already offers double the capacity of VFCache and it’s likely Fusion-IO have larger capacity cards in the pipeline as they already offer a range of SLC and MLC flash devices.

 

Thunder Follows Lightning

 

Surely Hitachi must be enjoying the irony of EMC choosing product code names based on already defunct HDS hardware (Thunder and Lightning were the mid-range and Enterprise products respectively that preceeded AMS and USP).  The next product announcement moves the flash-in-server story forward and explains how this technology is limited in terms of availability.  The move to centralised SAN environments was done precisely to fix the issues that occur with server-side SSD today.  Data is locked into the server, is difficult to expand (requiring downtime and physical intervention) and is isolated from access if a physical failure of the server should occur.  So, step up Project Thunder, EMC’s purpose build all-flash array.  This device allows multiple servers to share data across what EMC are calling the “server-area network”.  What they mean is a physically local, high-speed interconnect (such as Infiniband or Rapid-IO) between the server and a fast all-flash storage array.

The interconnect technology between devices already exists today (as already mentioned) but placing it into the server and using this for shared storage presents more of a challenge.  Where Lightning was a simple Filter Drive, Thunder will require deeper integration in order to manage consistency across all connected servers.  This isn’t something that comes new to EMC – think of how VMAX nodes interconnect and you have the model already there.  However, the implementation may require closer co-operation with server vendors than EMC can achieve, those same companies they are already in competition with for storage – IBM, HP and Dell.  This could mean Thunder becomes a VCE only product or is severely restricted when deployed in other manufacturers hardware.  We will have to wait and see. (Side Note: This also means that other all-SSD array manufacturers could become more attractive to HP, IBM & Dell as acquisition targets – check out here and here).

 

The Competition

 

Let’s not forget that dedicated all-flash arrays are already out there.  Recently I’ve discussed Pure Storage and SolidFire and there’s also Violin Memory, who have been going at this market for quite some time.  They already have the SSD array technology to a mature level including support for Infiniband; all that’s needed is a software driver to bring clustering to their products.

 

Futures

 

What can we expect going forward?  There are lots of gaps the product releases we’ve seen today.  Lack of vMotion support and no write-back cache to name only two.  The question we should be asking is what could be delivered in the future.  EMC have access to every piece of the I/O stack, from the hypervisor, the multi-path driver, through to the array.  Using PowerPath, EMC can develop more intelligent algorithms that choose whether to cache I/O locally in the server/hypervisor, destage to the array, leverage pre-fetching from disk and other clever ways to squeeze the best level of performance out of the hardware stack.

 

Conclusions

 

EMC have ratified both the PCIe SSD and dedicated flash array markets with their announcements of Lightning and Thunder.  At this stage they are bringing only “me too” products to the market, with other vendors out there having already delivered more advanced technology than being announced today.  However EMC have two big advantages; (a) they are a huge organisation, with access to the majority of customers in the market and a great marketing team.  They have the ability to place their products into customer environments and use price as the main differentiator (b) they have a huge R&D budget and never stand still on product development.  Today’s 1.0 releases will be superseded within months and address some of the shortcomings we can see today.  The future battle will not be over the hardware, but the software that integrates I/O in the server to I/O on the array, delivering the benefits of local flash with the safety of external storage.  The eventual winner will be the vendor who gets that software and hardware integration right.

Related Articles

You may be interested in the following related articles from this and other sites.

 

 

Read the original blog entry...

IoT & Smart Cities Stories
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
IoT is rapidly becoming mainstream as more and more investments are made into the platforms and technology. As this movement continues to expand and gain momentum it creates a massive wall of noise that can be difficult to sift through. Unfortunately, this inevitably makes IoT less approachable for people to get started with and can hamper efforts to integrate this key technology into your own portfolio. There are so many connected products already in place today with many hundreds more on the h...
The standardization of container runtimes and images has sparked the creation of an almost overwhelming number of new open source projects that build on and otherwise work with these specifications. Of course, there's Kubernetes, which orchestrates and manages collections of containers. It was one of the first and best-known examples of projects that make containers truly useful for production use. However, more recently, the container ecosystem has truly exploded. A service mesh like Istio addr...
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As automation and artificial intelligence (AI) power solution development and delivery, many businesses need to build backend cloud capabilities. Well-poised organizations, marketing smart devices with AI and BlockChain capabilities prepare to refine compliance and regulatory capabilities in 2018. Volumes of health, financial, technical and privacy data, along with tightening compliance requirements by...
Charles Araujo is an industry analyst, internationally recognized authority on the Digital Enterprise and author of The Quantum Age of IT: Why Everything You Know About IT is About to Change. As Principal Analyst with Intellyx, he writes, speaks and advises organizations on how to navigate through this time of disruption. He is also the founder of The Institute for Digital Transformation and a sought after keynote speaker. He has been a regular contributor to both InformationWeek and CIO Insight...
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
To Really Work for Enterprises, MultiCloud Adoption Requires Far Better and Inclusive Cloud Monitoring and Cost Management … But How? Overwhelmingly, even as enterprises have adopted cloud computing and are expanding to multi-cloud computing, IT leaders remain concerned about how to monitor, manage and control costs across hybrid and multi-cloud deployments. It’s clear that traditional IT monitoring and management approaches, designed after all for on-premises data centers, are falling short in ...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...