Power of 3: NetApp, VMware, and Broadcom Redefining Enterprise IT Architecture
Praneeth Kavuri
Co-authors: Naveen Krishnamurthy from VMware, Jayamohan Kallickal from Broadcom
General availability (GA) of VMware vSphere 7.0 is a major milestone for VMware. The new release supports many new features that can facilitate transformation of modern data centers. One key feature available with vSphere 7.0 is NVMe over Fabrics (NVMe-oF) support.
Broadcom, NetApp, and VMware have a long history of developing infrastructure technologies that modernize data centers. NVMe over Fabrics solutions based on Fibre Channel (FC) deliver the next level of performance required by traditional and real-time applications for various use cases such as AI, machine learning, and real-time analytics.
Highly optimized storage solutions have become a key piece in the digital transformation puzzle. In the first phase of storage modernization, administrators embraced flash storage to accelerate business-critical applications. Increasingly, organizations are looking for further ways to improve the speed and responsiveness of the applications that control their key operations. Because the performance of these applications is tightly linked to time to value and to stakeholder satisfaction, it’s critical that they operate at the highest levels of efficiency. Getting value and insights quickly and reliably from a range of mixed-workload environments can differentiate organizations and accelerate efficiency.
Enter NVMe (NVM Express), a new protocol for accessing high-speed storage media that offers many advantages over legacy protocols. NVMe has become the industry-standard interface for PCIe SSDs, with a streamlined protocol and command set and with fewer clock cycles per I/O operation. NVMe supports up to 64,000 queues and up to 64,000 commands per queue, which makes it more efficient than the existing SCSI-based protocols such as SAS and SATA. The NVMe-oF standard was created to extend the benefits of NVMe to shared storage. It was designed to support many transport protocols, including FC, RDMA over Converged Ethernet (RoCE), InfiniBand, and TCP. The NVMe/NVMe-oF protocol provides very high throughput, low CPU utilization, and ultralow latency, all of which required creating a lightweight NVMe stack that is parallel to SCSI, as well as a redesign of drivers. The NVMe-oF architecture supports many hosts connected to multiple targets through multiple switches.
Most enterprises already entrust their mission-critical applications to FC SANs because of their consistent performance, reliability, and resilience. As your organization considers accelerating business-critical applications, VMware, Broadcom, and NetApp have partnered to create a complete server-to-fabric-to-storage solution that is easy to deploy.
NVMe over Fibre Channel (NVMe/FC) brings the following enterprise-class storage services to NVMe-oF:
- Centralized discovery
- Sharing of discovery data across the network
- Zoning and isolation
- State-change notifications
- Storage network topology
- Sequence-level error recovery
Most enterprise data centers already deploy FC SANs, so you don’t need a rip-and-replace upgrade to move to an NVMe/FC solution. Broadcom’s Emulex FC HBAs and Brocade switches support NVMe/FC and SCSI Fibre Channel Protocol (FCP) simultaneously on the same port and fabric. This support simplifies the creation of multipath SANs that enterprises require. Further, Emulex Gen 7 FC HBAs are capable of round-trip latencies in the range of 10 microseconds while providing multimillion IOPS, making these HBAs an excellent fit for NVMe-oF solutions.
The Broadcom family of Gen 6/Gen 7 Brocade FC switches and Emulex FC HBAs has demonstrated more than 20 years of field-proven performance and reliability, making FC the trusted technology of choice. One of the biggest challenges of managing a dynamic infrastructure of physical and virtual resources is networkwide visibility. You can’t manage what you can’t measure, and you can’t measure what you can't see. Broadcom addresses this challenge through close integration with NetApp and VMware products. Broadcom’s Brocade and Emulex Gen 6/Gen 7 FC products not only deliver 32GFC performance, but also provide deep visibility into the health and performance of the SAN that can help you quickly respond to dynamic business demands. Taking advantage of the increased efficiency of NVMe, combined with the high performance, low latency, and actionable intelligence of Gen 6/Gen 7 FC, you can accelerate IOPS to deliver the performance, application response time, and scalability needed for next-generation data centers.
The NetApp AFF A800 all-flash storage array is a robust scale-out system that combines low-latency performance with comprehensive data management, built-in efficiencies, integrated data protection, multiprotocol support, and nondisruptive operations. It’s designed for the most demanding workloads that require ultralow latency, and it’s the first flash array on the market to support NVMe SSDs and NVMe-oF. It provides end-to-end NVMe connectivity between storage arrays and host servers for maximum bandwidth, high IOPS, and the lowest possible latency. AFF A800 systems offer support and deep integration for enterprise applications, virtual desktop infrastructure (VDI), and database and server virtualization, supporting Oracle, Microsoft SQL Server, VMware, SAP, MySQL, and more.
NetApp ONTAP®, the industry’s leading enterprise data management software, combines new levels of simplicity, flexibility, and security with powerful data management capabilities, proven storage efficiencies, and leading cloud integration. ONTAP 9.7 includes several enhancements that enable you to do more with less time and effort. ONTAP provides best-in-class integrations with vSphere, simplifying storage and data management for VMware environments. ONTAP already has enterprise-class storage for VMware, with support for both NFS and FCP/iSCSI on the same cluster to host VMware datastores. ONTAP also has robust support for VMware features such as Virtual Volumes (vVols), vStorage APIs - Array Integration (VAAI), inline deduplication/compaction/compression, vSphere integration through the Virtual Storage Console, backing up virtual machines (VMs) with NetApp SnapCenter®, NetApp FlexClone®, and VMware Storage Replication Adapter (SRA) plug-ins.
Figure 1) A common SAN configuration.
To compare the performance benefits of NVMe/FC versus SCSI FCP, we built the following test configuration, as shown in Figure 1:
- Dell PowerEdge R740 servers
- vSphere 7.0
- Red Hat Enterprise Linux 8.0 VMs running database software
- Trusted Platform Module (TPM) test suites
- NetApp AFF A800 cluster
- Emulex LPe35000 HBAs
- Brocade G620 switches
Test Results: Better Performance with NVMe/FC
On our test configuration, we ran performance tests on Microsoft SQL Server and Oracle 19c on Red Hat.Figure 2 shows the performance results with SQL Server: a 115% improvement for NVMe/FC compared with SCSI FCP. NVMe/FC delivers more transactions on the same hardware platform.
Figure 2) Performance improvement of NVMe/FC with Microsoft SQL Server on Red Hat 8.
Configuration: HammerDB 3.2/SQL Server 2017 on a Red Hat Enterprise Linux 8.0 VM running ESXi 7.0 hypervisor; Emulex LPe35002 HBA connected to two NetApp AFF A800 arrays through a Brocade G620 switch.
As depicted in Figure 3, Oracle online transactional processing (OLTP) TPC-C results show a 111% increase in TPM with NVMe/FC compared to SCSI FCP. Similar improvement is seen in NOPM for NVMe/FC.
Figure 3) Performance results of NVMe/FC with Oracle 19c.
Configuration: HammerDB 3.2/Oracle 19c on a Red Hat Enterprise Linux 8.0 VM running ESXi 7.0 hypervisor; Emulex LPe35002 HBA connected to two NetApp AFF A800 arrays through a Brocade G620 switch.
Delivering Enterprise-Class NVMe-oF with NVMe/FC
The consequence of ever-increasing reliance on data will be a never-ending expansion in the size of the global datasphere. Estimated to be 33ZB in 2018, IDC forecasts that the global datasphere will grow to 175ZB by 2025. As your business contends with the perpetual growth of data, you need to rethink how data is captured, preserved, and processed. Performance, economics, and endurance of data at scale are paramount.NVMe/FC with VMware, Emulex, Brocade, and NetApp delivers performance that is about two times better with just a software upgrade. The joint industry-leading NVMe-oF solution enables you to deploy an infrastructure that supports all of the following goals:
- Adopt modern, future-ready technologies nondisruptively.
- Accelerate your mission-critical applications that create business value.
- Simplify operations cost effectively.
Praneeth Kavuri
Praneeth Kavuri is a Product Manager at NetApp, responsible for SAN - Enterprise Applications. Praneeth focuses on enabling customers to deliver business outcomes and simplifying operations through NVMe-oF and other SAN product enhancements. He has 8 years of experience in the high-tech industry with previous stint at Ericsson and an MBA from University of Notre Dame. In addition to technology, traveling and reading fascinate him.