h1

Short-Take: Nexenta 3.1 Adds VAAI Support, Auto-Sync Resume

August 3, 2011

Nexneta Systems Inc released version 3.1 of its open storage software yesterday with a couple of VMware vSphere-specific feature enhancements. These enhancements are specifically targets at VMware’s vStorage API for Array Integration (VAAI) which promises to accelerate certain “costly” storage operations by pushing their implementation to the storage array instead of the ESX host.

From NexentaStor 3.1 Release Notes, the primitives implemented in 3.1 that contribute to VAAI support include:

  • SCSI Write Same: Supported in vSphere 4.1 and later
    Example. Accelerates zero block writes when creating new virtual disks.
  • SCSI ATS (Atomic Test & Set): Supported in vSphere 4.1 and later.
    Example. Enables specific LUN “region” to be locked instead of entire LUN when cloning a VM.
  • SCSI Block Copy: Supported in vSphere 4.1 and later.
    Example. Avoids reading and writing of block data “through” the ESX host during a block copy operation by allowing VMware to instruct the SAN to do so.
  • SCSI Unmap: Supported in vSphere 5 and later. Enables freed blocks to be returned to the zpool for new allocation when no longer used for VM storage.

Additional “optimizations” and improvements from Nexenta in 3.1 include:

  • In-flight deduplication
  • ARC performance enhancements
  • multiple connections per session for iSCSI
  • DMU fast path for iSCSI (i.e. no extra copy)
  • Auto-sync “resume” with progress bar in GUI/NMV and ability to change source/destination paths OTF
  • Parallel tasks in NMV (i.e. no more busy process “hangs”)
  • Improved CIFS performance
  • Support for multiple DC/DC fail-over for CIFS
  • Better cross-forrest trusts with CIFS
  • Configuration monitoring/reporting via “ConfGuard” plug-in
  • Multiple VIP per service for HA Cluster, fail-over of local users and elimination of separate heartbeat device
  • JBOD management for select devices from within the NMV

Given the addition of VAAI features, the upgrade offers some compelling reasons to make the move to NexentaStor 3.1 and at the same time removes obstacles from choosing NexentaStor as a VMware iSCSI platform for SMB/SME (versus low-end EMC VNXE, which at last look was still waiting on VAAI support.) However, for existing vSphere 4.1+ environments, a word of caution: you will want to “test, test, test” before upgrading to (or enabling) VAAI (fortunately, there’s a NexentaStor VSA available).

Auto-Sync Resume

In the past, NexentaStor’s auto-sync plug-in has been the only integrated means of block replication from one storage pool (or array) to another. In the past, the plug-in allowed for periodic replication events to be scheduled which drew from a marker snapshot until the replication was complete. Upon extended error (where the replication fails), the failure of the replication causes a roll-back to the marker point, eliminating any data that has transferred between the pools. For WAN replication, this can be costly as no check-points are created along the way.

More problematically, there has been no way to recreate a replication service in the event it has been either deleted or missing (i.e. zpool moved to a new host.) This creates a requirement for the replication to start over from scratch – a problem for very large datasets. With Auto-Sync 3.1, later problem is resolved, and provided NexentaStor can find at least one pair of identical snapshots for the file system.

Where I find this new “feature” particularly helpful is in seed replications to external storage devices (i.e. USB2.0 arrays, JBODs, etc.). This allows for a replication to external, removable storage to (1) be completed locally, (2) shipped to a central repository, and (3) a remote replication service created to continue the replication updates over the WAN.

Additionally, consider the case where the above local-to-WAN replication seeding takes place over the course of several months and the hardware at the central repository fails, requiring the replication pool to be moved to another NexentaStor instance. In the past, the limitation on auto-sync would have required a brand new replication set, regardless of the consistency of the replicated data on the relocated pool. Now, a new (replacement) service can be created pointing to the new destination, and auto-sync promises to find the data – intact – and resume the replication updates starting with the last identical marker snapshot.

NexentaStor Native Transport

The default transport for replication in NexentaStor 3.1 is now NexentaStor’s TCP-based Remote Replication protocol (RR). While SSH is still an option for non-NexentaStor destinations, netcat is no longer supported for auto-sync replications. While no indication of performance benefits are available, two tunable parameters are available for RR auto-sync services (per service): TCP connection count (-n) and TCP package size (-P). Defaults for each of these are 4 and 1024, respectively, meaning 4 connections and 1024KB PDU size for the replication session.

Conclusions

For VMware vSphere deployments in SMB, SME and ROBO environments, NexentaStor 3.1 looks to be a good fit, offering high-performance CIFS, NFS, iSCSI and Fiber Channel options in a unified storage environment complete with VAAI support to accelerate vStorage applications. For VMware View installations using NexentaStor, the VAAI/ATS feature should resolve some iSCSI locking behavior issues that have made NFS more attractive but remove SCSI-based VAAI features. That said, with the storage provisioning changes in View 4.5 and upcoming View 5, the ability to pick from FC, iSCSI or NFS (especially at 10G) from within the same storage platform has definite advantages (if not complexity implications.) Suffice to say, NexentaStor’s update is adding more open storage tools to the VMware virtualization architect’s bag of tricks.

NexentaStor 3.1 is available for download now.

Update, 8/12/2011:

Nexenta has found some problems with 3.1 post Q/A. They’ve released this statement on the matter:

Nexenta places the highest importance on maintaining access to and integrity of customer data. The purpose of this Technical Bulletin is to make you aware of an issue with the process of upgrading to 3.1. Nexenta has discovered an issue with the software delivery mechanism we use. This issue can result in errors during the upgrade process and some functionality not being installed properly. Please postpone upgrading to v3.1 until our next Technical Bulletin update. We are actively working to get this corrected and get it back to 100 % service as fast as possible. Until the issue is resolved we have removed 3.1 from the website and suspended upgrades. Thanks for your patience.

Nexenta Support, Aug. 6, 2011

According to sources from within Nexenta, the problems appear to be more related to APT repository/distribution issues “rather than the 3.1 codebase.” All ISO and repository distribution for 3.1 has been pulled until further notice and links to information about 3.1 on the corporate Nexenta site are no longer working…

Update, 8/17/2011:

Today, while working on a follow-up post, the lab systems (virtual storage appliances) were updated to NexentaStor 3.1.1 (both Enterprise and Community editions). Since a question was raised about the applicability of the VAAI enhancements to Community Edition (NexentaStor CE), I’ve got a teaser for you: see the following image of two identical LUNs mounted to an ESXi host from NexentaStor Enterprise Edition (NSEE) and NexentaStor Community Edition (NSCE). If you look closely, you’ll notice they BOTH show “supported” status.

vSphere VMFS5 Datastores provided by NexentaStor Community (VSA04) and Enterprise (VSA03) editions.

Update, 8/19/2011:

Nexenta officially re-released NexentaStor 3.1 today in the form of version 3.1.1 – it is available for download now.

10 comments

  1. Shame that VAAI is only available in Enterprise version, not in CE

    Like


    • Since the tweaks necessary for VAAI integration appear in the kernel and not the userland code, it’s likely that they exist as well in the CE version of NexentaStor. With the exception of some VDI deployments, I question the wisdom of pairing NexentaStor CE with vSphere 5 Enterprise or Enterprise Plus (since VAAI is not available in Essentials and Standard vSphere Editions). Typically, you would pair NexentaStor Enterprise with vSphere Enterprise/Enterprise Plus and the issue would be moot.

      Fortunately, you can load-up a VSA and evaluation version of vSphere 4.1 and see for yourself (i.e. run ESXi 4.1, vCenter 4.1 and NexentaStor 3.1in virtual machines under vSpher e 4.x).

      Like


    • RE: VAAI in Enterprise, only

      As I suspected, since the VAAI enhancements are kernel changes and NexentaStor Enterprise and Community Edition share the same kernel base, it did not make sense to me that VAAI would be present only in the Enterprise version. After some testing with both Enterprise and Community I can confirm that VAAI is present and active in both. I’ll be posting more detail in a follow-up blog post, but the proof is include in an update to the original announcement post.

      Like


  2. Yep, you are correct, I have had 3.1 CE running in my vSphere lab for couple days and VAAI works just fine. I based my original statement on a comment at NexentaStor forums which apparently was incorrect.

    Although it seems that NexentaStor supports only VAAI gen 2 as it works fine with vSphere 5 RC but not with vSphere 4.1.

    Like


  3. vSphere’s documentation on VAAI doesn’t mention, so I thought you might know. Does the VAAI in vSphere require an “Enterprise” license on that side? For instance I have vSphere Essentials. (The lowest end one) and would love to use VAAI with NS 3.1

    Like


  4. Yes, VAAI is available only in vSphere Enterprise and Enterprise Plus editions.

    Like


  5. Well crumb that sucks.

    Like


    • It is unfortunate that VAAI is not (yet) standard across the board, but neither is VAAI supported on most low-end storage associated with Essentials deployments (unless you’re one of SOLORI’s clients 😉

      If you’re Essentials for economic reasons, you’re going to have to wait until VAAI is mainstreamed (vSphere 6?), but if you have budget for features, it’s hard for me to make a case for Standard over Essentials – Enterprise Acceleration Kit seems to be the logical move from an Essentials Plus bundle in vSphere 5.

      Like


  6. […] EP: Quickly on that point we have to do a shout out to a fellow blogger, Collin Macmillan. He’s done sort of a paint by numbers breakdown of description of how he uses it in his home lab. You can read more about Collin’s experience with Nexenta here. […]

    Like



Comments are closed.

%d bloggers like this: