Archive for the ‘Short Take’ Category


Short-Take: vSphere Client for iPad, Preview

March 18, 2011

I highlighted the installation and use of VMware’s vCenter Moble Access (vCMA) appliance in a post in late February. For the most part, vCMA has not changed much since our initial download back in April of 2009. If you downloaded the OVF early this February and looked at the updated instructions from the “fling” site, you may have noticed the following “curious” statements:

  • Once it powers on, you will need to configure your iPad by going into Settings, the vSphere client (usually bottom left corner of screen, in the Apps section), then you enter the IP address of your mobile appliance.
  • Finally, you can access your environment from the vSphere iPad app by entering your vCenter server info or ESX server info, with appropriate username and password.

vSphere Client for iPad

Having a heads-up from the vExpert team briefing by Srinivas Krishnamurti, Sr. Director for Mobile Solutions and Marketing at VMware, plus earlier press coverage from VMworld 2010 (see below), I knew what this “information leak” was hailing. Fortunately, the offending section (text above) was quickly redacted and VMware managed to avoid spoiling the surprise pending today’s [press release].

However, that was not the only source of “information leakage” prior to today’s announcement: you just had to know where to look. For instance, while looking deeper into the virtual appliance for our vCMA how-to, I found bread-crumbs pointing to more “curious” iPad wanderings. The following “Easter egg” was discovered in the “action-config.xml” file (which we held back under the spirit of the information embargo):

<!-- VCMA iPad Actions -->
 <action name="vcmaAbout" type="com.vmware.vcma.action.VcmaAboutAction"></action>
 <action name="vcmaLogin" type="com.vmware.vcma.action.VcmaLoginAction"></action>
 <action name="vcmaLogout" type="com.vmware.vcma.action.VcmaLogoutAction"></action>
 <action name="vcmaHome" type="com.vmware.vcma.action.VcmaHomeAction"></action>
 <action name="vcmaHostInfo" type="com.vmware.vcma.action.VcmaHostInfoAction"></action>
 <action name="vcmaHostOp" type="com.vmware.vcma.action.VcmaHostOperationAction"></action>
 <action name="vcmaVmInfo" type="com.vmware.vcma.action.VcmaVmInfoAction"></action>
 <action name="vcmaVmQuestion" type="com.vmware.vcma.action.VcmaVmQuestionAction"></action>
 <action name="vcmaVmAnswer" type="com.vmware.vcma.action.VcmaVmAnswerAction"></action>
 <action name="vcmaVmOp" type="com.vmware.vcma.action.VcmaVmOperationAction"></action>
 <action name="vcmaSnapshot" type="com.vmware.vcma.action.VcmaSnapshotAction"></action>
 <action name="vcmaPerf" type="com.vmware.vcma.action.VcmaPerfAction"></action>
 <action name="vcmaSearch" type="com.vmware.vcma.action.VcmaSearchAction"></action>
 <action name="vcmaPing" type="com.vmware.vcma.action.VcmaPingAction"></action>
 <action name="vcmaTracert" type="com.vmware.vcma.action.VcmaTraceRouteAction"></action>
 <action name="vcmaVmsList" type="com.vmware.vcma.action.VcmaVmListAction"></action>
 <action name="vcmaMonitorTask" type="com.vmware.vcma.action.VcmaMonitorTaskAction"></action>

This grouping of action/command definitions identify 17 of 23 vCMA action classes. These classes meant four things to me: (1) the actions are tuned specifically for a non-HTML-only client; (2) the limitations of vCMA’s web interface do not bind the iPad client; (3) there is significant potential for “capabilities drift” between the iPad client the “generic” mobile access client (i.e. HTML) as time goes by (read: richer feature set, user options); and (4) other “tablet” or “mobile” clients can’t be too far behind.

Since it is not feasible to have iPad software previews for vExperts (i.e. via iTunes) for pre-release products, this “pre-view” is based on exposure to product briefing and other pre-launch sources (direct and indirect). We’ll be following-up within the week with actual hands-on experience… That said, here’s what’s going on with VMware and iPad:

vSphere Client for iPad

Today, VMware CIO Steve Herrod announced the launch of version 1.0 of the vSphere Client for iPad (vCiP). The aptly named utility runs on Apple’s current generations of iPad and provides access to many of the basic administrative functions available to vCenter and the standard vSphere Client. This release must be seen as a quick, 1-2-3 punch of mobile and management-centric releases for VMware in the span of two weeks: vCenter Ops, View Client for iPad and now vSphere Client for iPad.

This iPad application is not truly a “native” or “fat” client for vSphere in the “conventional Windows sense.” Instead, VMware’s new app deploys as a web service reliant application (typical of its iPad ilk), and it is accordingly “small, light and elegant.” As you might guess from the [leading] introduction, the “heavy lifting” is actually performed by VMware’s vCenter Mobile Access (vCMA) appliance through the set of new classes (conveniently listed above).

VMware diagram showing (optional) placement of firewall, vCMA, vCenter and vSphere clusters. The use of a VPN connection to your firewall is strongly recommended as vCMA deploys with its web service without SSL enabled.

This illustration depicts the “best practice” recommended deployment for the iPad client by way of a trusted VPN connection. Again, this information was provided to us from Srinivas and his team “pre-launch” and hence was also prior to the recently released enhancements in vCMA (see below). In either case, the connection from iPad to vCenter is always translated through vCMA.

Like the standard Windows “fat” client (now conveniently available as a ThinApp’d zero-install package), the iPad client login requires the following credentials:

  1. The IP address or DNS host name for your vCenter;
  2. A valid user name with rights to access/manage the target vCenter;
  3. The password for the vSphere user.

Unlike the Windows variant, the following must be configured into the iPad’s “Settings” for the vSphere app prior to initial connection:

  1. The IP address or DNS host name for your vCMA appliance (displayed as “Web Server” in “Settings”).

vCMA’s web service is not SSL encrypted, and these credentials could be passed “in the clear.” (see updated post, SSL added to vCMA this Tuesday.) Given this client is targeted for mobile use, the risk of exposure to insecure networks (Internet, public WiFi, etc) without SSL would have created “special” opportunities for man-in-the-middle attacks. However, the use of a mobile VPN connection for the iPad client is strongly recommended, but no longer strictly necessary.

Read the rest of this entry ?


Short-Take: Windows 7 for iPad, Free

March 9, 2011
Windows7 running on iPad

Windows7 running on iPad

Remember that announcement about View 4.6 and the PCoIP Software Gateway (PSG) a week or so back? If the existence of PSG got your imagination drifting towards running Windows7 over PCoIP on your iPad or Android tablet, then some of you are going to be very excited and some of you will have to wait a little bit longer.

Today VMware is taking mobile desktop to a new level by announcing the general availability of the View Client for iPad V1.0 – Android tablet users will have to wait! This is a iPad-native, PCoIP-only client for View 4.6 environments (i.e. PCoIP w/PSG support) with  gesture-enabled navigation and virtual mouse pad. If you liked accessing your View desktop in Wyse’s PocketCloud for iPhone & iPad (RDP mode only), you’re going to love the View Client for iPad because it unlocks the rich, PCoIP goodness that you’ve been missing.

Last week a group of vExperts were briefed on the iPad app by its development team leader Tedd Fox who came to VMware in August, 2010 after nearly 8 years of work at Citrix (co-inventor/designer of Citrix Reciever for iPad & iPhone). To say Tedd knows iPad/mobile and remote app/desktop is an understatement, and VMware has committed to an aggressive “feature update” schedule for the iPad app on the order of every 1-2 months (typical of mobile application norms.)

Needless to say, we had a few questions. Here’s just a few of the responses from our Q/A and demonstration session:

vExpert: Will there be a iPhone link for touchpad control?

Tedd: No. Due to some patent-pending issues, we decided not to tread on that ground.

vExpert: Has it been enhanced for the iPad2?

Tedd: No. It’s [iOS] 4.3 “ready” but nobody’s got an iPad2 so no one knows if there application’s going to work. We’ve tested on dual-core architecture before, just not Apple’s dual-core architecture.

vExpert: Dual-core tested? So there’s an Android app coming?

Tedd: Android app is coming! We’re looking at mid-year for the Android. I just spent a few weeks in China getting that in alpha-alpha mode; so we actually have a UI and everything – we’re just building-up the bits… it’s going to be tablet only. It works on a 7″ right now, but we’re not sure if that’s a useful form.

vExpert: Is that because it’s too small [i.e. 7″ screen]?

Tedd: It’s because of the mouse pad and everything… it just doesn’t feel right – the resolution and everything.

vExpert: Not even with panning and side scrolling [small screen]?

Tedd: Not really. Panning a windows desktop is “okay” for like 10 minutes, after which you develop something like Tourette syndrome with curse words and all. We actually ran tests on that to figure that out, but it could change [given the right demand/use case.]

vExpert: Will it support bi-directional audio?

Tedd: No, uh, uni-directional is definitely on the roadmap so doctors can dictate and stuff like that. Otherwise, we’re going to see how the protocol matches up for [more complex] audio applications.

vExpert: Can we get more information on the Android app?

Tedd: I don’t want to get into the Android client because everything is still “in flux” and we’re still designing it…

vExpert: Will [View Client for iPad] work with bluetooth mouse and keyboard?

Tedd: Yes… You have to go into the iPad settings and pair them… then with you do the three-finger tap on the screen – like to activate the on-screen keyboard – that’s how you activate the bluetooth keyboard [only, no mouse support per Apple policy], and the [on-screen] toolbar drops down to the bottom of the screen… It’s very nice to use.

vExpert: Will it support multitasking, multiple sessions and session swapping?

Tedd: No. We’re working with Teradici on full-multitasking for one of the feature revs this year.

vExpert: It seemed that logging-in and getting to your desktop seemed pretty quick. What would you say?

Tedd: This [demo] is on 3G – by the way – so it’s fairly quick. The only [downside] is if you’re using RSA tokens: you’ve got to read the token and put it in… If the broker policy allows users to save their passwords, then you’d only need the token code.

vExpert: Is there a way to transfer data to/from the iPad from the [View client desktop]?

Tedd: Working on that – that’ll be in the next rev or two. There’s a grey area there with the shared foldering system in iOS – some people are like “yeah, awesome” but if you talk to DoD they’re like “heck no” so we’re working on an elegant solution.

vExpert: What about dropbox or something like that?

Tedd: If we have an internal solution then yes. I don’t want to be [bound by a third party] on our app – I want to keep it as “pure VMware” as possible. If the market screams for it in enough number, then of course I’m going to listen… If it’s allowed in your desktop’s environment [dropbox will work.]

vExpert: How’s the performance of the View client while other programs are in the background on the iPad?

Tedd: You don’t even notice it. If you know me you know I’ve constantly got white earbuds on. One of my test cases was working on a desktop while running on Pandora in the background.

vExpert: Price is free?

Tedd: Yeah, as long as I’m with VMware it will always be free.

Over the course of the demonstration, we saw Tedd put the application through its paces. It’s fast – even on the original iPad. The gesture interface looks well thought-out, has been thoroughly tested – Tedd says “rock solid” – and repeated three-finger abuses [rapid toggling the keyboard] won’t crash the View iPad app. Can’t wait to get it into SOLORI’s lab…

Gesture Help for iPad View Client

View Client for iPad Keyboard (three-fingers to pop-up)

View for iPad soft mouse pad and cursor keys


Client support for tap-hold loupe: zoom near mouse pointer.

Related Links:

[Update: View 4.x -> View 4.6 (iPad Client designed for View 4.6 and PSG). Added community blog link, virtual keyboard and loupe screenshots. Remote add -> Remote app. Added link to Andre’s VDI calculator. Clarification on bluetooth mouse support. Related links section with PCoIP off-load.]


Short-Take: vCenter Operations, Launched

March 8, 2011

vCenter Operations Standard, Launced Today

I think “launched” is a good description of a product that represents a company’s first release from a product acquisition that was already somewhat mature. No surprising new features, no trend-setting advanced in interface or integration – just a solid, usable “pane of glass” to improve “visibility” into an existing product set. That’s how I’d describe VMware’s “new” vCenter Operations appliance for vSphere.

The product launches initially as a virtual appliance (similar to VMDR, vMA, vCMA, etc.) that enhances vCenter’s ability to track performance, capacity and changes in the vSphere environment. This initial offering is called VMware vCenter Operations Standard and is priced per-VM (I’ll get to those details later.) vCOPS Standard will be available for download and trial beginning March 14, 2011. Here’s how VMware describes it:

Proactively ensure service levels, optimum resource usage and configuration compliance in dynamic virtual and cloud environments with VMware vCenter Operations. Through automated operations management and patented analytics, you benefit from an integrated approach to performance, capacity and configuration management. You’ll gain the intelligence and visibility needed to

  • Get actionable intelligence to automate manual operations processes
  • Gain visibility across infrastructure and applications for rapid problem resolution
  • Proactively ensure optimal resource utilization and virtual/cloud infrastructure performance
  • Get ‘at-a-glance’ views of operational and regulatory compliance across physical and virtual infrastructure.

If you’re like me, that description won’t make you find a place in your strained IT budget for VMware’s new plug-in. Eventually, VMware will find the right messaging to sell this add-on, but let’s see if it can sell itself, shall we? Located deep within a “related whitepaper” there is an indication of how vCOPS differentiates itself from the crowd of “pretty statistics loggers” and delivers some real tasty goodness. I believe this is the real reason why VMware shelled-out $100M for the technology.

Dynamic Thresholds

Yeah, I thought that too. What the heck is a “dynamic threshold” and why do I care? For one thing, it takes VMware two pages of white paper just to describe what a “dynamic threshold” is, let alone describe how it adds value to vCenter. In short, VMware’s statistics logger applies eight proprietary algorithms to live and historical data to “predict” what “normal” operating parameters are for a specific VM, host, cluster, etc. and then make decisions as to whether or not anomalous conditions exist in the present operating state.

vCenter Operations' stats engine tries to see performance data as a seasoned admin would.

Effectively, VMware’s dynamic threshold takes a sophisticated look at the current trend data just like a seasoned IT admin would – except it does it across your entire virtual enterprise every 12 hours and predicts what the next 12 hours should look like. This “prediction” becomes the performance envelope, hour by hour, for the next 12 hours of operation. So long as your virtual object’s performance stays within the envelope, the likelihood of anomalous behaviour is low; however, when it is operating outside the envelope, outliers are likely to trigger performance alarms.

The following transforms are applied to statistical data every 12 hours:

•    An algorithm that can detect linear behaviour patterns (e.g., disk utilization, etc.).
•    An algorithm that can detect metrics that have only two states (e.g., availability measurements).
•    An algorithm that can detect metrics that have a discrete set of values, not a “range” of values, (e.g., “Number of DB User Connections,” “Number of Active JMVs,” etc.).
•    Two different algorithms that can detect cyclical behaviour patterns that are tied to calendar cycles (e.g., weekly, monthly, etc.)
•    Two different algorithms that can detect general non-calendar patterns (e.g., multi-modal)
•    An algorithm that works, not with time-series or frequently measured values, but with sparse data (e.g., daily, weekly, monthly batch data)

VMware claims this approach – to borrow a recently over-used term – “wins” versus typical bell-shape algorithm approaches many times over. In statistical analysis against real-world VM metrics, VMware says typical bell-shape analysis “barely shows up” and, in the few cases where it “wins” the bell-shape approach does so only slightly. In enterprise applications, being able to present “anomalous behaviour” of related systems in opposition can more quickly lead to root-cause identity. Here, VMware demonstrates how anomaly counts for separate, related application tiers can be compared and correlated visually:

Anomaly count comparison across separate tiers. Note "smart alert" gets triggered early in the process (Enterprise Edition).

Eric Sloof at NTPro.NL has posted a video (7 minutes) and screen shots that shows vCenter Operations Standard in operation. While Eric describes vCOPS as a “great new product,” Kendrick Coleman takes issue with VMware’s price model and questions its true value proposition (at least with the “Standard” edition.)

Do Fries Come with That?

From some of the back-peddling overheard in the vExpert pre-launch conference, VMware’s testing the waters on where the product fits at the low-end. Essentially, this is an enterprise class product offering that’s been paired-down to fit into a smaller IT budget. Like most VMware products, a generous “free” trial period will be granted to allow you to try before you buy. However, the introductory price (i.e. official pricing is not posted on VMware’s site) is set at $50/VM (hence Kendrick’s quandary) for up to 500 VMs (about $25K).

Since VMware intends to offer an inclusive pricing scheme, all registered VMs will need to be licensed into the Standard Edition’s footprint. In the vExpert call, there was “talk” about extending analysis only to specific VMs (and allowing for a paired-down licensing footprint) but that is conjecture today. In a typical enterprise where 70-80% of workloads are non-mission critical, the cost and license model for vCOPS could be an obstacle for some – or at least force the use of a separate vCenter and cluster arrangement. Let’s hope VMware comes-up with a mission-critical license model quickly.


Short-Take: New Oracle/Sun ZFS Goodies

November 17, 2010

I wanted to pass-on some information posted by Joerg Moellenkamp at – some good news for Sun/ZFS users out there about Solaris Express 2010.11 availability, links to details on ZFS encryption features in Solaris 11 Express and clarification on “production use” guidelines. Here’s the pull quotes from his posting:

“Darren (Moffat) wrote three really interesting articles about ZFS encryption: The first one is Introducing ZFS Crypto in Oracle Solaris 11 Express. This blog entry gives you a first overview how to use encryption for ZFS datasets. The second one…”

–  Darren Moffat about ZFS encryption,, 11-16-2010

“There is a long section in the FAQ about licensing and production use: The OTN license just covers development, demo and testing use (Question 14) . However you can use Solaris 11 Express on your production system as well…”

Solaris 11 Express for production use,, 11-16-2010

“A lot of changes found their way into the newest release of Solaris, the first release of Oracle Solaris Express 2010.11. The changes are summarized in a lengthy document, however…”

What’s new for the Administrator in Oracle Solaris  Express 2010.11,, 11-15-2010

Follow the links to Joerg’s blog for more details and links back to the source articles. Cheers!


Short-Take: ZFS version ZPOOL Versions

November 15, 2010

As features are added to ZFS – the ZFS (filesystem) code may change and/or the underlying ZFS POOL code may change. When features are added, older versions of ZFS/ZPOOL will not be able to take advantage of these new features without the ZFS filesystem and/or pool being updated first.

Since ZFS filesystems exist inside of ZFS pools, the ZFS pool may need to be upgraded before a ZFS filesystem upgrade may take place. For instance, in ZFS pool version 24, support for system attributes was added to ZFS. To allow ZFS filesystems to take advantage of these new attributes, ZFS filesystem version 4 (or higher) is required. The proper order to upgrade would be to bring the ZFS pool up to at least version 24, and then upgrade the ZFS filesystem(s) as needed.

Systems running a newer version of ZFS (pool or filesystem) may “understand” an earlier version. However, older versions of ZFS will not be able to access ZFS streams from newer versions of ZFS.

For NexentaStor users, here are the current versions of the ZFS filesystem (see “zfs upgrade -v”):

---  --------------------------------------------------------
 1   Initial ZFS filesystem version
 2   Enhanced directory entries
 3   Case insensitive and File system unique identifier (FUID)
 4   userquota, groupquota properties
 5   System attributes

For NexentaStor users, here are the current versions of the ZFS pool (see “zpool upgrade -v”):

---  --------------------------------------------------------
 1   Initial ZFS version
 2   Ditto blocks (replicated metadata)
 3   Hot spares and double parity RAID-Z
 4   zpool history
 5   Compression using the gzip algorithm
 6   bootfs pool property
 7   Separate intent log devices
 8   Delegated administration
 9   refquota and refreservation properties
 10  Cache devices
 11  Improved scrub performance
 12  Snapshot properties
 13  snapused property
 14  passthrough-x aclinherit
 15  user/group space accounting
 16  stmf property support
 17  Triple-parity RAID-Z
 18  Snapshot user holds
 19  Log device removal
 20  Compression using zle (zero-length encoding)
 21  Deduplication
 22  Received properties
 23  Slim ZIL
 24  System attributes
 25  Improved scrub stats
 26  Improved snapshot deletion performance

As versions change, upgrading the ZFS pool and filesystem is possible using the respective upgrade command. To upgrade all imported ZFS pools, issue the following command as root:

zpool upgrade -a

Likewise, to upgrade the ZFS filesystem(s) inside the pool and all child filesystems, issue the following command as root:

zfs upgrade -r -a

The new ZFS features available to these pool and filesystem version(s) will now be available to the upgraded pools/filesystems.


Short-Take: vSphere Multi-core Virtual Machines

November 8, 2010

Virtual machines were once relegated to a second class status of single-core vCPU configurations. To get multiple process threads, you had to add to add one “virtual CPU” for each thread. This approach, while functional, had potential serious software licensing ramifications. This topic drew some attention on Jason Boche’s blog back in July, 2010 with respect to vSphere 4.1.

With vSphere 4U2 and vSphere 4.1 you have the option of using an advanced configuration setting to change the “virtual cores per socket” to allow thread count needs to have a lesser impact on OS and application licensing. The advanced configuration parameter name is “cpuid.coresPerSocket” (default 1) and acts as a divisor for the virtual hardware setting “CPUs” which must be an integral multiple of the “cpuid.coresPerSocket” value. More on the specifics and limitations of this setting can be found in “Chapter 7, Configuring Virtual Machines” (page 79) of the vSphere Virtual Machine Administrator Guide for vSphere 4.1. [Note: See also VMware KB1010184.]

The value of “cpuid.coresPerSocket” is effectively ignored when “CPUs” is set to 1. In case “cpuid.coresPerSocket” is an imperfect divisor, the power-on operation will fail with the following message in the VI Client’s task history:


Virtual core count is imperfect divisor of CPUs

If virtual machine logging is enabled, the following messages (only relevant items listed) will appear in the VM’s log (Note: CPUs = 3, cpuid.coresPerSocket = 2):

Nov 08 14:17:43.676: vmx| DICT         virtualHW.version = 7
Nov 08 14:17:43.677: vmx| DICT                  numvcpus = 3
Nov 08 14:17:43.677: vmx| DICT      cpuid.coresPerSocket = 2
Nov 08 14:17:43.727: vmx| VMMon_ConfigMemSched: vmmon.numVCPUs=3
Nov 08 14:17:43.799: vmx| NumVCPUs 3
Nov 08 14:17:44.008: vmx| Msg_Post: Error
Nov 08 14:17:44.008: vmx| [msg.cpuid.asymmetricalCores] The number of VCPUs is not a multiple of the number of cores per socket of your VM, so it cannot be powered on.----------------------------------------
Nov 08 14:17:44.033: vmx| Module CPUID power on failed.

While the configuration guide clearly states (as Jason Boche rightly pointed out in his blog):

The number of virtual CPUs must be divisible by the number of cores per socket. The coresPerSocketsetting must be a power of two.

– Virtual Machine Configuration Guide, vSphere 4.1

We’ve found that “cpuid.coresPerCPU” simply needs to be a perfect divisor of the “CPUs” value. This tracks much better with prior versions of vSphere where “odd numbered” socket/CPU counts were allowed, so therefore odd numbers of cores-per-CPU allowed provided the division of CPUs by coresPerCPU is integral. Suffice to say, if the manual says “power of two” (1, 2, 4, 8, etc.)  then those are likely the only “supported” configuration available. Any other configuration that “works” (i.e. 3, 5, 6, 7, etc.) will likely be unsupported by VMware in the event of a problem.

That said, odd values of “cpuid.coresPerCPU” do work just fine. Since SOLORI has a large number of AMD-only eco-systems, it is useful to test configurations that match the physical core count of the underlying processors (i.e. 2, 3, 4, 6, 8, 12). For instance, we were able to create a single, multi-core virtual CPU with 3-cores (CPUs = 3, cpuid.coresPerSocket = 3) and run Windows Server 2003 without incident:

Virtual Tri-core CPU

Windows Server 2003 with virtual "tri-core" CPU

It follows, then, that we were likewise able to run a 2P virtual machine with a total of 6-cores (3-per CPU) running the same installation of Windows Server 2003 (CPUs = 6, cpuid.coresPerSocket = 3):

2P Virtual Tri-core

Virtual Dual-processor (2P), Tri-core (six cores total)

Here are the relevant vmware log messages associated with this 2P, six total core virtual machine boot-up:

Nov 08 14:54:21.892: vmx| DICT         virtualHW.version = 7
Nov 08 14:54:21.893: vmx| DICT                  numvcpus = 6
Nov 08 14:54:21.893: vmx| DICT      cpuid.coresPerSocket = 3
Nov 08 14:54:21.944: vmx| VMMon_ConfigMemSched: vmmon.numVCPUs=6
Nov 08 14:54:22.009: vmx| NumVCPUs 6
Nov 08 14:54:22.278: vmx| VMX_PowerOn: ModuleTable_PowerOn = 1
Nov 08 14:54:22.279: vcpu-0| VMMon_Start: vcpu-0: worldID=530748
Nov 08 14:54:22.456: vcpu-1| VMMon_Start: vcpu-1: worldID=530749
Nov 08 14:54:22.487: vcpu-2| VMMon_Start: vcpu-2: worldID=530750
Nov 08 14:54:22.489: vcpu-3| VMMon_Start: vcpu-3: worldID=530751
Nov 08 14:54:22.489: vcpu-4| VMMon_Start: vcpu-4: worldID=530752
Nov 08 14:54:22.491: vcpu-5| VMMon_Start: vcpu-5: worldID=530753

It’s clear from the log that each virtual core spawns a new virtual machine monitor thread within the VMware kernel. Confirming the distribution of cores from the OS perspective is somewhat nebulous due to the mismatch of the CPU’s ID (follows the physical CPU on the ESX host) and the “arbitrary” configuration set through the VI Client. CPU-z shows how this can be confusing:

CPU-z output, 1 of 2

CPU#1 as described by CPU-z

CPU-z CPU 2 of 2

CPU#2 as described by CPU-z

Note that CPU-z identifies the first 4-cores with what it calls “Processor #1” and the remaining 2-cores with “Processor #2” – this appears arbitrary due to CPU-z’s “knowledge” of the physical CPU layout. In (virtual) reality, this assessment by CPU-z is incorrect in terms of cores per CPU, however it does properly demonstrate the existence of two (virtual) CPUs. Here’s the same VM with a “cpuid.coresPerSocket” of 6 (again, not 1, 2, 4 or 8 as supported ):

Single 6-core (virtual) CPU

CPU-z demonstrating a 1P, six-core virtual CPU

Note again that CPU-z correctly identifies the underlying physical CPU as an Opteron 2376 (2.3GHz quad-core) but shows 6-cores, 6-threads as configured through VMware. Note also that the “grayed-out” selection for “Processor #1” demonstrates a single processor is enumerated in virtual hardware. [Note: VMware’s KB1030067 demonstrates accepted ways of verifying cores per socket in a VM.)

How does this help with per-CPU licensing in a virtual world? It effectively evens the playing field between physical and virtual configurations. In the past (VI3 and early vSphere 4) multiple virtual threads were only possible through the use of additional virtual sockets. This paradigm did not track with OS licensing and CPU-socket-aware application licensing since the OS/applications would recognize the additional threads as CPU sockets in excess of the license count.

With virtual cores, the underlying CPU configuration (2P, 12 total cores, etc.) can be emulated to the virtual machine layer and deliver thread-count parity to the virtual machine. Since most per-CPU licenses speak to the physical hardware layer, this allows for parity between the ESX host CPU count and the virtual machine CPU count, regardless of the number of physical cores.

Also, in NUMA systems where core/socket/memory affinity is a potential performance issue, addressing physical/virtual parity is potentially important. This could have performance implications for AMD 2400/6100 and Intel 5600 systems where 6 and 12 cores/threads are delivered per physical CPU socket.


Short-Take: Jeff Bonwick Leaves Oracle after Two Decades

September 29, 2010

Jeff Bonwick’s last day at Oracle may be September 30, 2010 after two decades with Sun, but his contributions to ZFS and Solaris will live on through Oracle and open source storage for decades to come. In 2007, Bill Moore, Jeff Bonwick (co-founders of ZFS) and Pawel Jakub Dawidek (ported ZFS to FreeBSD) were interviewed by David Brown for the Association for Computing Machinery and discussed the future of file systems. The discussion gave good insights into the visionary thinking behind ZFS and how the designers set out to solve problems that would plague future storage systems.

One thing that has changed, as Bill already mentioned, is that the error rates have remained constant, yet the amount of data and the I/O bandwidths have gone up tremendously. Back when we added large file support to Solaris 2.6, creating a one-terabyte file was a big deal. It took a week and an awful lot of disks to create this file.

Now for comparison, take a look at, say, Greenplum’s database software, which is based on Solaris and ZFS. Greenplum has created a data-warehousing appliance consisting of a rack of 10 Thumpers (SunFire x4500s). They can scan data at a rate of one terabyte per minute. That’s a whole different deal. Now if you’re getting an uncorrectable error occurring once every 10 to 20 terabytes, that’s once every 10 to 20 minutes—which is pretty bad, actually.

– Jeff Bonwick, ACM Queue, November, 2007

But it’s quotes like this from Jeff’s blog in 2007 that really resonate with my experience:

Custom interconnects can’t keep up with Ethernet.  In the time that Fibre Channel went from 1Gb to 4Gb — a factor of 4 — Ethernet went from 10Mb to 10Gb — a factor of 1000.  That SAN is just slowing you down.

Today’s world of array products running custom firmware on custom RAID controllers on a Fibre Channel SAN is in for massive disruption. It will be replaced by intelligent storage servers, built from commodity hardware, running an open operating system, speaking over the real network.

– Jeff Bonwick, Sun Blog, April 2007

My old business partner, Craig White, philosopher and network architect at BT let me in on that secret back in the late 90’s. At the time I was spreading Ethernet across a small city while Craig was off to Level3 – spreading gigabit Ethernet across entire continents. He made it clear to me that Ethernet – in its simplicity and utility – was like the loyal mutt that never let you down and always rose to meet a fight. Betting against Ethernet’s domination as an interconnect was like betting against the house: ultimately a losing proposition. While there will always be room for exotic interconnects, the remaining 95% of the market will look to Ethernet. Lookup “ubiquity” in the dictionary – it’s right there next to Ethernet, and it’s come a long way since it first appeared on Bob Metcalf’s napkin in ’73.

Looking back at Jeff’s Sun blog, it’s pretty clear that Sun’s “near-death experience” had the same profound change on the his thinking; and perhaps that change made him ultimately incompatible with the Oracle culture. I doubt a culture that embraces the voracious acquisition and marketing posture of former HP CEO Mark Hurd would likewise embrace the unknown risk and intangible reward framework of openness.

In each case, asking the question with a truly open mind changed the answer.  We killed our more-of-the-same SPARC roadmap and went multi-core, multi-thread, and low-power instead.  We started building AMD and Intel systems.  We launched a wave of innovation in Solaris (DTrace, ZFS, zones, FMA, SMF, FireEngine, CrossBow) and open-sourced all of it.  We started supporting Linux and Windows.  And most recently, we open-sourced Java.  In short, we changed just about everything.  Including, over time, the culture.

Still, there was no guarantee that open-sourcing Solaris would change anything.  It’s that same nagging fear you have the first time you throw a party: what if nobody comes?  But in fact, it changed everything: the level of interest, the rate of adoption, the pace of communication.  Most significantly, it changed the way we do development.  It’s not just the code that’s open, but the entire development process.  And that, in turn, is attracting developers and ISVs whom we couldn’t even have spoken to a few years ago.  The openness permits us to have the conversation; the technology makes the conversation interesting.

– Jeff Bonwick, Sun blog, April 2007

This lesson, I fear, cannot be unlearned, and perhaps that’s a good thing. There’s an side to an engineer’s creation that goes way beyond profit and loss, schedules and deadlines, or success and failure. This side probably fits better in the subjective realm of the arts than the objective realm of engineering and capitalism. It’s where inspiration and disruptive ideas abide. Reading Bonwick’s “fairwell” posting, it’s clear to see that the inspirational road ahead has more allure than recidivism at Oracle. I’ll leave it in his words:

For me, it’s time to try the Next Big Thing. Something I haven’t fully fleshed out yet. Something I don’t fully understand yet. Something way outside my comfort zone. Something I might fail at. Everything worth doing begins that way. I’ll let you know how it goes.

– Jeff Bonwick, Sun blog, September 2010