Just came across the release note from Cisco and VMware about the latest version Nexus 1000V R1.2 for VMware vSphere 4. According to the original blog there are new improvement on the area below

• GUI setup following software install
• Layer 3 control between VSM and VEMs
• Virtual Service Domains for classifying and separating traffic for network services
• iSCSI Multipath—supporting multipath feature introduced in vSphere 4
• XML API for developing client apps for managing/monitoring the Nexus 1000V
• DHCP Snooping for validating DHCP messages and filtering invalid responses
• Dynamic ARP Inspection for validating ARP requests and responses
• IP Source Guard for filtering traffic on interfaces to valid MAC and IP addresses
• MAC Pinning for assigning Ethernet port members to particular port channel subgroup (where upstream switches do not support port channels)
• Static Pinning

There are more new features in the latest release and please read more from here.

Cisco had officially launched the Palo adapter for their UCS platform. Now you will have more choice to select the adapter to be used by the UCS platform. While I am searching more information on the net to compare about the different within Palo and Menlo, I found this great article which provide a very detail overview of the UCS networking, nexus 1000v and etc.

Menlo adapter use the general CNA (Converge Network adapter) technology to provide FCoE connection from servers to switch level. Palo is working slightly different which support up to 128 vnic which user will able to present multiple virtual NIC to the UCS system and allow them to implement the VMware environment with multi-gigabits or interfaces VMware environment.  This will be similar to what Xsigo had done on their I/O virtualization solution which allow users to create multiple virtual NIC and virtual HBA to the dedicated physical server. While Cisco are using FCoE and SFP+ connection, Xsigo are base on HCA and infiniband technology.

I will recommend you to take a full view about the article here.

Benchmark Test Configuration
2 x VMware vSphere 4 hosts
2 x Dell PE2950 Hardware
– Intel(R) Xeon(R) CPU E5410 @ 2.33GHz
– 16GB RAM
– Qlogic QLE8042 10GbE Mercury Converged Network Adapter(CNA) – connected to PCIe 8x slot

Updated: PERC 6/i version. 6.2.0-0013 & BOIS version. 2.6.1

VMware vSphere vCenter
– running as virtual machine
– 4vCPU
– 6GB RAM
– Microsoft Windows Server 2003 Standard edition

2 x Microsoft Windows Server 2003 Standard edition virtual machines
– 4vCPU
– 4GB RAM

Software:
Netperf

MTU 9000 setup
VMware vSphere host
esxcfg-vswitch -m 9000 vSwitch2

Windows Server 2003 standard edition virtual machine
Device Manager -> Network Adapters -> VMXNET3 Ethernet Adapter -> Properties -> Advanced -> Jumbo Packet -> Jumbo 9000 and Speed / Duplex -> 10Gbps Full Duplex

Cisco Nexus 5000K – Enable MTU 9000 and Jumbo Frame

VMware vSphere hosts
name: esx05
name: esx06

Windows Server 2003 standard edition virtual machine
name: test1 (running on esx06)
name: test2 (running on esx05)

Virtual Switch
Port Group name – test262 (Connected with single vNICs detected as Intel 82598EB 10 Gigabit AF Dual Port Network) with VLAN ID 1.

Note: vSphere auto detect as ISP8432 4Gb FCoE PCI Express HBA & Intel 82598EB 10 Gigabit

And the result:

click to enlarge.

Updated: 10 July 2009
CNA connected back to back on 2 VMware vSphere servers.

Probably you may refer to VMware documentation as link below:
- 10GugE Performance
- ESX Networking Planning

Another tricky part is, I’m managed to get total of 6G out of 10G if I running 10 VMs instances with 8192 Message Size and 163840 Socket Size as suggested in documentation as above. Again, single netperf session will get about 2.7G for Linux and 1.5G for Windows either Nexus 5K nor back to back connection. And you may get better result with 1 CPU compare to 4 CPUs or 8 CPUs which I believed a limitation on netperf itself.

Updated: 27 July 2009

Windows 2008 Standard Edition with 1vCPU, E1000 vNIC and 5G single file transfer.

Windows 2008 Standard Edition with 1vCPU, E1000 vNIC and netperf.

Windows 2008 Standard Edition with 1vCPU, VMXNET3 vNIC, Internet Download Manager HTTP multiple sessions download.

Windows 2008 Data Center Edition with 1vCPU, VMXNET3 vNIC, 5G single file transfer.

Windows 2008 Data Center Edition with 8vCPU, VMXNET3 vNIC, 5G single file transfer with default TCP setup.

Windows 2008 Data Center Edition with 8vCPU, VMXNET3 vNIC, 5G single file transfer with TCP tuning enabled.

Summary:
The result are not consistent and I believed they may have some limitation on VMware or Microsoft Windows Operating System or Qlogic CNA card driver.

Updated:
Thanks to Maurizio & Craig comments. I’m fully agreed with you guys that the Cisco Nexus 5000 is not the bottleneck but other factors.

Here I would like to share how to remove Cisco Nexus 1000V Plugin

1. To view the extension key on the vCenter Server, open an Internet Browser, and enter the URL path http:///mob

2. Authentication dialog box open, Enter the username and password and click OK

3. The Managed Object Browser (MOB) open the Service Instance page, in the value column of the Properties table, click Content

4. In the Value column of the Properties table, Click extensionManager OR you can open browser with URL http://localhost/mob/?moid=ExtensionManager

5. In the Methods table, click UnregisterExtension

6. In the Value of the Parameters table, paste “Cisco_Nexus_1000V_1850055053″ and click Invoke Method.

Note: The actual value of “Cisco_Nexus_1000v_xxxxx” will wary. It should match the extension key from the Cisco_nexus_1000v_extension.xml file.

Enjoy!

Click here to enlarge the picture

Attach is the screen shot which is been captured during my test.  Our ESX server is connected with 2Gb fiber channel dual connection from the ESX host to the Fiber Switches in our environment. If we do a calculation, 190MB/s will be 1520 Mb/s which is 76 % of the 2Gbps FC connection utilization. In between, there will always be some latency lost from the host level to our FC storage. My SAN storage is currently 75% populated with disks in active, this performance is consider good enough for my personal point of view. Bear in mind, this is only utilize 1 physical FC connection from my ESX host. Another interesting we found, that my R900 with more than 35 VMs per host, in the peak hour, it will not go more than 50MB/s. In this case, it seems that our environment is still under utilize from the I/O perspectives. Now it came to the point that, we had invested so much money on the fiber switches but we did not fully utilize the resources we have. Imagine that each ESX server will consume 2 HBA connection, and we only utilize less than 50% of the available bandwidth. The next move, I will definitely looking for something to reduce the FC ports utilization on our VM farm and data center as well. I know Xsigo and Cisco both have their idea to do this. You may need to have a look on VP780 from Xsigo or Cisco Nexus 5000 or 7000 for the next generation data center switches.