I was busy setup the demo solution for the Cisco summit yesterday. The Demo we had were displaying the VMware, Cisco UCS, Nexus 5000, MDS 9124 & Netapp Storage Solution. 1 of the Surprise thing happened during the setup, which the power source for our MDS 9124 had been tripped during the installation yesterday. In this scenerio, all our connection to ESX host and VM were disconnected. It took us for 25 mins to recovered the power failure and the MDS Switch was back on line after that. I was thought to reboot all the ESX host as we are performing BOOT FROM SAN for all the ESX hosts that we setup. Surprise happened here, which I found all the ESX host were still continue running. I did the command uptime and check the system uptime from vcenter, it showed that the ESX host were not rebooted during the SAN connection drop from UCS to our Netapp FAS storage.

I further checked the virtual machines been power on in the ESX servers, which show all the VM were continue running without system crash or rebooted. Now I realize that the failure on SAN switch may not necessary result system crash or hung, in fact it may allow you resume the system state once the SAN switch are back online, of course, this is no guarantee assumption, just some surprise finding experienced yesterday would like to share here.

Recently I had deployed quite a number of EMC SAN with MDS Switches and Cisco UCS, I found tha FC SAN zoning might be a key consideration we may need to take a look for every deployment. This post will more focus on the MDS and UCS 6120 FC Uplink.

Cisco UCS interconnect fabric switches are utilizing FC uplink to allow the Cisco Unified Computing System to get access to the SAN fabric environment.

Here is the architecture diagram you can refer to. In order to make this work, 1st requirement will be the NPIV.

What is NPIV? ( N_Port ID Virtualization or NPIV is a Fibre Channel facility allowing multiple N_Port IDs to share a single physical N_Port)

For more details information, you can refer to the URL HERE

Once the NPIV had been enabled on the SAN Switch, you just need to enable the FC uplink from the Interconnect Fabric by the UCS Manager or CLI mode. Now you should able to see the WWN login from the command prompt, device manager or fabric manager. Any 1 of the method will work. To simplify the setup for the UCS, you can consider port base zoning or WWN zoning. I had done both setup before with VSAN in place, it looks similar to me except from security stand point. If we are only connecting host and storage via the SAN, the port base zoning with the Storage host group features should able to manage the host group to access the correct LUN. If there are additional devices such as tape library, this may create another layer of confusion. At the end, I configured all the zoning base on WWN id, which are still the best configuration in most environment today to prevent any device, host or target conflict in the fabric environment. Not to say the port base zoning will not work, it just the WWN base is more secure and reliable.

I had done the test with the port base zoning and end up, the backup device will be visible even from my ESX server which run on top of Cisco UCS. This is not cool

This new setup are included with 10 G Ethernet uplink from UCS 6120 to Catalyst 6509 through Fiber, and utilizing FC Uplinks from UCS 6120 to MDS 9124. In my test, I removed the VLAN that I created purposely for FCoE, and found that there are no impact to the configuration.  I had confirmed with the support guys that if you are doing an uplink with FC from UCS 6100 to MDS, you will not require to tag the specified VLAN that created for the VSAN in the configuration.