Is virtualization meant everything? Virtualization is not, but it had played the important role in the entire journey. Everyone may had faced different challenges during or after the server virtualization in the data center. The next thing which prompt up after server virtualization will often be I/O virtualization and utility computing model. I/O had become real challenges as server virtualization had increased the total bandwidth require to be allocated per physical servers which may easily contains 30 virtual machines in 1 single server. At the mean time, we often see the FC, ISCSI and NAS are seperated with the Ethernet LAN in data center.  Can we do it better? my answer is Definitely. Plenty of solutions today such as Xsigo or Cisco Nexus are really ready to address the challenges that everyone is facing today through Infiniband or FCoE.

FCoE had become an open standard as Qlogic, Emulex, Brocade and Cisco are officially supported today. Do we require to have multiple SAN cablings and Ethernet gigabits connected to each physical server? I will say Yes or No. For 90% of the environment, it should be the right thing to combine both FC and Ethernet into 1. I/O consolidation allowed everyone to simplify the data center environment and improve the bandwidth per connection from 1Gbps to 10Gbps for Ethernet and FC both. It help us to virtualize and simplify the I/O management within the data center.

Leave the technology away now, and look at the operation structure. How do we benefit from virtualization or private cloud computing? Virtualization had made a big move for many environments to go for the private cloud computing model. As my personal experience and the proven success in the past, the biggest benefit I had experienced was the way it transform the IT Services delivery. It had allowed everyone to work closely and demonstrated a better team work and understanding to delivery the right infrastructure for the business needs. Back to the traditional situation, the virtualization implementation team will have to deal with storage team, server team, application team, DBA, Network folks and others IT personnel within the organization, which require a long journey to kick off the virtualization project. We always noticed that for each individual functional team will have different standards, opinions and road maps for specified technology. We understood that standardization is always important, but it will require to further fine tune when it come to the virtualization, cloud computing and data center transformation era.

The collabration within each IT functional team had been improved due to the virtualization and cloud computing.  Everyone started to learn or understand about both virtualization and cloud computing. Infrastructures and solutions build today are always collaborate and align within different functional team to achieve the company goal. Is there a threat for server, network and storage guys due to virtualization and cloud computing? I will say NO, but it does help to improve the relation within the team and allow them to learn from each others. A pure networking background IT folk had known more about storage and servers technology because virtualization.

This picture had explained the relationship within the IT solutions. With the changes happen today, it allow IT to do more with less. It does simplify the IT infrastructure  and improve the services delivery.

This picture is a good reference which allow us to understand where are we now.

But I do want to point out some of weakness on the solutions proposed by the deployment team or vendors that suggest to start with 1 blade chassis to reduce the CAPEX require. To achieve the fully redundancy on the virtualization, we may require NIC teaming, redundant FC connection, redundant ISCSI HBA and etc. Some of the guys may had forgotten about the chassis issues on the blade today. If I would like to run the virtualization hosts or clustering servers on blade, I will not choose to go with only single blade chassis in this case. I will not believe there is 100 % guarantee of no failure on the chassis of the blade. I will definitely suggest to split the blade across to 2 different chassis as minimum to minimize the risk. But of course, we may still have a single point of failure on the SAN which require huge amount of capex investment to keep the production storage to be redundant. Imagine if you run 5 ESX hosts on single blade chassis today, you will easily achieve 15:1 per blade for consolidation, and if you are UNLUCKY enough and your chassis just burn or malfunction for any failure, you will have 75 VMs go down at 1 time and your HA and DRS is not worked at all. You will definitely screw and whack by your business or boss in this case. If we do not have enough capex to start with 2 blade chassis for HA in the virtualization, I will suggest to stick with 1U or 2U rack mount servers to minimize the risk of the business. I am impressed with the blade technology today, but just to start with 1 single chassis for HA and redundancy purpose, will not be my call.

Click here to enlarge the picture

Attach is the screen shot which is been captured during my test.  Our ESX server is connected with 2Gb fiber channel dual connection from the ESX host to the Fiber Switches in our environment. If we do a calculation, 190MB/s will be 1520 Mb/s which is 76 % of the 2Gbps FC connection utilization. In between, there will always be some latency lost from the host level to our FC storage. My SAN storage is currently 75% populated with disks in active, this performance is consider good enough for my personal point of view. Bear in mind, this is only utilize 1 physical FC connection from my ESX host. Another interesting we found, that my R900 with more than 35 VMs per host, in the peak hour, it will not go more than 50MB/s. In this case, it seems that our environment is still under utilize from the I/O perspectives. Now it came to the point that, we had invested so much money on the fiber switches but we did not fully utilize the resources we have. Imagine that each ESX server will consume 2 HBA connection, and we only utilize less than 50% of the available bandwidth. The next move, I will definitely looking for something to reduce the FC ports utilization on our VM farm and data center as well. I know Xsigo and Cisco both have their idea to do this. You may need to have a look on VP780 from Xsigo or Cisco Nexus 5000 or 7000 for the next generation data center switches.

Before the approach on Xsigo, I am actually working on the new design plan to improve the performance and capabilities of the existing virtual infrastructure for the large scale virtualization in our environment. I am actually think about the Cisco director switch on Fiber channel and the network switches of 6509 with 10Gb module, but the cost of doing this, is really going to kill us for the current economy down turn. Suddenly, I been introduce by my buddy with this brand new products which I setup the POC now and I found that is more make sense to look in this product to unlock the under utilize bandwidth in our DC include both Network and Fiber Channel.

Advantages:

1. HCA is cheaper than HBA as we may need 2 HBA per server for redundancy purpose, which only provide 8Gb bandwidth, the HCA will provide 20Gb per HCA, which will be powerful enough for virtualization purpose.
2. HCA will combine both network and fiber channel capabilities, reduce cabling requirement and improve bandwidth utilization.
3. With Vnic and Vhba, we will able to create and utilize existing available bandwidth by reduce the number of FC and Network switches in the Data Center.
4. I will able to reduce the cabling from 18 to 2 HCA connection per ESX in my environment

I am currently stress test the performance on the Xsigo chassis on our ESX, interesting that it will not sacrifice the performance as it able to perform as same as the previous configuration we done. Now, it provide the opportunity for me to utilize the FC ports in my Data Center without increase the number of FC switches, which is currently 95% utilize. Imagine, with 4Gb FC for HBA, except the high performance database server, we will not able to utilize the storage bandwidth on each FC ports. Same time, the gigabit network, will not be consume for more than 10% most of time, will provide more resources for the system which require extra bandwidth in the data center. Xsigo has the intelligent to manage the bandwidth and assign as needed.

A short summarize, it does looks similar to VMware itself as it provide better utilization with your existing infrastructure which is idle or low utilization most of the time. Again, 10Gb network and FCoE are still kinda expensive and Xsigo does provide an alternative solution for most of the customer needs.

The IBM x3950 M2 provide with scalable version on 1TB of memory and maximum of 96 cores. The concept of doing this is basically scale as on demand. Each of the X3950 M2 is contained 4 physical CPU sockets which support up to current 6 cores processors from Intel and 256GB of physical memory. The chassis itself allow for further expansion of 3 additional of X3950 M2, and able to combine the resources in term of CPU, Memory and I/O module to become single host. Why is scale on demand? The answer is simple, as you start with 1, and scale to 4. When you scale the machine, it scale the I/O, processing power, and total number of memory, still you are managing single host. As HA/DRS is always important to us, you may end up which only require to manage 2 ESX host from the virtual center with this design. If we compare with the traditional way we manage the ESX host, we might end up managing 8 different ESX host. Of course, there are always pros and cons, but I still think that this idea is not bad as we may looking at virtualizing some powerful machine in the future, which you may require more processors and memory per VM. Personally, I am aiming to even run the heavy database environment in a virtual mode as the reason of doing this is not cost but is the important and flexibility we gain from Virtualization Vs Physical Environment always. At the same time, the competition between Intel and AMD had drive the processors power to be increased to the level that we may not able to fully utilize it in 90% of the environment we have today, except Virtualization. VMware is going to support 256GB per VM in the next release. It will not surprise to have such powerful machine like IBM to provide the resources to those who may require having it. Again, this is quite similar to the Z system as IBM is working on SUSE Linux installable on Z system, and able to provision the Xen’s VM to achieve 1:100 easily( This had been communicated by Novell during the seminar in town previously ). Now we may able to do the same for x86 platform and will be interesting to see whether this concept will be another next hot item to the market.
I may think the combination of IBM X3950 M2 with Xsigo Virtual I/O director will be something as perfect match for massive performance and volume for Virtualization. Of course, I believe the cost of this will require additional consideration for us to proceed further.

Many enterprise who have implemented virtual environments as a way to consolidate servers and centralize management are running may facing I/O bottleneck issue. No matter how many servers, switches, SAN storage or operating systems that you have, you’re still require to go through IO device.

Imaging you are running on multiple VMware ESX hosts that have more than 10 nics and multiple HBA cards each. How do you going to manage your cabling? How many network switches or port require? What is the costs? What is the maximum I/O you can get? Now Xsigo can help you on virtualization I/O.

What is Virtualization I/O? As I/O today is specify on storage and network throughputs. In most cases, we used to manage with seperate environment for Storage connectivities and Network connectivities. Therefore, we will have HBA FC connection as well as NIC connection for every servers we have in the Data Center. You will also have different manage switch exist in your DC which require more administrator to manage seperately. In Virtualization, we speak about consolidate, increase utilization, centralized manage and etc. Xsigo Virtualization I/O will able to achieve the concept for us to standardize, consolidate, simplify, increase throughput and utlization for the environment we invested in DC.

Let’s imagine what happen today, you will have 4Gbps connection on FC from your server to the storage, but you will not necessary utulize that. You will not able to allocate the under utilize bandwidth to the rest of the client or connection which require, which is consider a waste. It happen the same thing to Network connection, in most cases, you may not require 1Gbps connection for each servers. Without Virtualization I/O, we will end up buying more manage switches, invest more bandwidth in Data Center and under utilize it. This technology even provide a better bandwidth for each connection and allow the combination for both storage and network. This will simplify the cable managment and adminstration time lead for long term solution. Host Channel Adaptor is cheaper if we compare the price of Host Bus Adaptor today. Is Virtualize I/O make sense? My personal answer is yes and it will just do exactly the same as what VMware provide, fully utilize your environment, for cosolidate, simplify and central manage for I/O in data center.