But I do want to point out some of weakness on the solutions proposed by the deployment team or vendors that suggest to start with 1 blade chassis to reduce the CAPEX require. To achieve the fully redundancy on the virtualization, we may require NIC teaming, redundant FC connection, redundant ISCSI HBA and etc. Some of the guys may had forgotten about the chassis issues on the blade today. If I would like to run the virtualization hosts or clustering servers on blade, I will not choose to go with only single blade chassis in this case. I will not believe there is 100 % guarantee of no failure on the chassis of the blade. I will definitely suggest to split the blade across to 2 different chassis as minimum to minimize the risk. But of course, we may still have a single point of failure on the SAN which require huge amount of capex investment to keep the production storage to be redundant. Imagine if you run 5 ESX hosts on single blade chassis today, you will easily achieve 15:1 per blade for consolidation, and if you are UNLUCKY enough and your chassis just burn or malfunction for any failure, you will have 75 VMs go down at 1 time and your HA and DRS is not worked at all. You will definitely screw and whack by your business or boss in this case. If we do not have enough capex to start with 2 blade chassis for HA in the virtualization, I will suggest to stick with 1U or 2U rack mount servers to minimize the risk of the business. I am impressed with the blade technology today, but just to start with 1 single chassis for HA and redundancy purpose, will not be my call.

As my environment, the virtual infrastructure are supporting our global business in Asia Pacific, EMEA and NA time zone. When I tracked back the performance history and found the busy hour for most of the machines will be different. Due to the time zone different, I notice that the hardware utilization on the ESX host are always remain 40 to 50 % all the time.

With the data record on hand, I decided to re-allocate the virtual machines in my virtual machine with VMotion to improve utilization on our virtual infrastructure. I had balanced allocated the Asia Pacific machine mix with Europe and North America servers on every esx host. In this case, we had able to consolidate more servers into every ESX host, and users do not experience slow performance after the changes been made. As now, the maximum CPU utilization on the ESX host always remain at 70% at the peak hour which is due to the time zone crossing from Asia pacific with EMEA hour. This peak utilization will not stay longer than 4 hours and it will back to normal once Asia Pacific office hour is end.

This trick may not work for everyone, but if your environment do similar with what happen here, this should work. As today, I do not utilize the resource pool after the virtual machines re-allocation I did. Of course, you may need to assign the resources carefully to every virtual machines.

The reason of virtualizing today, is really not about sacrifice performance. In our environment, we have most VMs running with 4 Vcpu as the result of customer compare VM to Physical Servers to ensure the performance is sustainable or improve when they switch to VM. When the VM itself is idle or less busy, it will not consume too much resources on the Physical CPU cores. Next release of ESX will go for 8Vcpu per VM. The number of Vcpu been assigned to the VMs which require extra CPU resources will increase too.

Updated: Configuration Maximums for VMware Infrastructure 3 – Number of Virtual CPU = 192

Here I am interesting to show 1 of the screen shot I capture during my stress test on the ESX host for our production machine which hit to 100% of the CPU  utilization on the ESX host level.

Click the picture above for full view

This will be really amazing if we can keep this kind of utilization without impact to the performance for the VM guests. Unfortunately, it does slow down the VM guests in my case as many of us know that we shouldn’t keep the ESX servers resource utilization actively more than 90%. In this round of test, I manage to achieve 40 : 1 ESX host consolidation at my ESX Servers. The normal day, we will always have 28 : 1 as minimal.

Price cut from Intel and 6 cores cpu will definately increase the consolidation ratio in our case moving forward. The integration with the latest technology and ESX 4 will really help in the case to achieve higher number consolidation ratio moving forward.

Personally I had been urging the storage and network vendor over and over again regarding the 10Ge products. The reason of it is pretty simple that the Fiber Channel is just too expensive to be deployed and managed VS ethernet 10Ge. But the concern is the vendor are very slow responding to the 10Ge release roadmap although customers are demanding on that.

With Intel moving to the market now, I will see that is a big opportunitty to make this 10Ge become inexpensive in the future with the high volume production provide by Intel in the servers’ market NIC. As virtualization become an important piece in the data center, the bandwidth will required to be grow as much as possible at the same time VS CPU Cores and Memory. Intel had proven their right track on the servers NIC as many of us can easily found that the server NICs are produced by Intel. Even in VMware ESX servers, your VM will be provided either AMD or Intel virtual NIC drivers on top of the hypervisor Virtualization.

As a IT Consumers, I am more than happy to see the technology become cheaper and cheaper and of course personally I do believe Intel will able to make the popularization for 10Ge moving forward.

Before you can use blade, you require higher power consumption per rack to support approximately 30 to 32 blades per racks on 42 servers rack. At the same time, the cooling unit design in you DC require to be customize to ensure your blade chassis is working in perfect condition. Once you have this, then you may able to start think about Blade.

ESX on Blade have been some idea I personally thought before. The products I specify looking is the latest DELL Blade M1000e. The power and cooling in my DC is not a big issue. When I do analyzed the possibilities, I found couple of show stopper to deny my decision to move forward on that. As the enterprise architect point of view, Blade will be more suitable to consolidate those machine which require to run on physical VS Virtualization. The reason of that, is not really the matter of CPU or Memory you can have in the single blade, is really about the redundancy and performance we focusing on our virtualization. The limited number of pass through, NIC, and FC per blade is really not able to meet the number of VM we tried to achieve per host. We require redundancy, teaming as well as performance through put in term of networking and storage with the ESX servers we have. When we do calculate in term of cost per VM, the number had not show up as significant saving as we expected.

In additional to that, the more ESX machine we have in our Virtualization farm, it always require additional efforts to manage it for long term basis. There are many cases that users had built the ESX server with only 2 gigabits NIC for VMnetwork, which end up facing the performance issue in term on the Networking as well as the single point of failure. Virtualization is not performance sacrification. If we do plan properly, we will gain performance in virtualization vs under utilization physical machine.

Here is those finding I have and I will say that the Blade will not fit the virtualization requirement to achieve High availability and performance requirement. If we will have enough money to be spent on Blade environment, I believe you should have sufficient budget to go for something more suitable like R900, R905 and others 4U servers which provide more memory and CPU you need.

After that, I try to check most of the setting and log file from putty and virtual center. Here is my finding. Due to some unknown reason, the .vmx file setting and the configuration on the virtual center show really different. I had to force restart the virtual center service, and reconfigure some of the resources setting in virtual center to ensure it had not reserve or limit any resources on CPU and memory, and I power up the VM again. This round, the VM able to power up and functioning. Originally, It shouldn’t reserve any resource as I had not configured that. For some unknown reason, the resources limit had been configured by the ESX itself. This had happened in the pass but it had not cause any issue.

The root cause here is clear again that the virtual center information is not tally with the information from the command. I will say this is not something new and is well known and really causing so much issue sometimes. A lesson really had been learnt today.

I do hope the Licensing Module will be remain and VMware may need to consider to reduce the licensing or provide additional software or services to be bundle together. Let’s keep an eye on this and soon will have some announcement on this by VMware.