FC NFS ISCSI
Posted by craig 
- on January 5th, 2009 in Hardware, Storage, Virtualization | 4 Comments »
There had been numbers of review and discussion regarding the choices of storage for us to run on Virtual Environment today. I believed that the hot discussion is always moving talk about the right storage with right design and implementation to the VM farm always. The most hottest topic are still regarding the NFS, ISCSI and FC implementation which provide better features, performance and reliability.
Personally, I had 3 of it running in my environment now and of course, I had spent a huge number of times to really test out all the solution and compare apple to apple to further study of my requirement. Here are some reading I get from my test for 3 solution above.
For I/O performance, I am using the IO meter as to generate the max output to the storage. I manage to get 190MB/s for my FC storage on 100 % read, and I only managed to get the output of 170MB/s on ISCSI, but on NFS, I only managed to get at 125MB/s. If we look at the number here, and compare to my high load R900 ESX host, actually it is more than enough to handle the number of VM I have in the ESX host. There are a lot of article introduce NFS, and the features of Netapps, but remember, ESX itself had improved from time to time. Thin Provisioning is no longer new thing, as you may get it with many vendor today by free as well as volume snap. But Dedupe will still be an interesting technology in netapps. If we look at FC, you will mostly lose all this technology, but it provide a stable performance as you need to virtualize some heavy environment moving forward. I am currently still test out the storage solution before I decide to go with a final choice. I will suggest to keep an eye on the technology trend as we may also remember that the VMware ESX 4 is on the way, which may change the game of storage strategy significantly.
4 Responses
You likely did not have proper network load balancing when you got 125MB/sec on NFS. It is a strange coincidence that your max NFS throughput is the max throughput of 1 Gig Ethernet.
I do agree with your suggestion as I am also finding a way to tweak it further. But in normal case, even with 1Gb connection, we will not getting 100% of 1Gb always. Just as similar to 2Gb FC, I am only getting 1.5Gbps performance as is consider good actually.
True, NFS is slower that FC because NFS requires that the storage host do some of the work (processing of information). However, as the number of VM’s increase on your Volumes/LUNS, FC LUN storage gets saturated and actually slows down incomparison to NFS with a large number of VM’s on a Volume/LUN.
One way to improve the throughput of your NFS storage is to increase the network connection speeds from 1Gbps to 10Gbps or use the Xsigo VP780-X2 (20Gbps) to connect all your ESX hosts. This will DRAMATICALLY improve the testing of your NFS environment.
HI James, some of your statement may be correct, but bear in mind, for performance and best practice to get the peak performance, we do not necessary to keep a big LUN in our environment. The more VMs per LUN will reduce the I/O performance from time to time. If you combine all in single datastore, it will also impact the performance as well as higher risk of the point of failure.
10Gbps are still not popular in the storage markets today, we will still yet to see the performance VS pricing always. You may need to consider the DISK device, Memory Disk for SAN will be common in another couple of years which will be 20 to 30 times faster than fiber disks, and that will really improved the I/O. 10Gbps not meant you will get full bandwidth through put, which will also require to consider the spindle power you have from the storage box, is a combination.