Calculation of Max LUN Supported in ESX Server
Posted by craig 
- on February 20th, 2009 in Tips, Virtualization | 5 Comments »
I found my ESX servers could not recover the 65th LUNs that I tried to present to it and myself did log a support call and still pending the reply from VMware. Beside that, I found another interesting article with the details below.
Article Copy from VMware
In Multipathing Configurations the Number of Paths Per LUN Is Inconsistent
The hpsa driver in ESX Server might reduce the number of supportable LUNs below the expected maximum limit of 256 when the controller is used in multipath configurations. In multipath configurations, if all four paths are configured, the total supportable LUNs is reduced to 64. In certain multipath configurations, because each target path consumes an available LUN slot, the total number of supportable LUNs might be reduced to 60.
Workaround
Reduce the number of LUNs on a server until the product of LUNs and paths is less than 256 (LUNs * Number of paths < 256), and if necessary, reduce the LUN count depending on use of multipath until each LUN has the expected number of paths.
The following example shows a configuration with the maximum supportable LUNs presented to an ESX Server installation on four paths, providing all LUNs with the expected number of usable paths:
Path 1: 63 LUNs seen through this path; Total LUN count (63 + 1 path) is less than 256
Path 2: 63 LUNs seen through this path; Total LUN count (63 + 63 + 2 paths) is less than 256
Path 3: 63 LUNs seen through this path; Total LUN count (63 + 63 + 63 + 3 paths) is less than 256
Path 4: 63 LUNs seen through this path; Total LUN count (63 + 63 + 63 + 63 + 4 paths) = 256
If I do use the formula above to calculate my environment, yes, I am at the full limit of 256 LUNs. I have 2 ESX servers which only have 2 HBA connection, and had no problem to present more than 67 physical LUNs to it until now. What I had done now is, I removed 2 HBA connection from each of my ESX servers, and run the rescan, and I found that the LUN is presented as I expected. Again, I am not confirmed with the solution yet and will do another round confirmation with the VMware engineer on this.
Related posts:
- Oracle officially supported on VMware Virtualization
- New Release from Cisco Nexus 1000V R1.2 on VMware vSphere 4
- Server Virtualization & Cloud Computing Model
5 Responses
The normal default for total number of paths on a server is 1024 but it looks like this driver has a much lower limit for some reason.
For a general explanation of the storage maximums you can see http://rodos.haywood.org/2008/12/fc-storage-maximums-explained.html
Just to let you know, I am not using the HP device and SAN, I am also facing the same problem. My ESX servers is running Qlogic HBA card with EMC Storages
I got a call with VMWare support today and confirm my thought on this post is correct. The physical connection of HBA on single ESX host will impact the total number of LUN to be able to present to single ESX host
Hi There
I’m not seeing the same limits. I have 256 LUNs (IDs 0-255) presented to 3 ESX servers:
- 2x Dell 1850, ESX 3.5u3, QLA2342
- 1x Dell 1950, ESX 4.0, QLA2462
Storage is NetApp FAS3070HA (2 controllers, 2 target ports each)
Each server has 4 paths (dual port HBA, each port on a different Brocade 4100. Each FAS controller has a port on each switch.)
ESX4 server sees 256 LUNs with 4 paths each. ESX 3.5 servers see almost all of the LUNs with 4 paths each, except LUN IDs 64, 128 and 192, which I find really weird. Also interesting is that 1, 65, 129, and 193 enumerate on the second path (vmhba0:1:x instead of vmhba0:0:x like all the rest). I don’t know if this is a ESX3.5 thing or a QLA2342 thing.
I’ll play with it more later if I get the chance.
Peter
Interesting finding from your end, but I am not sure whether they had fixed this in the latest patch or not, in my experience, we finally unplug 2 FC connection on our ESX host, and we able to present the extra LUN require. During that time, that is the feedback we received from VMware with the formula to calculate the available LUN per ESX host.