ESX host “/var” partition full
Posted by jlchannel - on February 10th, 2009 in Server, Tips | No Comments »

While I was trying to create a new virtual machine from template, I’m getting an error message: Failed to deploy template: A general system occured: Failed to create journal file provider.
I found the problem because of “/var” partition are running out of space. Damn!
During my previous VMware ESX setup, the partition setup are configure as DEFAULT which is 2G for “/var/log”.
[root@esxserver root]# df -kh
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 4.9G 2.1G 2.6G 45% /
/dev/sda1 99M 28M 67M 30% /boot
/dev/sda6 2.0G 496M 1.4G 27% /var/log
Normally, /var partition won’t be filled up easily(1st time encounter this problem) and for some reasons, total of 36 /var/log/vmkernel* related files are fully occupied the /var partition.
For avoid the similar incident, I make a changes on /etc/logrotate.d/vmkernel file.
/var/log/vmkernel{
create 0600 root root
missingok
compress (originally nocompress)
# keep a history over 1 years.
monthly
rotate 12 (originally 36 for 3 years)
# max log size of 200k (thus limiting total disk usage to under 8megs)
size 200k
sharedscripts
postrotate
/bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
endscript
Type # logrotate -f /etc/logrotate.d/vmkernel and forced it run on the spot.
Personally I don’t think I need to keep vmkernel log files for more than a year and for space concern, I would prefer compress/zip it. Again, the same logrotate option may apply to others log files.
Tags: ESX, full, logrotate, partition, var, vmkernel, VMware