During the upgrade of my home lab from vSphere 5.0 to vSphere 5.1 I have noticed that there is more memory consumption on my vCenter Server than before the upgrade. According to the vSphere Installation and Setup Guide, this happens because vCenter Server includes several Java services in the installation (VirtualCenter Management Webservices, Inventory Service, Profile-Driven Storage Service and Single Sign On) and when you install vCenter Server, you have an option to select the size of your vCenter Server inventory to allocate memory for these services. The inventory size determines the maximum Java Virtual Machine (JVM) heap settings for these services. Since I only have 3 hosts in my lab and just a couple of VMs I was surprised by the high memory consumption.
The default values for JVM heap settings in vSphere 5.1 can be found in table 2-5 of the vSphere Installation and Setup Guide.
Source: table 2-5 vSphere Installation and Setup Guide.
I think these values are a bit high for a small, home lab environment. Especially when you have limited resources. You can adjust the settings of the services after the installation by editing the corresponding configuration file.
vCenter Server
Change the wrapper.java.additional.9 setting in the wrapper.conf file located in <installation_directory>VMwareInfrastructuretomcatconf folder.
- Open the file and search for wrapper.java.additional.9
- Change the setting to desired value, in my case 512 MB.
- Save and close the file.
vCenter Single Sing On
Also here, change the wrapper.java.additional.9 setting in the wrapper.conf file located in <installation_directory>VMwareInfrastructureSSOServerconf folder.
- Open the file and search for wrapper.java.additional.9
- Change the setting to desired value, in my case once again 512 MB.
- Save and close the file.
vCenter Inventory Service
Change the wrapper.java.maxmemory setting in wrapper.conf located in the <installation_directory>VMwareInfrastructureInventory Serviceconf folder.
- Open the file and search for wrapper.java.maxmemory
- Change the setting to desired value, in my case 1024 MB.
- Save and close the file.
Profile-Drive Storage Service
Also here, change the wrapper.java.maxmemory setting in wrapper.conf located in the <installation_directory>VMwareInfrastructureProfile-Driven Storageconf folder.
- Open the file and search for wrapper.java.maxmemory
- Change the setting to desired value, in my case 512 MB.
- Save and close the file.
Now that you have changed the settings, restart the services or the vCenter Server for the changes to take effect. Try to tweak around with the settings and find out what best suits your environment.
Cheers!
– Marek.Z
Thanks Marek for these gr8 tips. Based on these I managed to do the same to the vCenter 5.1 SUSE appliance. I reduced the memory requirement from 8GB to 4GB.
1. After starting appliance, accept the EULA and press cancel
2. SSH to the server and modify the files as mentioned below. I recommend using WinSCP for those confortable with GUI:
vCenter Server: /etc/vmware-vpx/tomcat-java-opts.cfg – Change -Xmx1024m -XX:MaxPermSize=256m -Dvim.logdir=/var/log/vmware/vpx To -Xmx512m -XX:MaxPermSize=256m -Dvim.logdir=/var/log/vmware/vpx
/usr/lib/vmware-vpx/tomcat/bin/setenv.sh – Change JVM_OPTS=”-Xss1024K -Xincgc -XX:+ForceTimeHighResolution -XX:PermSize=64M” To JVM_OPTS=”-Xss512M -Xincgc -XX:+ForceTimeHighResolution -XX:PermSize=64M”
vCenter Single Sing On: /usr/lib/vmware-sso/bin/setenv.sh – Change JVM_OPTS=”$JVM_OPTS -XX:MaxPermSize=256M –Xms2048m –Xmx2048m” to JVM_OPTS=”$JVM_OPTS -XX:MaxPermSize=256M -Xms512m -Xmx512m”
vCenter Inventory Service: /usr/lib/vmware-vpx/inventoryservice/wrapper/conf/wrapper.conf – Change wrapper.java.maxmemory=3072 to wrapper.java.maxmemory=1024
Profile-Drive Storage Service: /usr/lib/vmware-vpx/sps/wrapper/conf/wrapper.conf – Change wrapper.java.maxmemory=1024 to wrapper.java.maxmemory=512
3. Shutdown the appliance and modify the memory from 8Gb to 4GB
4. Configure the appliance as you would normally do
Hope it helps someone.
Hi Ashvin,
Awesome! Thanks for sharing.
Cheers!
Hold on, do not apply Ashvins setting in /usr/lib/vmware-vpx/tomcat/bin/setenv.sh – he changed Xss (thread stack) from 1024k (kilobyte!) to 512M – not a good idea…
With de default settings of “small”, the VC 5.1.0a Server (just VC, no DB) ate up so much memory that even 16GB RAM for the vCenter Server was no where near enough. It was at 97% physical memory all the time. The machine, obviously, was slow as hell as a consequence.
After these exact tweaks (as suggested in this article), physical memory is now stable at 89%. The Server still has 16GB of RAM (i cannot reproduce a machine that is even remotely satisfied with 4 or 8GB RAM). Not a large improvement unfortunately. Others on the Net have experimented with 384MB as the max. values where this article suggests 512 and so on. It accomplisches very little as well. Again only a small reduction in memory consumption. The Java processes still eat memory like they have not eaten for weeks…
No matter what, the memory consumption of VC5.1.0a remains astronomical. I’d say the programmers that coded this piece of crap should be !
VC 5.0.x on the same machine was happy with 7GB RAM (which is still a lot, but it borders on “acceptable”. Needing more than 16GB after the upgrade is idiotic to say the least. And tweaks like in this article hardly helps and the side-effects are not fully understood at this time.
The choice of Java was a poor one to begin with. vCenter and products like View Connection Server never leave the Win64 platform anyway, so why code in a multi-platform language (Java was meant to “write once, run everywhere”) and the quality of the code is obviously appaling, churning away CPU and Memory.
Even on state-of-the-art hardware it’s slow, memory consumption is huge. And for what?
Sorry VMware, bad job. VERY bad job…
Hi Steven,
What you describe is quite unusual, I’m running several vCenter Servers in my lab with 6 GB of memory with everything installed (SQL, SSO, Web Client, VUM etc) and have no issues after tweaking the memory, but that is not a production infra, just a home lab. According to VMware vSphere installation guide the recommended size of memory for your vCenter Server with its components is 10 GB at the moment. Perhaps you could split the load over 2 servers of create a support ticket at VMware.
Good luck with it.
Cheers!
Thank you sooo much. My memory usage has been crazy. I only have 6 hosts and this helps a lot.
Hi Mike,
Great! Just remember that it is not supported by VMware for production environments. Otherwise, you OK 🙂
Cheers!
I am Searching the same issue but there is no Kb article for this. Anybody can help me with a KB Article.
There will never be a KB Article, this is not a supported change for production environments.
The VMware documentation unfortunately does not cover all configuration files. I found more settings for vCenter on Windows as well as the appliance. Both run fine in small environments with 5 GB of memory and still some headroom. Check out my recent posts on that topic:
http://vaspects.blogspot.com/2013/04/minimizing-vcenter-memory-footprint.html
http://vaspects.blogspot.com/2013/04/minimizing-vcenter-memory-footprint_24.html
Blog has moved:
http://www.vaspects.com/2013/04/24/minimizing-vcenter-memory-windows/
http://www.vaspects.com/2013/04/24/minimizing-vcenter-memory-appliance/
Hi Marek,
W are having the VMware setup with 37 ESX and and vcenter with 32 GB RAM runnig on Windows 2008R2. Most of the time the memory utilization is 90%. We have vcenter 5.1. Could you please suggest the proper tweaking parameters for our setup.
Thanks,
Vinod
Hi Vinod,
Well, in a production environment, you should scale your vCenter Server according to VMware best practice. Do not use java tweaks as it not supported by VMware.
90% of 32 GB is quite a lot. Move the SQL database to a separate server is you running it locally, you could also consider moving the Update Manager and any other 3rd party software from the vCenter Server. If this does not help, I suggest you open a support ticket at VMware.
Hope this helps.
Cheers!
Hi Marek,
Thanks for your reply. We are running SQL in the local server. We shall open a case with VMware.
Thanks & regards,
Vinod
is there sucha a thing as too big for JVM heaps? We have a large env and have our system tweaked above the recommended large env setting, (vcenter server is 4GB instead if recommended 3GB,etc…)Thanks,
GB
Hi Glenn,
Could be, if you set more memory than recommended, other processes could be jeopardized on your server.
Cheers!
Too much heap size could cause performance degradation due to garbage collection, which will take more time. Depends on the code quality if the heap gets so much fragmented that the GC will take a noticable amount of time. Symptom would be short “hangs” corresponding to CPU load peaks on the vC. Besides swapping caused by configuring more JVM mem than assigned to the vC VM.
Personally I would stick with the VMware “large” recommendations if you run an environment as large as outlined in the docs. I suspect they choose too large settings anyway, no need to waste more RAM. 😉
Thank for the reply. I will try this.
I’ve had the same problem this week on my home lab when upgrading to 5.5b. I’ve tried your settings, but it would take 2hours before the vcneter server started to lower the RAM usage. After that I’ve restarted my server, with the same problem agian.
After a while i’ve tried a different approach; i’ve deleted the iscsi-discovery adresses (not the adapter itself!!!), rebooted the vcenter server and after 5minutes everything was ready and running fine.
Once done, I’ve added the adresses again for dynamic discovery, and now the vcenter server is using 1,04GB of RAM. note: i’ve kept the settings as you described 🙂
The biggest problem of course is when you need iscsi for your VM’s. I’m using VSA on 3hosts, with SSD cache, better performance and no iscsi needed. I only need iscsi for my backup’s and clones. VSAN is another solution, but never tried that before.
I hope this is a little help to anyone out there with the same problem 🙂
Hi,
Great! Thanks for sharing!
Cheers!
Hi There,
I changed heap memory size in every wrapper.conf and still my vCenter is running on low memory. My vCenter is running on 4 GB.
Kindly suggest.
Hi Prem,
I think 4 GB is a bit low, my vCenter Server run fine with 6 GB back than if I remember correctly.