Dell PowerEdge R710 BIOS settings for VMware vSphere 4.x.
Posted by Marek.Z on 14 April 2011
Recently, I was involved in a big infrastructure refreshment project for one of our customers across different locations in Europe. The old hosts were replaced with the brand new Dell PowerEdge R710 hosts with Intel X5650 processor and 96 GB of memory. All hosts were installed with the vSphere Hypervisor (ESXi). Here are some best practice BIOS settings that we used during the project.
Power on the server and press F2 during the initialization process to enter the BIOS and let’s start with:
Memory Settings:
- Set Node Interleaving to Disabled
Processor Settings:
- Set Virtualization Technology to Enabled
- Set Turbo Mode to Enabled
- Set C1E to Disable
Integrated Devices:
- Set the Embedded NIC1 to Enabled without PXE
Serial Communication:
- Set Serial Communication to Off
- Set Redirection After Boot to Disabled
Power Management:
- Set for Maximum Performance
Save the settings and reboot the server. You can now start the installation of the VMware vSphere Hypervisor 
Cheers!
- Marek.Z
Mihir S. said
Marek,
We are about to do a similar upgrade project, and I am wondering if you could possibly elaborate just a bit about why you configure these settings this way. Just a line or two, and/or a link would be really helpful. I’d really appreciate it.
Thanks!
Marek.Z said
Hi Mihir,
Sure, of course. Thanks for visiting.
The processor and memory settings can easily be found in the Performance Best Practices for VMware vSphere 4.1 whitepaper (see link below). I suggest you read it. There is a lot of information for setting up your hardware with vSphere 4.x.
Regarding the Integrated devices and Serial Communication settings, they will not be used on the servers so I turned it off to save the IRQ’s
Cheers!
http://www.vmware.com/pdf/Perf_Best_Practices_vSphere4.1.pdf
Dunedin IT said
Having a couple of performance issues with r710 with the same processors but with 64GB RAM. I’m hoping some BIOS tweaks will help to fix this problem.
What time of RAID hardware and setup do you have?
Thanks for posting
Jamie
Marek.Z said
Hi there,
I’m sorry. But what do you mean by “time of RAID”? Type of RAID setting? If so, we used flash cards to boot the ESXi.
Cheers!
Dunedin IT said
Yeah, I meant Type! Been a long day!
I was having a problem with VM hosts going really really slow and I was think I’ve done something wrong on the hardware selection and setup. Turns out the RAID card battery hadn’t fully charged. This morning it has charged and now the hosts are running fine.
Thanks for your BIOS tips
Jamie
Wilson said
Hi Mihir,
After couple hours by setting up the BIOS as you wrote up, it probably solved the BIGGEST problem on my R710 + ESXi 4.1 ever. The setup is 16GB RAM, 3 x 450G 15Krpm in RAID 5, guest OSs are Ubuntu 10.04 64-bit or CentOS 5.6 64-bit.
The problem was the guest OSs download (uplink) are progressive (massively) slowing down after each reboot. e.g. after each boot up, it’s around 110M/s by http or sftp transfer. By some instances, some guests will drop to 150K/s by transferring the same file. Then it spread to all guests. The files sizes are vary from 20MB to 380MB, sometime some are okay, but not all. Really can’t replicated / isolate the problem.
Initially, I thought it was the NIC problem, and spending almost a week to track down how to disable the LRO on VMXNET3, but no luck at all. Few days ago, I thought it might due to the “sleeping” issues on either NIC or hard drives on linux level. CentOS guest was the worst by hdparm -tT /dev/sda test, the reading buffer is only 3MB/s… really can’t serve as any purpose. Some other Ubuntu lucid 10.04 LTS is better in the very beginning. hdparm can up to 160MB/s or up. However, couple hours later, it drops back to 2~3MB.. and wget http from a bare metal linux is down to 150K/s (compare 110M/s after the guestOS freshly reboot.) In short, it really can’t isolate the problem on R710. So, I headed back installing ESXi 4.1 on 2950 old server. Wao, it rocks. Everything go so smooth for over 24 hours. Downloading speed is keep running 110MB/s, but R710 keeps 120K/s.
By this noon, I stopped all guest and try the last time (before buying a new server or calling Dell), I set the BIOS as you said. Wao, it rocks (at least for couple hours.) Thank you very much. I don’t know will it last for another day or not, I will report this back to here. It might be very useful to the others whose having the same issues for weeks and weeks.
Good work and thanks again.
Wilson.
Ref. testing hd test
dd if=/dev/zero of=/tmp/output.img bs=8k count=256k
sudo hdparm -tT /dev/sda
Wilson said
Stupid me. After another 18 hours, everything drops back to 150K or less. Then I realized assigned total guest RAM is larger than physical RAM on the host. After adjustment, everything back to the normal fast speed as they are. Hope this might help others when they come to this page!
Marek.Z said
Hi Wilson,
Common mistake
Anyway, thank for sharing!
Cheers!
michaelbbrown3 said
Hi Marek,
Any idea how to clear the chassis intrusion alert on the R710? My ESXi 4.1U1 host keeps notifying me of this in the Hardware Status tab of the vSphere Client. The bezel is on securely as well as the top. We have been into the server to replace PCIe cards and RAM, but everything seems to be physically secure.
All the best,
VirtuallyMikeB
http://virtuallymikebrown.com
Marek.Z said
Hi Mike,
Really, I have no idea. I have never encounter that issue before. There must be a option in the BIOS to reset the warning or maybe a failing sensor?!
I suggest you call Dell Support, they got to know this
Cheers!
michaelbbrown3 said
Hi Marek,
Thanks for the quick response. I thought there should be something in the BIOS, as well, but alas – there was nothing.
VirtuallyMikeB
michaelbbrown3 said
Hi Marek,
Nevermind! The problem with the chassis intrusion alert was that the micro-switch was physically broken off, not leaving enough protruding from the top of the riser card for the server cover to touch, and therefore, close the switch.
I had a hard time finding any documentation on the chassis intrusion mechanism for the Dell PowerEdge R710. I found documentation on the R610, however, which told me the micro-switch was on the 2nd riser card. Since the R610 and R710 are similar, I checked it out and sure enough, the micro-switch was in the same spot. We have spare servers so I’ll just replace the riser card and order another one against a different service tag.
Keep up the good work.
VirtuallyMikeB
Marek.Z said
Hi Mike,
Glad you solved that!
Thanks for visit and of course sharing your findings with us.
Cheers!
vanitylicenseplate said
Thanks for the great post.
Still holds for R710 on ESXi 5, except for power management, which should be “OS control” instead of “Maximum Performance”. More reading at http://www.vmware.com/files/pdf/hpm-perf-vsphere5.pdf
Marek.Z said
Indeed, thanks for pointing that out!
Cheers!