Transcript
Networking Division Technical Advisory TA-201
2111 N.E. 25th Avenue Hillsboro, OR 97124-5961
Tuning Windows Server® 2008 R2 with Intel® Ethernet 1 GbE/10 GbE/40 GbE Controllers Drops Link Under Heavy Network Load Products Affected All 1 GbE, 10 GbE, and 40 GbE Intel network adapters running Windows Server® 2008 R2.
Problem Description In some situations when Windows Server® 2008 R2 is under consistent high stress (both CPU and network use), the NDIS driver might force a reset on the network, which might cause a link drop and a reconnect.
Implications Repeated link drop due to NDIS driver network resets.
Corrective Action/Resolution For all corrective actions, make one change at a time and include an incremental backup and/or restore points. 1. Before proceeding with any changes, create a restore point or make sure all relevant files are backed up. 2. Review the error logs in the event viewer. Make changes to avoid any errors. Corresponding services could be disable or delay the restart. For example, if there are Windows Error Reporting (WER), change its service to restart after 10 minutes or longer. Since the system is already under heavy load, repeated event logging increases CPU cycles. Do not change critical services. For more information go to: http://msdn.microsoft.com/en-us/library/windows/desktop/bb513641(v=vs.85).aspx. 3. Use perfmon and observe DPC queued/sec, rate, interrupts/sec on each core (
). View it in a report format and not histogram or line (these are less meaningful formats). 4. In perfmon, add % C state time. Analyze this data and if possible turn off C states. Note that the BIOS might provide options to change C states.
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular, purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps. The products and services described may contain defects or errors which may cause deviations from published specifications. Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries. * Other names and brands may be claimed as the property of others. © 2015 Intel Corporation.
May 2015
1
TA-201 5. For each Intel 1 GbE, 10 GbE, or 40 GbE Ethernet adapter, change the registry value *RSSProfile to four. For each port spread, change the RSS base processor number by setting the registry value of *RssBaseProcNumber to different values. For example, one for the first port, three for the second port, etc. Avoid setting this value to zero. Instead, change the settings to reduce the load on core zero. Note that these two settings need to be done manually in the registry. For more details, go to: http://msdn.microsoft.com/en-us/library/windows/hardware/ff570864(v=vs.85).aspx 6. For each Intel 1 GbE/10 GbE/40 GbE adapter, turn off interrupt moderation. Go to the Properties page and turn off interrupt moderation in both the Advanced and Performance Options tabs. 7. Change the maximum number of RSS processors. Set the RSS processor for each network adapter to two and observe the performance as to how each of the core DPCs are queued. Depending on performance, try changing the RSS processors a value of four. Select a value with a DPC load that gets spread across other cores and minimizes the load on core zero. 8. Set RSS queues to two queues on each network adapter/port. Again use perfmon to see how DPCs are getting queued and spread the load on different cores leaving core zero with the least number of DPCs or interrupts. 9. For each network adapter/port, set the preferred Numa Node (registry name *NumaNodeId) to match *RssBaseProcNumber. To reduce the number of DPCs or interrupts in core zero, you can select a different node. 10. If the problem continues after preforming the previous tuning instructions: Upgrade the operating system to Windows Server® 2012 R2. OR Upgrade to a virtualized solution using a non-W2K8R2 host operating system and W2K8R2 as a guest operating system.
2
May 2015
