How we test: LaptopsBattery life and application performance are two of the more important performance-based attributes of a laptop computer, and they can significantly influence each other. While most laptops modulate processor speed and other resource-intensive components such as screen brightness to save energy, models with more powerful processors or larger displays typically sacrifice battery life for raw power. CNET Labs' tests are designed to gauge a laptop's expected battery life as well as application and gaming performance using many of the same applications you might use on a regular basis, such as Windows Media Player or DVD Player (on the Mac), Apple iTunes, and Adobe Photoshop. Our test results, performance charts, and analysis compare a laptop's performance with that of other systems in its class.
You can read the full description of our testing procedures below, but if you want to get the short version, check out this video.
Test environmentWhen a laptop arrives at CNET Labs for testing, we try to make as few changes as possible to its operating system and software configuration. There are a number of basic changes we must make, however, in order to ensure that certain system settings won't adversely impact a system's performance. The adjustments we make also allow us to compare the performance of different laptops on as even a playing field as possible. Before installing our tests, we uninstall any preinstalled programs that will be used by our benchmarks, ensuring that we always test with the correct version of our test applications.
On Windows systems, we install all available critical Windows Updates, and then turn off automatic updating; on Mac systems, we install all available software updates. For Windows systems, we also disable System Protection (Vista) or System Restore (XP), Problem Reporting (Vista) or Error Reporting (XP), Remote Assistance, Remote Desktop, and all Windows Security Center alerts.
- For Vista systems, we also disable the following Visual Effects:
- Animate controls and elements inside windows
- Animate windows when minimizing and maximizing
- Fade or slide menus into view
- Fade or slide ToolTips into view
- Fade out menu items after clicking
- Slide open combo boxes
- Slide taskbar buttons
We run all tests with the laptop disconnected from any network and with wireless networking and Bluetooth (if any) disabled. Unless the particular test being run requires security software to be present, we deactivate all security applications while the tests are running. The system's screen resolution is set to the native resolution of the laptop and at a 32-bit color depth. For all application testing, the laptop's power management settings are set to allow the system to run at maximum speed, with no components permitted to power down. The laptop's screensaver is also disabled.
Unless otherwise specified, all tests are run a minimum of three times. Following the initial run on Windows systems, we defragment the system's hard drive using Diskeeper 2007 and force Windows to run its ProcessIdleTasks routine so that the system is fully optimized for the applications used by the tests. We report the average of the three scores that are within +/-5 percent of each other. If scores consistently fluctuate outside of the +/-5 percent range, we run additional iterations of the test and instead report the overall average of all the test runs, throwing out the highest and lowest scores.
We first set the brightness of each laptop screen to 60 to 70 nits (or 60cd/m² to 70cd/m²), using a Konica Minolta LS-100 luminance meter to calibrate laptop brightness. If a laptop supports autodim, we disable the feature in order to ensure that the laptop remains at the same brightness level thought testing. We enable adaptive CPU, throttling those laptops that include this feature as user configurable.
The system volume is set to 100 percent; the media playback software's volume is set to 50 percent; and headphones are plugged into the laptop's headphone jack.
For Windows Vista systems, we test with a customized version of Vista's Balanced power scheme. The system, display, and hard disk are not allowed to sleep; the processor state is allowed to scale from 0 to 100 percent; the system is set to remain on until the battery level is 0 percent; and all alarms are disabled.
For Windows XP systems, we test with a customized version of XP's Portable/Laptop power scheme. The system, display, and hard disk are not allowed to sleep, and all alarms are disabled.
For laptops that use a power conservation utility other than the one built into Windows, we choose a profile and custom settings that comes closest to the above schemes.
For Mac laptops, we set the computer, display, and hard disk never to sleep.
For Windows laptops, we test using whichever playback codec is already installed. If one is not installed, we install and use the codec from InterVideo's WinDVD 8. By default, Mac laptops already include the necessary software to play movies. The media playback software we use on Windows laptops is Windows Media Player 11; we use DVD Player on Mac laptops. The video we use is a DivX conversion of the Collector's Edition of King Kong.
We time how long the laptop's power lasts before its battery dies. We set the file to repeat from the beginning in case the power lasts longer than the movie. We run this test only twice if the two scores are +/-5 percent of each other. Performance is reported in minutes.
This test's score is based on how long it takes a system to perform only the QuickTime conversion. The iTunes conversion taking place in the background is designed to significantly increase the overall CPU workload and to create a true multitasking environment. This test exercises nearly every major subsystem, including the CPU, the memory, and the hard drive. Desktops with multicore CPUs are likely to perform better than comparable systems that use CPUs with fewer cores or single-core CPUs.
This test primarily exercises a system's CPU, memory, and chipset subsystem, but it also utilizes the graphics and hard drive subsystems to a degree. Some of the filters we use in the Photoshop CS3 test can use multithreading, so desktops with multicore CPUs are likely to perform better than comparable systems that use CPUs with fewer cores or single-core CPUs.
3D games tests
Quake 4 settings:
- Video quality: High quality
- Full screen: Yes
- High-quality special effects: Yes
- Enable shadows: Yes
- Enable specular: Yes
- Enable bump maps: Yes
- Vertical sync: No
- Multiple CPU/core: Yes (if available)
- Single player physics: Maximum
- Multiplayer physics: Maximum
- Max software sounds: Maximum
- Particle bouncing: Maximum
- Shell casings: On
- World detail: Maximum
- Corpse detail: Maximum
- Effects detail: Maximum
- Model decals: Maximum
- Water resolution: Maximum
- Reflections and displays: Maximum
- Volumetric lights: On
- Volumetric light density: Maximum
- FSAA: 4x *
- Light Detail: Maximum
- Enable shadows: On
- Shadow detail: Maximum
- Soft shadows: Off *
- Texture filtering: Anisotropic 8x
- Texture resolution: Maximum
- Videos: Maximum
- Pixel doubling: Off
- DX8 shadows: Off
- Shaders: Maximum
We play back the game's built-in performance test at resolutions of 1,024x768; 1,280x1,024; 1,600x1,200; and 2,048x1,536, with antialiasing at 4x and anisotropic filtering at 8x. We run the test with the graphics chip's driver interface settings for antialiasing and anisotropic filtering set to application controlled and with vSync off. The test generates an average frame rate score, reported in frames per second (fps); a higher frame rate is better.