Version: 2008
  • On TechRepublic: Windows 7: Slower to boot than Vista?
advertisement

How we test:

Desktops

Whether you're looking to purchase a budget PC for simple tasks, such as sending e-mail and browsing the Web, or you're drooling over the fastest gaming desktop money can buy, system speed should be one of your key considerations--especially as the demands of tomorrow's software challenge today's hardware. To help you find the right desktop for your needs, we run each system through a series of benchmark tests that use many of the same applications you might use on a regular basis. Our test results and analyses compare one system's performance with that of other systems in its class.

Test environment

We test throughput and range in a multiple -node network environment consisting of a 3.4GHz Pentium 4 550-based desktop system and two Dell Precision M4300 laptops. All three computers run Windows XP professional. We typically test a router or an access point with a PC Card adapter from the same manufacturer; if no PC Card adapter is provided by the manufacturer, we use a Linksys WPC600N card. The reason this card is chosen is because it's one of a few that support all wireless specifications and both 5GHz and 2.4GHz wireless bands.

On Windows systems, we install all available critical Windows Updates, then turn off automatic updating; on Mac systems, we install all available software updates. For Windows systems, we also disable System Protection (Vista) or System Restore (XP), Problem Reporting (Vista) or Error Reporting (XP), Remote Assistance, Remote Desktop, and all Windows Security Center alerts.
    For Vista systems, we also disable the following Visual Effects:
  • Animate controls and elements inside windows
  • Animate windows when minimizing and maximizing
  • Fade or slide menus into view
  • Fade or slide ToolTips into view
  • Fade out menu items after clicking
  • Slide open combo boxes
  • Slide taskbar buttons
We run all tests with the desktop disconnected from any network and with wireless networking and Bluetooth (if any) disabled. Unless the particular test being run requires security software to be present, we deactivate all security applications while the tests are running. If a system includes an integrated display, we set the screen resolution to the native resolution of the display; otherwise, Windows is set to 1,024x768. We set the color depth to the highest setting the system supports and the refresh rate to 75Hz. For all application testing, the desktop's power management settings are set to allow the system to run at maximum speed, with no components permitted to power down. The system's screensaver is also disabled.

Occasionally, some desktops arrive for testing at CNET Labs with nonstandard system settings--such as overclocked CPUs, GPUs, or graphics memory--which can potentially affect the performance. As long as the system manufacturer demonstrates that the test system's configuration is identical to how it ships to paying customers, we test systems with these nonstandard settings and mention the settings in the review.

Unless otherwise specified, all tests are run a minimum of three times. Following the initial run on Windows systems, we defragment the system's hard drive using Diskeeper 2007 and force Windows to run its ProcessIdleTasks routine so that the system is fully optimized for the applications used by the tests. We report the average of the three scores that are within +/-5 percent of each other. If scores consistently fluctuate outside of the +/-5 percent range, we run additional iterations of the test and instead report the overall average of all the test runs, throwing out the highest and lowest scores.

Application tests

Multimedia multitasking test

We use Apple's QuickTime to convert a high-definition source video using QuickTime's "Movie to iPod" selection. The source file is an H.264-encoded, 30fps, 1,920x1,072, 302MB MOV file. While the video conversion takes place in the foreground, iTunes converts a group of 128Kbps MP3 files into 128Kbps AAC files.

This test's score is based on how long it takes a system to perform only the QuickTime conversion. The iTunes conversion taking place in the background is designed to significantly increase the overall CPU workload and to create a true multitasking environment. This test exercises nearly every major subsystem, including the CPU, the memory, and the hard drive. Desktops with multicore CPUs are likely to perform better than comparable systems that use CPUs with fewer cores or single-core CPUs.

Adobe Photoshop CS3 image-processing test

Using our own custom Action file, we time how long it takes for Adobe Photoshop CS3 to execute the Action file on a collection of seven, 12.7MB Camera RAW image files captured from an 8.2-megapixel camera. The Action file represents the automated tasks that a portrait or wedding photographer might undertake to prepare black-and-white proofs for a client, such as running the Unsharp Mask, Lens Correction, and Dust and Scratches filters; as well as reducing image noise and converting the images to grayscale JPEGs.

This test primarily exercises a system's CPU, memory, and chipset subsystem, but it also utilizes the graphics and hard drive subsystems to a degree. Some of the filters we use in the Photoshop CS3 test can use multithreading, so desktops with multicore CPUs are likely to perform better than comparable systems that use CPUs with fewer cores or single-core CPUs.

Apple iTunes encoding test

Using iTunes, we time how long it takes to convert 19, 320Kbps MP3 tracks to 128Kbps AAC files, totaling 169MB. This test almost exclusively exercises a system's CPU capabilities. Apple iTunes supports multithreading, so desktops with multicore CPUs are likely to perform better than comparable systems that use CPUs with fewer cores or single-core CPUs.

Microsoft Office productivity test

This test utilizes Microsoft's Word 2003, Excel 2003, and PowerPoint 2003 from Microsoft Office 2003. It is run only on Windows XP and Vista-based systems.

The test starts with Word running a macro, which performs a number of functions on a document, such as searching and replacing, changing font sizes, and creating columns. Once the Word macro completes, Excel launches and runs a macro, which performs functions on a spreadsheet, such as editing formulas and creating charts. Next, PowerPoint runs a macro, which adds graphics and text and moves images around on three different presentations.

As soon as the test starts, a file copy also starts in the background, copying from one set of folders on the system's hard drive to another set of folders on the same drive. The resulting folders total 2.57GB in size. Once the copy completes, the 2.57GB folder is then compressed into a single 2.04GB ZIP file, also in the background.

This test's score is based on how long it takes a system to perform all of these tasks. Time is kept until the last task running completes.

This test exercises nearly every major subsystem, including CPU, memory, and hard drive. Desktops that have multicore CPUs are likely to perform better than comparable systems that use CPUs with fewer cores or single-core CPUs.

Cinebench test

Cinebench is a 3D rendering test based on Maxon's 3D animation application, Cinema 4D. This test almost exclusively exercises a system's CPU capabilities. Cinebench supports multithreading up to 16 CPU cores, so desktops with multicore CPUs are likely to perform better than comparable systems that use CPUs with fewer cores or single-core CPUs.

3D games tests

Quake 4 test

Quake 4, a fairly demanding 3D game developed by Raven Software, is one of the few that is available for both Windows and Mac platforms. The game uses the OpenGL engine of Doom 3, which features its own lighting model (where lighting effects and shadows are generated in real time). The result is impressive graphics with realistic lighting and shadows that respond accurately to moving objects. After installing the retail game, we patch it to version 1.2, which is available for download here on GameSpot.

Quake 4 settings:
  • Video quality: High quality
  • Full screen: Yes
  • High-quality special effects: Yes
  • Enable shadows: Yes
  • Enable specular: Yes
  • Enable bump maps: Yes
  • Vertical sync: No
  • Multiple CPU/core: Yes (if available)
We play back our own custom netdemo of actual gameplay using Quake 4's Bloodwork map. The test generates an average frame rate score, reported in frames per second (fps); a higher frame rate is better. We run the test at resolutions of 1,024x768; 1,280x1,024; 1,600x1,200; and 2,048x1,536, with antialiasing at 4x and anisotropic filtering at 8x, using the game's internal settings; therefore, the graphics chip's driver interface settings for antialiasing and anisotropic filtering are set to application controlled. We run the test with vSync off. Occasionally, we run this test at additional resolutions and settings, which we note in the review.

F.E.A.R. test

F.E.A.R. is a very demanding 3D game that can bring many systems to their knees. Developed by Monolith Productions, F.E.A.R. is a DirectX 9-based game that uses its own proprietary graphic-and-physics engine. As it is a DX9 game, this test is only run on Windows XP and Vista-based systems. The game features complex particle effects, realistic physics, and real-time lighting- and shadow-effects models. After installing the retail game, we patch it to version 1.08, which is available for download here on GameSpot.

F.E.A.R. settings:
  • Single player physics: Maximum
  • Multiplayer physics: Maximum
  • Max software sounds: Maximum
  • Particle bouncing: Maximum
  • Shell casings: On
  • World detail: Maximum
  • Corpse detail: Maximum
  • Effects detail: Maximum
  • Model decals: Maximum
  • Water resolution: Maximum
  • Reflections and displays: Maximum
  • Volumetric lights: On
  • Volumetric light density: Maximum
  • FSAA: 4x *
  • Light Detail: Maximum
  • Enable shadows: On
  • Shadow detail: Maximum
  • Soft shadows: Off *
  • Texture filtering: Anisotropic 8x
  • Texture resolution: Maximum
  • Videos: Maximum
  • Pixel doubling: Off
  • DX8 shadows: Off
  • Shaders: Maximum
* Note that when we run the F.E.A.R. test at a resolution of 1,024x768, we turn Soft shadows on and turn FSAA off.

We play back the game's built-in performance test at resolutions of 1,024x768; 1,280x1,024; 1,600x1,200; and 2,048x1,536, with antialiasing at 4x and anisotropic filtering at 8x. We run the test with the graphics chip's driver interface settings for antialiasing and anisotropic filtering set to application controlled and with vSync off. The test generates an average frame rate score, reported in frames per second (fps); a higher frame rate is better.

Anecdotal testing

In addition to the benchmark tests mentioned above, we run additional tests that are designed to evaluate the performance of the CPU, the memory, and the hard drive subsystems of a Windows desktop. If any of the results from these additional tests are especially relevant to a particular system, we comment on these findings in the review. We are constantly evaluating new tools to assist us in this process. At present, the benchmark we use for our anecdotal testing is SiSoftware Sandra.
advertisement