Manage updates with the Download App
Whether you're looking to purchase a budget PC for simple tasks, such as sending e-mail and browsing the Web, or you're drooling over the fastest gaming desktop money can buy, system speed should be one of your key considerations. To help you find the right desktop for your needs, we run each system through a series of industry-standard benchmarks and actual applications designed to gauge real-world performance. Our benchmark results and analyses compare a PC's performance to that of other systems in its class, typically budget, mainstream, performance, or business.
Our regular testing procedure uses two different sets of benchmark tests, focusing on several key areas of desktop performance. BAPCo's SysMark 2004 measures a system's overall application performance using real-world office-productivity and content-creation software. To determine a system's high-end 3D graphics capabilities, we run the popular Half-Life 2 and Doom 3 games at settings designed to push a PC's graphics subsystem to its limits (in a measurement to which gamers can relate: frames per second). And to give you an idea of a system's gaming performance with less demanding material, we also run these games at more conservative settings.
For systems that contain multiple-core processors, we run additional tests that focus on multimedia applications and multitasking. These are the areas that multiple-core processors are designed to excel at above and beyond their single-core brethren.
If a system does not come with a restore disc or a partition, we back up the system's hard disk before changing any settings or installing our benchmarks. We use a variety of tools to back up a system's hard disk, including Symantec Ghost Corporate Edition, Acronis True Image 8.0 Corporate Workstation, and PowerQuest's PartitionMagic 8.0.
We use a Dell PowerEdge 4600 file server to store the hard disk images of the systems we test. Additionally, we store our benchmark test files on the server and download the files from the server to the test systems. Our Dell PowerEdge 4600 uses two 2GHz Intel Xeon processors, 2GB of ECC DDR SDRAM, an integrated Dell Perc 3/Di RAID Controller, four Fujitsu Ultra160 hot-pluggable SCSI hard drives set to RAID 5 and providing 50GB of storage, two Seagate Ultra160 hot-pluggable SCSI hard drives set to RAID 1 as the boot partition, four 300W hot-pluggable redundant power supplies, integrated Fast Ethernet and Gigabit Ethernet network adapters, and Microsoft Windows 2000 Server.
System settings
Before running our tests, we disable all applications that launch at start-up and uninstall any programs that conflict with our tests, including apps that will be installed and used by the benchmarks. We set all power-management settings for maximum performance, with no components allowed to power down. We run Windows Update, then turn off automatic updating. We also disable Windows' System Restore, Error Reporting, Remote Assistance, and Remote Desktop. We run all tests with the desktop unattached to any network and with wireless networking (if any) disabled.
Occasionally, some desktops arrive for testing at CNET Labs with nonstandard system settings--such as overclocking of the CPU, GPU, or graphics memory--that can potentially affect the performance. As long as the system manufacturer can demonstrate that the test system's configuration is identical to how it is shipped to paying customers, we test systems with these nonstandard settings and specifically mention these settings in the review.
The SysMark 2004 benchmark measures a system's overall performance using off-the-shelf applications. SysMark comprises two workload scenarios, reflecting usage models of mainstream business users ("office productivity") and Web designers ("Internet content creation"). The benchmark runs multiple applications simultaneously, continually switching between applications, with some tasks running in the background.
SysMark measures the response time between the user's submission of a task and the completion of that task by the application. It records only the response time for each operation, not the time to reposition the cursor to perform the operation. SysMark includes application-load times as part of the measurement for its ratings. We first perform a trial run of SysMark, which forces Windows to run its ProcessIdleTasks routine at the end of the test run, so that the system is fully optimized for all applications used by the benchmark.
SysMark's scoring methodology is broken down into groups or subcategories, such as 3D creation and Web publication (we do not report these scores but use them for our analysis). SysMark determines each of these subcategory ratings by dividing the sum of the total time to perform all of the operations in that subcategory on the calibration system by the total time to perform these same operations on the test system, then multiplying by 100. SysMark's calibration platform is as follows: a 2GHz Pentium 4, Intel's D845GBV motherboard chipset, 512MB of 266MHz DDR SDRAM, an ATI Radeon 9700 Pro graphics card with 64MB of DDR memory, an ATA/100 hard drive using the NTFS file system, Windows XP Professional SP1, Windows' display properties set to 1,024x768 and 32-bit color at 75Hz, and Windows' power scheme set to Always On.
The office-productivity and Internet-content-creation scenario ratings, which we report, are generated by taking the geometric mean of the respective subcategory ratings. Finally, the SysMark 2004 rating, which we also report, is the geometric mean of the Internet-content-creation and office-productivity scenario ratings; higher scores are better. A rating of 100 means the test system's SysMark performance equals that of the calibration platform. Scores above or below 100 indicate performance relative to the calibration platform.
Depending on the class of the system, we may report only the office-productivity or Internet-content-creation portions of SysMark.
We run SysMark with Windows' display properties set to a resolution of 1,024x768 and a 32-bit color depth. (Systems with CRT monitors have the refresh rate set to 85Hz; systems with LCDs are set to 60Hz.)
Valve's Half-Life 2 is generally considered one of the most advanced 3D games available. In addition to advanced physics and AI engines, Half-Life 2 uses Valve's own realistic, shader-based render engine, Source Engine. This DX9-based game's popularity with the gaming set is unparalleled, and Source Engine has even been licensed to other game developers--so you can expect to see more games using this same render engine in the not-too-distant future. With the proper selection of game levels and image-quality settings, Half-Life 2 can be fairly demanding on a graphics subsystem.
After installing the game, we download and install the latest updates from Valve. We then make the following settings within the game:
Activision's Doom 3 is currently one of the most advanced 3D games on the market. The game uses OpenGL with its own lighting model, where lighting effects and shadows are generated in real time--a fairly new rendering technique for games. The result is very impressive-looking graphics with realistic-looking lighting and shadows that respond accurately to moving objects. With the quality settings cranked up high, the game can be very demanding on the graphics subsystem. After installing the retail game, we patch it to version 1.1, which is available for download here. Doom 3 settings:
Increasing the clock speed of a CPU is not the only way to eke out faster performance from a processor. In fact, both AMD and Intel are moving away from the gigahertz war and focusing their attention on producing processors with multiple cores. This approach spreads the processing power around to multiple processing cores, so that multiple simultaneous tasks can execute quicker. Multimedia applications, in particular, are well suited for such a scenario. However, for a single application to take advantage of a multicore environment, the application must be designed specifically to be multithreaded. Unfortunately, not many of today's mainstream multimedia applications are multithreaded and therefore are not specifically optimized to take advantage of multicore technology. As multicore processors become more prevalent in mainstream systems, the availability of mainstream multithreaded applications should increase significantly.
Systems that utilize multicore processors are put through a few additional paces on our tests. We use a collection of multithreaded and nonmultithreaded multimedia applications to determine how well multicore systems perform multimedia-type workloads, such as ripping MP3 files and encoding video files. We also include a multitasking test where two different applications are working simultaneously in order to see how well a multicore system can handle such a workload.
Multitasking test
We use McAfee VirusScan 9 and DivX 6.1.1 together to test the multitasking capabilities of a multicore system. Using the latest updates and VirusScan's default settings, we scan 42.1GB worth of files on the system's hard drive. While the virus scan takes place in the background, Dr. DivX converts a 2-minute, 1-second VOB file, also stored on the system's hard drive, to a high-quality DivX file. The Dr. DivX video encoding takes place in the foreground. We start the virus scan first, then start the video encoding immediately after, timing how long it takes for the system to perform the completion of all tasks from start to finish.
Photoshop CS test
Using our own custom Action file, we time how long it takes for Adobe Photoshop CS to execute the Action file on a collection of 15 JPEG and TIFF files, ranging in size from 1.83MB to 49.2MB. The Action file represents a Web-image production work flow, by performing such tasks as converting color bit-depths, resizing images, applying a number of Photoshop's built-in filters, and exporting the resulting images as moderately compressed JPEG files.
iTunes MP3-encoding test
Using iTunes version 6.0.4.2, we time how long it takes to rip 19 audio tracks to 192Kbps MP3 files. So as to better isolate the system's CPU and eliminate the optical drive as a potential bottleneck, we cheat a bit with this test: instead of ripping directly from an audio CD, we rip from WAV files already stored on the system's hard drive.
In addition to the benchmark tests mentioned above, we run additional tests that are designed to evaluate the performance of the CPU, the memory, and the hard disk subsystems of a Windows desktop. If any of the results from these additional tests are especially relevant to a particular system, we comment on these findings in the review. We are constantly evaluating new tools to assist us in this process. At present, the benchmark we use for our anecdotal testing is SiSoftware Sandra 2005.
Our regular testing procedure uses two different sets of benchmark tests, focusing on several key areas of desktop performance. BAPCo's SysMark 2004 measures a system's overall application performance using real-world office-productivity and content-creation software. To determine a system's high-end 3D graphics capabilities, we run the popular Half-Life 2 and Doom 3 games at settings designed to push a PC's graphics subsystem to its limits (in a measurement to which gamers can relate: frames per second). And to give you an idea of a system's gaming performance with less demanding material, we also run these games at more conservative settings.
For systems that contain multiple-core processors, we run additional tests that focus on multimedia applications and multitasking. These are the areas that multiple-core processors are designed to excel at above and beyond their single-core brethren.
| Test environment |
 |
If a system does not come with a restore disc or a partition, we back up the system's hard disk before changing any settings or installing our benchmarks. We use a variety of tools to back up a system's hard disk, including Symantec Ghost Corporate Edition, Acronis True Image 8.0 Corporate Workstation, and PowerQuest's PartitionMagic 8.0.
We use a Dell PowerEdge 4600 file server to store the hard disk images of the systems we test. Additionally, we store our benchmark test files on the server and download the files from the server to the test systems. Our Dell PowerEdge 4600 uses two 2GHz Intel Xeon processors, 2GB of ECC DDR SDRAM, an integrated Dell Perc 3/Di RAID Controller, four Fujitsu Ultra160 hot-pluggable SCSI hard drives set to RAID 5 and providing 50GB of storage, two Seagate Ultra160 hot-pluggable SCSI hard drives set to RAID 1 as the boot partition, four 300W hot-pluggable redundant power supplies, integrated Fast Ethernet and Gigabit Ethernet network adapters, and Microsoft Windows 2000 Server.
System settings
Before running our tests, we disable all applications that launch at start-up and uninstall any programs that conflict with our tests, including apps that will be installed and used by the benchmarks. We set all power-management settings for maximum performance, with no components allowed to power down. We run Windows Update, then turn off automatic updating. We also disable Windows' System Restore, Error Reporting, Remote Assistance, and Remote Desktop. We run all tests with the desktop unattached to any network and with wireless networking (if any) disabled.
Occasionally, some desktops arrive for testing at CNET Labs with nonstandard system settings--such as overclocking of the CPU, GPU, or graphics memory--that can potentially affect the performance. As long as the system manufacturer can demonstrate that the test system's configuration is identical to how it is shipped to paying customers, we test systems with these nonstandard settings and specifically mention these settings in the review.
| BAPCo SysMark 2004 |
|
The SysMark 2004 benchmark measures a system's overall performance using off-the-shelf applications. SysMark comprises two workload scenarios, reflecting usage models of mainstream business users ("office productivity") and Web designers ("Internet content creation"). The benchmark runs multiple applications simultaneously, continually switching between applications, with some tasks running in the background.
- The office-productivity scenario runs the following applications: Adobe Acrobat 5.0.5, Microsoft Access 2002 SP2, Microsoft Excel 2002 SP2, Microsoft Internet Explorer 6 SP1, Microsoft Outlook 2002 SP2, Microsoft PowerPoint 2002 SP2, Microsoft Word 2002 SP2, Network Associates McAfee VirusScan 7.0, ScanSoft Dragon NaturallySpeaking 6.0 Preferred, and WinZip Computing WinZip 8.1. File compression and virus scanning runs in the background.
- The Internet-content-creation scenario runs the following applications: Adobe After Effects 5.5, Adobe Photoshop 7.01, Adobe Premiere 6.5, Discreet 3ds Max 5.1, Macromedia Dreamweaver MX, Macromedia Flash MX, Microsoft Windows Media Encoder 9.0 series, Network Associates McAfee VirusScan 7.0, and WinZip Computing WinZip 8.1. Video encoding, WinZip, and VirusScan run in the background.
) |
SysMark's scoring methodology is broken down into groups or subcategories, such as 3D creation and Web publication (we do not report these scores but use them for our analysis). SysMark determines each of these subcategory ratings by dividing the sum of the total time to perform all of the operations in that subcategory on the calibration system by the total time to perform these same operations on the test system, then multiplying by 100. SysMark's calibration platform is as follows: a 2GHz Pentium 4, Intel's D845GBV motherboard chipset, 512MB of 266MHz DDR SDRAM, an ATI Radeon 9700 Pro graphics card with 64MB of DDR memory, an ATA/100 hard drive using the NTFS file system, Windows XP Professional SP1, Windows' display properties set to 1,024x768 and 32-bit color at 75Hz, and Windows' power scheme set to Always On.
The office-productivity and Internet-content-creation scenario ratings, which we report, are generated by taking the geometric mean of the respective subcategory ratings. Finally, the SysMark 2004 rating, which we also report, is the geometric mean of the Internet-content-creation and office-productivity scenario ratings; higher scores are better. A rating of 100 means the test system's SysMark performance equals that of the calibration platform. Scores above or below 100 indicate performance relative to the calibration platform.
Depending on the class of the system, we may report only the office-productivity or Internet-content-creation portions of SysMark.
We run SysMark with Windows' display properties set to a resolution of 1,024x768 and a 32-bit color depth. (Systems with CRT monitors have the refresh rate set to 85Hz; systems with LCDs are set to 60Hz.)
| Half-Life 2 |
|
) |
After installing the game, we download and install the latest updates from Valve. We then make the following settings within the game:
- Combat difficulty: Easy
- Enabled developer console (~): Enabled
- Aspect ratio: Normal (4:3)
- Display mode: Full screen
- Model detail: High
- Texture detail: High
- Water detail: Reflect all
- Shadow detail: High
- Shader detail: High
- Wait for vertical sync: Disabled
| Doom 3 |
|
) |
- Video quality: High quality
- Full screen: Yes
- High-quality special effects: Yes
- Enable shadows: Yes
- Enable specular: Yes
- Enable bump maps: Yes
- Vertical sync: No
| Multimedia performance tests |
|
Increasing the clock speed of a CPU is not the only way to eke out faster performance from a processor. In fact, both AMD and Intel are moving away from the gigahertz war and focusing their attention on producing processors with multiple cores. This approach spreads the processing power around to multiple processing cores, so that multiple simultaneous tasks can execute quicker. Multimedia applications, in particular, are well suited for such a scenario. However, for a single application to take advantage of a multicore environment, the application must be designed specifically to be multithreaded. Unfortunately, not many of today's mainstream multimedia applications are multithreaded and therefore are not specifically optimized to take advantage of multicore technology. As multicore processors become more prevalent in mainstream systems, the availability of mainstream multithreaded applications should increase significantly.
Systems that utilize multicore processors are put through a few additional paces on our tests. We use a collection of multithreaded and nonmultithreaded multimedia applications to determine how well multicore systems perform multimedia-type workloads, such as ripping MP3 files and encoding video files. We also include a multitasking test where two different applications are working simultaneously in order to see how well a multicore system can handle such a workload.
)
We use McAfee VirusScan 9 and DivX 6.1.1 together to test the multitasking capabilities of a multicore system. Using the latest updates and VirusScan's default settings, we scan 42.1GB worth of files on the system's hard drive. While the virus scan takes place in the background, Dr. DivX converts a 2-minute, 1-second VOB file, also stored on the system's hard drive, to a high-quality DivX file. The Dr. DivX video encoding takes place in the foreground. We start the virus scan first, then start the video encoding immediately after, timing how long it takes for the system to perform the completion of all tasks from start to finish.
)
Using our own custom Action file, we time how long it takes for Adobe Photoshop CS to execute the Action file on a collection of 15 JPEG and TIFF files, ranging in size from 1.83MB to 49.2MB. The Action file represents a Web-image production work flow, by performing such tasks as converting color bit-depths, resizing images, applying a number of Photoshop's built-in filters, and exporting the resulting images as moderately compressed JPEG files.
)
Using iTunes version 6.0.4.2, we time how long it takes to rip 19 audio tracks to 192Kbps MP3 files. So as to better isolate the system's CPU and eliminate the optical drive as a potential bottleneck, we cheat a bit with this test: instead of ripping directly from an audio CD, we rip from WAV files already stored on the system's hard drive.
)
Sorenson Squeeze video-encoding test
Using our own custom project file, we time how long it takes Sorenson Squeeze 4.0.301.11 to convert a 30-second DV AVI file to MPEG-2 and MPEG-4 files. Our source AVI file was originally captured from a DV camera and its attributes are:
- Frame size: 720x480 pixels
- Video data rate: 1,536Kbps
- Frame rate: 29fps
- Audio: 16-bit
For the MPEG-2 file creation, we use Sorenson Squeeze's default DVD_NTSC_Lg selection.
For the MPEG-4 file creation, we use Sorenson Squeeze's default 384K_Stream selection, with the following deviations:
- Frame size: 480x360
- Video data rate: 1,000Kbps
- Audio data rate: 128Kbps
- Audio sample rate: 44100
- Audio Channels: stereo
- Video method: Sorenson 2-Pass VBP
| Anecdotal testing |
|
In addition to the benchmark tests mentioned above, we run additional tests that are designed to evaluate the performance of the CPU, the memory, and the hard disk subsystems of a Windows desktop. If any of the results from these additional tests are especially relevant to a particular system, we comment on these findings in the review. We are constantly evaluating new tools to assist us in this process. At present, the benchmark we use for our anecdotal testing is SiSoftware Sandra 2005.
