Home Phone and Internet $60/mo
 | Windows desktops |
 | |
By Daniel A. Begun
Folks are always bragging about how fast their systems are. But how much of that bragging is based merely on gut feeling? The only true way to judge how fast a system performs is to measure its performance with a benchmark--an objective test designed to perform specific tasks and that generates a repeatable result you can then compare to the performance of other systems.
But why even bother to measure performance? If all you care about is light word processing, infrequent e-mails, and occasional Web surfing, performance shouldn't be your main concern. But if you typically have many programs open at once, run memory-hungry applications, work with large data files, play the latest 3D games, or merely want to keep up with the geeks in the next cubicle, you should care about performance.
But why even bother to measure performance? If all you care about is light word processing, infrequent e-mails, and occasional Web surfing, performance shouldn't be your main concern. But if you typically have many programs open at once, run memory-hungry applications, work with large data files, play the latest 3D games, or merely want to keep up with the geeks in the next cubicle, you should care about performance.
| Before you begin |
|
Before you start testing, there are number of things you need to consider. Many variables can affect performance, so you want to be careful to use the same methodology on the system you are testing as was used for the system(s) you are comparing your system to. If you own the systems being compared, then you have full control over this. This gets a little more difficult, however, if you want to compare your system's performance against someone else's. If you want to compare your system to that of the systems reviewed on CNET, ZDNet, and in the pages of Computer Shopper, you can simply follow our testing methodology. The guidelines presented in this article conform to this methodology.
Some of the variables that can affect a system's performance are the display properties' color depth and resolution settings, applications that launch at start-up, power-management settings, virtual memory and paging file sizes, and the hard drive's DMA setting. Ideally, you want these settings to be same on any systems you are comparing, so you can rule them out as variables. This is what we mean when speak of a testing methodology.
A benchmark score by itself doesn't mean anything unless you have an object for comparison. But you also need to be careful about overgeneralizing the meaning of the results. There is no disputing that a system that garners an application-performance score of 150 is twice as fast as a system with a score of 75. But just because the system with a score of 150 might be a 1.7GHz P4 and the system with the score of 75 might be PIII-700 does not necessarily mean that a typical 1.7GHz P4 system is twice as fast as a typical PIII-700. You need to look at the scores from multiple systems before you can start making generalizations about the performance of a particular class of system. A system is made up of various subsystems that can all affect performance: CPU, memory, system bus, graphics, hard disk, and operating system. Take two identically configured systems and makes changes to any one of these subsystems and you are likely to see a significant change in the system's performance.
| Back it up |
|
Many benchmarks make significant changes to your system, such as modifying the registry and disabling start-up apps. Some benchmarks won't even install properly if you already have certain application installed--so you'll have to uninstall those problem apps first. Therefore, we highly recommend that before you install any benchmark, you perform a full system backup of your entire hard drive using an application such as Symantec's Norton Ghost to a second hard drive or to removable media such as CD-RW. Once you are done benchmarking your system, you can restore your hard drive to its original condition.
| Application performance |
|
Of all the various types of benchmark tests you can run on a system, an application-based benchmark is the best suited to provide a sense of your system's overall performance. By running real-world applications and performing tasks as a typical user would, the benchmark puts nearly all of a desktop's subsystems through the paces. There are a number of application-based benchmarks with industry-wide acceptance, but the one that we have chosen to use is BAPCo's SysMark2002 ($199.95).
SysMark2002 uses both-office productivity and Internet-content-creation applications and breaks up the test into two workload scenarios, reflecting usage models of office-productivity users and content creators. The benchmark runs multiple applications simultaneously, continually switching between applications, with some tasks running in the background.
Before you install SysMark2002, you'll need to uninstall Microsoft Office and McAfee VirusScan if you have them loaded on your system. The next step is to unzip the SysMark installation files to your hard drive from the self-extracting files on the two SysMark CDs. Assuming you accept the defaults, all of the installation files will be placed in the sm2k2src directory in your C: drive. Once you've done that, you need to install DirectX 8.1, which is located in the PreInstall\DX8 folder of the same directory--make sure that you install the version that is appropriate for your operating system (if the system you are testing has Windows XP, you can skip the DX 8.1 installation). You can now commence installing SysMark by launching its setup.exe file. Make sure you follow the onscreen prompts carefully; depending on your operating system, you might have to manually configure the NUL print driver--and note that it is case sensitive. The final step is to download and install the latest SysMark2002 patch from BAPCo's Web site.
Now are you ready to run the test. You can run the office-productivity or Internet-content-creation workload or choose an Official Run, which includes both workloads. If the system you are testing has Windows XP, you'll first want to run a Trial Run, which runs both workloads and initiates XP's ProcessIdleTasks utility. This utility optimizes XP for loading any applications that have been recently used. Since SysMark2002 measures application load time (among other items), you'll want to run a Trial Run first to assure that your system is optimized for the apps the SysMark2002 uses. Note that SysMark2002's Trial Run does not produce any scores.
Tip: To guarantee that the scores you generate can be reproduced, we recommend that you run any benchmark test multiple times. Be sure to defragment the hard drive and reboot the system before you commence a test run.
| 3D graphics performance |
|
If your idea of playing games on PC is limited to solitaire or Minesweeper, you needn't be concerned with your system's 3D graphics performance. On the other hand, if you are a hard-core 3D gamer who enjoys the latest games that push the technological envelope, 3D performance is all the world to you. While there are more 3D graphics benchmarks out there than you can shake a virtual stick at, we settled on two: 3DMark2001 Pro ($39.95 at Futuremark, or download the shareware version for free) and Quake III Arena's built-in benchmark test.
3DMark2001 uses the DX8 interface, which some might argue makes this benchmark a bit ahead of its time. Some of 3DMark2001's tests even require DX8 hardware support, so a system that lacks DX8 hardware support will generate a lower score than one that does. This being the case, we also run the less hardware-demanding, mini OpenGL-based Quake III Arena test to balance the results.
Before you install 3DMark2001, you need to install DX8 (if it isn't already installed). You can choose to install it from the 3DMark2001 CD or download the most recent version. Once DX8 is installed, you can proceed with installing 3DMark2001 by running setup.exe from the CD.
There are a fair number of settings you can select within the benchmark, such as color depth, resolution, and Z-buffer depth. Make sure that you choose the settings that match what were used on the system you are comparing to.
Quake III Arena also has a handful of settings to choose from that can drastically affect the score. Even though the game has been around for some time now, it is still popular enough to warrant periodic patch releases. While logic dictates that you want the most recent patch available, the benchmarking reality is that you want to use the same patch version on your test system as was used on the comparison system--even if it is an older version. You can download Quake patches from GameSpot; we test using patch version 117.
Before you run the Quake III benchmark, we recommend that you confirm that VSync is off; otherwise, performance will be limited to only how fast the monitor can refresh the image and won't push the graphics subsystem to its limits. Hit the keyboard's tilde key to access Quake's command-line interface, then type r_swapinterval. If the result is "r_swapinterval" is: "0" default: "0", then VSync is disabled. If the result is "r_swapinterval" is: "1" default: "0", then VSync is enabled. You need to disable it by typing r_swapinterval 0, then vid_restart.
Quake III Arena includes two built-in benchmarks. At Quake's command line, first type timedemo 1, then type either demo demo001 or demo demo002, depending on which test you want to run (we run demo001). Once the test is completed, hit the tilde key (located to the right of the 1 key) to bring back the command line, and the bottom line will reveal the benchmark score in frames per second (fps).
Tip: Display drivers can vastly affect performance. For the best possible performance, make sure that you download and install the latest driver from your graphics card manufacturer's Web site.
| Video-encoding performance |
|
If you use your computer to make home movies, you're probably interested in your system's ability to encode video clips. But even if you don't encode video clips, these tasks are so CPU intensive that they're an excellent means of isolating the performance of a system's CPU and memory subsystems. For this test, we chose Futuremark's Video2000, which you can download for free from Download.com.
Video2000 requires that DX7 or higher is installed first; it's conveniently included on the CD. Installing the benchmark is as simple as running setup.exe and following the onscreen prompts. The benchmark includes a plethora of video tests, but we are interested in only one of them. Launch Video2000, choose Video2000 Project, then click Select. Choose the Clear All button, then MPEG-2 Encoding and OK, then hit the Benchmark button. On a fast system, the test should take a few seconds to run. Once the test completes, click the Show Details button and scroll down until you see the results for the test; the result will be given in fps.
| Stay tuned |
|
Technologies change quickly, and systems keep getting faster. To keep up with these changes, benchmark technology has to change too, and as a result, benchmarks are typically updated every year--sometimes even quicker. As such, it is important to compare only results generated from the same version of a particular benchmark.
Daniel A. Begun is CNET Labs manager in New York.
