Windows 7 Performance Questions Don't Add Up

A new study claims that Windows 7 creates serious performance issues for PC users. I'm not sure the authors are even asking the right questions here -- never mind their answers.Here is how a Computerworld article summarizes the study: Citing data from Devil Mountain Software's community-based Exo.performance.network (XPnet), Craig Barth, the company's chief technology officer, said that new metrics reveal an unsettling trend. On average, 86% of Windows 7 machines in the XPnet pool are regularly consuming 90%-95% of their available RAM, resulting in slow-downs as the systems were forced to increasingly turn to disk-based virtual memory to handle tasks.

The 86% mark for Windows 7 is more than twice the average number of Windows XP machines that run at the memory "saturation" point, said Barth. The most recent snapshot of XPnet's 23,000-plus PCs -- taken yesterday -- pegs only 40% of XP systems as running low on memory.

"The vast majority of Windows 7 machines over the last several months are very heavily-memory saturated," Barth told Computerworld. "From a performance standpoint, that has an immediate impact on the machine."

It's a provocative claim, and at first glance the numbers appear to back it up. Given the questions this study fails to answer, however, I'm not prepared to buy into Barth's conclusions.

First, let's get one thing straight: High resource usage does not automatically equal degraded performance. In fact, it can mean exactly the opposite.

The fact is, an operating system that takes full advantage of installed memory is a good thing. After all, RAM is faster than disk-based virtual memory; when a system relies on the former and reduces its use of the latter, users will actually enjoy better system performance.

Of course, the only way to know whether this is happening is to take a closer look at a given system's virtual memory usage. One key metric -- page fault activity -- can give users a good idea of how often their OS is forced to rely on virtual memory. If a system's RAM usage is high yet its page fault activity is low, this indicates that the operating system is good at making the most efficient use of the available RAM.

In other words, high RAM usage might be a bug, or it might be a feature. One needs more data to decide which is the case, and page fault activity can provide that data.

Yet Devil Mountain doesn't report any statistics on page fault activity. And neither the article nor the company's blog entry on the study even bother to mention it as a factor. That's a major omission, and it makes me question the value of anything this company has to say about Windows 7 performance issues.

The same caveats, by the way, apply two two other metrics Devil Mountain reported: CPU usage and I/O activity. Neither metric provides any meaningful insights into Windows 7 performance when they are taken out of context, yet this is exactly how Devil Mountain reports them.

Also, according to the Computerworld article, "Barth acknowledged that XPnet's data couldn't determine whether the memory usage was by the operating system itself, or an increased number of applications." This is a crucial question; most researchers would think twice about drawing any larger conclusions until they found some way to answer it.

I would like to believe that Devil Mountain is really interested in learning the truth about this issue. Yet everything the company has reported so far, and everything Barth has said, suggests that it simply wants to grab headlines by pushing a hot-button topic as hard as it can.

At least for now, I would look elsewhere for trustworthy Windows 7 performance metrics.