I gotcha. So the only way to tell if something is up is by diff’ing the current load against historical averages?

In this particular case, the server was basically unusable, so I knew something was up. “top” load usually hovered around 1 historically but had jumped to 15.

Was basically looking for a way to “prove” that the server was acting foo bar.

i.e. that it wasn’t any of my scripts that started acting rogue.

One thing you could do is monitor and trend the page-load time for pages on your site over time. This would allow you to see when there were instances of your relatively low utilization, but relatively poor performance.

The real problem with Load is that on servers like the Web servers I described and Mail servers is that when lots of processes run to handle individual requests, this can drive the load up under semi-normal operations. One of the primary things you should see are relatively slow changes in load. Load shouldn’t change from 1 to 15 over 10 minutes unless your traffic has also spiked substantially (not too likely unless you discover some event, like a blog post or a new link, and that should show up in your analytics).

How do I tell then if the CPU really is overutilized?

-David