Up until recently using Data Collector Sets in Performance Monitor with Windows Server 2008 R2 was a bit of a mystery. I understood the purpose, but wasn’t particularly motivated to learn how to use it because resource monitor or task manager gave me the basic information I needed to monitor performance in realtime for a quick fix. However, I’m evaluating disk configurations for a SAN to implement Hyper-V virtualization and needed to know just how much disk speed or IOPS I needed to support a virtualized environment without reaching an I/O bottleneck. I’ve always just followed the credo of using SAS / 15K drives for SQL or traditionally high I/O applications and then using SATA / 7.2K drives for file sharing, print sharing, or low I/O applications. Yet I couldn’t help but wonder if this advice applies more to medium to large organizations. I’ve jotted some notes down on how I’m using Performance Monitor to calculate production server IOPS and then use IOPS calculators to determine a strategy for purchasing disks in a SAN environment.

Calculating IOPS on production servers

  1. Open Performance Monitor and create a user defined data collector set
  2. Choose Create manually after giving a name for your collector set
  3. Create data logs and check Performance counter
  4. Add the following counters (taken from this article http://blogs.technet.com/b/cotw/archive/2009/03/18/analyzing-storage-performance.aspx)
    \LogicalDisk\Avg. Disk Sec/Read
    \LogicalDisk\Avg. Disk Sec/Write
    \LogicalDisk\Disk Bytes/Sec
    \LogicalDisk\Disk Reads/Sec
    \LogicalDisk\Disk Writes/Sec
    \LogicalDisk\Split IO/sec
    \LogicalDisk\Disk Transfers/sec
  5. Follow the rest of the wizard
  6. Right-click on the newly created collector set, navigate to the Stop Condition tab and change the stop condition to the period of time you wish to test IOPS.
  7. You can manually run the collection process by right-clicking the collector set and clicking start or schedule a task by right-clicking, choosing properties, and navigating to the schedule tab.  Note that the Performance Counter service must NOT be disabled on the server for this to work
  8. Once you’ve run the collection process you can view the report by going to the Reports section of Performance monitor and navigating to your user defined data collector set.  When you open the report you can view the summary with top disk by IO Rate and notate the IO/sec.  If you have multiple disks or want more detailed information you can go to the Disk section and choose Disk Breakdown.
  9. Optionally, you can export and import your custom data collector set to other servers quickly to recreate the test on all servers.  Right-click on the data collector set and choose Save Template.  On all other servers you can start the new data collector set wizard, choose create from a template, and then click the browse button to select your custom template.

Now that you have an understanding of how much a I/O load is running on your production server you can then match that up by using an IOPS calculator to determine disk speed and RAID configuration needed to meet that demand.  I used this resource http://www.wmarow.com/strcalc or you can contact the SAN vendor and ask for IOPS specs.  I found this article on transactional I/O regarding Exchange helpful as well http://technet.microsoft.com/en-us/library/bb738147%28EXCHG.80%29.aspx.  So what did I learn, although a 15K SAS drive would be great, SATA will often fit the bill for small to medium sized installations.  Add up the IO/sec for all your servers and then add a buffer (I used 25%, but each organization will have to factor in fluctuation for high demand periods).  It feels much better to make the decision based on logic vs “best practices” from vendors who profit from the upsell.

  1. Puja McGriddle says:

    Thank you for this post!

  2. OliT says:

    Thank you I was looking for something exactly like this!

  3. Veeresh says:

    I was looking for this. Thanks a lot

  4. Brian says:

    For reporting, this stats collection has worked great. Thanks for that! I needed this in CSV format, so I am replying here.
    1. click on the named data collector set
    2. right click “DataCollector01”, properties.
    3. change log format to comma separated.
    4. profit.

    also, can run ‘relog datacollector01.blg -f CSV -o stats.csv’ on the file and get a copy of csv of existing data.

    hope this helps!
    -Brian

  5. Brent says:

    Thanks! The help I needed.

  6. […] As well as the VDI’s, I was interested to know what sort of disks I needed for my servers. I ran a couple of tests for this. Firstly the SAN itself allowed me to set some monitoring on the various Raid Groups and LUNs. I ran this for a few days to get an idea of what was being used. I also used Windows performance monitor to get rough, and I do mean rough, figures of IOPS for each server. You can do this by checking “LogicalDiskDisk Transfers/sec”, see Seth’s article here. […]

  7. […] Using Performance Monitor to calculate IOPS requirements […]

  8. Ben C says:

    What does “top disk by IO Rate” mean here? In perfmon, i only see a chart for the 7 counters. Not sure how to get to the IOPS from the chart?

  9. Julio Colon says:

    Hi, I’m looking for the Performance Monitor, where I can download.

    thank a lot for the help.

  10. Philip Planas says:

    Thanks for the post.
    This really helps me a lot for my task.
    Happy blogging.!

  11. Thanks for writing this up. It gets the basics down so is a good starting point. EMC has an article that may be useful, https://community.emc.com/docs/DOC-28792. They use physical disk as the object, so make sure you understand when to use logical or physical disk objects.

  12. BI Consultant says:

    What if I am currently using a disk subsystem which is able to attain only 200 IOPS and I have a serious disk bottleneck on my critical aplications using that disk ? How can I determine the exact IOPS I need to provide so that I solve the disk botleneck ? I am interested for counters that would tell me that the current demand on the system is 3548 IOPS and the max achieved is 200 ! This way I know that I need to RAID or find a solution to achieve 3548 IOPS or higher. My understanding is that the counters you described here show only the max IOPS achieved by the disk subsystem which in my case will never go beyond 200. Are there counters that can help me scale my disk upgrade to the right IOPS based on the current bottleneck observed on a running system ?