 |
Home > SPC Benchmark Results Overview
SPC Benchmark Results
SPC benchmark results provide a source of comparative storage performance
information that is objective, relevant, and verifiable. That information
will provide value throughout the storage product lifecycle, which includes
development of product requirements, product implementation, performance
tuning, capacity planning, market positioning, and purchase evaluations. SPC
benchmarks are designed to be vendor/platform independent and are applicable
across a broad range of storage configuration and topologies. Any vendor
should be able to sponsor and publish an SPC benchmark result, provided their
tested configuration satisfies the requirements of the appropriate SPC
benchmark specification.
There are currently four SPC benchmarks, which have been released:
SPC Benchmark 1 (SPC-1), SPC Benchmark 2 (SPC-2), SPC Benchmark 1C (SPC-1C), and SPC Benchmark 2C (SPC-2C). A brief overview
of each benchmark appears below. Immediately following the benchmark
overviews are links to access the SPC Results. In
addition, the following information is available to aid in comparing SPC
Results as well understanding status levels and categorization applicable for
each SPC result:
SPC Benchmark 1
(SPC-1)
SPC-1 consists of a single workload designed to demonstrate the performance
of a storage subsystem while performing the typical functions of business
critical applications. Those applications are characterized by predominately
random I/O operations and require both queries as well as update operations.
Examples of those types of applications include OLTP, database operations,
and mail server implementations.
SPC Benchmark 2
(SPC-2)
SPC-2 consists of three distinct workloads designed to demonstrate the
performance of a storage subsystem during the execution of business critical
applications that require the large-scale, sequential movement of data. Those
applications are characterized predominately by large I/Os organized into one
or more concurrent sequential patterns. A description of each of the three
SPC-2 workloads is listed below as well as examples of applications
characterized by each workload.
- Large File Processing: Applications in a wide range of fields,
which require simple sequential process of one or more large files such as
scientific computing and large-scale financial processing.
- Large Database Queries: Applications that involve scans or joins
of large relational tables, such as those performed for data mining or
business intelligence.
- Video on Demand: Applications that provide individualized video
entertainment to a community of subscribers by drawing from a digital film
library.
SPC Benchmark 1C
(SPC-1C)
SPC-1C is the first SPC component-level benchmark applicable across a broad
range of storage component products such as disk drives, host bus adapters
(HBAs) intelligent enclosures, and storage software such as Logical Volume
Managers. SPC-1C utilizes an identical workload as SPC-1, which is designed
to demonstrate the performance of a storage component product while
performing the typical functions of business critical applications. Those
applications are characterized by predominately random I/O operations and
require both queries as well as update operations. Examples of those types of
applications include OLTP, database operations, and mail server
implementations.
SPC-1C configurations consist of one or more HBAs/Controllers and one of
the following storage device configurations:
- One (1), two (2), or (4) storage devices in a "standalone"
configuration. An external enclosure may be used but only to provide power
and/or connectivity for the storage devices.
- A "Small Storage Subsystem" configured with a maximum of twenty-four
(24) storage devices.
SPC Benchmark 2C
(SPC-2C)
SPC-2C is the second SPC component-level benchmark, which is also applicable
across a broad range of storage component products such as disk drives, host
bus adapters (HBAs) intelligent enclosures, and storage software such as
Logical Volume Managers. SPC-2C consists of three distinct workloads,
identical to SPC-2, which are designed to demonstrate the performance of a
storage component during the execution of business critical applications that
require the sequential movement of data. Those applications are characterized
predominately by large I/Os organized into one or more concurrent sequential
patterns. A description of each of the three SPC-2 workloads is listed below
as well as examples of applications characterized by each workload.
- Large File Processing: Applications in a wide range of fields,
which require simple sequential process of one or more large files such as
scientific computing and large-scale financial processing.
- Large Database Queries: Applications that involve scans or joins
of large relational tables, such as those performed for data mining or
business intelligence.
- Video on Demand: Applications that provide individualized video
entertainment to a community of subscribers by drawing from a digital film
library.
SPC-2C configurations consist of one or more HBAs/Controllers and one of
the following storage device configurations:
- One (1), two (2), or (4) storage devices in a "standalone"
configuration. An external enclosure may be used but only to provide power
and/or connectivity for the storage devices.
- A "Small Storage Subsystem" configured with a maximum of twenty-four
(24) storage devices.
Comparing SPC Results
When comparing SPC Results, the difference between the Tested Storage
Configuration and the Tested Storage Product of each SPC Result
must be considered.
Tested Storage Configuration (TSC)
The TSC consists of all storage hardware and software used to produce a SPC
Result. Included in the TSC are all storage devices, controllers/adapters,
enclosures, storage software, etc. The TSC represents a complete storage
solution.
Tested Storage Product (TSP)
The TSP is a distinct, customer orderable storage product, which is the focal
point of a SPC Result. Each SPC Result is labeled with the formal name of the
TSP. The TSP may essentially be identical to the Tested Storage
Configuration. The TSP in those cases would typically include all of the
storage hardware and software as a single customer orderable storage product.
In other cases, the TSP may be a component of the Tested Storage
Configuration such as a storage software product. In such cases, the Tested
Storage Configuration consists of multiple, distinct customer orderable
products including the TSP.
Accurate SPC Result Comparison
Comparison between SPC Results that only considers the Tested Storage
Product may, in some cases, compare a total storage solution against a
single component of a storage solution, which would be an inaccurate
comparison. Accurate comparison of SPC Results compares the Tested Storage
Configuration rather than the Tested Storage Product of each
result, ensuring comparison between the total storage solution required to
achieve each SPC Result.
SPC Peer Review Status
An SPC benchmark measurement becomes a new SPC benchmark result upon
successful completion of the required SPC Audit and SPC submission process.
The new SPC result is initially given a Submitted for Review status
for a minimum of 60 days during which the SPC Peer Review occurs. The SPC
Peer Review allows SPC members an opportunity to review the details of the
SPC benchmark result and raise any compliance issues resulting from that
review. If there are no compliance issues raised at the end of the SPC Peer
Review, the status of the SPC benchmark result will transition to an
Accepted status. If compliance issues are raised during the SPC Peer
Review and the SPC benchmark result is found to be compliant with the
appropriate SPC specification, the status of the SPC benchmark result will
transition to an Accepted status. If the SPC benchmark result is found
to be non-compliant during the SPC Peer Review, the SPC benchmark will either
be withdrawn or revised to become compliant
and then will transition to an Accepted status.
Active/Inactive Status
A new SPC benchmark result is automatically designated as Active.
An existing SPC benchmark result may be designated as Inactive by the
Test Sponsor. An Active SPC benchmark result typically becomes
Inactive when the Test Sponsor no longer supports all components that
were priced in the SPC benchmark result, but those components are available
and supported via third-party suppliers. A Test Sponsor may also designate an
existing SPC benchmark result as Inactive in cases where the SPC
benchmark result references a product that has been superceded by a more
recent model/version, but the original product is still available and/or
supported by the Test Sponsor. Inactive SPC results remain on the SPC
Benchmark Results website pages and may continue to be publicly
referenced.
Revision Status
A Test Sponsor may voluntarily submit a revised Full Disclosure Report
(FDR) to update pricing, component availability, or any other allowable
change to an existing SPC benchmark result. A Test Sponsor may also be
required to submit a revised FDR to address confirmed compliance issues that
were identified during the SPC Peer Review of an existing SPC benchmark
result. All revisions will be listed in the Key Dates and Revision
Information section of the FDR and Executive Summary.
The TSP is a distinct, customer orderable product, which is the focal
point of an SPC benchmark result. Each SPC benchmark result is labeled with
the formal name of the TSP.
If all of the storage devices, priced in an SPC benchmark result, are
included as components of TSP, the SPC benchmark result will be categorized
as "TSP includes all storage". Inclusion of all storage devices must
be via one or more selectable options when ordering the TSP.
If the TSP does not offer any selectable options to include all storage
devices priced in an SPC benchmark result, the SPC benchmark result will be
categorized as "TSP does not include all storage". In such cases, the
storage devices are not considered to be part of the TSP and are ordered
separately to complete the priced configuration.
SPC benchmark results in the first category typically feature a TSP that is
primarily a hardware storage product, which includes all configured storage
devices. The second category of SPC benchmark results typically features a
TSP that is primarily a software storage product and requires the configured
storage devices to be ordered separately from the TSP.
Withdrawn Status
An SPC benchmark result may be withdrawn voluntarily by the Test Sponsor
or as directed by the SPC. Each withdrawn SPC benchmark will be placed in one
of the following three categories:
-
Withdrawn 1: The SPC benchmark result was voluntarily withdrawn
by the Test Sponsor without any compliance issues raised concerning the SPC
benchmark result. This action is typically taken when an existing SPC
benchmark result references a product that is no longer supported or has
been superceded by a more recent model/version.
-
Withdrawn 2: The SPC benchmark result was withdrawn by the Test
Sponsor after one or more compliance issues were raised during the SPC Peer
Review. The SPC benchmark result was withdrawn prior to resolution of the
compliance issue(s).
- Withdrawn 3: The SPC benchmark result was withdrawn by the SPC
after being ruled non-compliant with the appropriate SPC
specification.
A withdrawn SPC benchmark result will remain on the SPC Benchmark
Results website pages for 120 days from the date of withdrawal and may be
publicly referenced during that period as a withdrawn SPC benchmark
result. At the end of the 120-day period, the withdrawn SPC benchmark
result will be removed from the SPC Benchmark Results website pages and is no
longer considered a SPC benchmark result and should not be referenced as
such.
|
|
