 |
Home > SPC Benchmark Results Overview
SPC Benchmark Results
SPC benchmark results provide a source of comparative storage performance
information that is objective, relevant, and verifiable. That information
will provide value throughout the storage product lifecycle, which includes
development of product requirements, product implementation, performance
tuning, capacity planning, market positioning, and purchase evaluations. SPC
benchmarks are designed to be vendor/platform independent and are applicable
across a broad range of storage configuration and topologies. Any vendor
should be able to sponsor and publish an SPC benchmark result, provided
their tested configuration satisfies the requirements of the appropriate SPC
benchmark specification.
There are currently two SPC benchmarks, which have been released:
SPC Benchmark 1 (SPC-1) and SPC Benchmark 2 (SPC-2). A brief
overview of each benchmark appears below. Immediately following the
benchmark overviews are links to access the SPC
results. In addition, the following information is available to aid in
comparing SPC Results as well understanding status levels and categorization
applicable for each SPC result:
SPC Benchmark 1 (SPC-1)
SPC-1 consists of a single workload designed to demonstrate the performance
of a storage subsystem while performing the typical functions of business
critical applications. Those applications are characterized by
predominately random I/O operations and require both queries as well as
update operations. Examples of those types of applications include
OLTP, database operations, and mail server implementations.
SPC Benchmark 2
(SPC-2)
SPC-2 consists of three distinct workloads designed to demonstrate the
performance of a storage subsystem during the execution of business critical
applications that require the large-scale, sequential movement of data.
Those applications are characterized predominately by large I/Os organized
into one or more concurrent sequential patterns. A description of each of
the three SPC-2 workloads is listed below as well as examples of
applications characterized by each workload.
- Large File Processing: Applications in a wide range of
fields, which require simple sequential process of one or more large files
such as scientific computing and large-scale financial processing.
- Large Database Queries: Applications that involve scans or
joins of large relational tables, such as those performed for data mining
or business intelligence.
- Video on Demand: Applications that provide individualized
video entertainment to a community of subscribers by drawing from a digital
film library.
Comparing
SPC Results
When comparing SPC Results, the difference between the Tested Storage
Configuration and the Tested Storage Product of each SPC Result
must be considered.
Tested Storage
Configuration (TSC)
The TSC consists of all storage hardware and software used to produce a SPC
Result. Included in the TSC are all storage devices, controllers/adapters,
enclosures, storage software, etc. The TSC represents a complete storage
solution.
Tested Storage Product
(TSP)
The TSP is a distinct, customer orderable storage product, which is the
focal point of a SPC Result. Each SPC Result is labeled with the formal name
of the TSP. The TSP may essentially be identical to the Tested Storage
Configuration. The TSP in those cases would typically include all of the
storage hardware and software as a single customer orderable storage
product. In other cases, the TSP may be a component of the Tested Storage
Configuration such as a storage software product. In such cases, the Tested
Storage Configuration consists of multiple, distinct customer orderable
products including the TSP.
Accurate SPC Result Comparison
Comparison between SPC Results that only considers the Tested Storage
Product may, in some cases, compare a total storage solution against a
single component of a storage solution, which would be an inaccurate
comparison. Accurate comparison of SPC Results compares the Tested
Storage Configuration rather than the Tested Storage Product of
each result, ensuring comparison between the total storage solution required
to achieve each SPC Result.
SPC
Peer Review Status
An SPC benchmark measurement becomes a new SPC benchmark result upon
successful completion of the required SPC Audit and SPC submission process.
The new SPC result is initially given a Submitted for Review status
for a minimum of 60 days during which the SPC Peer Review occurs. The SPC
Peer Review allows SPC members an opportunity to review the details of the
SPC benchmark result and raise any compliance issues resulting from that
review. If there are no compliance issues raised at the end of the SPC Peer
Review, the status of the SPC benchmark result will transition to an
Accepted status. If compliance issues are raised during the SPC Peer
Review and the SPC benchmark result is found to be compliant with the
appropriate SPC specification, the status of the SPC benchmark result will
transition to an Accepted status. If the SPC benchmark result is
found to be non-compliant during the SPC Peer Review, the SPC benchmark will
either be withdrawn or revised to become
compliant and then will transition to an Accepted status.
Active/Inactive Status
A new SPC benchmark result is automatically designated as Active. An
existing SPC benchmark result may be designated as Inactive by the
Test Sponsor. An Active SPC benchmark result typically becomes
Inactive when the Test Sponsor no longer supports all components that
were priced in the SPC benchmark result, but those components are available
and supported via third-party suppliers. A Test Sponsor may also designate
an existing SPC benchmark result as Inactive in cases where the SPC
benchmark result references a product that has been superceded by a more
recent model/version, but the original product is still available and/or
supported by the Test Sponsor. Inactive SPC results remain on the SPC
Benchmark Results website pages and may continue to be publicly
referenced.
Revision
Status
A Test Sponsor may voluntarily submit a revised Full Disclosure Report (FDR)
to update pricing, component availability, or any other allowable change to
an existing SPC benchmark result. A Test Sponsor may also be required to
submit a revised FDR to address confirmed compliance issues that were
identified during the SPC Peer Review of an existing SPC benchmark result.
All revisions will be listed in the Key Dates and Revision
Information section of the FDR and Executive Summary.
The TSP is a distinct, customer orderable product, which is the focal point
of an SPC benchmark result. Each SPC benchmark result is labeled with the
formal name of the TSP.
If all of the storage devices, priced in an SPC benchmark result, are
included as components of TSP, the SPC benchmark result will be categorized
as "TSP includes all storage". Inclusion of all storage devices must
be via one or more selectable options when ordering the TSP.
If the TSP does not offer any selectable options to include all storage
devices priced in an SPC benchmark result, the SPC benchmark result will be
categorized as "TSP does not include all storage". In such cases, the
storage devices are not considered to be part of the TSP and are ordered
separately to complete the priced configuration.
SPC benchmark results in the first category typically feature a TSP that is
primarily a hardware storage product, which includes all configured storage
devices. The second category of SPC benchmark results typically features a
TSP that is primarily a software storage product and requires the configured
storage devices to be ordered separately from the TSP.
Withdrawn
Status
An SPC benchmark result may be withdrawn voluntarily by the Test Sponsor or
as directed by the SPC. Each withdrawn SPC benchmark will be placed in one
of the following three categories:
-
Withdrawn 1: The SPC benchmark result was voluntarily
withdrawn by the Test Sponsor without any compliance issues raised
concerning the SPC benchmark result. This action is typically taken when
an existing SPC benchmark result references a product that is no longer
supported or has been superceded by a more recent model/version.
-
Withdrawn 2: The SPC benchmark result was withdrawn by the
Test Sponsor after one or more compliance issues were raised during the
SPC Peer Review. The SPC benchmark result was withdrawn prior to
resolution of the compliance issue(s).
- Withdrawn 3: The SPC benchmark result was withdrawn by the
SPC after being ruled non-compliant with the appropriate SPC
specification.
A withdrawn SPC benchmark result will remain on the SPC Benchmark
Results website pages for 120 days from the date of withdrawal and may be
publicly referenced during that period as a withdrawn SPC benchmark
result. At the end of the 120-day period, the withdrawn SPC benchmark
result will be removed from the SPC Benchmark Results website pages and is
no longer considered a SPC benchmark result and should not be referenced as
such.
|
|
