Metrics and Limitations

In this section, some basic notions that dictate various characteristics of architectures are examined. The section opens with a list of key metrics and some brief explanations, and closes with a statement of three important laws.

The key metrics used to characterize high-performance computers are

• Peak operation rate-the fastest achievable rate for the computer.
• Sustainable operation rate-this rate ideally is close to peak operation rate, but could be orders of magnitude below peak operation rate for problems that are ill-suited for a particular petacomputer architecture.
• Operation latency-the delay between the time of initiation of an operation on a set of data and the time at which the results of the operation appear at the output of the operation unit.
• Average memory system latency-the delay between request for an item and the arrival of the item. Because delays may vary depending on where the item is located when requested, latency is averaged over all the possible locations with the probabilities taken into account.
• Memory chip drain time-the time required to read all locations in a memory chip and to present the data read at the external interface of the chip.
• Memory chip cost per bit-the ratio of chip cost to number of bits per chip.
• Ratio of memory latency to operation latency-This ratio expresses the balance between memory and operation units. When the ratio is high, the memory latency may be too high to maintain full utilization of the operation units. When the ratio is very low, the operation units may saturate well before they are able to take full advantage of the memory system. At a ratio of one, one access path of a memory system can supply one operand per operation per cycle, which is a typical design point for high-performance computer systems.
• Memory capacity-the total main memory of the computer.
• Ratio of memory capacity to performance-the ratio of gigabytes of memory to GigaFLOPS of performance.
• Network bisection bandwidth-the peak rate of data transfer between one half of the computer system and the other half of the computer system.
• Per processor network bandwidth-the peak rate of data transfer between one processor and all of the other processors in a system.
• Local memory bandwidth versus network bandwidth-The bandwidth for transfers between local memory and a processor or between closely coupled local memories is generally high when such transfers are carried out on high-bandwidth local interconnections. By comparison, the bandwidth for data transfers between a processor and remote regions of a computer system that are accessible only via network interconnections may be much lower.
• Machine diameter-Measured in processor cycles, this is the length of the longest path from one processor to another processor or to an addressable region of the memory system.
• Average dependence latency-the average time that one operation stalls at a dependence until the dependence is removed. This latency arises because dependences within execution streams force some operations to wait for others.
• Best case dependence latency-the minimum latency that will be incurred when a dependence forces an operation to stall.
• Recurring Cost-the replication cost of a single machine.
• Development cost-the investment cost to develop a machine through the first few copies.
• Price-The price of a machine to a customer is a function of the replication cost plus a fraction of the development cost, plus other components, including profit. When development costs are very large, to bring price down, the number of copies over which the development costs are shared should be as large as possible. Doubling the number of copies in use may have as great an impact on price as reducing the replication cost by a factor of two, depending on the relative size of replication and development cost.

Basic Laws

1. Concurrency=latency times bandwidth.

   Assume that a processor is fed operands at the bandwidth of operations per cycle, each of which requires an elapsed time of cycles. That is, each cycle the processor accepts sets of input operands and produces outputs, and the time elapsed between a specific input and its corresponding output is cycles. Then the internal concurrency of the processor is . In other words, at any given cycle, there are distinct computations proceeding concurrently within the processor. This formula specifies how much concurrency must exist within a processor for a specified bandwidth and latency.

   The formula also can be applied to the latency and bandwidth of a memory system. To sustain operations at maximum rate from a memory system whose bandwidth is and whose latency is requires concurrent access streams from the memory system to the processors, and thus the memory must be able to support concurrent operations internally.

   Corollary: If operation latency is on the order of 1 ns ( s), the concurrency required to achieve a processing bandwidth of PetaFLOPS () must be on the order . To reduce concurrency below 10,000, the operation latency must be less than 10 ps ( s), and to reduce concurrency below 10, the operation latency must be less than 10 femtoseconds ( s).

2. In a machine limited by the speed of light, i.e., when the machine diameter is much greater than one cycle, if bisection bandwidth is nearly the maximum possible, the average dependence latency grows at least as fast as where is the number of processors. Physical constraints on packaging and layout may raise this bound to

   In a machine not limited by the speed of light, that is, in a machine whose diameter is a small number of cycles close to unity, or in a machine with a very small bisection bandwidth, the dependence latency can grow at a rate proportional to

3. In a system that makes use of multiprogramming of independent processes on a single processor to improve performance, memory requirements grow linearly with the degree of multiprogramming, and thus scale proportionally with performance. Parallel applications can save memory relative to multiprogramming because memory is shared. Also, as bandwidth increases in shared-memory systems, memory requirements diminish because sharing of data can replace making copies of data.