While a myriad of issues present themselves as relevant to the broad question of PetaFLOPS computing systems, four major areas were identified as having been of critical importance to past generational changes in high-performance computing. These areas are
Device technology determines the maximum clock rate of a computing system and the density of its component packaging. Conventionally, semiconductor technology has provided the basis for many of the important advances over the preceding two decades, but important alternatives are available that may enable significant performance advances. Among these alternatives are optical devices and superconducting devices. Optical devices provide alternative approaches to communications, storage, and processing with important advantages over their semiconductor counterparts in each case. Josephson-Junction superconducting devices have been explored as a basis for computing systems for over a decade and have been shown to yield substantial performance advantage compared to conventional semiconductor devices. Other exotic technologies may merit consideration also. Beyond the device material physics, the form of the basic computing elements may be subject to change. The conventional use of Boolean gates may need rethinking. Hybrids of digital and analog processing devices may provide significant potential for speedup. Technologies that permit much denser packing or higher interconnectivity, such as those proposed for neural nets, might enable a scale of parallelism unprecedented in today's system framework.
While useful for establishing a baseline, the approach of harnessing off-the-shelf workstation processors in large highly interconnected ensembles is unlikely to move the performance curve to PetaFLOPS levels, with the exception of special but possibly important large widely distributed heterogeneous applications in science, engineering, and information management. The issues of latency, overhead, and load balancing that are already proving to be major challenges in achieving TeraFLOPS-scale computing with MPPs will dominate systems where the speed-of-light distance of one clock cycle will be a fraction of a Wafer Scale Integration (WSI) die. The underlying model of computation reflected by both the system architecture and the programming model may involve serious alteration or even replacement as ease-of-use issues for managing these highly complex systems dominate effectiveness. Much functionality currently found in software will migrate to hardware as runtime information becomes critical for optimizing scheduling and data migration. Assuming that parallelism exploitation will be key to success of PetaFLOPS execution, management of fine-grain tasks and synchronization will further encourage hardware realization of new support mechanisms. Many advanced concepts in parallel architecture have been studied but have failed to compete on an economic basis with conventional incremental advances. A close examination of the underlying concepts of the best of these architecture models will reveal new directions that may dominate system structures for PetaFLOPS class platforms.
The early experience with massively parallel processing is revealing the importance of the interface between the user and the high-performance computing system on which the application is performed. Many difficulties are being encountered that show the need for improved tools to assist in achieving program correctness and performance optimization. Much of the difficulty results from the incremental approach programmers have taken from uniprocessor programming methodologies to parallel programming techniques. Message-passing methods are yielding only slowly to data-parallel programming techniques and these are not likely in and of themselves to fully satisfy the needs of all Grand Challenge problems. Fundamental problems still exist in this highly complex relationship and these must be highlighted. Research in alternative methods has been pursued but little of it has found its way into day-to-day scientific computing. But, these advanced concepts may prove essential for organizing computation on systems logically ten-thousand times larger than today's most substantial systems. Object-oriented, message-driven, and functional programming models may be required in a single framework to manage complexity and master scalable correctness.
PetaFLOPS computing systems are only justified if problems of major importance can be identified requiring such capabilities. Even as the HPCC program works toward the goal of usable TeraFLOPS, it is becoming apparent that many problems of engineering and scientific importance will require performance and storage capacity in excess of that anticipated for the TeraFLOPS machines. In speculating on such problems, the balance of resources as they scale through five orders of magnitude must be understood for real problems. For example, communication requirements can be anticipated to scale nonlinearly for interprocessor and I/O needs for some problems. But the degree of change is poorly understood. Is it possible that a PetaFLOPS system will be largely an I/O hub? Or instead of memory intensive, will interprocessor communication be the dominant resource investment? Without direction from application program scaling evaluation, the entire organization of PetaFLOPS computer systems will be in doubt.
By focusing on these four areas, the workshop worked to provide the basis for understanding the challenges, opportunities, and approaches to achieving and effectively using PetaFLOPS performance. These areas are not orthogonal to each other, but rather are mutually supportive. Each contributes to the context of the others. Each provides constraints on the others. And each may supply some of the solutions to problems presented by the others. It was the task of the workshop to untangle these relationships in the regime of PetaFLOPS operation and establish new directions that reflect the insights gained from such an evaluation.