\documentstyle{article} % \setlength{\textheight}{9.7truein} \setlength{\textwidth}{6.25truein} \setlength{\topmargin}{1truein} \setlength{\oddsidemargin}{1truein} \setlength{\evensidemargin}{1truein} \setlength{\headheight}{0truein} \setlength{\headsep}{0truein} \hoffset -1truein \voffset -1truein % % \begin{document} % \pagestyle{empty} \title{PARALLEL COMPUTING IN INDUSTRY --- \\ AN INITIAL SURVEY} \author{Geoffrey C. Fox \\ \\ Syracuse University \\ Northeast Parallel Architectures Center \\ Syracuse, New York 13244-4100 \\ gcf@nova.npac.syr.edu \\ (315) 443-2163 \\ (315) 443-1973 (FAX)} \maketitle \thispagestyle{empty} \begin{abstract} Progress in parallel computing will be determined by the way and timescale in which it is used in industry. We describe the results of a major project funded by New York State to accelerate the introduction of parallel computing into the State's industry. We have surveyed forty different industrial applications from some 18 different economic sectors. We describe some initial results of the survey, which shows very different machine and software models needed in the different sectors. We find that the most promising applications in New York State industry involve much fewer simulations and much more information processing than the current academic and research activities under the U.S. high performance computing and communications initiative. \end{abstract} \section{Background} \label{sec.back} We believe that the progress in parallel computing will be determined by the way and time scale in which it is used by industry. We also believe that high performance computing offers an important opportunity for industry to obtain economic advantage in new and old products. High performance computing (HPC) is a technology where currently the U.S. has a major lead over both Europe and Japan. Thus, we can speculate that use of HPC could allow (the U.S.) industry to obtain an edge in the global economic ``battle'' that many see as the new challenge following the end of the cold war. There is much experience with parallel computing, but so far this is mainly in research and academic applications. How do these differ from the ``real world'' of industrial and government problems? Here, we can learn by examining some feature of existing sequential codes. For industrial and government applications, we typically find large software systems with perhaps one hundred thousand to a million lines of code, not the one to ten thousand line size typical of research academic problems. Correspondingly, industrial codes last longer and evolve more slowly. It is not clear what implications these size and timescale differences have for the software support needs. We note that ``software engineering'' is an underlying principle behind many new government and large industrial software projects. I see little communication between the national high performance computing and software engineering communities---this seems a serious issue which needs to be addressed. DoD estimates that the costs of a typical software project divide into 10\% initial development, 20\% testing and 70\% maintenance during the life time of the code. Most HPC efforts (including mine) address the initial 10\%---is this wise? One could argue that the low ratio for initial development costs could suggest that one should not worry about redeveloping codes from scratch and this should be preferred to porting existing codes. This conclusion is countered by the observation that the existing code embodies ``expertise'' of previous developers and this expertise is not easily available except from the code as the original designers quite likely have moved on. Such nontechnical issues are critical in evaluating the introduction of HPC into industry. In the area of simulation, e.g., industrial design of new materials, air frames, etc., the needed parallel algorithms have probably already been studied in the research community and so one can be confident that parallel computing will be effective as long as we can address the software engineering issues coming from the special demands of the larger and longer lived industrial codes. On the other hand, one initial conclusion of my evaluation of industrial opportunities for HPC is that simulation is very promising, but not the largest market in the long run. Rather, we live in the ``information era'' and it is in the processing of information where HPC will have its largest opportunity. This is not (just) transaction processing for the galaxy-wide network of automatic teller machines; rather, it is the storage and access of information followed by major processing (``number-crunching''). Examples include the interpretation of data from NASA's ``mission to planet earth'' where the processing is large scale image analysis; the scanning and correlation of technical and electronic information from the world's media to give early warning for economic and social crises; the integration of Medicaid databases to lower the burden on doctors and patients and identify inefficiencies. Interestingly, such information processing is currently not stressed in the national HPC initiative. \section{Examples} New York State has funded a project ACTION-NYS (Advanced Computing Technology is an Innovative Opportunity Now for New York State) which is designed to accelerate the integration of HPC into New York industry. Table~1 presents some initial results \cite{Carroll:92a} of a survey dating from the Summer of 1992 which is currently being extended. This quantifies some of the issues described in Section~\ref{sec.back}. The information obtained in the survey typically came from meetings of at least an hour with individuals within particular industries. In some cases, we have had major interaction including research projects to develop prototype applications. The principles behind ACTION-NYS came from a study of several state and federal documents on the role of high technology and parallel computing in industry \cite{Battelle}, \cite{Agencies}, \cite{DofC}. The funding of ACTION-NYS is motivated by economic development and our success will be judged in the long term by the impact of our program in creation and preservation of jobs in the State's industry. Let us now describe the format of the table. We have divided industrial applications into 18 areas (rows of the table). This is somewhat arbitrary and many examples span more than one area, but these areas provide a helpful organization. In the first column of the table, we give both the area title and some example corporations or government agencies with applications in this area. Most of the companies come from New York State, which sponsored this survey. We put in brackets those companies where our interaction was indirect. The second column lists typical applications in this area. The third and fourth column describes the parallelism present in the problem, and the natural machine architectures that support it. This subject has been a particular interest for me \cite{Fox:91d}--\cite{Choudhary:92d}. We will not dwell on it here, but note that we have studied this more precisely with the concept of problem architectures summarized in Table~2. The concepts of Table~1 and Table~2 are related in Table~3. We use the somewhat less precise terminology in Table~1 as it is more familiar. Table~2 summarizes the software paradigms that are suitable for the different problem architectures. The final column, five, of Table~1 contains comments on particular applications we studied in our survey. In the following, we refer to the numerical field number (from one to 18) given for the rows in Table~1. In this short article, we will only comment on some of the fields contained in Table~1. \setcounter{table}{1} \vspace{0.2in} \begin{table} \begin{center} Table 2:\ \ Architectures for Five Problem Classifications \end{center} \begin{itemize} \item{\bf Synchronous:\ \ Data Parallel} Tightly coupled. Software needs to exploit features of problem structure to get good performance. Comparatively easy, as different data elements are essentially identical. {\em Candidate software paradigms: High Performance Fortran, Parallel Fortran 77D, Fortran 90D, CMFortran, Crystal, APL, C++.} \item{\bf Loosely Synchronous:\ \ Data Parallel} As above but data elements are not identical. Still parallelizes due to macroscopic time synchronization. {\em Candidate software paradigms:\ may be extensions of the above. C (Fortran) message passing is currently only guaranteed method!} \item{\bf Asynchronous} Functional (or data) parallelism that is irregular in space and time. Often loosely coupled and so need not worry about optimal decompositions to minimize communication. Hard to parallelize (massively). {\em Candidate software paradigms:\ PCN, Linda, object-oriented approaches.} \item{\bf Embarrassingly Parallel} Independent execution of disconnected components. {\em Candidate software paradigms:\ Several approaches work? PCN, Linda, Network Express, ISIS.} \item{\bf A=LS (Loosely Synchronous Complex)} Asynchronous collection of loosely synchronous components where these program modules can be parallelized. {\em Candidate software paradigms:\ PCN, Linda, ADA, controlling modules written in synchronous or loosely synchronous fashion.} \end{itemize} \end{table} \vspace{0.2in} \vspace{0.2in} \begin{table} \begin{center} Table 3:\ \ Relation of Terminology between Table 1 and Table 2 \vspace{0.2in} \begin{tabular}{||l|l||} \hline & \\ \multicolumn{1}{||c|}{``Problem Parallelism''}&\multicolumn{1}{c||}{Problem Architecture} \\ \multicolumn{1}{||c|}{and Machine Architecture}& \multicolumn{1}{c||}{Classification} \\ \multicolumn{1}{||c|}{(Table 1)}&\multicolumn{1}{c||}{(Table 2)} \\ & \\ \hline & \\ Data Parallel, SIMD&Synchronous \\ Data Parallel, MIMD&Loosely Synchronous \\ Data Parallel, Irregular and or Adaptive&Loosely Synchronous \\ Functional Parallelism&Asynchronous \\ Embarrassingly Parallel&Embarrassingly Parallel \\ Data and Functional Parallelism&Loosely Synchronous Complex \\ & \\ \hline \end{tabular} \end{center} \end{table} \vspace{0.2in} Computational fluid dynamics, structural analysis and electromagnetic simulation (fields 1,3,4) are a major emphasis in the national high performance computing initiative---especially within NASA and DOE. However, the industries that can use this application are typically facing major cutbacks and the integration of new technology faces major hurdles. How do you use parallelism when the corporation would like to shut down its current supercomputer center and, further, has a hiring freeze preventing personnel trained in this area entering the company? We are collaborating with NASA in helping industry with a new consortium MADIC (Multidisciplinary Analysis and Design Industrial Consortium) where several ($\sim 14$) companies are banding together to accelerate the integration of parallelism into their working environment. An interesting initial concept was a consortium project to develop a non-proprietary software suite of generic applications which would be modified by each company for their particular needs. One company would optimize the CFD code for a new commercial transport, another for aircraft engine design, another for automobile air drag simulation, another for automobile fan design, another for minimizing noise in air conditioners or more efficient exhaust pumps. The electromagnetic simulation could be optimized either for stealth aircraft or the simulation of electromagnetics properties for a new high frequency printed circuit board. In the latter case, we use simulation to identify problems which otherwise would require time consuming fabrication cycles. Thus, HPC can accelerate the introduction of products to market and so give competitive edge to corporations using HPC. Currently, the MADIC consortium is not concentrating on particular applications but rather on developing the technology and standards for an integrated multidisciplinary analysis and design environment. This is a key component in national and corporate initiatives in ``concurrent engineering'' and ``agile manufacturing'' which are believed by many to be key guiding principles for the manufacturing principles that will give leadership in the 21st century (field~14). Discussions with industrial members of the consortium suggested that parallelism of individual components of the analysis environment was not sufficient. ``Amdahl's law'' applied to design means one cannot just speed up the CFD simulation by parallelizing this one code when, for instance, mesh generation is an equally or more time consuming step. A parallelized integrated design environment is a ``paradigm shift'' enabled by HPC. Virtual reality (field~16) is likely to be another key component of the future manufacturing environment. Neural nets and related optimization methods \cite{Fox:91j} provide naturally parallel scheduling algorithms (field~18) and this is likely to be a nearer term application of HPC to manufacturing and distribution. Environmental Modelling (field 2) has many similar algorithmic issues to those in field~1. Unfortunately, there are many important applications to simulations of past, present and future contamination of the Earth. There is also a defense application to underseas warfare with the modelling of the ocean environment as a large scale partial differential equation problem, which can determine dynamically the acoustic environment in which sonar signals are propagating. Large scale (teraflop) machines in a submarine would allow real time simulation and lead to dramatic improvement in detection efficiency of other submarines or surface ships. The application of HPC to the oil and chemical industry (fields~5,6,13) is being explored vigorously as part of the national HPCCI and within particular companies. Currently, we have not quantified this opportunity for New York State, although there are important chemical companies here. Power Utilities (field 8) have several interesting applications of HPC including nuclear power safety simulation, as well as gas and electric transmission problems. Here the huge dollar value of power implies that small percentage savings can warrant large HPC systems. There are many electrical transmission problems suitable for HPC which are built around sparse matrix operations. For Niagara Mohawk, a New York utility, the matrix has about 4,000 rows (and columns) with approximately 12 nonzero elements in each row (column). We are designing a parallel transient stability analysis system now. Niagara Mohawk's problem (matrix size) can only use a modest (perhaps 16 node) parallel system. However, one could use large teraflop machines (10,000 nodes?) to simulate larger areas--such as the sharing of power over a national grid. HPC also has great potential for more general systems simulations (field 9). This is already evident from war games in the defense community (as implemented in the DoD distributed simulation SIMNET). More interesting is the educational opportunities of say climate models that we return to in field~17, as such simulations are of greatest interest when coupled with powerful visualization interfaces, which themselves need HPC. We also see a technology transfer from past defense to new civilian applications in the area of signal processing, vision and feature extraction (field~10). Currently, special purpose hardware---typically with built in parallelism---is used for military problems. We can expect that use of commercial parallel architectures will aid the software development process and enhance re-use. HPC in acoustic beam forming for submarine detection should allow adaptive on-line signal processing to maximize signal to noise ratio dynamically as a function of angle and time. Currently, the INTEL iWarp is being used although SIMD architectures would be effective in this and most low level signal processing problems. A SIMD initial processor would be augmented with a MIMD machine to do the higher level vision functions. Currently, JointStars uses a VAX for the final tracking stage of their airborne synthetic aperture radar system. This was used very successfully in the gulf war. However, HPC could enhance the performance of JointStars and allow it to track many moving targets---one may remember the difficulties in following the movement of Skud launchers in the gulf war. We already know good MIMD algorithms for multi-target tracking \cite{Gottschalk:88a}, \cite{Gottschalk:90b}. We can expect DoD to reduce the purchases of new planes, tanks and ships over the next decade. However, we see a significant opportunity to integrate new HPC systems into existing systems at all levels of defense. This includes both avionics and mission control in existing aircraft, and the hierarchy of control centers within the armed services. HPC can be used both in the delivered systems and perhaps even more importantly in the simulation of their performance. Civilian applications include the support of HDTV and a suite of applications built around GIS (Geographic Information Systems). There are currently substantial projects devoted to feature extraction from maps so that we can recover the information (roads, rivers, location of power and telephone lines, etc.) present in maps. Usually this information only exists in (paper) map form as the original data that produced the map has been ``lost''. However, computers and GIS Systems need the basic information from a map was made---not the complicated overlays that make up a map. We have found, not surprisingly, that the dominant use of computers in industry involves the storing and processing of data (fields 11,12). Although this is currently not the major focus of high performance computing, as stressed in the introduction, we believe that information processing is the most promising application for parallel computer vendors. We need the coupling of parallel databases such as the new ORACLE~7.0 with front ends that perform sophisticated numerical and symbolic processing. This will often require the development of new ways of ``doing business''. For instance, the MONY Insurance Company spends \$70M a year on data processing---largely on COBOL applications where they have some 15 million lines of code and a multi-year backlog. They see no immediate need for HPC, but surely a more productive software environment would be a great value! Similarly, Empire Blue Cross/Shield (field 11) processes 6.5 million medical insurance transactions every day. Their IBM 3090-400 handles this even with automatic optical scanning of all documents. HPC could only be relevant if one could develop a new approach with perhaps HPC examining the database with an expert system or neural network to identify anomalous situations. The states and federal government are burdened by the major cost of Medicaid and small improvements would have great value. Consumer Reports Magazine estimates that the U.S. health care system ``wastes'' \$200B each year. We need to use HPC to examine the mounds of paperwork generated by regulations to find pointers to both efficient and inefficient aspects of the system. We have identified the health care industry as a particularly promising area for HPC. Other interesting applications in this field include planning of cancer therapy (field~7) where we told of a major \$40M facility whose use was limited by the time taken to optimize the therapy with the tradeoff between destruction to the tumor and incidental damage to healthy tissue. Also important is the use of HPC image processing and database systems for images generated via different medical instruments (field~10,12). Virtual reality is very promising in medical rehabilitation with pioneering work at Loma~Linda Medical Center in Southern California (field~17). Otis is another example of information processing discussed generally in Section~\ref{sec.back}. They are interested in setting up a database of elevator monitoring data which can be analyzed for indicators of future equipment problems. This would lead to improved reliability---an area where Otis and Japanese companies compete. In this way, HPC can lead to competitive advantage in the ``global economic war''. The major computing problem for Wall Street (fields 11,15) is currently centered on the large databases. SIAC runs the day-to-day operation of the New York and American Stock exchanges. Two acres (about 300) of Tandem computers handle the calls from brokers to traders on the floor. The traders already use an ``embarrassingly parallel'' decomposition with some 2000 stocks of the New York Stock Exchange decomposed over about five hundred personal computers with about one PC per trader. For SIAC, the major problem is reliability and network management with essentially no down time ``allowed''. HPC could perhaps be used as part of a heterogeneous network management system to simulate potential bottlenecks and strategies to deal with faults. The brokerages already use parallel computers for economic modelling \cite{Mills:92a}, \cite{Mills:92b}, \cite{Zenios:91b}. This is obviously glamorous with integration of sophisticated optimization methods very promising (field~15). As our final example (field 17), we have the entertainment and education industries. Here HPC is linked to areas such as multimedia and virtual reality with high bandwidth and sophisticated visualization and delivery systems. Some applications can be viewed as the civilian versions of military flight simulators with HPC systems replacing the special purpose hardware now used. Parallelism will appear in the low end with future extensions of Nintendo like systems; at a medium scale for computer generated stages in a future theater; at the high end with parallel supercomputers controlling simulations in tomorrow's theme parks. HPC should also appear in all major sports stadiums to perform image analysis as a training aid for coaches or providing new views for cable TV audiences. We can imagine sensors and tracking systems developed for Strategic Defense Initiative being adapted to track players' on a football field rather than a missile launch from an unfriendly country. Many might consider this appropriate with American football considered as aggressive as many military battles! In this brief article, we can only highlight some results from our survey. We are currently writing a longer version of this report and, further, working on some applications we have isolated as promising near term opportunities. These include the pre-college education simulation, as well as the Power Utility and Wall Street applications. \bigskip \noindent{\bf Acknowledgments} \medskip This work was supported by the State of New York and the Center for Research on Parallel Computation with National Science Foundation Cooperation Agreement No. CCR-8809165---the Government has certain rights to this material. \bigskip \begin{thebibliography}{9} \bibitem{Carroll:92a} Carroll,~P. (1992), ``Industrial applications for parallel computing'', {\em Numerical Aerodynamic Simulation Program Newsletter}, {\bf 7}, 3, April 1992. \bibitem{Battelle} ``A Framework for Technology-Based Development in New York State'', a report prepared for the New York State Science and Technology Foundation by Battelle. \bibitem{Agencies} ``The Federal High Performance Computing Program'', produced by a collaboration between four federal agencies: Defense Advanced Research Projects Agency (DARPA), Department of Energy (DOE), National Aeronautical and Space Agency (NASA) and the National Science Foundation (NSF). This report was produced in 1991 and updated in 1992. \bibitem{DofC} ``Emerging Technologies: A Survey of Technical and Economic Opportunities'', produced by the U.S. Department of Commerce. \bibitem{Fox:91d} Fox,~G.~C. (1991), ``FortranD as a portable software system for parallel computings'', in {\em Proceedings of Supercomputing USA/Pacific 91}, held in Santa Clara, California. Syracuse University Technical Report SCCS-91, June 1991. \bibitem{Fox:91g} Fox,~G.C. (1991), ``The architecture of problems and portable parallel software systems''. Syracuse University Technical Report SCCS-134, July 1991. \bibitem{Fox:91n} Fox,~G.C. (1992), ``Lessons from massively parallel architectures on message passing computers'' in {\em The 37th Annual IEEE International Computer Conference, COMPCON '92}, IEEE Computer Society Press. Syracuse University Technical Report SCCS-214, December 1991. \bibitem{Choudhary:92d} Choudhary,~A., Fox,~G.~C., Hiranandani,~S., Kennedy,~K., Koelbel,~C., Ranka,~S., and Saltz,~J. (1992), ``A classification of irregular loosely synchronous problems and their support in scalable parallel software systems'', in {\em Proceedings from Darpa Software Technology Conference}, 1992, pp.~138--149. Syracuse University Technical Report SCCS-255. \bibitem{Fox:91j} Fox, G.~C. (1991), ``Physical computation'' {\em Concurrency: Practice and Experience}, {\bf 3}(6):627--653, Special Issue:\ Practical Parallel Computing: Status and Prospects. Syracuse University Technical Report SCCS-2b, December 1991. \bibitem{Gottschalk:88a} Gottschalk,~T.~D. (1988), ``Concurrent multiple target tracking'', in {\em The Third Conference on Hypercube Concurrent Computers and Applications, Volume 2}, ACM Press, New York, 1988, pp.~1247--1268. California Institute of Technology Technical Report C3P-567. \bibitem{Gottschalk:90b} Gottschalk,~T.~D. (1990), ``Concurrent multi-target tracking'', in {\em The Fifth Distributed Memory Computing Conference, Volume~I}, D. W. Walker and Q. F. Stout, editors, IEEE Computer Society Press, 1990, pp~85--88. California Institute of Technology Technical Report C3P-908. \bibitem{Mills:92a} Mills,~K., Vinson,~M., Cheng,~G., and Finucane,~T. (1992), ``A large scale comparison of option pricing models with historical market data'', Syracuse University Technical Report SCCS-260 (March, 1992). \bibitem{Mills:92b} Mills,~K., Cheng,~G., Vinson,~M., Ranka,~S., and Fox,~G.~C. (1992), ``Software issues and performance of a stock option pricing model on the Connection Machine-2 and DEC MPP-12000'', in Proceedings of the Fifth Australian Supercomputing Conference, September 23, 1992. Syracuse University Technical Report SCCS-273b. \bibitem{Zenios:91b} Zenios,~S. (1991), ``Massively Parallel Computations for Financial Planning under Uncertainty'', Chapter 18 in {\em Very Large Scale Computation in the 21st Century}, edited by Jill P. Mesirov, Philadelphia: SIAM. \end{thebibliography} \end{document}