Architecture

Reviewer: Peter C. Patton

Clearly, the future of high-performance computing lies in building highly parallel computers out of high-volume, low-cost components rather than systems of limited multiplicity using low-volume, high-cost components. This paper is an excellent review of scalable parallel processing (SPP)—past, present, and future—by six of its leading practitioners. They review the history of SPP architectures and their applications in science, and then turn to the limits on SPP performance based on looming constraints on growth in processor, memory, and software performance. If we define the goal of computer architecture as increasing performance by an order of magnitude using system organization features alone, the promise of architecture is still considerable. The architect must exploit improvements in component technology yet design around performance limits. Our experience in applying large commercial parallel systems in online transaction processing with multitier client/server applications indicates that, today, operating system limits to scalability obtain before hardware limits do. The basic issue is preemptive multithreading. For example, the operating system may saturate at 1000 or so threads, and some experiments with Java indicate saturation at 110 to 120 threads per Java Virtual Machine. The authors point out in a sidebar that the Tera MTA-1 hardware offers a solution to this problem. Fortunately, there is at least one software solution as well, namely the CHoPP asynchronous non-preemptive tasking system (ANTs). The trick to gaining parallel performance is hiding latency, but if task switching requires thousands of instruction cycles it eventually becomes self-defeating. Techniques like Tera's hardware to reduce task switching to a couple of instruction cycles or ANTs software to reduce it to 10 to 20 instruction cycles on an Intel or RISC processor may prove to be significant as overall hardware and software architectural techniques.