In Chapter 1, we discussed an abstract CPU/GPU architecture with a CPU and CPU memory, a GPU and GPU memory, and a data path that connects the two.
This chapter will give more detailed descriptions of CUDA platforms, from the system level to the functional units within the GPUs.
Section 2.1 builds on Chapter 1’s abstract CPU/GPU architecture with a discussion of the many different ways that CUDA systems may be built, from single-GPU workstations to multi-GPU servers.
Section 2.2 examines integrated GPUs, which share a memory pool with the CPU, and follows that arrangement from the early chipset GPUs through today’s Arm-based superchips.
Section 2.3 considers systems with more than one GPU, from a handful in a workstation to the rack-scale machines of the data center.
Section 2.4 discusses address spaces and how CUDA’s memory model is implemented in hardware and software.
Section 2.5 discusses CPU/GPU interactions, with special attention paid to how commands are submitted to the GPU and how CPU/GPU synchronization is performed.
Section 2.6 gives a high-level description of the GPUs themselves: functional units such as Copy Engines and Streaming Multiprocessors, with block diagrams of the different types of Streaming Multiprocessor.
Finally, Section 2.7 surveys the interconnects that move data between GPUs and out across a cluster—PCI Express, NVLink, and the RDMA-capable network adapters that scale CUDA beyond a single server.