The CUDA Handbook ▸ Chapter 14. N-Body

14.8 Conclusion

Since instruction sets and architectures differ, performance is measured in body-body interactions per second rather than GFLOPS. Except for the multi-GPU scaling of Table 14-3—measured on the four-GK104 server of Section 14.6—the results in this section were all taken on a single modern desktop machine: an 8-core AMD Ryzen 7 7700X (Zen 4) paired with a GeForce RTX 3060 (Ampere).

Table 14-4 summarizes the speedups due to CPU optimizations, all measured on the 7700X.

Table 14-4. Speedups due to CPU optimizations

On this machine, the generic CPU code of Listing 14-2 performs about 0.48 billion interactions per second; the single-threaded AVX code performs 3.6 billion interactions per second, some 7.5x faster. As expected, multithreading the AVX code across the eight cores achieves a further near-linear speedup, delivering 26 billion interactions per second, about 7.2x as fast as one thread. Between porting to AVX and multithreading, the total speedup for the CPU is roughly 54x.

Even after these CPU optimizations, the GPU retains a wide lead, though the optimized CPU narrows it more than most CUDA papers would suggest⁹. The highest-performing kernel (the shared memory implementation of Listing 14-4, with a loop unroll factor of 4) reaches about 291 billion body-body interactions per second on the RTX 3060—roughly 11x the fastest CPU result, the multithreaded AVX implementation at 26 billion. The gap is narrow enough that even brute-force AOS traversal on the GPU (about 275 billion) comes within a few percent of the tuned kernel. If anything, this understates CUDA’s advantage, since the RTX 3060 is a midrange part while the 7700X is a current-generation 8-core desktop CPU.

Furthermore, future improvements can be had for both technologies: for CPUs, the AVX-512 extensions double the vector width again where they are available, and the optimized CPU implementation still does not exploit symmetry. For GPUs, larger and newer parts scale up the throughput reported here. Comparing the source code for Listings 14-1 and 14-8 (the GPU and AVX implementations of the core body-body interaction code), though, it becomes clear that performance isn’t the only reason to favor CUDA over optimizing CPU code. Dr. Vincent Natoli alluded to this tradeoff in his June 2010 article Kudos for CUDA:

Similarly, we have found in many cases that the expression of algorithmic parallelism in CUDA in fields as diverse as oil and gas, bioinformatics and finance is more elegant, compact and readable than equivalently-optimized CPU code, preserving and more clearly presenting the underlying algorithm. In a recent project we reduced 3,500 lines of highly-optimized C code to a CUDA kernel of about 800 lines. The optimized C was peppered with inline assembly, SSE macros, unrolled loops and special cases, making it difficult to read, extract algorithmic meaning and extend in the future. By comparison the CUDA code was cleaner and more readable. Ultimately it will be easier to maintain.

Although it was feasible to develop an AVX implementation of this application, with a core body-body computation that takes about 50 lines of code to express (Listing 14-8), it’s hard to imagine what the source code would look like for an AVX-optimized implementation of something like Boids, where each body must evaluate conditions and, when running on CUDA hardware, the code winds up being divergent. AVX supports divergence both in the form of predication (using masks and Boolean instruction sequences such as VANDPS/VANDNPS/VORPS to construct the result) and branching (often using VMOVMSKPS to extract evaluated conditions), but getting the theoretical speedups on such workloads would require large engineering investments unless they can be extracted automatically by a vectorizing compiler.