The CUDA Handbook ▸ Chapter 6. Streams, Events, and Graphs

6.4 CUDA Events: Timing

CUDA events work by submitting a command to the GPU that, when the preceding commands have been completed, causes the GPU to write a 32-bit memory location with a known value. The CUDA driver implements cuEventQuery() and cuEventSynchronize() by examining that 32-bit value. But besides the 32-bit “tracking” value, the GPU also can write a 64-bit timer value that is sourced from a high-resolution, GPU-based clock.

Because they use a GPU-based clock, timing using CUDA events is less subject to perturbations from system events such as page faults or interrupts; and the function to compute elapsed times from timestamps is portable across all operating systems. That said, the so-called “wall clock” times of operations are ultimately what users see; so CUDA events are best used in a targeted fashion to tune kernels or other GPU-intensive operations, not to report absolute times to the user.

When using CUDA events for timing, it is best to record them in the NULL stream. The rationale is similar to the reason the machine instructions in superscalar CPUs to read time stamp counters (e.g., RDTSC on x86) are serializing instructions that flush the pipeline: Forcing a “join” on all the GPU engines eliminates any possible ambiguity on the operations being timed⁵.

Just make sure the cu(da)EventRecord() calls bracket enough work that the timing delivers meaningful results.

Finally, note that CUDA events are intended to time GPU operations; any synchronous CUDA operations will result in the GPU being used to time the resulting CPU/GPU synchronization operations.

    cuda(EventRecord( startEvent, NULL ) );
    // synchronous memcpy – invalidates CUDA event timing
  cuda(Memcpy( deviceIn, hostIn, N*sizeof(int) ) );
    cuda(EventRecord( stopEvent, NULL ) );

The example explored in the next section will illustrate how to use CUDA events for timing.