The CUDA Handbook ▸ Chapter 3. Software Architecture

3.11 External Memory and Graphics

A CUDA program often needs to share GPU memory with another API running on the same device—a Vulkan or Direct3D 12 renderer whose images CUDA post-processes, a hardware video decoder, a compositor. The external-resource API is the modern, API-agnostic path for sharing GPU memory in place, avoiding superfluous copies to and from host memory.¹⁴

Rather than registering a resource through API-specific calls, CUDA imports memory and synchronization objects through operating-system shareable handles—file descriptors on Linux, NT handles on Windows—the same primitives Vulkan and Direct3D 12 already use to share resources among themselves. Two properties set it apart from the older graphics-interop path: it is API-agnostic, so the CUDA context need not be bound up front to one specific graphics API, and it synchronizes CUDA with the other API entirely on the GPU, with no CPU round-trip.

A resource allocated by the other API—a Vulkan VkDeviceMemory, a Direct3D 12 heap or committed resource—is exported by that API as a handle and imported with cudaImportExternalMemory(). The import is then mapped into CUDA’s address space in one of two forms: cudaExternalMemoryGetMappedBuffer() returns a device pointer for a linear buffer, and cudaExternalMemoryGetMappedMipmappedArray() returns a CUDA mipmapped array for an image, whose channel format, extent, and level count must match the source. The underlying allocation stays owned by the exporting API; cudaDestroyExternalMemory() releases only CUDA’s mappings, not the memory.

cudaExternalMemoryHandleDesc memDesc = { };
memDesc.type = cudaExternalMemoryHandleTypeOpaqueFd;   // or OpaqueWin32, D3D12Heap, D3D12Resource...
memDesc.handle.fd = fd;                                // handle exported by Vulkan / D3D
memDesc.size = bytes;
cudaExternalMemory_t extMem;
cudaImportExternalMemory( &extMem, &memDesc );

cudaExternalMemoryBufferDesc bufDesc = { 0, bytes, 0 };   // offset, size, flags
void *devPtr;
cudaExternalMemoryGetMappedBuffer( &devPtr, extMem, &bufDesc );

Synchronization objects are imported the same way. cudaImportExternalSemaphore() takes a Vulkan binary or timeline semaphore, a Direct3D 12 fence, or a keyed mutex, and cudaSignalExternalSemaphoresAsync() and cudaWaitExternalSemaphoresAsync() then signal and wait on it in stream order—so CUDA can pass work to, and take work from, the other API without the CPU ever blocking. Timeline semaphores and fences carry a monotonically increasing value; binary semaphores simply signal and wait. Whether the device supports timeline-semaphore interop is reported by the timelineSemaphoreInteropSupported device attribute.

cudaExternalSemaphoreWaitParams w = { };   w.params.fence.value = frame;
cudaWaitExternalSemaphoresAsync( &extSem, &w, 1, stream );    // wait for the renderer
//  ... CUDA kernels operate on the shared image ...
cudaExternalSemaphoreSignalParams s = { };  s.params.fence.value = frame;
cudaSignalExternalSemaphoresAsync( &extSem, &s, 1, stream );  // hand it back

Put together, these enable a zero-copy pipeline in which, say, Vulkan renders an image and CUDA post-processes it. Vulkan and CUDA first confirm they are driving the same physical GPU by matching device UUIDs—cudaDeviceProp.uuid against Vulkan’s VkPhysicalDeviceIDProperties, since the two runtimes may enumerate devices in different orders. Vulkan then exports the image and two semaphores, and CUDA imports all three. Each frame, Vulkan signals a render-complete semaphore, CUDA waits on it, processes the image in place, signals a CUDA-complete semaphore, and Vulkan waits on that before presenting—no host copies, and no CPU synchronization anywhere in the loop.

Two constraints are easy to overlook. The importing and exporting APIs must target the same physical device, which is why the UUID (or LUID, on Windows) match matters and a device ordinal will not do. And handle ownership is asymmetric: importing an opaque file descriptor transfers ownership to CUDA, so the application must not close it afterward, whereas importing a Windows NT handle does not, so the application must close that handle itself. Because exercising any of this requires a second graphics API to produce the handles, the complete worked examples are the CUDA SDK’s simpleVulkan and vulkanImageCUDA samples and their Direct3D 12 counterparts.