Devices correspond to physical GPUs. When CUDA is initialized (either
by explicitly calling the driver API’s cuInit() function or implicitly
by calling a CUDA runtime function), the CUDA driver enumerates the
available devices and creates a global data structure that contains
their names and immutable capabilities such as the amount of device
memory and maximum clock rate.
For some platforms, NVIDIA includes a tool that can set policies with
respect to specific devices. The nvidia-smi tool sets the policy with
respect to a given GPU. For example, nvidia-smi can be used to enable
and disable ECC (error correction) on a given GPU. nvidia-smi also can
be used to control the number of CUDA contexts that can be created on a
given device. The possible modes are as follows:
Default: Multiple CUDA contexts may be created on the device.
“Exclusive” mode: One CUDA context may be created on the device.
“Prohibited”: No CUDA context may be created on the device.
If a device is enumerated but you are not able to create a context on that device, it is likely the device is in “prohibited” mode, or it is in “exclusive” mode and another CUDA context already has been created on that device.
The application can discover how many CUDA devices are available by
calling cuDeviceGetCount() or cudaGetDeviceCount(). Devices can then be
referenced by an index in the range [0..DeviceCount-1].
The driver API requires applications to call cuDeviceGet() to map the
device index to a device handle (CUdevice).
Driver API applications can query the name of a device by calling
cuDeviceGetName(), and can query the amount of global memory by calling
cuDeviceTotalMem(). The major and minor compute capabilities of the
device (i.e., the SM version, such as 2.0 for the first Fermi-capable
GPUs) can be queried by calling cuDeviceComputeCapability().
CUDA runtime applications can call cudaGetDeviceProperties(), which
will pass back a structure containing the name and properties of the
device. Table 3-2 gives the descriptions of the members of
cudaDeviceProp, the structure passed back by
cudaGetDeviceProperties().
The driver API’s function for querying device attributes,
cuDeviceGetAttribute(), can pass back one attribute at a time, depending
on the CUdevice_attribute parameter. The CUDA runtime provides the same
function in the form of cudaDeviceGetAttribute(), presumably because the
structure-based interface was too cumbersome to run on the device.
cudaDeviceProp member |
Description |
|---|---|
char name[256]; |
ASCII string identifying device |
cudaUUID_t uuid; |
16-byte unique identifier. |
char luid[8]; |
8-byte locally unique identifier. Value is undefined on TCC and non-Windows platforms. |
unsigned int luidDeviceNodeMask; |
LUID device node mask. Value is undefined on TCC and non-Windows platforms. |
size_t totalGlobalMem; |
Global memory available on device in bytes |
size_t sharedMemPerBlock; |
Shared memory available per block in bytes |
int regsPerBlock; |
32-bit registers available per block |
int warpSize; |
Warp size in threads |
size_t memPitch; |
Maximum pitch in bytes allowed by memory copies |
int maxThreadsPerBlock; |
Maximum number of threads per block |
int maxThreadsDim[3]; |
Maximum size of each dimension of a block |
int maxGridSize[3]; |
Maximum size of each dimension of a grid |
int clockRate; |
Clock frequency in kilohertz |
size_t totalConstMem; |
Constant memory available on device in bytes |
int major; |
Major compute capability |
int minor; |
Minor compute capability |
size_t textureAlignment; |
Alignment requirement for textures |
size_t texturePitchAlignment; |
Pitch alignment requirement for a texture object over pitched memory |
int deviceOverlap; |
Device can concurrently copy memory and execute a kernel. Deprecated. Use instead asyncEngineCount. |
int multiProcessorCount; |
Number of multiprocessors on device |
int kernelExecTimeoutEnabled; |
Specifies whether there is a runtime limit on kernels |
int integrated; |
Device is integrated as opposed to discrete |
int canMapHostMemory; |
Device can map host memory with cudaHostAlloc/cudaHostGetDevicePointer |
int computeMode; |
Compute mode (See ::cudaComputeMode) |
int maxTexture1D; |
Maximum 1D texture size |
int maxTexture1DMipmap; |
Maximum 1D mipmapped texture size |
int maxTexture1DLinear; |
Maximum size for 1D textures bound to linear memory |
int maxTexture2D[2]; |
Maximum 2D texture dimensions |
int maxTexture2DMipmap[2]; |
Maximum 2D mipmapped texture dimensions |
int maxTexture2DLinear[3]; |
Maximum dimensions (width, height, pitch) of 2D textures bound to pitch-linear memory |
int maxTexture2DGather[2]; |
Maximum 2D texture dimensions if texture gather operations have to be performed |
int maxTexture3D[3]; |
Maximum 3D texture dimensions |
int maxTexture3DAlt[3]; |
Maximum alternate 3D texture dimensions. |
int maxTextureCubemap; |
Maximum Cubemap texture dimensions |
int maxTexture1DLayered[2]; |
Maximum 1D layered texture dimensions |
int maxTexture2DLayered[3]; |
Maximum 2D layered texture dimensions |
int maxTextureCubemapLayered[2]; |
Maximum Cubemap layered texture dimensions |
int maxSurface1D; |
Maximum 1D surface size |
int maxSurface2D[2]; |
Maximum 2D surface dimensions |
int maxSurface3D[3]; |
Maximum 3D surface dimensions |
int maxSurface1DLayered[2]; |
Maximum 1D layered surface dimensions |
int maxSurface2DLayered[3]; |
Maximum 2D layered surface dimensions |
int maxSurfaceCubemap; |
Maximum Cubemap surface dimensions |
int maxSurfaceCubemapLayered[2]; |
Maximum Cubemap layered surface dimensions |
size_t surfaceAlignment; |
Alignment requirements for surfaces |
int concurrentKernels; |
Device can possibly execute multiple kernels concurrently |
int ECCEnabled; |
Device has ECC support enabled |
int pciBusID; |
PCI bus ID of the device |
int pciDeviceID; |
PCI device ID of the device |
int pciDomainID; |
PCI domain ID of the device |
int tccDriver; |
1 if device is a Tesla device using TCC driver |
int asyncEngineCount; |
Number of asynchronous engines |
int unifiedAddressing; |
Device shares a unified address space with the host |
int memoryClockRate; |
Peak memory clock frequency in kilohertz |
int memoryBusWidth; |
Global memory bus width in bits |
int l2CacheSize; |
Size of L2 cache in bytes |
int persistingL2CacheMaxSize; |
Device’s maximum l2 persisting lines capacity setting in bytes. |
int maxThreadsPerMultiProcessor; |
Maximum resident threads per multiprocessor |
int streamPrioritiesSupported; |
Device supports stream priorities. |
int globalL1CacheSupported; |
Device supports caching globals in L1. |
int localL1CacheSupported; |
Device supports caching locals in L1. |
size_t sharedMemPerMultiprocessor; |
Shared memory available per multiprocessor in bytes. |
int regsPerMultiprocessor; |
32-bit registers available per multiprocessor. |
int managedMemory; |
Device supports allocating managed memory on this system. |
int isMultiGpuBoard; |
Device is on a multi-GPU board. |
int multiGpuBoardGroupID; |
Unique identifier for a group of devices on the same multi-GPU board. |
int hostNativeAtomicSupported; |
Link between the device and the host supports native atomic operations. |
int singleToDoublePrecisionPerfRatio; |
Deprecated. Ratio of single precision performance (in floating-point operations per second) to double precision performance. |
int pageableMemoryAccess; |
Device supports coherently accessing pageable memory without calling
cudaHostRegister() on it. |
int concurrentManagedAccess; |
Device can coherently access managed memory concurrently with the CPU. |
int computePreemptionSupported; |
Device supports Compute Preemption. |
int canUseHostPointerForRegisteredMem; |
Device can access host registered memory at the same virtual address as the CPU. |
int cooperativeLaunch; |
Device supports launching cooperative kernels via
cudaLaunchCooperativeKernel(). |
int cooperativeMultiDeviceLaunch; |
Deprecated. cudaLaunchCooperativeKernelMultiDevice() is
deprecated |
size_t sharedMemPerBlockOptin; |
Per device maximum shared memory per block usable by special opt in. |
int pageableMemoryAccessUsesHostPageTables; |
Device accesses pageable memory via the host’s page tables. |
int directManagedMemAccessFromHost; |
Host can directly access managed memory on the device without migration |
int maxBlocksPerMultiProcessor; |
Maximum number of resident blocks per multiprocessor. |
int accessPolicyMaxWindowSize; |
The maximum value of cudaAccessPolicyWindownum_bytes |
size_t reservedSharedMemPerBlock; |
Shared memory reserved by CUDA driver per block in bytes. |
int hostRegisterSupported; |
Device supports host memory registration via cudaHostRegister() |
int sparseCudaArraySupported; |
1 if the device supports sparse CUDA arrays and sparse CUDA mipmapped arrays, 0 otherwise. |
int hostRegisterReadOnlySupported; |
Device supports using the cudaHostRegister() flag
cudaHostRegisterReadOnly to register memory that must be mapped as
read-only to the GPU. |
int timelineSemaphoreInteropSupported; |
External timeline semaphore interop is supported on the device. |
int memoryPoolsSupported; |
1 if the device supports using the cudaMallocAsync() and cudaMemPool
family of APIs, 0 otherwise. |
int gpuDirectRDMASupported; |
1 if the device supports GPUDirect RDMA APIs, 0 otherwise. |
unsigned int gpuDirectRDMAFlushWritesOptions; |
Bitmask to be interpreted according to the cudaFlushGPUDirectRDMAWritesOptions enum. |
int gpuDirectRDMAWritesOrdering; |
See the cudaGPUDirectRDMAWritesOrdering enum for numerical values. |
unsigned int memoryPoolSupportedHandleTypes; |
Bitmask of handle types supported with mempool-based IPC. |
int deferredMappingCudaArraySupported; |
1 if the device supports deferred mapping CUDA arrays and CUDA mipmapped arrays. |
int ipcEventSupported; |
Device supports IPC Events |
int clusterLaunch; |
Indicates device supports cluster launch. |
int unifiedFunctionPointers; |
Indicates device supports unified pointers. |
Table 3-2. cudaDeviceProp members.