CUDA arrays are allocated from the same pool of physical memory as device memory, but have an opaque layout that is optimized for 2D and 3D locality. The graphics drivers use these layouts to hold textures; by decoupling the indexing from the addressing, the hardware can operate on 2D or 3D blocks of elements instead of 1D rows. For applications that exhibit sparse access patterns, especially patterns with dimensional locality (for example, computer vision applications), CUDA arrays are a clear win. For applications with regular access patterns, especially ones with little to no reuse or whose reuse can be explicitly managed by the application in shared memory, device pointers are the obvious choice.
Some applications, such as image processing applications, fall into a gray area where the choice between device pointers and CUDA arrays is not obvious. All other things being equal, device memory is probably preferable to CUDA arrays; but the following considerations may be used to help in the decision-making process:
CUDA arrays do not consume CUDA address space.
On WDDM drivers (Windows Vista and later), the system can automatically manage the residence of CUDA arrays.
CUDA arrays can reside only in device memory, and the GPU can convert between the two representations while transferring the data across the bus. For some applications, keeping a pitch representation in host memory and a CUDA array representation in device memory is the best approach.
A texture object sets up the texturing hardware to interpret
and sample the contents of underlying memory. It is an ordinary runtime
value – a cudaTextureObject_t, in practice a 64-bit handle
– created before a launch, passed to the kernel as an argument, and read
from device code through the tex* family of intrinsics.
Through CUDA 11, the same role was filled by compile-time texture
references: file-scope objects declared with a
texture<> template and bound to memory
before use. References were deprecated in favor of objects and then
removed outright in CUDA 12.0, so texture objects – available since SM
3.0 – are now the only interface.
A texture object is created by cudaCreateTextureObject(), which takes
two descriptors. A cudaResourceDesc names the underlying
memory and its layout – a CUDA array, one-dimensional linear device
memory, or pitched two-dimensional device memory (Table 3-5). A
cudaTextureDesc gives the sampling attributes (Table 3-6):
the filtering mode, an addressing mode per dimension, whether
coordinates are normalized to the range [0, 1), whether an
sRGB-to-linear conversion is applied on read, and the read mode that
decides whether integer texels are returned as integers or converted to
normalized floating point. Because every attribute is fixed when the
object is created, the reference era’s split between
compile-time-immutable attributes (such as dimensionality) and
runtime-mutable attributes (such as the filtering mode) is
gone: the object carries all of them, and a kernel that must read the
same memory with different attributes simply uses a second object.
cudaResourceType |
Underlying memory |
|---|---|
cudaResourceTypeLinear |
1D linear device memory |
cudaResourceTypePitch2D |
2D pitched device memory |
cudaResourceTypeArray |
CUDA array |
Table 3-5. Texture-object resource types.
cudaTextureDesc attribute |
Meaning |
|---|---|
filterMode |
point or linear filtering |
addressMode[3] |
wrap, clamp, mirror, or border, per dimension |
normalizedCoords |
coordinates in [0, 1) rather than [0, size) |
sRGB |
apply an sRGB-to-linear conversion on read |
readMode |
return integer texels as integers or as normalized floats |
Table 3-6. Texture-object sampling attributes.
The hardware textures most efficiently from CUDA arrays, but a texture object built over linear or pitched device memory is useful in its own right:
it enlists the texture cache, which serves as a bandwidth aggregator;
it lets a kernel work around coalescing restrictions; and
it avoids superfluous copies when reading from memory that is otherwise best written as device memory – a video codec, for example, may emit frames into device memory yet read them back through a texture.
Inside a kernel, the tex* intrinsics read through the
object, which they take as their first argument (Table 3-7). When the
object is no longer needed, it is released with
cudaDestroyTextureObject().
| Texture type | Intrinsic |
|---|---|
| Linear device memory | tex1Dfetch(texObj, int) |
| 1D | tex1D(texObj, float x) |
| 2D | tex2D(texObj, float x, float y) |
| 3D | tex3D(texObj, float x, float y, float z) |
| Cubemap | texCubemap(texObj, float x, float y, float z) |
| Layered (1D) | tex1DLayered(texObj, float x, int layer) |
| Layered (2D) | tex2DLayered(texObj, float x, float y, int layer) |
| Layered (Cubemap) | texCubemapLayered(texObj, float x, float y, float z, int layer) |
Table 3-7. Texture intrinsics.
NOTE: there are no coherency guarantees between texture reads and writes performed through global load/store or surface load/store. A CUDA kernel must take care not to texture from memory that is also being written by other means during the same launch.
The driver API mirrors the runtime: cuTexObjectCreate() takes a
CUDA_RESOURCE_DESC and a CUDA_TEXTURE_DESC and
returns a CUtexObject that a kernel uses exactly as it
would a cudaTextureObject_t (Table 3-8). Neither API has a
binding step, and neither keeps any file-scope texture state, so
texturing composes cleanly with the driver API’s stricter separation of
host and device code. Chapter 10 covers texturing – addressing and
filtering modes, mipmapped and cubemap textures, and the gather path –
in full.
| Operation | CUDA runtime | Driver API |
|---|---|---|
| Create | cudaCreateTextureObject() |
cuTexObjectCreate() |
| Destroy | cudaDestroyTextureObject() |
cuTexObjectDestroy() |
Table 3-8. Texture-object functions.
Surface objects let CUDA kernels read and write CUDA arrays
through the surface load/store intrinsics (surf2Dread(),
surf2Dwrite(), and their 1D and 3D kin). Like a texture
object, a surface object is a runtime handle – a
cudaSurfaceObject_t – created with
cudaCreateSurfaceObject() from a cudaResourceDesc and
passed to the kernel as an argument; the older surface
references were removed alongside texture references in CUDA 12.0.
A surface object’s primary purpose is to let a kernel write a CUDA
array: before surface load/store became available, a kernel had to write
to device memory and then perform a device-to-array memcpy to copy and
convert the output into a CUDA array.
Compared to a texture object, which can transform everything from the input coordinates to the output format depending on how it is configured, a surface object exposes a plain, bitwise interface to the contents of the CUDA array: coordinates are given in bytes, and values are read and written verbatim. CUDA never gave surface load/store the converting access that OpenCL’s image reads and writes provide – translating between a sample value and the array’s stored format – so as to stay future-proof to conversions of a more sophisticated kind, such as splatting samples into the array at fractional coordinates or interoperating with an antialiased graphics surface.