Pool of pinned CPU memory buffers for efficient GPU-to-CPU transfers

Pinned (page-locked) memory enables faster async GPU-to-CPU data transfers by allowing the GPU to directly access the CPU memory without involving the CPU. This class maintains a pool of such buffers to avoid the overhead of repeated allocation and deallocation.

Value

A PinnedBufferPool object.

Methods

`PinnedBufferPool$new()`

Usage

PinnedBufferPool$new(max_buffers_per_shape = 4L)

Arguments

max_buffers_per_shape: Integer. Maximum buffers per (shape, dtype)

`PinnedBufferPool$get()`

Usage

PinnedBufferPool$get(shape, dtype)

Arguments

shape: Integer vector. Shape of the tensor.
dtype: torch.dtype. Data type of the tensor.

Returns

A pinned CPU tensor.

`PinnedBufferPool$put()`

Usage

PinnedBufferPool$put(buf)

Arguments

buf: Tensor. Buffer to return.

`PinnedBufferPool$clear()`

Usage

PinnedBufferPool$clear()

Returns

A cleared private pool

`PinnedBufferPool$clone()`

The objects of this class are cloneable with this method.

Usage

PinnedBufferPool$clone(deep = FALSE)

Arguments

deep: Whether to make a deep clone.

Examples

if (FALSE) { # \dontrun{
pool <- pinned_buffer_pool$new(max_buffers_per_shape = 8L)
buf <- pool$get(c(100L, 128L), torch_float())
# ... use buffer ...
pool$put(buf)  # Return to pool for reuse
} # }