Manages asynchronous GPU-to-CPU copies using CUDA streams

Uses a dedicated CUDA stream for device-to-host (D2H) transfers, allowing GPU computation to overlap with data movement.

Value

An AsyncCopyManager object.

Methods

reset_bytes_counter()

Reset the bytes written counter

clear()

Clear pending copies without completing them

Methods

Public methods

• AsyncCopyManager$new()

• AsyncCopyManager$submit_copy()

• AsyncCopyManager$drain_all()

• AsyncCopyManager$get_bytes_written()

• AsyncCopyManager$reset_bytes_counter()

• AsyncCopyManager$clear()

• AsyncCopyManager$clone()

---

AsyncCopyManager$new()

Usage

AsyncCopyManager$new(device, max_pending = 4L, buffer_pool = NULL)

Arguments

device

torch.device. CUDA device.

max_pending

Integer. Max pending copies.

buffer_pool

PinnedBufferPool or NULL.

---

AsyncCopyManager$submit_copy()

Usage

AsyncCopyManager$submit_copy(gpu_tensor, target, indices)

Arguments

gpu_tensor

Tensor. Source tensor on GPU.

target

Tensor or DiskTensor. Target storage.

indices

List. Indices where to write in target.

---

AsyncCopyManager$drain_all()

Usage

AsyncCopyManager$drain_all()

Returns

Numeric. Total bytes written in MB.

---

AsyncCopyManager$get_bytes_written()

Usage

AsyncCopyManager$get_bytes_written()

Returns

Numeric.

---

AsyncCopyManager$reset_bytes_counter()

Usage

AsyncCopyManager$reset_bytes_counter()

Returns

NULL

---

AsyncCopyManager$clear()

Usage

AsyncCopyManager$clear()

Returns

NULL

---

AsyncCopyManager$clone()

The objects of this class are cloneable with this method.

Usage

AsyncCopyManager$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.