Concatenate Multiple KVCacheEntry Objects

Concatenate multiple KVCacheEntry objects along a dimension.

Usage

kv_cache_entry_concat(entries, dim = 1)

Arguments

entries

List of KVCacheEntry instances. All valid entries are concatenated; invalid ones are silently skipped.

dim

Integer. Dimension to concatenate along. Default is 1 (batch dimension in R 1-based indexing; corresponds to Python dim 0).

Value

A new KVCacheEntry with concatenated key and value tensors. Returns an empty entry if no valid entries are found.

Examples

if (FALSE) { # \dontrun{
  e1 <- KVCacheEntry$new(key = torch_randn(2, 4, 10, 64),
                         value = torch_randn(2, 4, 10, 64))
  e2 <- KVCacheEntry$new(key = torch_randn(3, 4, 10, 64),
                         value = torch_randn(3, 4, 10, 64))
  e_cat <- kv_cache_entry_concat(list(e1, e2), dim = 1)
} # }