Concatenate multiple KVCache objects along a dimension.
Usage
kv_cache_concat(caches, dim = 1)
Arguments
- caches
List of KVCache instances. All must have the same
layer indices.
- dim
Integer. Dimension to concatenate along (batch dimension).
Default is 1 (R 1-based; corresponds to Python dim 0).
Value
A new KVCache with concatenated entries at each layer index.