A compact tabular in-context learning model with column, row, and ICL attention blocks.
Usage
NanoTabICLv2(
max_classes,
out_dim,
embed_dim = 128L,
col_n_block = 3L,
row_n_block = 3L,
icl_n_block = 12L,
col_n_head = 8L,
row_n_head = 8L,
icl_n_head = 8L,
feature_group_size = 3L,
col_n_cls = 4L,
row_n_cls = 128L
)Arguments
- max_classes
Maximum number of classes (0 for regression)
- out_dim
Output dimension (n_classes for classification, n_quantiles for regression)
- embed_dim
Embedding dimension for features
- col_n_block
Number of column transformer blocks
- row_n_block
Number of row transformer blocks
- icl_n_block
Number of ICL transformer blocks
- col_n_head
Number of attention heads for column blocks
- row_n_head
Number of attention heads for row blocks
- icl_n_head
Number of attention heads for ICL blocks
- feature_group_size
Size of feature groups for repeated grouping
- col_n_cls
Number of CLS tokens per column
- row_n_cls
Number of inducing vectors for column attention