NanoTabICL v2 Model

A compact tabular in-context learning model with column, row, and ICL attention blocks.

NanoTabICLv2(
  max_classes,
  out_dim,
  embed_dim = 128L,
  col_n_block = 3L,
  row_n_block = 3L,
  icl_n_block = 12L,
  col_n_head = 8L,
  row_n_head = 8L,
  icl_n_head = 8L,
  feature_group_size = 3L,
  col_n_cls = 4L,
  row_n_cls = 128L
)

Arguments

max_classes

Maximum number of classes (0 for regression)

out_dim

Output dimension (n_classes for classification, n_quantiles for regression)

embed_dim

Embedding dimension for features

col_n_block

Number of column transformer blocks

row_n_block

Number of row transformer blocks

icl_n_block

Number of ICL transformer blocks

col_n_head

Number of attention heads for column blocks

row_n_head

Number of attention heads for row blocks

icl_n_head

Number of attention heads for ICL blocks

feature_group_size

Size of feature groups for repeated grouping

col_n_cls

Number of CLS tokens per column

row_n_cls

Number of inducing vectors for column attention

Value

An nn_module ready for training/inference