Temporarily enable or disable Flash Attention 3

This function provides a simple save/restore pattern to control whether Flash Attention 3 is used during attention computation.

Temporarily enable or disable Flash Attention 3 during inference.

Usage

flash_attn3_toggle(enabled)

flash_attn3_toggle(enabled)

Arguments

enabled

Logical. Whether to enable FA3.

Value

A function that, when called, restores the previous setting.

Function. Restoration function to call when done.

Examples

if (FALSE) { # \dontrun{
# Temporarily disable Flash Attention 3
restore <- flash_attn3_toggle(FALSE)
# ... run inference ...
restore()  # Restore previous setting

# Or use with on.exit() for automatic restoration
old <- flash_attn3_toggle(FALSE)
on.exit(old(), add = TRUE)
# ... code that uses attention ...
} # }

if (FALSE) { # \dontrun{
# Temporarily disable FA3
restore <- flash_attn3_toggle(FALSE)
# ... run inference ...
restore()  # Restore previous setting
} # }