Glossary · Term

associative scan

← all terms

Definition

A way to speed up sequence computations by combining results in parallel chunks instead of one at a time.

A parallel-prefix-sum algorithm that computes cumulative results of an associative operation in logarithmic depth, used in state-space models for parallel training.

Mentioned in 1 episode

  1. 032
    A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking