Definition
Plain language
A simple kind of two-way multiplication that depends on both inputs together.
As stated in the literature
A function linear in each of two arguments separately; in attention, queries and keys interact bilinearly via dot product, capturing pairwise structure that linear features alone cannot represent.
Why it matters: Bilinear interactions let models capture relationships between pairs of items, which a purely linear or additive combination of features can't express.
For example, in attention, the score between a query and a key is the dot product of two learned vectors — a bilinear function of the query and key.