David's raw ML reference notes

00 Constants used in analysis of The Annotated Transformer

1 min read

  • The batch size
  • The number of attention heads
  • The length of the sequence
  • The dimension of the inputs and outputs to each block in the model
  • The context vector length for each attention head

Graph View

Backlinks