Define B/D/F upfront in the roofline matmul section - #127
Conversation
…ne matmul section In the "Matrix multiplication" section the activation X has shape [B, D], so B is the number of tokens (the rows of X) -- i.e. batch_size x sequence_length, the same quantity written B*T in the Transformer chapters. Labeling this B as the "batch size" is easy to confuse with the usual batch size (a count of *sequences*) and with the book's own use of B for the sequence count elsewhere. - Define B, D, F explicitly where the matmul is introduced. - Refer to B as the "number of tokens" consistently instead of "batch size". - Keep one contrasting note that this is not the usual per-sequence batch size. No math or numbers change; wording/clarity only.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
Signed-off-by: Future-Outlier <eric901201@gmail.com>
c387f74 to
6ae6a90
Compare
|
cc @jacobaustin123 to take a look, tks! |
Future-Outlier
changed the title
Future-Outlier
changed the title
Future-Outlier
changed the title
JiangJiaWei1103
left a comment
JiangJiaWei1103
left a comment
There was a problem hiding this comment.
Overall LGTM. Defining the notations at the top makes it easy to follow!
thanks for the review |
|
Thanks for sending this. I'll try to find a way to tweak this section. I'm not a fan of emphasizing tokens since this section isn't even about Transformers, it's just about matrix multiplication as an abstract algorithm. |
Thank you! |
3 participants
Summary
Define
B,D,Fonce at the top of the Matrix multiplicationsection — one short sentence each — so the reader knows each dimension
right away:
Bis the batch size — the number of tokens (not sequences).Dis the model dimensiond_model(the embedding size).Fis the MLP hidden dimensiond_ff.The key point is that the batch size
Bcounts tokens, not sequences.Since it's now defined once upfront, the rest of the section keeps the
familiar term "batch size
B" and drops the repeated "(not sequences)"asides — which reads more cleanly than renaming
Bto "number of tokens"everywhere.