You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This PR gives us a way to handle duplicated cost between lookups that
share the destination selector. It applies the framework to some lookups
and makes `log_derivative_inverse_commitments_round` ~30% faster.
## The Problem
Given a lookup instance, the inverses are computed (and then committed
to) for any row which has either the source or destination selector on.
This means that if you have 2 lookups (or 100+ in case of sha, etc)
into, say, `bitwise.sel`, and you add 1 row to it because you did one
AND in your execution, you will pay for 2 (or 100+) and not just 1 new
inverse.
This can be mitigated by adding new selectors like `bitwise.sel_sha`
but... ah, we don't want to have to provide that information in
simulation! In this PR I use the same idea I used in multipermutations,
to do this almost transparently.
## The solution
I introduce the concept of a "coarse" (or global selector) like
`bitwise.sel` and a "fine grained" selector like `bitwise.sel_sha256`.
You add the new selectors columns in PIL, e.g., `bitwise.sel_sha256`.
You use that selector in your lookup in PIL. And in tracegen you specify
the coarse selector in the lookup definition, e.g.,
`.add<lookup_sha256_w_s_0_xor_0_settings,
InteractionType::LookupGeneric>(Column::bitwise_sel)`.
That's it in terms of usage.
## The implementation
Lookup builders
* Dummy inverses are not added anymore. It was previously only done to
force the correct size, so that later when computing the polynomials we
would not need to think about this. See later how this is solved.
* I added the concept of "outer" selector (the coarse grained one). By
default, it is set the same as the fine grained one (which is taken from
the settings). There is some new code where we say "ok, if I'm
incrementing a count and it's the first time, set the fine grained
selector as well". This is what allows us to avoid passing information
in simulation.
Computation of logderivative inverses: if you are familiar with this,
you know that as opposed to the lookup builders, the inverses are
computed _after_ wire commitment, because the challenges depend on that,
and the inverses depend on the challenges. This didn't change. What
changes is that now there is a call to resize the inverse polynomials to
the right size, at this point. It's actually the right thing to do, the
tracegen option was a bit of a hack.
## Results
These are the results with just a few changes. I think we still have a
lot of room for improvement by just adding a few columns. Especially for
precomputed tables. I might do a few more in subsequent PRs.
```
Incoming lookups* (a bit outdated)
117 bitwise_start
54 range_check_sel
51 gt_sel
28 public_inputs_sel
25 poseidon2_hash_end
24 memory_sel
11 precomputed_sel_range_8
9 precomputed_sel_range_16
8 merkle_check_start
...
```
Before
```
Sum of all column rows: 31751512
------- STATS -------
prove/log_derivative_inverse_commitments_round_ms: 2666 <--
prove/log_derivative_inverse_round_ms: 379
prove/pcs_rounds_ms: 3278
prove/public_inputs_round_ms: 0
prove/sumcheck_ms: 10427
prove/wire_commitments_round_ms: 3322
proving/all_ms: 21626
```
After
```
Sum of all column rows: 22541631
------- STATS -------
prove/log_derivative_inverse_commitments_round_ms: 1826 <--
prove/log_derivative_inverse_round_ms: 376
prove/pcs_rounds_ms: 3145
prove/public_inputs_round_ms: 0
prove/sumcheck_ms: 9902
prove/wire_commitments_round_ms: 3330
proving/all_ms: 20217
```
*`% grep -r "DST_SELECTOR = Column::" . 2>/dev/null | sed -n
's/.*DST_SELECTOR = Column::\([^;]*\);.*/\1/p' | sort | uniq -c | sort
-nr`
0 commit comments