-
-
Notifications
You must be signed in to change notification settings - Fork 97
Open
Labels
enhancementimportantperf impactedThis issues impacts performance of duplicut (either positively of negatively)This issues impacts performance of duplicut (either positively of negatively)
Description
MEDIUM_LINE_BYTES is currently hardcorded in const.h, to a value of 8.
The hasmap & chunks chunks are then made in such way that if real medium length of lines is MEDIUM_LINE_BYTES, the hashmap will be filled by a factor defined by HMAP_LOAD_FACTOR (currently set to 0.5, for 50% hmap filling).
Therefore, we could read some random pages in the file (e.g: start/middle/end of file), and get a better guess of MEDIUM_LINE_BYTES from there.
It would greatly improve performance in wordlists with a lot of very long lines (for example, a list of md5).
Because if lines are 32bytes long, hmap will be filled 12.5% only (50%/2/2). And a lot more chunks are needed.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementimportantperf impactedThis issues impacts performance of duplicut (either positively of negatively)This issues impacts performance of duplicut (either positively of negatively)