-
Notifications
You must be signed in to change notification settings - Fork 1.9k
fix(cubejs-server-core): fix schema caching in SQL API with DAP on #9204
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
5b05694 to
bc2ca42
Compare
| }, | ||
| })); | ||
| const visibiliyMask = JSON.stringify(isMemberVisibleInContext, Object.keys(isMemberVisibleInContext).sort()); | ||
| const visibilityMaskHash = crypto.createHash('md5').update(visibiliyMask).digest('hex').slice(0, 8); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's basically zero reason to use MD5 today for any application. I get that there's no performant siphash built in Node.js, but why not at least SHA256? Would probably be faster as well, here's openssl speed output on my laptop:
> openssl speed md5 sha256
...
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
md5 140935.16k 353522.47k 701603.62k 932349.94k 1027145.29k 1032898.87k
sha256 193046.72k 585457.47k 1373412.94k 2059078.04k 2404984.89k 2433511.68k
In any case: can you add comment about why this hashing is here, and what are the expectations from hash function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
md5 is already used in other places in the same file. For this use-case crypto strength isn't important so, any hash would do. Can switch it to sha256
| // Mix hash into the first 4 bytes and the last 4 bytes in a way that doesn't change the UUID version | ||
| for (let i = 0; i < 4; i++) { | ||
| // eslint-disable-next-line no-bitwise | ||
| uuidBytes[i] ^= hashBytes[i]; // First 4 bytes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's not use XOR mixing, and not do this patching manually, please.
- One can provide 16 random bytes to
uuid.v4withrandomoption. See link - If we want to derive new pseudorandom value deterministically, simplest and safest way would be to just concatenate and hash them again (because we know/expect the length of parts to be fixed):
crypto.createHash('sha256').update(uuidBytes).update(hashBytes).digest()
So, why not just uuidv4({random: c.createHash('sha256').update(uuidBytes).update(hashBytes).digest().slice(0,16)})?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The native side expects compilerId to be a valid uuid (it tries to parse it)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also important is that if DAP isn't applied, compilerId stays the same. This is to make sure we don't accidentally introduce any bug in non-DAP use-cases
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will remain valid UUID, that's exactly what uuid.v4 is doing with incoming random bytes. Here. That's what I meant by "not do this patching manually" - keep UUID-related stuff in uuid package.
And compilerId will remain same, because metaConfig will not mix in visibilityMaskHash when it's null.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, ok. Now I see your point
updated the code
| }, | ||
| })); | ||
| const visibiliyMask = JSON.stringify(isMemberVisibleInContext, Object.keys(isMemberVisibleInContext).sort()); | ||
| const visibilityMaskHash = crypto.createHash('md5').update(visibiliyMask).digest('hex').slice(0, 8); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why slicing just 4 first bytes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
to make it shorter
bc2ca42 to
290c122
Compare
mcheshkov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I forgot to click "Submit review" yesterday 😞
| // Mix hash into the first 4 bytes and the last 4 bytes in a way that doesn't change the UUID version | ||
| for (let i = 0; i < 4; i++) { | ||
| // eslint-disable-next-line no-bitwise | ||
| uuidBytes[i] ^= hashBytes[i]; // First 4 bytes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will remain valid UUID, that's exactly what uuid.v4 is doing with incoming random bytes. Here. That's what I meant by "not do this patching manually" - keep UUID-related stuff in uuid package.
And compilerId will remain same, because metaConfig will not mix in visibilityMaskHash when it's null.
74d9837 to
2877226
Compare
2877226 to
20f11f2
Compare
Check List
Issue Reference this PR resolves
[For example #12]
Description of Changes Made (if issue reference is not provided)
[Description goes here]