Skip to content

Conversation

@KSDaemon
Copy link
Member

@KSDaemon KSDaemon commented Nov 15, 2024

This PR brings some performance improvements to Cube:

  • Move cubestore websocket response parsing to native (rust)
    • Now Cubestore results processing in some cases may be faster by up to 50%
  • All query results are now transformed on the native side
  • All query results are now transformed asynchronously, thus offloading the main node.js event loop
  • In case of queries from SQL API, the driver's query results (from cubestore/native or js) are transparently delivered back to native invoker.

This improves the overall Cube performance, reducing the node.js event loop pressure, moving some possible computations to rust native threads.

Here are some fancy pictures.

Current master:
Screenshot 2025-01-16 в 20 57 30

Improved:
Screenshot 2025-01-16 в 20 58 09

@vercel
Copy link

vercel bot commented Nov 15, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

8 Skipped Deployments
Name Status Preview Comments Updated (UTC)
examples-angular-dashboard ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-react-d3 ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-react-dashboard ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-react-data-table ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-react-highcharts ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-react-material-ui ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-react-pivot-table ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm
examples-vue-query-builder ⬜️ Ignored (Inspect) Visit Preview Nov 29, 2024 6:48pm

@codecov
Copy link

codecov bot commented Nov 15, 2024

Codecov Report

Attention: Patch coverage is 89.77273% with 9 lines in your changes missing coverage. Please review.

Project coverage is 83.07%. Comparing base (924a17c) to head (892ca42).
Report is 10 commits behind head on master.

Files with missing lines Patch % Lines
rust/cubesql/cubesql/src/compile/engine/df/scan.rs 86.95% 9 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8961      +/-   ##
==========================================
+ Coverage   83.03%   83.07%   +0.03%     
==========================================
  Files         226      226              
  Lines       79482    79607     +125     
==========================================
+ Hits        66000    66132     +132     
+ Misses      13482    13475       -7     
Flag Coverage Δ
cubesql 83.07% <89.77%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@KSDaemon KSDaemon force-pushed the feat/cubestore-driver-rust-parsing branch from 3f3e17d to 3ef1192 Compare November 20, 2024 17:12
@KSDaemon KSDaemon force-pushed the feat/cubestore-driver-rust-parsing branch 5 times, most recently from 9c18870 to bec29e9 Compare November 29, 2024 18:04
@KSDaemon KSDaemon force-pushed the feat/cubestore-driver-rust-parsing branch 2 times, most recently from e3179ce to be5e22c Compare December 9, 2024 18:09
@KSDaemon KSDaemon force-pushed the feat/cubestore-driver-rust-parsing branch 11 times, most recently from 52ce278 to 5f5cc6c Compare December 17, 2024 15:41
@KSDaemon KSDaemon changed the title feat(cubestore-driver): parse cubestore response in rust feat(api-gateway): async native query results transformations Dec 17, 2024
@KSDaemon KSDaemon force-pushed the feat/cubestore-driver-rust-parsing branch 7 times, most recently from 1a39a49 to f5425ff Compare December 18, 2024 15:58
@KSDaemon KSDaemon force-pushed the feat/cubestore-driver-rust-parsing branch from 00ced53 to ebad189 Compare January 23, 2025 17:30
@KSDaemon KSDaemon merged commit 3822107 into master Jan 24, 2025
78 checks passed
@KSDaemon KSDaemon deleted the feat/cubestore-driver-rust-parsing branch January 24, 2025 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants