Skip to content

Conversation

@dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Oct 7, 2025

What changes were proposed in this pull request?

This PR aims to exclude org/apache/spark/ui/static/** pattern from Spark operator jar.

Why are the changes needed?

Apache Spark K8s operator doesn't require Apache Spark static web resources. We had better clean them up.

BEFORE (Apache Spark K8s Operator 0.5.0)

$ jar tvf spark-kubernetes-operator.jar | grep 'org/apache/spark/ui/static/'
     0 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/
 21808 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/jquery.mustache.js
 80650 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/bootstrap.bundle.min.js
  4282 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/table.js
 11639 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/timeline-view.js
  5496 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/jquery.cookies.2.2.0.min.js
  6326 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/webui.js
  8929 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/utils.js
  3077 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/spark-logo-77x50px-hd.png
  9262 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/jquery.blockUI.min.js
  1120 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/historypage-common.js
279633 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/d3.min.js
  7363 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/dataTables.rowsGroup.js
 11663 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/dataTables.bootstrap4.min.css
 13666 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/streaming-page.js
  1412 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/streaming-page.css
576053 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/vis-timeline-graph2d.min.js
725261 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/dagre-d3.min.js
 87278 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/jquery.dataTables.min.js
 89476 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/jquery-3.5.1.min.js
  2139 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/webui-dataTables.css
  8672 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/historypage.js
  5651 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/stagespage-template.html
 32360 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/executorspage.js
  2340 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/dataTables.bootstrap4.min.js
 22999 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/jquery.dataTables.min.css
 20264 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/spark-dag-viz.js
  8067 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/executorspage-template.html
 52432 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/stagepage.js
 71789 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/graphlib-dot.min.js
    80 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/package.json
 19781 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/vis-timeline-graph2d.min.css
159470 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/bootstrap.min.css
  3097 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/spark-dag-viz.css
  5272 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/structured-streaming-page.js
  4548 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/log-view.js
  5304 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/timeline-view.css
   912 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/initialize-tooltips.js
  1820 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/flamegraph.js
  8950 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/webui.css
 17677 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/sorttable.js
 58677 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/d3-flamegraph.min.js
  2767 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/historypage-template.html
   865 Fri Feb 01 00:00:00 PST 1980 org/apache/spark/ui/static/d3-flamegraph.css

AFTER

$ jar tvf spark-kubernetes-operator.jar | grep 'org/apache/spark/ui/static/' | wc -l
       0

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the CIs.

Was this patch authored or co-authored using generative AI tooling?

No.

@dongjoon-hyun
Copy link
Member Author

Thank you, @viirya .

@dongjoon-hyun
Copy link
Member Author

Merged to main.

@dongjoon-hyun dongjoon-hyun deleted the SPARK-53828 branch October 8, 2025 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants