Skip to content

Remove blocks truly asynchroniously#170

Closed
EnricoMi wants to merge 1 commit intomasterfrom
remove-blocks-async
Closed

Remove blocks truly asynchroniously#170
EnricoMi wants to merge 1 commit intomasterfrom
remove-blocks-async

Conversation

@EnricoMi
Copy link

@EnricoMi EnricoMi commented Feb 9, 2026

Calling driverEndpoint.askSync waits spark.rpc.askTimeout / spark.network.timeout seconds for the future, hence removeRdd, removeShuffle and removeBroadcast are ignoring blocking = false. With blocking = true, they still first wait spark.rpc.askTimeout / spark.network.timeout, then they wait spark.cleaner.referenceTracking.blocking.timeout.

If the former two are greater than the latter, the latter is effectively ignored.

@github-actions
Copy link

github-actions bot commented Feb 9, 2026

⚠️ Pull Request Title Validation

This pull request title does not contain a JIRA issue ID.

Please update the title to either:

  • Include a JIRA ID: [SPARK-12345] Your description
  • Mark as minor change: [MINOR] Your description

For minor changes that don't require a JIRA ticket (e.g., typo fixes), please prefix the title with [MINOR].


This comment was automatically generated by GitHub Actions

@github-actions github-actions bot added the CORE label Feb 9, 2026
@EnricoMi
Copy link
Author

invalid

@EnricoMi EnricoMi closed this Feb 12, 2026
@EnricoMi EnricoMi deleted the remove-blocks-async branch February 12, 2026 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant