File tree Expand file tree Collapse file tree 2 files changed +15
-0
lines changed
Expand file tree Collapse file tree 2 files changed +15
-0
lines changed Original file line number Diff line number Diff line change @@ -21,6 +21,12 @@ def spark_shuffle_partitions_heuristic(dataset_size: int) -> int:
2121
2222@contextmanager
2323def set_job_description (desc : str | None , spark_context ):
24+ """Set the Spark job description.
25+
26+ This context manager sets the Spark job description to the given string,
27+ then restores the job description to its previous value on exit. Passing
28+ desc=None resets the job description to the Spark default.
29+ """
2430 previous_desc = spark_context .getLocalProperty ("spark.job.description" )
2531 spark_context .setJobDescription (desc )
2632 try :
Original file line number Diff line number Diff line change 33The format of this changelog is based on [ Keep A Changelog] [ keep-a-changelog ] .
44Hlink adheres to semantic versioning as much as possible.
55
6+ ## Not Yet Released
7+
8+ ### Added
9+
10+ * Started setting custom Spark job descriptions for some of hlink's long-running
11+ jobs. These descriptions appear in the Spark UI and make it easier to understand
12+ what hlink is doing. [ PR #228 ] [ pr228 ]
13+
614## v4.2.1 (2025-08-18)
715
816### Fixed
@@ -453,6 +461,7 @@ and false negative data in model exploration. [PR #1][pr1]
453461[ pr215 ] : https://github.com/ipums/hlink/pull/215
454462[ pr219 ] : https://github.com/ipums/hlink/pull/219
455463[ pr222 ] : https://github.com/ipums/hlink/pull/222
464+ [ pr228 ] : https://github.com/ipums/hlink/pull/228
456465
457466[ household-matching-docs ] : config.html#household-matching
458467[ household-training-docs ] : config.html#household-training-and-model-exploration
You can’t perform that action at this time.
0 commit comments