Skip to content

Commit 8fe811d

Browse files
committed
[#227] Add docs for set_job_description and update the changelog
1 parent c59400a commit 8fe811d

File tree

2 files changed

+15
-0
lines changed

2 files changed

+15
-0
lines changed

hlink/linking/util.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,12 @@ def spark_shuffle_partitions_heuristic(dataset_size: int) -> int:
2121

2222
@contextmanager
2323
def set_job_description(desc: str | None, spark_context):
24+
"""Set the Spark job description.
25+
26+
This context manager sets the Spark job description to the given string,
27+
then restores the job description to its previous value on exit. Passing
28+
desc=None resets the job description to the Spark default.
29+
"""
2430
previous_desc = spark_context.getLocalProperty("spark.job.description")
2531
spark_context.setJobDescription(desc)
2632
try:

sphinx-docs/changelog.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,14 @@
33
The format of this changelog is based on [Keep A Changelog][keep-a-changelog].
44
Hlink adheres to semantic versioning as much as possible.
55

6+
## Not Yet Released
7+
8+
### Added
9+
10+
* Started setting custom Spark job descriptions for some of hlink's long-running
11+
jobs. These descriptions appear in the Spark UI and make it easier to understand
12+
what hlink is doing. [PR #228][pr228]
13+
614
## v4.2.1 (2025-08-18)
715

816
### Fixed
@@ -453,6 +461,7 @@ and false negative data in model exploration. [PR #1][pr1]
453461
[pr215]: https://github.com/ipums/hlink/pull/215
454462
[pr219]: https://github.com/ipums/hlink/pull/219
455463
[pr222]: https://github.com/ipums/hlink/pull/222
464+
[pr228]: https://github.com/ipums/hlink/pull/228
456465

457466
[household-matching-docs]: config.html#household-matching
458467
[household-training-docs]: config.html#household-training-and-model-exploration

0 commit comments

Comments
 (0)