Skip to content

Commit e81531c

Browse files
authored
More longs for triggering workflows (#1772)
## Changes Add logs when running jobs ### Linked issues None ### Functionality - [ ] added relevant user documentation - [ ] added new CLI command - [ ] modified existing command: `databricks labs ucx ...` - [ ] added a new workflow - [ ] modified existing workflow: `...` - [ ] added a new table - [ ] modified existing table: `...` ### Tests - [ ] manually tested - [ ] added unit tests - [ ] added integration tests - [ ] verified on staging environment (screenshot attached) --------- Co-authored-by: Eric Vergnaud <[email protected]>
1 parent 98ba2c9 commit e81531c

File tree

1 file changed

+5
-1
lines changed
  • src/databricks/labs/ucx/source_code

1 file changed

+5
-1
lines changed

src/databricks/labs/ucx/source_code/jobs.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -182,11 +182,15 @@ def __init__(
182182

183183
def refresh_report(self, sql_backend: SqlBackend, inventory_database: str):
184184
tasks = []
185-
for job in self._ws.jobs.list():
185+
all_jobs = list(self._ws.jobs.list())
186+
logger.info(f"Preparing {len(all_jobs)} linting jobs...")
187+
for job in all_jobs:
186188
tasks.append(functools.partial(self.lint_job, job.job_id))
187189
problems: list[JobProblem] = []
190+
logger.info(f"Running {tasks} linting tasks in parallel...")
188191
for batch in Threads.strict('linting workflows', tasks):
189192
problems.extend(batch)
193+
logger.info(f"Saving {len(problems)} linting problems...")
190194
sql_backend.save_table(f'{inventory_database}.workflow_problems', problems, JobProblem, mode='overwrite')
191195

192196
def lint_job(self, job_id: int) -> list[JobProblem]:

0 commit comments

Comments
 (0)