Feature: Add progress bars by peterjdolan · Pull Request #5692 · beetbox/beets

peterjdolan · 2025-03-26T17:08:18Z

Description

Many long-running commands produce little or no feedback in the terminal to indicate that they're progressing, and none of them provide estimates of how long the operation will run. This change introduces the enlighten python package, which displays progress bars akin to TQDM below the existing terminal output. This PR does not modify the import workflow, rather only modifies several built-in plugins. Modifying the import workflow to display a progress bar will be done in a follow-up PR.

See tracking issue

To Do

Documentation. (If you've added a new command-line flag, for example, find the appropriate page under docs/ to describe it.)
Changelog. (Add an entry to docs/changelog.rst to the bottom of one of the lists near the top of the document.)
Tests. (Very much encouraged but not strictly required.)

Example screenshot

github-actions · 2025-03-26T17:08:34Z

Thank you for the PR! The changelog has not been updated, so here is a friendly reminder to check if you need to add an entry.

peterjdolan · 2025-04-10T03:51:47Z

Import operation partially implemented; still needs some work. Example screenshot above.

Copilot

Copilot reviewed 24 out of 25 changed files in this pull request and generated 2 comments.

Files not reviewed (1)

docs/changelog.rst: Language not supported

beetsplug/duplicates.py

beetsplug/badfiles.py

Copilot

Copilot reviewed 24 out of 25 changed files in this pull request and generated no comments.

Files not reviewed (1)

docs/changelog.rst: Language not supported

Comments suppressed due to low confidence (1)

beetsplug/replaygain.py:1560

Removal of the SystemExit and KeyboardInterrupt exception handling in the replaygain command may affect how user interruptions are managed. Please confirm that propagating these exceptions, rather than silencing them, is the intended behavior.

finally:

peterjdolan · 2025-04-17T04:29:15Z

@snejus I'm not sure why the latest push is failing the automatic lint and formatting; it passes 'poetry run poe lint && poetry run poe format' on my machine just fine. Any suggestions? I've already sync'd my fork to beets master and rebased.

wisp3rwind · 2025-04-22T15:30:33Z

This seems to be related to some combination of (although I'm not seeing the full picture yet)

you regenerated the lockfile, which now chooses a newer version of ruff
https://github.com/astral-sh/ruff/blob/6bdffc3cbff2194afdff0e46fccb03cf6147d367/CHANGELOG.md?plain=1#L987
[flake8-type-checking] Adds implementation for TC006 astral-sh/ruff#14511
maybe, you're still running another ruff version locally

Someone should probably open a new PR to fix all TC006 warnings, include TC in our ruff rule selection instead of TCH, and probably fix ruff to a major version (a 0.X.*). Since major ruff updates might contain breaking rule/style changes, I'd guess that we'd only want to bump it manually.

I do get the same TC006 error if I locally run the system-installed ruff 0.11.0.

semohr · 2025-09-15T09:27:19Z

Is there a way to disable the progress bar?

We’re running in a containerized environment without a cli UI, where all stdout and stderr are redirected into log files. In similar cases, progress libraries (e.g. tqdm) tend to generate a lot of noisy log output, as each terminal update is written as a new line. I’m not sure how enlighten handles this and was not able to derive this from their docs.

Just wanted to mention this here to spare us some trouble down the line. Do you see any other problems regarding this use case?

jackwilsdon · 2025-09-15T09:32:17Z

It looks like this needs to replicate the check enlighten.get_manager does - disabling the manager if the output is not a TTY.

You might be best actually just using enlighten.get_manager and passing enabled to that, as it will only enable the manager if the provided enable value was true and the stream is a TTY.

peterjdolan · 2025-09-16T13:31:13Z

It looks like this needs to replicate the check enlighten.get_manager does - disabling the manager if the output is not a TTY.

You might be best actually just using enlighten.get_manager and passing enabled to that, as it will only enable the manager if the provided enable value was true and the stream is a TTY.

Great points, thank you. I'll add that condition as well.

peterjdolan

I've addressed your comments to the best of my ability, and left unresolved the comments that I had questions on. Thanks for your review!

peterjdolan · 2026-03-02T22:49:38Z

beetsplug/badfiles.py

+                for line in error_lines:
+                    ui.print_(line)
+
+        with concurrent.futures.ThreadPoolExecutor() as executor:


The base implementation uses par_map to parallelize the operation instead of simply looping in the main thread. I'm trying to maintain all existing logic, so I'm maintaining the thread-parallel structure

However, par_map's implementation is incompatible with the enlighten progress bar - it delegates work to threads and it's not possible to update the progress bar from different threads. So, instead we use a thread pool executor to achieve the same type of parallelism while keeping the progress bar iterator operating in the main thread.

If you'd rather, I could simplify this by removing the thread-parallelism, but in my testing it improves speed (over a NAS connection) by about an order of magnitude.

peterjdolan · 2026-03-02T22:50:37Z

beetsplug/badfiles.py

+        return None

-    def check_item(self, item):
+    def check_item(self, item: library.Item) -> tuple[bool, list[str]]:


Ah, I thought I'd do a little code health improvements while I'm here, but yes it's muddying the commit's changes. I'll revert them.

beets/ui/__init__.py

peterjdolan · 2026-03-02T23:15:17Z

beetsplug/duplicates.py

+        ):
            values = [getattr(obj, k, None) for k in keys]
-            values = [v for v in values if v not in (None, "")]
+            values = list(filter(lambda v: v not in (None, ""), values))


peterjdolan · 2026-03-02T23:23:30Z

beetsplug/missing.py

        if count:
            fmt += ": $missing"
-
-        for album in albums:
-            if count:
+            for album in ui.iprogress_bar(
+                albums, desc="Analyzing albums", unit="albums"
+            ):
                if _missing_count(album):
                    print_(format(album, fmt))
-
-            else:
+        else:
+            for album in albums:


Ah, my mistake. I had misread the original logic and thought there was a bug, but I understand it now. Reverted.

peterjdolan · 2026-03-02T23:27:59Z

beetsplug/unimported.py

                os.path.join(lib.directory, x.encode())
                for x in self.config["ignore_subdirectories"].as_str_seq()
            ]
-            in_folder = set()


Reverted. Thank you, and will do.

peterjdolan · 2026-03-02T23:29:02Z

beetsplug/unimported.py

+                    and file not in art_files
+                )
+
+            with ui.changes_and_errors_pbars(


My mistake, it was left over from a previous revision. I've resolved this error, and we should improve test coverage for this plugin.

pyproject.toml

poetry.lock

Many long-running commands produce little or no feedback in the terminal to indicate that they're progressing, and none of them provide estimates of how long the operation will run. This change introduces the `enlighten` python package, which displays progress bars akin to TQDM below the existing terminal output. To support consistent use and presentation of the progress bars, and to allow for future modification, we introduce a method to beets.ui - beets.ui.iprogress_bar - which can be used by Beets' core commands and all Beets plugins. Example usage is provided in the methods' documentation and in a new 'further reading' doc for developers. The Enlighten library does not work as well in Windows PowerShell as it does in a linux terminal (manually tested in Zsh), so the progress bars are disabled in Windows environments. Resolving these issues and enabling them in Windows is left as future work. Progress Bars are disabled also when not in a TTY, to prevent interference with logging when running as a web server, for example.

…hout crashing.

…ify command.

- Add lib.items_with_progress, lib.albums_with_progress - Code formatting and style - Update docs

peterjdolan · 2026-03-03T00:56:44Z

@snejus it seems that 'poe format-docs' reformats virtually all of the docs; should I do so, or would you like another PR that does so that I can submit first?

…ten to the database.

codecov · 2026-03-03T01:33:13Z

Codecov Report

❌ Patch coverage is 87.05882% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.59%. Comparing base (30d5157) to head (e66c89a).
⚠️ Report is 4 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
beetsplug/badfiles.py	40.00%	3 Missing ⚠️
beetsplug/duplicates.py	33.33%	2 Missing ⚠️
beetsplug/unimported.py	33.33%	2 Missing ⚠️
beets/ui/__init__.py	96.29%	0 Missing and 1 partial ⚠️
beetsplug/chroma.py	0.00%	1 Missing ⚠️
beetsplug/lyrics.py	0.00%	1 Missing ⚠️
beetsplug/missing.py	66.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5692      +/-   ##
==========================================
+ Coverage   69.33%   69.59%   +0.26%     
==========================================
  Files         141      141              
  Lines       18788    18831      +43     
  Branches     3061     3067       +6     
==========================================
+ Hits        13026    13105      +79     
+ Misses       5117     5078      -39     
- Partials      645      648       +3

Files with missing lines	Coverage Δ
beets/library/library.py	`94.02% <100.00%> (+0.37%)`	⬆️
beets/ui/commands/modify.py	`93.75% <100.00%> (ø)`
beets/ui/commands/move.py	`73.97% <100.00%> (+0.36%)`	⬆️
beets/ui/commands/remove.py	`100.00% <100.00%> (ø)`
beets/ui/commands/update.py	`79.06% <100.00%> (ø)`
beets/ui/commands/write.py	`76.00% <100.00%> (ø)`
beetsplug/autobpm.py	`75.00% <100.00%> (ø)`
beetsplug/embedart.py	`73.38% <100.00%> (ø)`
beetsplug/fetchart.py	`74.36% <100.00%> (+0.18%)`	⬆️
beetsplug/ftintitle.py	`85.71% <100.00%> (ø)`
... and 13 more

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

peterjdolan changed the title ~~Add progress bars to several commands.~~ Feature: Add progress bars Apr 7, 2025

peterjdolan mentioned this pull request Apr 8, 2025

Add progress bars for long-running item- or album-level operations #5711

Open

snejus requested review from Copilot and snejus and removed request for snejus April 14, 2025 01:04

Copilot AI reviewed Apr 14, 2025

View reviewed changes

beetsplug/duplicates.py Outdated Show resolved Hide resolved

beetsplug/badfiles.py Outdated Show resolved Hide resolved

peterjdolan force-pushed the tqdm branch 3 times, most recently from bc02408 to c64e577 Compare April 14, 2025 18:52

peterjdolan requested a review from Copilot April 17, 2025 03:10

Copilot AI reviewed Apr 17, 2025

View reviewed changes

peterjdolan force-pushed the tqdm branch from 1369b80 to 3ad823f Compare April 22, 2025 03:48

matmut7 mentioned this pull request Jun 24, 2025

chore: fix some ruff warnings since version upgrade peterjdolan/beets#1

Open

peterjdolan force-pushed the tqdm branch 2 times, most recently from 7ab870e to f42279b Compare July 27, 2025 20:54

peterjdolan force-pushed the tqdm branch from f42279b to f5ae6d7 Compare September 13, 2025 16:23

peterjdolan requested a review from a team as a code owner September 13, 2025 16:23

peterjdolan force-pushed the tqdm branch from f5ae6d7 to 523c8f1 Compare September 13, 2025 16:49

peterjdolan force-pushed the tqdm branch from 523c8f1 to 373e372 Compare September 16, 2025 13:30

peterjdolan force-pushed the tqdm branch 4 times, most recently from c698875 to 83155c0 Compare September 25, 2025 00:44

peterjdolan added 3 commits March 2, 2026 19:22

Plural units -> singular units

5fe3740

Fix units

51ccc89

Fix tests

994144b

peterjdolan commented Mar 3, 2026

View reviewed changes

peterjdolan added 19 commits March 2, 2026 19:49

Formatting

1f08b32

Fix small counter error

7b86304

Fix import

c61cb9e

Add progress bar to 'beet mv', and handle missing file exceptions wit…

b85e77a

…hout crashing.

Add progress bars to 'beet update', 'beet rm', 'beet mv', and the mod…

46ad63c

…ify command.

Code review feedback

a1299f1

- Add lib.items_with_progress, lib.albums_with_progress - Code formatting and style - Update docs

Revise documentation formatting

eb7aa50

Review feedback

71ff77e

Review feedback

3904e33

Fix poetry, and lint

403ba4d

Formatting

aebe5b6

Plural units -> singular units

eb01939

Fix units

47fff1f

Fix tests

049deac

Formatting

157c668

Fix small counter error

e8ad9d1

Fix import

6e20d3a

Merge branch 'tqdm' of https://github.com/peterjdolan/beets into tqdm

3c5ced9

peterjdolan requested a review from snejus March 3, 2026 00:55

peterjdolan marked this pull request as ready for review March 3, 2026 00:56

peterjdolan added 3 commits March 2, 2026 19:59

Format progressbars.rst

0d437d1

Revert unintended change to format-docs command

d74e291

Check if revision is not None, as it will be 0 for items not net writ…

e66c89a

…ten to the database.

Conversation

peterjdolan commented Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

To Do

Example screenshot

Uh oh!

github-actions bot commented Mar 26, 2025

Uh oh!

peterjdolan commented Apr 10, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

peterjdolan commented Apr 17, 2025

Uh oh!

wisp3rwind commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

semohr commented Sep 15, 2025

Uh oh!

jackwilsdon commented Sep 15, 2025

Uh oh!

peterjdolan commented Sep 16, 2025

Uh oh!

peterjdolan left a comment

Choose a reason for hiding this comment

Uh oh!

peterjdolan Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

peterjdolan Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

peterjdolan Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

peterjdolan Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

peterjdolan Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

peterjdolan Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

peterjdolan commented Mar 3, 2026

Uh oh!

codecov bot commented Mar 3, 2026

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

peterjdolan commented Mar 26, 2025 •

edited

Loading

wisp3rwind commented Apr 22, 2025 •

edited

Loading