Unexpected Error [Errno 12] Out of memory - when trying to start/auto import library

[kyxap]

I want to move my library to beets and decide to use this project to make it happen.
But when I'm trying to start the the container im getting errors longs/ui.

Expected behavior
Expecting that it start the import based on default config (tag + copy)

To Reproduce

• Start to docker
• See the logs/ui

Technical Details

68047e148380 pspitzner/beets-flask:stable

logs:

[ERROR] beets-flask: Unhandled exception: [Errno 12] Out of memory
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/quart/app.py", line 1500, in full_dispatch_request
    result = await self.dispatch_request(request_context)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/quart/app.py", line 1597, in dispatch_request
    return await self.ensure_async(handler)(**request_.view_args)  # type: ignore[return-value]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/repo/backend/beets_flask/server/routes/inbox.py", line 47, in get_tree
    folders.append(path_to_folder(folder, subdirs=False))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/cachetools/_cached.py", line 100, in wrapper
    v = func(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/repo/backend/beets_flask/disk.py", line 274, in path_to_folder
    return Folder.from_path(root_dir, subdirs=subdirs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/repo/backend/beets_flask/disk.py", line 128, in from_path
    album_folders = all_album_folders(path, subdirs=subdirs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/repo/backend/beets_flask/disk.py", line 385, in all_album_folders
    if all(is_archive_file(i.decode("utf-8")) for i in items):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/repo/backend/beets_flask/disk.py", line 385, in <genexpr>
    if all(is_archive_file(i.decode("utf-8")) for i in items):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/repo/backend/beets_flask/disk.py", line 227, in is_archive_file
    return ArchiveImportTask.is_archive(str(path))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/beets/importer/tasks.py", line 824, in is_archive
    if path_test(os.fsdecode(path)):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 2822, in is_tarfile
    t = open(name)
        ^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 1834, in open
    return func(name, "r", fileobj, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 1963, in xzopen
    t = cls.taropen(name, mode, fileobj, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 1884, in taropen
    return cls(name, mode, fileobj, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 1744, in __init__
    self.firstmember = self.next()
                       ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 2662, in next
    raise e
  File "/usr/local/lib/python3.11/tarfile.py", line 2635, in next
    tarinfo = self.tarinfo.fromtarfile(self)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 1296, in fromtarfile
    buf = tarfile.fileobj.read(BLOCKSIZE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/lzma.py", line 200, in read
    return self._buffer.read(size)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/_compression.py", line 68, in readinto
    data = self.read(len(byte_view))
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/_compression.py", line 97, in read
    rawblock = self._fp.read(BUFFER_SIZE)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 12] Out of memory
[ERROR] beets-flask: Unhandled exception: [Errno 12] Out of memory
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/quart/app.py", line 1500, in full_dispatch_request
    result = await self.dispatch_request(request_context)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/quart/app.py", line 1597, in dispatch_request
    return await self.ensure_async(handler)(**request_.view_args)  # type: ignore[return-value]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/repo/backend/beets_flask/server/routes/inbox.py", line 47, in get_tree
    folders.append(path_to_folder(folder, subdirs=False))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/cachetools/_cached.py", line 100, in wrapper
    v = func(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/repo/backend/beets_flask/disk.py", line 274, in path_to_folder
    return Folder.from_path(root_dir, subdirs=subdirs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/repo/backend/beets_flask/disk.py", line 128, in from_path
    album_folders = all_album_folders(path, subdirs=subdirs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/repo/backend/beets_flask/disk.py", line 385, in all_album_folders
    if all(is_archive_file(i.decode("utf-8")) for i in items):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/repo/backend/beets_flask/disk.py", line 385, in <genexpr>
    if all(is_archive_file(i.decode("utf-8")) for i in items):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/repo/backend/beets_flask/disk.py", line 227, in is_archive_file
    return ArchiveImportTask.is_archive(str(path))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/beets/importer/tasks.py", line 824, in is_archive
    if path_test(os.fsdecode(path)):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 2822, in is_tarfile
    t = open(name)
        ^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 1834, in open
    return func(name, "r", fileobj, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 1963, in xzopen
    t = cls.taropen(name, mode, fileobj, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 1884, in taropen
    return cls(name, mode, fileobj, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 1744, in __init__
    self.firstmember = self.next()
                       ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 2662, in next
    raise e
  File "/usr/local/lib/python3.11/tarfile.py", line 2635, in next
    tarinfo = self.tarinfo.fromtarfile(self)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/tarfile.py", line 1296, in fromtarfile
    buf = tarfile.fileobj.read(BLOCKSIZE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/lzma.py", line 200, in read
    return self._buffer.read(size)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/_compression.py", line 68, in readinto
    data = self.read(len(byte_view))
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/_compression.py", line 97, in read
    rawblock = self._fp.read(BUFFER_SIZE)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 12] Out of memory

yaml:

services:
    beets-flask:
        image: pspitzner/beets-flask:stable
        container_name: beets-flask
        restart: unless-stopped
        ports:
            - "5001:5001"
        environment:
            # 502 is default on macos, 1000 on linux
            TZ: ${TZ}
            USER_ID: 1000
            GROUP_ID: 1000
        volumes:
            - ./config/:/config
            # for music folders, match paths inside and out of container!
            - ${NAS_MUSIC_ROOT}:${NAS_MUSIC_ROOT}

conf:

cat config/beets/config.yaml
# ------------------------------------------------------------------------------------ #
#                                      BEETS CONFIG                                    #
# ------------------------------------------------------------------------------------ #
# Opinionated example beets configuration. This file was automatically copied to
# /config/beets/config.yaml. Feel free to edit this file to customize the beets
# configuration. For more information on the beets configuration, see
# https://beets.readthedocs.io/en/stable/reference/config.html

plugins: [
        info,
        the,
        fetchart,
        embedart,
        ftintitle,
        lastgenre,
        missing,
        albumtypes,
        scrub,
        zero,
        mbsync,
        duplicates,
        convert,
        fromfilename,
        inline,
        edit,
        #spotify, # needs authentication https://docs.beets.io/en/latest/plugins/spotify.html
        musicbrainz, # needs to be enabled explicitly since beets 2.4.0
#        lyrics
    ]

lyrics:
  auto: yes
  synced: yes
  sources:
    - lrclib
    - genius

directory: /mnt/music/music-beets
# library: /config/beets/library.db # default location in the container

import:
    move: no
    copy: yes
    write: yes
    log: /mnt/music/music-beets/last_beets_imports.log
    quiet_fallback: skip
    detail: yes
    duplicate_action: ask # ask|skip|merge|keep|remove

ui:
    color: yes

# fix up output file paths
replace:
    '[\\]': ""
    "[_]": "-"
    "[/]": "-"
    '^\.+': ""
    '[\x00-\x1f]': ""
    '[<>:"\?\*\|]': ""
    '\.$': ""
    '\s+$': ""
    '^\s+': ""
    "^-": ""
    "’": ""
    "′": ""
    "″": ""
    "‐": "-"

per_disc_numbering: no
asciify_paths: yes

# adjusting the `threaded` setting of beets should currently have no effect.
# we launch our own workers for previews and only have one
# import worker that runs one import (file moving) at a time.
threaded: no

fetchart:
    minwidth: 500
    enforce_ratio: 10px # yes|no or tolerance around 1:1 ratio
    sources: coverart filesystem itunes amazon spotify albumart fanarttv

embedart:
    auto: yes
    ifempty: yes # whether to avoid embedding album art for files that already have art embedded.
    remove_art_file: yes

ftintitle:
    auto: yes
    format: (feat. {0})

lastgenre: # get genres from last fm
    auto: yes
    count: 4
    prefer_specific: yes # Sort genres by the most to least specific, rather than most to least popular.
    force: yes # By default, beets will always fetch new genres, even if the files already have one
    source: track # album|track
    separator: "; "
    fallback: ""

match:
    # autotagger tolerance [0, 1], default 0.04
    # for example, 0.1 means 90% similarity required.
    # inbox-folder autotag setting 'auto' respects this and imports above the threshold (like the beets cli would)
    strong_rec_thresh: 0.1

    # customize how penalties affect the match score
    distance_weights:
        # source: 2.0
        # artist: 3.0
        # album: 3.0
        # media: 1.0
        # mediums: 1.0
        # year: 1.0
        # country: 0.5
        # label: 0.5
        # catalognum: 0.5
        # albumdisambig: 0.5
        # album_id: 5.0
        # tracks: 2.0
        data_source: 0.0 # do not apply penalty to any data source plugin
        missing_tracks: 0.2 # If your prefer not being so picky about missing tracks. default 0.9
        # unmatched_tracks: 0.6
        # track_title: 3.0
        # track_artist: 2.0
        # track_index: 1.0
        # track_length: 2.0
        # track_id: 5.0

musicbrainz:
    external_ids:
        discogs: yes
        bandcamp: yes
        spotify: yes
        deezer: yes
        beatport: yes
        tidal: yes

flask

# ------------------------------------------------------------------------------------ #
#                                   BEETS GUI CONFIG                                   #
# ------------------------------------------------------------------------------------ #
# Example file, this file was automatically copied to /config/beets-flask/config.yaml.
# Feel free to edit this file to customize the gui configuration.
# Especially the `folders` section in the `inbox` section are important to set up your inbox
# folders. You can add as many folders as you like, but don't forget to volume-map them in
# your docker-compose.yml.

gui:
    num_preview_workers: 4 # how many previews to generate in parallel

    library:
        # Use to split artists in the library view if using multiple artists in a field.
        # Set to an empty list to disable this feature.
        artist_separators: [",", ";", "&"]

    terminal:
        start_path: "/mnt/music/music-beets" # the directory where to start new terminal sessions

    inbox:
        folders:
            # --------------------------------- README -------------------------------- #
            # Before using the inbox feature, you need to create the folders
            # and decide on an inbox type. Have a look at the examples below.

            #Inbox1:
            #    name: "Dummy inbox"
            #    path: "/music/dummy"
            #    autotag: no
            #    # do not automatically trigger tagging and do not automatically import
            Inbox2:
                name: "Auto Inbox"
                path: "/mnt/music/music-lidarr"
                autotag: "auto"
                # trigger tag and import if a good match is found based on `auto_threshold`
                auto_threshold: 0.15
                # if set to null, uses the value in beets config (match.strong_rec_thresh)
                # define the distance from a perfect match, i.e. set to 0.1 to import
                # matches with 90% similarity or better.
           # Inbox3:
           #     name: "An Inbox that only generates the previews"
           #     path: "/music/inbox_preview"
           #     autotag: "preview"
           #     # trigger tag but do not import, recommended for most control

[semohr]

It seems like your inbox is not empty and you have some tar files in it (which are bigger than your machines memory). I would say this is not really a bug on our side but a usage error. To confirm this could you have a look at the content in your /mnt/music/music-lidarr folder?

We might be able to create and push a fix in beets tho but I dont think this is an issue we should fix in beets-flask

[kyxap]

interesting... I was not able to find any .tar so far but still looking. Also I decided to try cli beets and got similar error - Out of memory but per some albums, most of the data was not imported at all

[kyxap]

anyway no tar files, I have some 600+mb flac files but everything else is much more smaller

[kyxap]

Another test, I've created another folder with just 1 Artist, getting same error:

have not Idea what I'm doing wrong