How do we import skipped files from log-file?

[arsaboo]

I saw this feature to read skipped files from logfile merged, but have not been able to use it.

I tried several combinations, but they all give me some error:

$ beet import --from-logfile=/home/arsaboo/.config/beets/beet.log
error: malformed logfile /home/arsaboo/.config/beets/beet.log: Can't mix absolute and relative paths
$ beet import --from-logfile="/home/arsaboo/.config/beets/beet.log"
error: malformed logfile /home/arsaboo/.config/beets/beet.log: Can't mix absolute and relative paths
$ beet import --from-logfile=beet.log
error: unreadable logfile beet.log: [Errno 2] No such file or directory: b'/home/arsaboo/beet.log'
$ beet import --from-logfile=".config/beets/beet.log"
error: malformed logfile .config/beets/beet.log: Can't mix absolute and relative paths

The log file is /home/arsaboo/.config/beets/beet.log. Not sure why it is not working.

[sampsyo]

Can you please try enabling verbose mode (beet -vv import ...)?

[arsaboo]

$ beet -vvv import --from-logfile=/home/arsaboo/.config/beets/beet.log
user configuration: /home/arsaboo/.config/beets/config.yaml
data directory: /home/arsaboo/.config/beets
plugin paths:
inline: adding item field initial_char
inline: adding item field folder
lyrics: Disabling google source: no API key configured.
rewrite: adding template field artist Rahul Dev Burman
Sending event: pluginload
library database: /home/arsaboo/.config/beets/musiclibrary.blb
library directory: /data/music
Sending event: library_opened
error: malformed logfile /home/arsaboo/.config/beets/beet.log: Can't mix absolute and relative paths

[sampsyo]

Hmm; that's too bad! I was hoping that the error would come with an exception traceback that explains where the error originally arose. 😕

It's hard to say exactly what's going wrong, but this error does not come from how you're specifying the option—it's from the contents of the file. It's one of these errors, if the code makes sense to you:

beets/beets/ui/commands.py

Lines 88 to 99 in 2e18f84

	verb, sep, paths = line.rstrip("\n").partition(" ")
	if not sep:
	raise ValueError(f"line {i} is invalid")
	# Ignore informational lines that don't need to be re-imported.
	if verb in {"import", "duplicate-keep", "duplicate-replace"}:
	continue
	if verb not in {"asis", "skip", "duplicate-skip"}:
	raise ValueError(f"line {i} contains unknown verb {verb}")
	yield os.path.commonpath(paths.split("; "))

So it might be worth looking inside the file to see if it looks like a proper beets log file.

[arsaboo]

So, I added some additional logging and I think here's the issue. My log files included paths outside the library directory. These are temporary folders that I used to import music. Ideally, it should just skip paths that are outside the library directory.

[JOJ0]

Thanks to this discussion I also started to use this feature ;-))
I'm not sure though if the reason for your error @arsaboo is the fact that you have a path outside the library, for me paths outside the library directory seem to work.

Also none of the error messages in the code snippet seem to reflect what the error message actually says: "Can't mix abosolute an relative paths" is nowhere to be found in the posted code.

I think the error is thrown out of this this logfile parser function that is actually the caller of the function posted above: https://github.com/beetbox/beets/blob/master/beets/ui/commands.py#L102-L116

Hope that helps somehow and maybe once you figure out what the problem is, you want to improve this error handling or spit out a better error message or just skip the paths that can't be processed?

For me this feature works nice since sometimes I have rubbish paths still in the logfile that don't exist anymore. Then I would get a nice error telling me this and it is just ignored. For example this doesn't interrupt the importer, it just continues with the next line in the logfile:

$ beet -P beatport4,chroma import -t --from-logfile=$BEETSDIR/remotebeets-import.log 
No such file or directory: /remote/data/music-library/MatlMusicSpeicher/Applescal - 2009 - A Slaves Commitment
No such file or directory: /remote/data/music-library/MatlMusicSpeicher/Jori Hulkkonen, Jesper Dahlback - 2010 - Kebacid - Rippes Dreams

[theraspb3rry]

I ran across this issue and my problem was that one of the folders in the log actually had a semicolon ; in the actual folder name, so it was literally trying to mix absolute and relative paths because of the way the log file is parsed.

i just edited the yield in the _paths_from_logfile function to except the warning and just log the path so I can continue chomping down on this massive log file rather than just deleting the path(s) from my log file. not a proper fix, but at least avoids the issue for now.

            try:
                yield os.path.commonpath(paths.split("; "))
            except Exception as error:
                log.warning(
                    "Path can not be parsed: {} {}".format(
                        error,
                        paths
                    )
                )
                return []