Cannot relocate unicode

[elijahr]

Issue description

Enjinn was unable to relocate a file with this name: RAW - RAW Compilation II- Second Breath 1-4 - 09 Raär - Strobes.flac.
Enjinn was able to relocate files with similar names which did not have unicode chars.

Steps to reproduce

# file exists here, used to be in `2021-05` directory
Music Production/trax/trax
❯ find . -name "*Raär*" -print
./2021-05 pt 2/RAW - RAW Compilation II- Second Breath 1-4 - 09 Raär - Strobes.flac

# enjinn relocate tries and fails to discover the new location of the track 
Music Production/trax/trax
❯ enjinn relocate .
✔ Detected Engine library [v1.6]
    /Users/elijah/Music/Engine Library
✔ Connected to Engine DB

✔ Found 1 missing tracks
    ../../Google Drive/Music Production/trax/trax/2021-05/RAW - RAW Compilation II- Second Breath 1-4 - 09 Raär - Strobes.flac

✔ What folder would you like to search for your tracks in? … .

✖ Couldn't find 1 tracks
    ../../Google Drive/Music Production/trax/trax/2021-05/RAW - RAW Compilation II- Second Breath 1-4 - 09 Raär - Strobes.flac

Environment

• enjinn version: enjinn/1.3.1 darwin-x64 node-v14.16.1
• Engine PRIME version: 1.6.1
• Operating System: macOS 11.2.3 (arm64)

[shayded-exe]

Good catch. I'll investigate

[elijahr]

@rshea0 it may not be unicode, just seemed like it. If I can help debug further let me know. Cheers!

[shayded-exe]

So it looks like that special character isn't a single character. It's a character with a modifier.

I created a song with that character in it's title and Engine wouldn't even load it into a deck. This explains a lot as I've been confused why some people aren't able to load Unicode songs while others can. Special characters work in Engine if they are in fact a single character. These modifier characters mess it up.

How did you add that song into your library in the first place? Does it not show up red (assuming the file is in the right place)?

[elijahr]

I think I just dragged it in. I'm not using any library integrations. I have a few tracks with ä in the file name:

They load and play okay in the software on my laptop. Haven't tried them on a device yet.

[shayded-exe]

Please do me a favor and paste here the filenames of all three of those tracks you highlighted.

[elijahr]

From the terminal:

❯ find . -name "*ä*"
./2021-01 (pt 5)/Berg Jaär - Dome.flac
./2021-01 (pt 5)/Berg Jaär - DNA.flac
./2021-05 pt 2/RAW - Clouds - AKW Stränge.flac

Interestingly, the file that enjinn couldn't relocate doesn't show for that query. So I think you're onto something that there is some difference between ä and ä. In the JS shell here I do see:

encodeURIComponent('ä')
"%C3%A4"
encodeURIComponent('ä')
"a%CC%88"

Here's a query that finds the file enjinn could not relocate:

❯ find . -name "*Strobes*"
./2021-05 pt 2/RAW - RAW Compilation II- Second Breath 1-4 - 09 Raär - Strobes.flac

I am able to drag that file from Finder into Engine Prime, and it can be loaded/played without issue.

I just tested moving and relocating with enjinn for one of the files in the first set (Berg Jaär - DNA.flac), and it worked fine:

❯ enjinn relocate
 ›   Warning: Update available from 1.3.1 to 1.3.2
✔ Detected Engine library [v1.6]
    /Users/elijahrutschman/Music/Engine Library
✔ Connected to Engine DB

✔ Found 2 missing tracks
    ../../Google Drive/Music Production/trax/trax/2021-05/RAW - RAW Compilation II- Second Breath 1-4 - 09 Raär - Strobes.flac
    ../../Google Drive/Music Production/trax/trax/2021-01 (pt 5)/Berg Jaär - DNA.flac

✔ What folder would you like to search for your tracks in? … .

⚠ Relocated 1 tracks, couldn't find 1 tracks
    ../../Google Drive/Music Production/trax/trax/2021-01 (pt 5)/new location/Berg Jaär - DNA.flac
  [still missing]
    ../../Google Drive/Music Production/trax/trax/2021-05/RAW - RAW Compilation II- Second Breath 1-4 - 09 Raär - Strobes.flac

✔ Saved relocated tracks to Engine

[shayded-exe]

So that's the weird part. A file with that character doesn't load in Engine for me. Could there be an OS inconsistency here?

Either way, ENJINN isn't working with it. I'll figure out what's going on.

[shayded-exe]

Ensure all three of those tracks are in your library then please send your m.db file to me at shayded@shayded.com.

I found this interesting article: https://eclecticlight.co/2021/05/08/explainer-unicode-normalization-and-apfs/

It appears there may be some file system fuckery going on.

I think the solution might be for ENJINN to do its own Unicode normalization when searching. That combined with the upcoming "strip accents" feature to remove all accents from the filenames and Engine tags.