Skip to content

Conversation

@c0ball
Copy link
Contributor

@c0ball c0ball commented Nov 27, 2024

No description provided.

Copy link
Collaborator

@timrae timrae left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this matching algorithm of just looking at the name would be too error prone to be used reliably. When matching strings like this you generally have to use multiple fields in order to avoid false positives and false negatives, as per the match() function. For albums you should also use the ipc code if it exists

@c0ball
Copy link
Contributor Author

c0ball commented Nov 29, 2024

I've improved the artist and album syncing. When syncing a users' album, the algorithm preferable uses the UPC code to retrieve it from the tidal API, if none is present it uses the search function with a query string and afterwards verifies the name, the artist and the number of tracks (I decided not to include the check for the release date as this reduced the syncing quality as for some reason some albums don't have the same release date as on Spotify...). The artist syncing matches artists by retrieving a few albums of the artist from the Spotify API. Afterwards it searches these albums on tidal and checks whether the artists are the correct one. I've also added a fallback to match an artist by their top tracks if none albums are available.

@c0ball
Copy link
Contributor Author

c0ball commented Nov 29, 2024

If that works for you, I can create a cleanup commit.

@timrae
Copy link
Collaborator

timrae commented Dec 3, 2024

Sure that's a reasonable starting point, I'd need to go through and do some testing to find the right balance between speed and accuracy though. In general the PR needs quite a lot of cleaning up so if you could do your best to clean up, simplify, refactor, and match the existing style that would be appreciated, then I can take over the PR.

try:
albums = spotify_session.artist_albums(artist_id, album_type="album,single", limit=5)['items']
return albums
except Exception as e:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general it's very unlikely that you should need to catch a general Exception in a function like this. If there's a specific exception that needs to be handled here then please catch that specific exception. If not then just remove the try block.

Copy link
Collaborator

@timrae timrae Dec 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect a maximum of ONE instance of catching except Exception as e for the entire project

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that right. I should have thrown a more specific exception like spotipy.exceptions.SpotifyException. But I removed this function in my last commit.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are still 6 other instances of catching Exception though ;)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I refactored the code to remove most of the try catch blocks. The one remaining in search_album_on_tidal is needed as the method get_albums_by_barcode is raising an ObjectNotFound exception when it cannot find an album with the given UPC.

@timrae
Copy link
Collaborator

timrae commented Dec 3, 2024

Btw have you tested this?

@c0ball
Copy link
Contributor Author

c0ball commented Dec 5, 2024

Yes, I have tested this and can confirm that both the artists and the album syncing do work. I've did some clean up, improved the artist syncing and reduced the number of methods.

Copy link

@julence julence left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

worked for me

@c0ball
Copy link
Contributor Author

c0ball commented Jan 2, 2025

@timrae does this look good to you now?

@c0ball
Copy link
Contributor Author

c0ball commented Feb 15, 2025

@jlindbergh can you look into this and merge the PR?

@jlindbergh
Copy link
Contributor

@jlindbergh can you look into this and merge the PR?

Hi! Unfortunately I don't have time to look deeper into this, nor have I the privileges to approve a merge. It does look pretty reasonable to me though (without having really tested it...). Not sure what happens with saved spotify sessions when the SPOTIFY_SCOPES changes though? Might cause unexpected issues if spotipy doesn't refresh the session?

@c0ball
Copy link
Contributor Author

c0ball commented Feb 28, 2025

Hi @jlindbergh,
that's unfortunate. Maybe I'll have to continue the project in my fork for now.
As far as I can remember, you had to accept the new permission in the browser, as with the initial setup.

@timrae
Copy link
Collaborator

timrae commented Feb 28, 2025

Sorry I haven't had capacity to look into this. Currently I'm the only maintainer, and am going to be pretty occupied with other projects for at least the next few months. I'd say continuing in your fork for now with the goal of merging back at a later point would be pragmatic for now

@EBendinelli
Copy link

Thanks @c0ball for the work on this feature! I've run --album-sync without any issue but I'm encountering a problem with --artists-sync:

Invalid ISRC code 'ushm82295225'
API Response: 'Invalid ISRC: Must be 3 groups, 12 symbols in total'
Adding new artists to Tidal:   3%|██▍                                                                                   | 8/281 [00:15<08:38,  1.90s/it]
Traceback (most recent call last):
  File "/home/p13/Documents/Dev/spotify_to_tidal/.venv/lib/python3.13/site-packages/tidalapi/session.py", line 951, in get_tracks_by_isrc
    res = self.request.request(
          ~~~~~~~~~~~~~~~~~~~~^
        "GET",
        ^^^^^^
    ...<2 lines>...
        base_url=self.config.openapi_v2_location,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ).json()
    ^
  File "/home/p13/Documents/Dev/spotify_to_tidal/.venv/lib/python3.13/site-packages/tidalapi/request.py", line 151, in request
    request.raise_for_status()
    ~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/p13/Documents/Dev/spotify_to_tidal/.venv/lib/python3.13/site-packages/requests/models.py", line 1026, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://openapi.tidal.com/v2/tracks?sessionId=REMOVED&countryCode=REMOVED&limit=1000&filter%5Bisrc%5D=ushm82295225

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/p13/Documents/Dev/spotify_to_tidal/src/spotify_to_tidal/__main__.py", line 70, in <module>
    main()
    ~~~~^^
  File "/home/p13/Documents/Dev/spotify_to_tidal/src/spotify_to_tidal/__main__.py", line 63, in main
    _sync.sync_artists_wrapper(spotify_session, tidal_session, config)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/p13/Documents/Dev/spotify_to_tidal/src/spotify_to_tidal/sync.py", line 529, in sync_artists_wrapper
    asyncio.run(sync_artists(spotify_session=spotify_session, tidal_session=tidal_session, config=config))
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/usr/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/lib/python3.13/asyncio/base_events.py", line 725, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/home/p13/Documents/Dev/spotify_to_tidal/src/spotify_to_tidal/sync.py", line 448, in sync_artists
    matched_artist = await match_artist_with_tidal_tracks(spotify_artist, tidal_candidates)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/p13/Documents/Dev/spotify_to_tidal/src/spotify_to_tidal/sync.py", line 415, in match_artist_with_tidal_tracks
    tidal_track = await find_tidal_track_by_spotify_track(track, tidal_session)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/p13/Documents/Dev/spotify_to_tidal/src/spotify_to_tidal/sync.py", line 387, in find_tidal_track_by_spotify_track
    isrc_results = tidal_session.get_tracks_by_isrc(isrc)
  File "/home/p13/Documents/Dev/spotify_to_tidal/.venv/lib/python3.13/site-packages/tidalapi/session.py", line 968, in get_tracks_by_isrc
    raise InvalidISRC
tidalapi.exceptions.InvalidISRC

I added a print to see which artist was causing the problem and OKRAA is the last artist loaded before the error. Happy to do a bit more debugging if you need help.

@eignatenkov
Copy link

+1 on the above, getting the same error

@c0ball
Copy link
Contributor Author

c0ball commented Aug 18, 2025

Hi @EBendinelli,
thank you for providing the log.
I'm very busy at the moment, but I'll try to have a look at it in the coming week.

@c0ball
Copy link
Contributor Author

c0ball commented Aug 19, 2025

@EBendinelli @eignatenkov I was in a hurry, trying to reproduce the error, but was not able to. Could you try the latest commit in which I added ISRC validation and let me know if the error is still thrown and if synchronization works?

@eignatenkov
Copy link

eignatenkov commented Aug 19, 2025

it fails differently now

Adding new artists to Tidal:   4%|██                                                      | 11/296 [00:13<05:58,  1.26s/it]
Traceback (most recent call last):
  File "/home/egor.ignatenkov/anaconda3/bin/spotify_to_tidal", line 8, in <module>
    sys.exit(main())
  File "/home/egor.ignatenkov/private/stt_branch/spotify_to_tidal/src/spotify_to_tidal/__main__.py", line 63, in main
    _sync.sync_artists_wrapper(spotify_session, tidal_session, config)
  File "/home/egor.ignatenkov/private/stt_branch/spotify_to_tidal/src/spotify_to_tidal/sync.py", line 552, in sync_artists_wrapper
    asyncio.run(sync_artists(spotify_session=spotify_session, tidal_session=tidal_session, config=config))
  File "/home/egor.ignatenkov/anaconda3/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/egor.ignatenkov/anaconda3/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/home/egor.ignatenkov/private/stt_branch/spotify_to_tidal/src/spotify_to_tidal/sync.py", line 471, in sync_artists
    matched_artist = await match_artist_with_tidal_tracks(spotify_artist, tidal_candidates)
  File "/home/egor.ignatenkov/private/stt_branch/spotify_to_tidal/src/spotify_to_tidal/sync.py", line 438, in match_artist_with_tidal_tracks
    tidal_track = await find_tidal_track_by_spotify_track(track, tidal_session)
  File "/home/egor.ignatenkov/private/stt_branch/spotify_to_tidal/src/spotify_to_tidal/sync.py", line 406, in find_tidal_track_by_spotify_track
    isrc_results = tidal_session.get_tracks_by_isrc(formatted_isrc)
  File "/home/egor.ignatenkov/anaconda3/lib/python3.10/site-packages/tidalapi/session.py", line 961, in get_tracks_by_isrc
    raise ObjectNotFound
tidalapi.exceptions.ObjectNotFound

@eignatenkov
Copy link

After the last fix it stopped failing. It gave out a gigantic output of the kind like below just for one artist but then it went okay with the next ones

No matching tracks found for ISRC 'TCAAW-11-23760'
No tracks found on Tidal for ISRC 'TCAAW-11-23760' - continuing with text search
No matching tracks found for ISRC 'TCAAW-11-23781'
No tracks found on Tidal for ISRC 'TCAAW-11-23781' - continuing with text search
No matching tracks found for ISRC 'TCAAW-11-23757'
No tracks found on Tidal for ISRC 'TCAAW-11-23757' - continuing with text search
Invalid ISRC code 'TCAAW-11-23766'
API Response: 'The service encountered an error'
Invalid ISRC error for 'TCAAW1123766' (formatted as 'TCAAW-11-23766'): 
Invalid ISRC code 'TCAAW-11-23756'
API Response: 'The service encountered an error'
Invalid ISRC error for 'TCAAW1123756' (formatted as 'TCAAW-11-23756'): 
No matching tracks found for ISRC 'TCAAW-11-23760'
No tracks found on Tidal for ISRC 'TCAAW-11-23760' - continuing with text search
No matching tracks found for ISRC 'TCAAW-11-23781'
No tracks found on Tidal for ISRC 'TCAAW-11-23781' - continuing with text search
No matching tracks found for ISRC 'TCAAW-11-23757'
No tracks found on Tidal for ISRC 'TCAAW-11-23757' - continuing with text search
Invalid ISRC code 'TCAAW-11-23766'
API Response: 'The service encountered an error'

(this is just a small part of it)

@c0ball
Copy link
Contributor Author

c0ball commented Aug 19, 2025

Give me some time for further debugging, I'll hit you up

@EBendinelli
Copy link

Thank @c0ball for your work. I tired just now and I think I got rate limited after 183 artists?

Added artist 'Phon.o' to Tidal.
Adding new artists to Tidal:  66%|███████████████████████████████████████████████████████▏                            | 180/274 [09:51<02:06,  1.35s/it]Added artist 'Ouri' to Tidal.
Adding new artists to Tidal:  66%|███████████████████████████████████████████████████████▍                            | 181/274 [09:54<05:05,  3.28s/it]

urllib3.exceptions.SSLError: [SSL] record layer failure (_ssl.c:2657)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/p13/Documents/Dev/spotify_to_tidal/.venv/lib/python3.13/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
        method=request.method,
    ...<9 lines>...
        chunked=chunked,
    )
  File "/home/p13/Documents/Dev/spotify_to_tidal/.venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 841, in urlopen
    retries = retries.increment(
        method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2]
    )
  File "/home/p13/Documents/Dev/spotify_to_tidal/.venv/lib/python3.13/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.tidal.com', port=443): Max retries exceeded with url: /v1/users/[REDACTED]/favorites/artists?sessionId=[REDACTED]&countryCode=FR&limit=1000 (Caused by SSLError(SSLError(1, '[SSL] record layer failure (_ssl.c:2657)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/p13/Documents/Dev/spotify_to_tidal/src/spotify_to_tidal/__main__.py", line 70, in <module>
    main()
    ~~~~^^
  File "/home/p13/Documents/Dev/spotify_to_tidal/src/spotify_to_tidal/__main__.py", line 63, in main
    _sync.sync_artists_wrapper(spotify_session, tidal_session, config)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/p13/Documents/Dev/spotify_to_tidal/src/spotify_to_tidal/sync.py", line 557, in sync_artists_wrapper
    asyncio.run(sync_artists(spotify_session=spotify_session, tidal_session=tidal_session, config=config))
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/usr/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/lib/python3.13/asyncio/base_events.py", line 725, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/home/p13/Documents/Dev/spotify_to_tidal/src/spotify_to_tidal/sync.py", line 478, in sync_artists
    tidal_session.user.favorites.add_artist(matched_artist.id)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/p13/Documents/Dev/spotify_to_tidal/.venv/lib/python3.13/site-packages/tidalapi/user.py", line 325, in add_artist
    return self.requests.request(
           ~~~~~~~~~~~~~~~~~~~~~^
        "POST", f"{self.base_url}/artists", data={"artistId": artist_id}
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ).ok
    ^
  File "/home/p13/Documents/Dev/spotify_to_tidal/.venv/lib/python3.13/site-packages/tidalapi/request.py", line 148, in request
    request = self.basic_request(method, path, params, data, headers, base_url)
  File "/home/p13/Documents/Dev/spotify_to_tidal/.venv/lib/python3.13/site-packages/tidalapi/request.py", line 101, in basic_request
    request = self.session.request_session.request(
        method, url, params=request_params, data=data, headers=headers
    )
  File "/home/p13/Documents/Dev/spotify_to_tidal/.venv/lib/python3.13/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/p13/Documents/Dev/spotify_to_tidal/.venv/lib/python3.13/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/home/p13/Documents/Dev/spotify_to_tidal/.venv/lib/python3.13/site-packages/requests/adapters.py", line 698, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='api.tidal.com', port=443): Max retries exceeded with url: /v1/users/[REDACTED]/favorites/artists?sessionId=[REDACTED]&countryCode=FR&limit=1000 (Caused by SSLError(SSLError(1, '[SSL] record layer failure (_ssl.c:2657)')))

Relaunching the command processed the remaining 100 artists without issue. I also got a few errors similar to @eignatenkov :

Adding new artists to Tidal:  29%|████████████████████████▌                                                            | 79/274 [03:14<04:30,  1.39s/it]No matching tracks found for ISRC 'US25X-10-89656'
No tracks found on Tidal for ISRC 'US25X-10-89656' - continuing with text search
No matching tracks found for ISRC 'US25X-10-90471'
No tracks found on Tidal for ISRC 'US25X-10-90471' - continuing with text search
No matching tracks found for ISRC 'US25X-10-90474'
No tracks found on Tidal for ISRC 'US25X-10-90474' - continuing with text search

@eignatenkov
Copy link

I have a question about album sync. Re-running it, to be specific.

when I re-run artist-sync, it only tries again to add the artists which had errors on the previous run. I got 20 something left, only they are processed again, and they fail again :)
But when I re-run album sync something different happens

Opening Spotify session
Opening Tidal session
Loading saved albums from Spotify
Fetching additional data chunks: 100%|█████████████████████████████████████████████████████| 85/85 [00:03<00:00, 23.60it/s]
Found 1702 albums saved on Spotify
Adding new albums to Tidal:   0%|▏                                                         | 3/731 [00:01<05:48,  2.09it/s]

you can see that I'm at a point where the tool wants to synchronize only 731 albums out of 1702, it goes through the list, fails sometimes, other entries are processed without any comments, but when I try to re-run it, I'm back at 731 albums it wants to add, and the output is the same. So I'm not sure what is happening there.

@c0ball
Copy link
Contributor Author

c0ball commented Aug 20, 2025

Thanks for pointing that out: I'll also have an eye on this

@c0ball
Copy link
Contributor Author

c0ball commented Sep 22, 2025

@eignatenkov @EBendinelli Sorry that it took a little longer than expected.
I've improved the ISRC validation, which is used for the artists synchronization as well as the error handling and rate limiting during synchronization. Also improved the matching algorithm used for syncing a user's album.

Therefore, I would be extremely grateful if you could test this new version and provide feedback. 🙏

@eignatenkov
Copy link

eignatenkov commented Sep 25, 2025

Ran album-sync. In the beginning it said that it found 1798 albums on spotify. Then it started trying to add 875. In the end it said

Album synchronization complete. Successfully added: 669, Failed: 206

I can see 16 new albums added in tidal interface
Then I re-ran it, it again says that it found 1798 albums and now tries to add 876.

Album synchronization complete. Successfully added: 423, Failed: 453

I see one new album
re-ran again, 1798/876 as before

Album synchronization complete. Successfully added: 763, Failed: 113

Another album added

I'd say it's kind of unclear what is happening. Is 876 a maximum number of albums that can be processed in one go? And it still looks like it re-adds again the albums there were already synced

@julence
Copy link

julence commented Oct 9, 2025

not syncing cases

artist name western / cyrillic

spoti track/5dlycXEoqr38kYWYvEuxBJ / tid track/451213310

removed from spoti / available on tid track & name upper / lowercase diff

spoti track/1CjhaYTXfU4fwyiOhwDfNq / tid track/297316033

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants