Local files playback

[SapiensAnatis]

Firstly, apologies if there is already a thread about this -- I had a quick look and couldn't find a relevant issue/discussion.

I gather from the enum member StateError::UnsupportedLocalPlayback that librespot doesn't currently support playing back local files:

librespot/connect/src/state.rs

Lines 57 to 58 in be37402

	#[error("playback of local files is not supported")]
	UnsupportedLocalPlayBack,

Predictably when I try to play back a local file it blows up (although interestingly with a 404 Not Found error rather than this specific error).

Is there any interest in supporting local file playback and is it something that would be accepted as a community contribution? I'm aware it's quite a large amount of work and I'm unsure if the local files syncing protocol is supported by the public API.

[roderickvd]

Is there any interest in supporting local file playback and is it something that would be accepted as a community contribution?

That's be a yes for me, because it'll make librespot more feature-complete with respect to the official client.

[SapiensAnatis]

@photovoltex I think the best reference is the official support page: https://support.spotify.com/uk/article/local-files/

Looking at this, it seems to have changed since I last used the feature. It used to be that you could sync files between devices if they were on the same network. For example, you could add a folder on your PC and 'download' that music to your phone and it would be playable on both devices. It was very finicky though. It looks like this has been removed and you are now required to copy the local files to each device you want to play them on. That probably simplifies the work behind this feature.

Somewhat concerningly though this help page appears to suggest Spotify Connect doesn't support local files: https://community.spotify.com/t5/FAQs/Local-Files/ta-p/5186118. However I don't think this is accurate, I am able to use connect on my phone to play local files that are stored on both my PC and my phone. If I try and play a file that my PC doesn't have the PC client just shows an error snackbar.

---

Assuming it is feasible, since sync is no longer possible I was thinking we enable support by accepting a --local-file-dir argument when starting librespot - does that sound OK?

[photovoltex]

Would make sense to make it an extra argument in the binary.

But where to put it in the library? Did you look into how the local tracks are propagated from connect? Could we easily match them with the local files?

[SapiensAnatis]

I haven't yet studied the protocol and the detailed structure of the library -- all I can say for now is it blows up with some local files here because the uri contains unexpected characters:

librespot/connect/src/state/context.rs

Lines 357 to 376 in 8b72954

	let tracks = page
	.tracks
	.iter()
	.enumerate()
	.flat_map(|(i, track)| {
	match self.context_to_provided_track(
	track,
	Some(new_context_uri),
	context_length.map(|l| l + i),
	Some(&page.metadata),
	provider.clone(),
	) {
	Ok(t) => Some(t),
	Err(why) => {
	error!("couldn't convert {track:#?} into ProvidedTrack: {why}");
	None
	}
	}
	})
	.collect::<Vec<_>>();

Here we can see the track representation for local files is:

{
    uri: Some(
        "spotify:local:Takeru+Kanazaki%2C+Yasuhisa+Baba%2C+Takafumi+Wada%2C+%26+Shoh+Murakami:Fire+Emblem+Echoes%3A+Shadows+of+Valentia+Original+Sound+Version:Twilight+of+the+Gods:238",
    ),
    uid: Some(
        "83daa405a4ba15c8",
    ),
    gid: None,
    metadata: {
        "added_by_username": "63plob58lis4xqojfoeaxjr0l",
        "highlight_id": "83daa405a4ba15c8",
        "added_at": "1733667914000",
    },
    special_fields: SpecialFields {
        unknown_fields: UnknownFields {
            fields: None,
        },
        cached_size: CachedSize {
            size: 0,
        },
    },
}

The URI there appears to look like

spotify:local:<artist>:<album name>:<song title>:<some number>

I don't know what the number at the end represents (it's always different) or how the uid is derived either.

I'll do some more digging into the code and see what the approximate flow looks like -- I'm not too familiar with the overall protocol but it would be good to read up.

[SapiensAnatis]

On a closer look, it appears like the number at the end is the duration of the track in seconds

[photovoltex]

You probably need to take a look at several places. The player which will need to load the files from disk and the spirc which relays the message from connect to the player. In the spirc "transfer" is when we receive a request to take control of the playback. And I think the "play" request can have data on what to play. One of them should receive the info and handle it.

[SapiensAnatis]

@photovoltex @roderickvd I have made some progress and identified the capability that allows you to start playing a local file on librespot. On dev as of now you have to 'trick' it into playing a local file by starting with a non-local track and then attempting to play a local file. With the capability change you can start playing a local file on a supported client and then directly switch the playback to librespot -- which on dev is not possible as Spotify says the device "can't play this track".

I am not sure how you would prefer to review this work -- this is a tiny change but is it worth making a small incremental PR? Or would you rather the feature be implemented all in one PR, so that librespot isn't lying about its capabilities?

I was also thinking about updating the docs/ directory with what I have found out about the connect endpoint, since it would be good to expand the documentation there and also check my own understanding. Is that something you would be open to?

[roderickvd]

This is the same list as its support for external podcasts, I believe, and should cover most of cases. We may think about adding FLAC to the list, as that seems quite popular for offline archival. I don’t know whether the Spotify client will “index” formats it doesn’t support itself?

[SapiensAnatis]

I've just done a quick test with FLAC on the iOS client and it isn't getting indexed. Adding a new MP3 does work though so I haven't forgotten the process 🙂

There are some people online saying FLAC does work so it may just not like my specific FLAC file - not sure. I could also see it being desktop-only, I will try on desktop this evening.

[SapiensAnatis]

No luck on desktop Linux either. Strange as there's stuff online about it working but just nothing official, only forum posts from moderators and that sort of thing

[photovoltex]

In this post they mention something about updating the codec pack. Maybe that helps get it working, as it seems that it might work when they specifically reference FLAC? Tho I wouldn't know what they mean by updating the codec in any offical client.

Update your codec pack. This can also solve issues trying to play FLAC.

[SapiensAnatis]

Could well be my system codecs. I use Spotify in the flatpak which I think requires it to ship with all the codecs and the situation is a bit weird. Does anyone else want to try FLAC on their machine? Would be good to test different FLAC files as well.

[SapiensAnatis]

Now that we have the URI PR merged in I can take another look at this functionality.

I've managed to get local file URIs parsed and for the client to report it can play local files, the next tricky bit is matching a URI to a file.

I managed to make a snippet to derive a local file URI from a path to an audio file using Symphonia:

Symphonia example

use std::fs::File;

use symphonia::core::formats::{FormatOptions};
use symphonia::core::io::MediaSourceStream;
use symphonia::core::meta::{
    MetadataOptions, StandardTagKey, Tag,
};
use symphonia::core::probe::{Hint, ProbeResult};

fn main() {
    let path = "/home/jay/Music/Spotify/Dialga's Fight To the Finish!.mp3";

    let src = File::open(path).expect("failed to open media");

    let mss = MediaSourceStream::new(Box::new(src), Default::default());

    let mut hint = Hint::new();
    hint.with_extension("mp3");

    let meta_opts: MetadataOptions = Default::default();
    let fmt_opts: FormatOptions = Default::default();

    let mut probed = symphonia::default::get_probe()
        .format(&hint, mss, &fmt_opts, &meta_opts)
        .expect("unsupported format");

    let mut artist: Option<String> = None;
    let mut album_title: Option<String> = None;
    let mut track_title: Option<String> = None;
    let mut duration_secs: Option<u64> = None;

    if let Some(tags) = get_tags(&mut probed) {
        for tag in tags {
            if let Some(std_key) = tag.std_key {
                match std_key {
                    StandardTagKey::Album => {
                        album_title.replace(tag.value.to_string());
                    }
                    StandardTagKey::Artist => {
                        artist.replace(tag.value.to_string());
                    }
                    StandardTagKey::TrackTitle => {
                        track_title.replace(tag.value.to_string());
                    }
                    _ => {
                        continue;
                    }
                }
            }
        }
    }

    if let Some(first_track) = probed.format.tracks().iter().next() {
        let tb = first_track.codec_params.time_base;

        let time = tb
            .expect("tb")
            .calc_time(first_track.codec_params.n_frames.unwrap());
        duration_secs.replace(time.seconds);
    }

    let url = format!(
        "spotify:local:{}:{}:{}:{}",
        artist.as_deref().unwrap_or(""),
        album_title.as_deref().unwrap_or(""),
        track_title.as_deref().unwrap_or(""),
        duration_secs.unwrap_or(0)
    );

    assert_eq!(url, "spotify:local:Arata Iiyoshi:Explorers of Sky:Dialga's Fight To the Finish!:257");

    println!("{}", url);
}

fn get_tags(probed: &mut ProbeResult) -> Option<Vec<Tag>> {
    if let Some(metadata_rev) = probed.format.metadata().current() {
        return Some(metadata_rev.tags().to_vec());
    }

    if let Some(metadata_rev) = probed.metadata.get().as_ref().and_then(|m| m.current()) {
        return Some(metadata_rev.tags().to_vec());
    }

    None
}

I'm thinking that on initialisation the PlayerInternal can create and store a HashMap of URIs to file paths which can then be used when attempting to play a local file.