More Track Infos

[martenumberto]

Hey!

Is it possible do output more TrackInfos? I've Tracks that have the same Title but different "Mix Name". I wanna clearly identify if this is the Original Song (e.g. no "Mix Name") or a Remix. If its a remix "Mix Name" under Track Infos is set.
Thanks! :)

[grufkork]

Hm, interesting use case. I don't currently have the ability to do so unfortunately. I'm reluctant to add more values to extract, as updating the program for a new Rekordbox version already is a significant amount of work. I probably won't work on this specifically, but there's a possibility that I'll stick it in if I manage to get #17 working. It's possible to read the analysis data, but I don't currently have a good way to find the analysis file of the current playing track. If I figure something out, it would be able to extract pretty much all data in a stable way.

Is the remix tag an ID3 tag in the audio file? The idea I have for the above would involve extracting the file path of the track. Unless the mix tag is in the analysis file, I don't think I want to pull in an ID3 parser dependency too, but I could send the path over OSC and you could take it from there. Mind if I ask what you're building?

[martenumberto]

Sounds great. The goal is do get the current Phrase of the Master Deck. I already have a script which can search the DB, get the corresponding ANLZ file and parse the PSSI Data. But I only can search for what rkbx_link gives me as title. That's where Mix Name kicks in. But I don't find that name in RAM. I'm now looking for a TrackID of the Deck in RAM so I can search the DB and find the corresponding ANLZ File.
But I'm not very familiar with RAM Addresses and CheatEngine.
I don't know if it's stored in the ID3Tags. Most of the time I use Streamed Audio from Beatport or SoundCloud.
If I get the ID everything is fine. I can share my Python Script with you.

[grufkork]

I'd love to see your script, might be something to help with finding the analysis files. I'm pretty sure I have previously found the currently playing file path in memory, so my idea is to scan through all of the analysis files (which contain the file name) and from there build a mapping from file name -> analysis file name. A couple of problems:

• Finding all analysis files, as these can be both on USB:s and the local DB. It seems RB only allows one DB per drive though, so it might not be too difficult to catch all
• The analysis files only contain the file name, not the full path. Not sure how duplicates between drives should be/are handled. If your remix even has the same filename but in a different folder, that would also be a problem
• How quickly does the files update? Mostly important for the beatgrid, you don't want to be running on old data

I spoke some to the DeepSymmetry people. They haven't dug into desktop RB that much, but they didn't really have a good way to map that without exporting/decrypting the database which I don't want users to have to do every time they change something. The database otherwise is what contains the mapping from track to anlz. The anlz paths look a lot like MD5 hashes, but they don't seem to be trivially recreateable (if not fully random). I have found the all the anlz paths in memory, but they are just in a big lump. There's a possibility that you could find an index or pointer somewhere that references which one of the paths is active, but I haven't looked in to that.

I'm looking to do exactly what you are doing - extract the phrases for driving lights, so I'd be very happy to get this working too :)

[martenumberto]

I think the most effecient way is to go this way:

1. Finding the ID of the Track of the MasterDeck
2. Looking up the DB for the Track ID
3. Get the corresponding ANLZ File
4. Parse the Data (See Script below)

The scirpt is not the best, but works for a proof of concept.
Here you go (Thx to Gemini helping me :D):

import os
import struct
from pyrekordbox import Rekordbox6Database

PHRASE_MAP = {
    1: "Intro", 2: "Verse", 3: "Chorus", 4: "Bridge",
    5: "Up", 6: "Down", 7: "Outro", 8: "Fill-in",
    9: "Verse 2", 10: "Chorus 2", 11: "Verse 3", 12: "Chorus 3"
}

try:
    db = Rekordbox6Database()
except Exception as e:
    print(f"Fehler beim Laden der Rekordbox-Datenbank: {e}")
    exit()

search_title = input("Gib den Titel des Tracks ein (lokal oder streaming): ").strip().lower()

matching_tracks = []
for content in db.get_content():
    if content.Title and search_title in content.Title.lower():
        matching_tracks.append(content)

if not matching_tracks:
    print(f"\nKein Track mit dem Titel '{search_title}' in der Datenbank gefunden.")
else:
    print(f"\n{len(matching_tracks)} passende Einträge für '{search_title}' gefunden. Teste sie nacheinander...")
    
    found_and_processed = False
    
    for i, content in enumerate(matching_tracks):
        print(f"\n--- Versuch #{i+1}: Verarbeite '{content.Title}' von {content.Artist.Name if content.Artist else 'Unbekannt'} ---")
        print(f"Track ID: {content.ID}") # Hinzugefügte Zeile

        # Methode 1: Versuche, die Daten direkt aus der DB zu lesen (bevorzugt)
        if hasattr(content, 'SongAnalysis') and content.SongAnalysis and content.SongAnalysis.Phrase:
            # (Code für direktes DB-Parsing, falls es je benötigt wird)
            print("-> Info: Direkte DB-Phrasen gefunden (dieser Teil ist noch nicht voll implementiert).")
        
        try:
            anlz_path = db.get_anlz_path(content, type_='EXT')
            if not anlz_path or not os.path.exists(anlz_path):
                anlz_path = db.get_anlz_path(content, type_='DAT')

            if anlz_path and os.path.exists(anlz_path):
                print(f"-> ANLZ-Datei gefunden unter: {os.path.basename(anlz_path)}. Lese Datei...")
                
                with open(anlz_path, 'rb') as f:
                    binary_content = f.read()
                
                file_size = len(binary_content)
                pssi_offset = binary_content.find(b'PSSI')
                
                if pssi_offset != -1:
                    # Header-Analyse
                    header_start = pssi_offset + 4
                    entry_size = struct.unpack('>H', binary_content[header_start+10:header_start+12])[0]
                    num_phrases = struct.unpack('>H', binary_content[header_start+12:header_start+14])[0]
                    
                    if num_phrases > 0:
                        print(f"--> Erfolg! Header-Info: {num_phrases} Phrasen, jede ist {entry_size} Bytes groß.")
                        data_start = header_start + 32
                        parsed_phrases = []

                        for j in range(num_phrases):
                            entry_offset = data_start + (j * entry_size)
                            if entry_offset + entry_size > file_size:
                                print(f"Warnung: Datenblock #{j+1} ist unvollständig und wird übersprungen.")
                                continue

                            phrase_chunk = binary_content[entry_offset : entry_offset + entry_size]
                            
                            phrase_num = struct.unpack('>H', phrase_chunk[0:2])[0]
                            phrase_type_id = struct.unpack('>H', phrase_chunk[20:22])[0]
                            start_bar = struct.unpack('>H', phrase_chunk[22:24])[0]
                            
                            phrase_name = PHRASE_MAP.get(phrase_type_id, f"Unbekannt ({phrase_type_id})")
                            parsed_phrases.append({"num": phrase_num, "name": phrase_name, "bar": start_bar})

                        sorted_phrases = sorted(parsed_phrases, key=lambda p: p['bar'])

                        print("\nSong-Struktur (chronologisch sortiert):\n")
                        for phrase in sorted_phrases:
                            print(f"🎵 {phrase['name']} (Phrase #{phrase['num']})")
                            print(f"   - Beginnt bei Takt: {phrase['bar']}")
                        
                        found_and_processed = True
                        break 
                    else:
                        print("--> Info: ANLZ-Datei enthält einen PSSI-Block, aber keine Phrasen-Einträge.")
                else:
                    print("--> Info: ANLZ-Datei enthält keinen PSSI-Block.")
            else:
                print("-> Fehler: ANLZ-Datei nicht gefunden oder Pfad ist ungültig.")
        except FileNotFoundError:
             print("-> Fehler: Der Pfad zur ANLZ-Datei konnte nicht gefunden werden (FileNotFoundError).")
        except Exception as e:
            print(f"-> Ein unerwarteter Fehler ist aufgetreten: {e}")

    if not found_and_processed:
        print(f"\nVerarbeitung abgeschlossen. Es konnten keine gültigen Phrasen-Daten für '{search_title}' gefunden werden.")

EDIT: I'm not familiar with exporting Data to USB. I think the interval of updating the DB is not mandatory when Rekordbox or the DJ is in Performance mode :)

[grufkork]

I haven't completely parsed what your script is doing, but I see now that pyrekordbox can open the database without having to export it first. I didn't know this was possible, as the Rust library I've been toying with does not support it. I'll have to ponder for a bit what's the best way to do this. My number one priority is reliability: there shouldn't be any funny edge cases, lags or desyncs that cause trouble when performing live. Ideally I wouldn't have to go through the database, that data should already be loaded within Rekordbox. I'll poke around some more and see what I can find...

[grufkork]

Hey, I just think I managed to pull the current .DAT location from memory!! Have a look here:

It's only for Deck 1 so far, but it seems to persist between restarts. Surprised it was such a short pointer... I'll check how it plays with USB:s, if it just straight up gives the path we are pretty much at home!

edit: This is for 7.1.4 btw
edit2: And it works for the rest of the decks too. Just increment the second offset (8, 10, 18, 20) to get them all

[martenumberto]

Wow great! In this case we don't need to mess with the DB which makes it more efficient.
Ill have a look this evening if I could it implement it my self. As I said I'm not really familiar with CheatEngine, Addresses and Rust.
For the DB part: I got the keys while running "python pyrekordbox --download-keys" or something like that.
Can I help in some way? :)

EDIT: It would be great to get some more OSC Datapoints:

• Phrase currently active (maybe also as numbers since not all software can handle text in logic. Im using DMXControl3)
• Next Phrase
• Beats to next Phrase

[grufkork]

Those data points are pretty much what I have planned! I'd like to avoid touching the DB completely. In regards to helping out, just getting some feedback on what is wanted from the tool is great. I'll send you a test build when I get it up and running. If you find any other discoveries in the meantime just shoot me a ping 👍

[martenumberto]

I'm happy to hear this!
You should have a look at my script how the ANLZ.EXT File is parsed. I think there was a change at this file Format. The documentation out in the web are not correct for 7.1.4.
Also the OSC Datapoints mentioned in my previous comment would me great. (For e.g. to trigger a strobo 4 beats before the drop). Also an alternative output for the Phrases would be great, since not all Software does support String OSC. Maybe we could use integer (0=Unknown, 1=Intro, 2=BuildUp,...).
I try integrate it in your project to learn rust. But don't expect much :D
Do you have any time plans for the test build? I have a gig at the end of August and I wanna get in to work for it so bad :D

EDIT: I could now get the .DAT Files into your Project. See #27
Tomorrow im looking for a Parser for the corresponding .EXT File which includes the Data we can use (PSSI).
Is it known that it's sometimes necessary to reboot windows to get a working connection to Rekordbox?

[grufkork]

Hm, you should not need to reboot windows, not sure what's up with that... What version are you on? If you're on 7.1.4, I just published a new offset version which might have fixed what you're experiencing. I will have several options for output, both integers (which are used internally in RB) and strings. I still haven't looked at how much to data to actually export. I probably won't dump the entire contents, but rather create a nice interface to the data in a format that is useful for live shows. If you still need ALL the data, you can just grab the ANLZ file path and read it externally.

I just now saw your PR! Unfortunately I just finished up beat tracking using the beat grid, just pushed to the dev branch... It's good code though, honestly better quality than what I just threw together. I'll put some feedback over there to explain some details however (please keep it in English though, my German isn't great :P )

You can check out and try the code here: https://github.com/grufkork/rkbx_link/tree/dev (there's loads of dead code in the beat tracker update I haven't removed yet)

It's very much an MVP but it's already significantly more stable and simple. This is in part because the logic to calculate the beat is now stateless.

It seems that beatgrid edits are written to disk immediately so there's no problem there. I'm going to finish up the beat tracking first and check that it actually works for all songs and for USB´s. If you'd like to, give it a spin with some strange songs (eg. shifting tempo, long time until first beat) and see if it breaks.

I'll then throw in a file watcher that monitors the ANLZ files currently in use for changes, so they can be reloaded as soon as they update. From there it should be straightforward to export the phrasing too, as long as the library I'm using (Rekordcrate) is somewhat up to date. The ANLZ files don't change much over time as they need to remain backwards compatible. Instead they seem to add more files (EXT, 2EX, 3EX).

I'll see what I get done tomorrow, and thanks for your interest!

[martenumberto]

Hey that are great news!
Im going through your feedback in a few hours. The latest changes on the dev branch looks verry prommising.
Sadly i dont get any Track Information in DebugMode

Rekordbox Link v1.0.1
Updates          https://github.com/grufkork/rkbx_link/releases/latest
Get a license    https://3gg.se/products/rkbx_link
Repo and docs    https://github.com/grufkork/rkbx_link
Missing a feature? Spotted a bug? Just shoot me a message!

=====================

[Update]  Checking for updates...
[Update]  Program up to date (v1.0.1)
[Update]  Auto update disabled, skipping offset update check
[Parser]  Masterdeck index
[Parser]  Parsing pointer: 057EA5A8 A0 0 DF0
[Parser]  Current BPM
[Parser]  Parsing pointer: 055F8068 0 2B0 1A0
[Parser]  Sample position
[Parser]  Parsing pointer: 05779F60 8 8 198 718 8 1E2C
[Parser]  Track info
[Parser]  Parsing pointer: 05779F60 8 8 198 718 8 1E28
[Parser]  ANLZ path
[Parser]  Parsing pointer: 05779F60 8 8 198 808
[Parser]  Current BPM
[Parser]  Parsing pointer: 055F8068 0 2B0 130
[Parser]  Sample position
[Parser]  Parsing pointer: 056E43A8 20 410 80 168 F0 0
[Parser]  Track info
[Parser]  Parsing pointer: 05715AD8 8 3F0 0
[Parser]  ANLZ path
[Parser]  Parsing pointer: 055F8068 8 2B0 1A0
[Parser]  Current BPM
[Parser]  Parsing pointer: 05779F60 8 10 198 718 8 1E2C
[Parser]  Sample position
[Parser]  Parsing pointer: 05779F60 8 10 198 718 8 1E28
[Parser]  Track info
[Parser]  Parsing pointer: 05779F60 8 10 198 808
[Parser]  ANLZ path
[Parser]  Parsing pointer: 055F8068 8 2B0 130
[Parser]  Current BPM
[Parser]  Parsing pointer: 056E43A8 20 410 88 168 F0 0
[Parser]  Sample position
[Parser]  Parsing pointer: 05715AD8 10 3F0 0
[Parser]  Track info
[Parser]  Parsing pointer: 055F8068 10 2B0 1A0
[Parser]  ANLZ path
[Parser]  Parsing pointer: 05779F60 8 18 198 718 8 1E2C
[Parser]  Current BPM
[Parser]  Parsing pointer: 05779F60 8 18 198 718 8 1E28
[Parser]  Sample position
[Parser]  Parsing pointer: 05779F60 8 18 198 808
[Parser]  Track info
[Parser]  Parsing pointer: 055F8068 10 2B0 130
[Parser]  ANLZ path
[Parser]  Parsing pointer: 056E43A8 20 410 90 168 F0 0
[Parser]  Current BPM
[Parser]  Parsing pointer: 05715AD8 18 3F0 0
[Parser]  Sample position
[Parser]  Parsing pointer: 055F8068 18 2B0 1A0
[Parser]  Track info
[Parser]  Parsing pointer: 05779F60 8 20 198 718 8 1E2C
[Parser]  ANLZ path
[Parser]  Parsing pointer: 05779F60 8 20 198 718 8 1E28
[Parser]  Current BPM
[Parser]  Parsing pointer: 05779F60 8 20 198 808
[Parser]  Sample position
[Parser]  Parsing pointer: 055F8068 18 2B0 130
[Parser]  Track info
[Parser]  Parsing pointer: 056E43A8 20 410 98 168 F0 0
[Parser]  ANLZ path
[Parser]  Parsing pointer: 05715AD8 20 3F0 0
[App]  Rekordbox versions available: ["7.1.4"]
[App]  Targeting Rekordbox version: 7.1.4
[BeatKeeper]  Active modules:
[BeatKeeper]   - Ableton Link
[BeatKeeper]   - OSC
[BeatKeeper]  Looking for Rekordbox...


[BeatKeeper]  Connected to Rekordbox!
[BeatKeeper]  Master track changed: TrackInfo { title: "", artist: "", album: "" }
[Ableton Link]  Link peers: 0
[OSC]  Sending 127.0.0.1:4450 -> 127.0.0.1:4460
[BeatKeeper]  Master track changed: TrackInfo { title: "", artist: "", album: "" }
[BeatKeeper]  Master track changed: TrackInfo { title: "", artist: "", album: "" }

But i think you are working on it. :)

[grufkork]

That is strange... Are you running the code from the dev branch? What exact RB version are you on? Check help -> about, I'm on 7.1.4.0308 rev.a8bec28f1-6e4057e-7c27cd6-09629382-d5604cc-d08e809

[martenumberto]

Yes its the code from the dev branch and im running the exact RB as you mentioned.
EDIT: Doesnt matter if its a local or a streamed song.

[martenumberto]

I just checked a whole other Laptop with the Dev Branch. Also got the Empty Title there.
Can i provide some logs/Debug Information?

[grufkork]

First off, that is good info. I think I've just picked bad offsets, I'll try fixing it tomorrow.

Second, basic phrase support is on the dev branch now! There's three messages:
/phrase/master/current (string) : Current phrase
/phrase/master/next (string) : Next phrase coming up
/phrase/master/countin (int) : Whole beats until next phrase starts. 1 means that the next beat will be a new phrase, so it's never equal to 0

Check it out and see if it works with your tracks. I think I've aligned the off-by-ones so that the phrases should hit on the right beat. It's sometimes unclear if it's the analysis that's wrong or my parsing...

For int values of the phrases: there's 3 sets of phrase names depending on the song intensity called "moods". I'm thinking of condensing them into an overall "energy" value. You would then get two values: one for which mood (lo/mid/hi) and then one energy representing the current intensity within that mood. Not sure if "Chorus 2" is meant to have more energy than "Chorus 1", they are not always in order.