v0.3.1: readme

telperion · telperion · commit e6974735df3c · 2023-03-18T15:52:07.000-07:00
- CLI with no args invokes GUI. that's better, I think
diff --git a/.gitignore b/.gitignore
@@ -150,5 +150,3 @@ cython_debug/
 #  and can be added to the global gitignore or merged into this file.  For a more nuclear
 #  option (not recommended) you can uncomment the following to ignore the entire idea folder.
 #.idea/
-
-*.png
diff --git a/README.md b/README.md
@@ -1,4 +1,66 @@
-# itl-rhythm
- Testing rhythmic complexity algorithms for ITL.
- 
- The "+9ms or null" sync analysis also ended up in here. Oops!
+# +9ms or Null?
+## +9ms or Null? is a StepMania simfile unbiasing utility.
+
+This utility can determine whether the sync bias of a simfile or a pack is +9ms (In The Groove) or null (general StepMania). A future version will also offer to unify it under one of those two options.
+
+***It is not meant to perform a millisecond-perfect sync!*** Please don't depend on the exact value you get from it. Instrument attacks vary significantly, and the algorithm I'm using is not smart enough to know what to focus on.
+
+You can read more about the origins of the +9ms sync bias here:
+- [Club Fantastic Wiki's explanation](https://wiki.clubfantastic.dance/Sync#itg-offset-and-the-9ms-bias)
+- [Ash's discussion of solutions @ meow.garden](https://meow.garden/killing-the-9ms-bias)
+
+Needless to say, it's about time to put the nine millisecond nail in this coffin.
+
+## How does it work?
+I'll get into more detail on this elsewhere, but the seed concept is a visual representation first implemented (as far as I can tell) by beware, of beware's DDR Extreme fame.
+1. Identify the time that each beat occurs - I used ashastral's excellent [`simfile`](https://simfile.readthedocs.io/en/latest/) library to do this.
+1. Load the audio (`pydub`) and calculate its spectrogram (`numpy` & `scipy`).
+1. Snip a small window around each beat time out of the spectrogram and stack them in some way - this becomes the **sync fingerprint**.
+    - Time will always be on the X axis for this exercise.
+    - **Beat digest**: Flatten the frequencies (after applying a simple filter to avoid ultra-high/-low bands) and let the Y coordinate be the index of the beat. The most helpful visual, and the original inspiration.
+    - **Accumulator**: Sum up the window and keep the frequency of the sound energy as the Y coordinate. Not as useful but a good sanity check.
+1. Apply a time-domain convolution to identify the audio feature the simfile is sync'd to.
+    - **Rising edge**: This one seems like the most reliable.
+    - **Local loudness**: A more naive approach - and a bit easier to fool. But if you want to see what it does, it's here.
+1. Take the time at the highest response of the convolution as the bias of the simfile's sync.
+1. Decide whether the simfile was sync'd to the +9ms (In The Groove) or the null (StepMania) paradigm, or to neither in particular, by checking whether that bias lies within a small interval around +9ms or 0ms, respectively.
+1. Visualize it all for the user (`matplotlib`).
+
+## How to use
+
+### Setting up the project
+You have a couple options:
+- Clone the repository and use `poetry` to set up the `nine-or-null` package.
+  1. `poetry install`
+  1. (CLI) `poetry run python -m nine_or_null "C:\Games\ITGmania\Songs\pack-name"`
+  1. (GUI) `poetry run python -m nine_or_null.gui` or `poetry run python -m nine_or_null`
+- Download the executable (made with PyInstaller).
+
+### Command-line interface (CLI)
+It's not as configurable as the GUI yet but that's coming soon. For now you can just call the main routine of the `nine_or_null` package, and pass it the full path to the pack as a command-line argument. If you call it without a path, it'll invoke the GUI.
+
+### Graphical user interface (GUI)
+![Screenshot of +9ms or Null v0.2.0](doc/nine-or-null-v0.2.0.png)
+
+The intended workflow:
+1. Select the path to the pack or simfile you want to check using the directory button in the upper-right corner (or manually enter in the text box next to it).
+1. You can probably leave all the parameters alone for now.
+1. Press the big button to **let that sync in**.
+1. Let the GUI do some heavy lifting. It'll probably take a few seconds per simfile with the default settings - and it's probably gonna act like it's unresponsive. (There's some threading I could do to fix that, but I'll get to that later.)
+1. Watch the status bar, results table, and sync fingerprint plots for updates.
+    - From top to bottom, the plots represent the frequency domain accumulator, the beat digest, and the convolved fingerprint.
+    - The red line indicates where the maximum kernel response lies, and thus the sync bias of this simfile. The white line is the kernel response for the whole fingerprint vs. local time.
+1. Once the status bar indicates the job is "Done!", the number of paradigm-adherent simfiles will appear above the results table.
+1. Feel free to "View logs", "View plots", or double-click on individual simfiles in the results table to reload plots in the neighboring pane.
+1. In the future, the two arrows above the paradigm counts will allow you to batch adjust the sync bias on simfiles (either from +9ms to null, or vice versa). ***Again, not a millisecond-accurate sync utility! It's only offering to add or subtract 9ms.***
+
+(If your computer starts really chugging during the bias check, bump the "Spectral step" up or the "Spectral window" down - both of these sacrifice a bit of spectrogram precision but the results are still generally good.)
+
+## Future plans
+- Batch adjust the sync bias on simfiles (either from +9ms to null, or vice versa)
+- Handle split timing properly when it's present
+- If a straight vertical line "fit" can identify bias, then a line fit with both local time and beat index dependence could also identify sync drift...hmm...
+
+
+----
+Also there's a little broom closet in this repo where I was doing some testing on rhythmic complexity algorithms. Don't worry about that for now ;)
diff --git a/doc/nine-or-null-v0.2.0.png b/doc/nine-or-null-v0.2.0.png
diff --git a/nine-or-null/nine_or_null/__init__.py b/nine-or-null/nine_or_null/__init__.py
@@ -1,4 +1,4 @@
-_VERSION = '0.3.0'
+_VERSION = '0.3.1'
 
 from datetime import datetime as dt
 from enum import IntEnum
diff --git a/nine-or-null/nine_or_null/__main__.py b/nine-or-null/nine_or_null/__main__.py
@@ -3,84 +3,92 @@
 import logging
 
 from . import BiasKernel, KernelTarget, batch_process, guess_paradigm, _VERSION
+from .gui import start_gui
 
 if __name__ == '__main__':
     print(f'+9ms or Null? v{_VERSION}')
-    root_path = os.getcwd()
-    if len(sys.argv) > 1:
-        root_path = sys.argv[1]
-    
-    # Verify existence of root path
-    if not os.path.isdir(root_path):
-        sys.exit(f'Root directory doesn\'t exist: {root_path}')
-    else:
-        print(f"Root directory exists: {root_path}")
 
-    # Verify existence of root path
-    report_path = os.path.join(root_path, '__bias-check')
-    if not os.path.isdir(report_path):
-        try:
-            os.makedirs(report_path)
-            print(f"Report directory created: {report_path}")
-        except Exception as e:
-            sys.exit(f'Report directory can\'t be created: {report_path}')
+    if len(sys.argv) <= 1:
+        print('No path to simfile or pack provided. Entering GUI mode...')
+        start_gui()
+        print('Exiting GUI mode. Thank you for playing!')
     else:
-        print(f"Report directory exists: {report_path}")
+        print('Entering CLI mode...')
+        
+        root_path = sys.argv[1]
+
+        # Verify existence of root path
+        if not os.path.isdir(root_path):
+            sys.exit(f'Root directory doesn\'t exist: {root_path}')
+        else:
+            print(f"Root directory exists: {root_path}")
+
+        # Verify existence of root path
+        report_path = os.path.join(root_path, '__bias-check')
+        if not os.path.isdir(report_path):
+            try:
+                os.makedirs(report_path)
+                print(f"Report directory created: {report_path}")
+            except Exception as e:
+                sys.exit(f'Report directory can\'t be created: {report_path}')
+        else:
+            print(f"Report directory exists: {report_path}")
 
-    # Set up logging
-    log_path = os.path.join(report_path, 'nine-or-null.log')
-    log_fmt = logging.Formatter(
-        '[%(asctime)s.%(msecs)03d] %(levelname)-8s %(message)s',
-        datefmt='%Y-%m-%d %H:%M:%S'
-    )
-    logging.basicConfig(
-        filename=log_path,
-        encoding='utf-8',
-        level=logging.INFO
-    )
-    logging.getLogger().addHandler(logging.StreamHandler())
-    for handler in logging.getLogger().handlers:
-        handler.setFormatter(log_fmt)
+        # Set up logging
+        log_path = os.path.join(report_path, 'nine-or-null.log')
+        log_fmt = logging.Formatter(
+            '[%(asctime)s.%(msecs)03d] %(levelname)-8s %(message)s',
+            datefmt='%Y-%m-%d %H:%M:%S'
+        )
+        logging.basicConfig(
+            filename=log_path,
+            encoding='utf-8',
+            level=logging.INFO
+        )
+        logging.getLogger().addHandler(logging.StreamHandler())
+        for handler in logging.getLogger().handlers:
+            handler.setFormatter(log_fmt)
 
-    # Default parameters.
-    params = {}
-    params['root_path']      = root_path
-    params['report_path']    = report_path
-    params['consider_null']  = True
-    params['consider_p9ms']  = True
-    params['tolerance']      = 3.0
-    params['fingerprint_ms'] = 50
-    params['window_ms']      = 10
-    params['step_ms']        = 0.20
-    params['kernel_target']  = KernelTarget.DIGEST
-    params['kernel_type']    = BiasKernel.RISING
-    params['magic_offset']   = 2.0
+        # Default parameters.
+        params = {}
+        params['root_path']      = root_path
+        params['report_path']    = report_path
+        params['consider_null']  = True
+        params['consider_p9ms']  = True
+        params['tolerance']      = 3.0
+        params['fingerprint_ms'] = 50
+        params['window_ms']      = 10
+        params['step_ms']        = 0.20
+        params['kernel_target']  = KernelTarget.DIGEST
+        params['kernel_type']    = BiasKernel.RISING
+        params['magic_offset']   = 2.0
 
-    # Recall parameters.
-    header_str = f'+9ms or Null? v{_VERSION} (CLI)'
-    logging.info(f"{'=' * 20}{header_str:^32s}{'=' * 20}")
-    logging.info('Parameter settings:')
-    for k, v in params.items():
-        logging.info(f'\t{k} = {v}')
+        # Recall parameters.
+        header_str = f'+9ms or Null? v{_VERSION} (CLI)'
+        logging.info(f"{'=' * 20}{header_str:^32s}{'=' * 20}")
+        logging.info('Parameter settings:')
+        for k, v in params.items():
+            logging.info(f'\t{k} = {v}')
 
-    fingerprints = batch_process(**params)
+        fingerprints = batch_process(**params)
 
-    logging.info('-' * 72)
-    logging.info(f"Sync bias report: {len(fingerprints)} fingerprints processed in {root_path}")
+        logging.info('-' * 72)
+        logging.info(f"Sync bias report: {len(fingerprints)} fingerprints processed in {root_path}")
 
-    paradigm_count = {}
-    for paradigm in ['+9ms', 'null', '????']:
-        paradigm_map = {k: v for k, v in fingerprints.items() if guess_paradigm(v['bias_result']) == paradigm}
-        logging.info(f"Files sync'd to {paradigm}: {len(paradigm_map)}")
-        for k, v in paradigm_map.items():
-            logging.info(f"\t{k:>50s}")
-            logging.info(f"\t\tderived sync bias = {v['bias_result']:+0.1f} ms")
-        paradigm_count[paradigm] = len(paradigm_map)
+        paradigm_count = {}
+        for paradigm in ['+9ms', 'null', '????']:
+            paradigm_map = {k: v for k, v in fingerprints.items() if guess_paradigm(v['bias_result']) == paradigm}
+            logging.info(f"Files sync'd to {paradigm}: {len(paradigm_map)}")
+            for k, v in paradigm_map.items():
+                logging.info(f"\t{k:>50s}")
+                logging.info(f"\t\tderived sync bias = {v['bias_result']:+0.1f} ms")
+            paradigm_count[paradigm] = len(paradigm_map)
 
-    paradigm_most = sorted([k for k in paradigm_count], key=lambda k: paradigm_count.get(k, 0))
-    logging.info('=' * 72)
-    logging.info(f'Pack sync paradigm: {paradigm_most[-1]}')
-    logging.info('-' * 72)
+        paradigm_most = sorted([k for k in paradigm_count], key=lambda k: paradigm_count.get(k, 0))
+        logging.info('=' * 72)
+        logging.info(f'Pack sync paradigm: {paradigm_most[-1]}')
+        logging.info('-' * 72)
 
-    # Done!
+        # Done!
+        print('Exiting CLI mode. Thank you for playing!')
     
diff --git a/nine-or-null/nine_or_null/gui.py b/nine-or-null/nine_or_null/gui.py
@@ -23,9 +23,9 @@ def __init__(self, *args, **kwargs):
         self.SetSize((540, 300))
 
         label_preceding = wx.StaticText(self,
-            label=re.sub('\n[ \t]+', '\n', """+9ms or Null? is a StepMania simfile sync bias utility.
+            label=re.sub('\n[ \t]+', '\n', """+9ms or Null? is a StepMania simfile unbiasing utility.
 
-            This utility can determine whether the sync bias of a simfile or a pack is +9ms (In The Groove) or null (general StepMania) and offer to unify it under one of those two options.
+            This utility can determine whether the sync bias of a simfile or a pack is +9ms (In The Groove) or null (general StepMania). A future version will also offer to unify it under one of those two options.
             It is not meant to perform a millisecond-perfect sync!
 
             You can read more about the origins of the +9ms sync bias here:"""),
@@ -327,7 +327,7 @@ def __init__(self, *args, **kwargs):
         self.panel_plot.figure = Figure(dpi=48)
         gs = self.panel_plot.figure.add_gridspec(3)         # hspace
         self.panel_plot.axes = gs.subplots(sharex=True, sharey=False)
-        self.panel_plot.figure.suptitle('Audio fingerprint\nArtist - "Title"\nSync bias: +0.003 (probably null)')
+        self.panel_plot.figure.suptitle('Sync fingerprint\nArtist - "Title"\nSync bias: +0.003 (probably null)')
         x_test = np.linspace(-50, 50, 101, endpoint=True)
         # self.panel_plot.axes[0].plot(x_test, np.sin(x_test / 10), 'r-')
         # self.panel_plot.axes[1].plot(x_test, np.sin(x_test / 12), 'g-')
@@ -610,9 +610,12 @@ def OnAbout(self, event):
         dlg.ShowModal()
 
 
-if __name__ == '__main__':
+def start_gui():
     app = wx.App()
     frame = NineOrNull(None, title=f'+9ms or Null? v{_VERSION}', style=wx.DEFAULT_FRAME_STYLE & ~(wx.RESIZE_BORDER | wx.MAXIMIZE_BOX))
     frame.Show()
     app.MainLoop()
 
+
+if __name__ == '__main__':
+    start_gui()

Original file line number	Diff line number	Diff line change
`@@ -1,4 +1,4 @@`
`1`		`-_VERSION = '0.3.0'`
	`1`	`+_VERSION = '0.3.1'`
`2`	`2`
`3`	`3`	`from datetime import datetime as dt`
`4`	`4`	`from enum import IntEnum`