spreites and rotations

Author: mkalkbrenner

AGENTS.md

@@ -0,0 +1,56 @@
+# Codex Notes: ColorizingDMD Editor + libserum
+
+## Scope
+- This file documents development context and non-obvious legacy-format behavior.
+- It must not change cROM/cROMc data formats.
+- The editor must remain a drop-in replacement for the legacy app.
+- The new editor targets Serum V2 only; libserum still supports V1 for other clients.
+
+## Data format constraints (legacy compatibility)
+- Do not alter the binary layout of cROM/cROMc files.
+- All editor changes must map to existing fields in the legacy format.
+
+## Key serum relationships (from libserum)
+- Dynamic masks are per-frame pixel maps:
+  - `dynamasks[frame][pixel]` (SD) and `dynamasks_extra[frame][pixel]` (HD).
+  - Each pixel stores a dynacouche index (0..31) or 255 for none.
+- Dynamic colors are per-frame sets:
+  - `dyna4cols_v2[frame][dynacouche * nocolors + originalPixel]` (SD)
+  - `dyna4cols_v2_extra[frame][...]` (HD)
+- Sprite dynamic colors are per-sprite:
+  - `dynaspritemasks[sprite][pixel]` selects a dynacouche index.
+  - `dynasprite4cols[sprite][dynacouche * nocolors + originalPixel]`.
+- Sprite detection:
+  - Frame detection zones: `framespriteBB[frame][slot*4..]`.
+  - Per-sprite detection areas: `spritedetareas[sprite][area*4..]`.
+  - Matching + placement logic is in libserum `Check_Spritesv2`.
+- Backgrounds:
+  - `backgroundIDs[frame][0]` selects background.
+  - `backgroundmask[frame][pixel]` masks background visibility.
+  - Applied before dynamic masks; sprites applied after.
+
+## Rendering order (v2, runtime behavior)
+1) Background (if original pixel is 0 and mask allows).
+2) Dynamic mask replacement (dyna4cols).
+3) Color rotations (if enabled).
+4) Sprites overlay (spritecolored / dynasprite4cols).
+
+## Editor goals
+- WYSIWYG: render canvas + preview using libserum logic.
+- Avoid data duplication for large projects.
+- Preview row only renders the visible subset of frames.
+
+## Editor mask behavior
+- Frame uses exactly one comparison mask and one dynamic mask assignment.
+- Mask editing happens on the original (orange) frame.
+- Background mask is independent from mask/dynamic mask toggles.
+
+## HD/SD rules
+- HD is 256x64 (SD is 128x32).
+- HD previews are double size vs SD previews.
+- When HD is absent, treat all-black HD backgrounds as non-existent.
+- HD background masks should be double-scaled when generated from SD.
+
+## Open integration task
+- Add libserum editor API (non-owning view) to render frames + sprite matches.
+- Keep libserum free of new dependencies (no Qt/OpenCV).

README.editor.md

@@ -0,0 +1,51 @@
+# Editor + libserum Development Notes
+
+This document captures legacy-format behavior and editor migration decisions.
+It complements `readme.md` (user-facing info) and must not change file formats.
+
+## Guiding constraints
+- Keep cROM/cROMc formats 100% compatible with existing tools.
+- The editor should be a drop-in replacement for the legacy app.
+- Avoid new dependencies in libserum (no Qt/OpenCV).
+- The editor targets Serum V2; libserum retains V1 for other consumers.
+
+## Current direction
+- Use libserum for canvas + preview rendering (WYSIWYG).
+- Add a libserum editor API with non-owning data views to avoid RAM blowups.
+- Preview row renders only the visible subset of frames.
+
+## libserum editor API concept
+- New header: `serum-editor.h` (separate from `serum-decode.h`).
+- Core functions (names TBD):
+  - Create/destroy editor context.
+  - Attach a non-owning `SerumDataView` (raw pointers + sizes).
+  - Render a frame by ID using project originals.
+  - Match sprites for a frame and return positions.
+  - Optional: render with precomputed matches to avoid repeat matching.
+- Color rotations should be off by default, enabled only on demand.
+
+## Legacy data relationships (v2)
+- Dynamic masks: `dynamasks[frame][pixel]` -> dynacouche index.
+- Dynamic colors: `dyna4cols_v2[frame][dynacouche * nocolors + originalPixel]`.
+- Sprite dynamic colors:
+  - `dynaspritemasks[sprite][pixel]` -> dynacouche index.
+  - `dynasprite4cols[sprite][dynacouche * nocolors + originalPixel]`.
+- Sprite detection:
+  - Frame detection zones: `framespriteBB[frame][slot*4..]`.
+  - Per-sprite detection areas: `spritedetareas[sprite][area*4..]`.
+
+## Rendering order (runtime behavior)
+1) Background (when original pixel is 0 and mask allows).
+2) Dynamic mask replacement.
+3) Color rotations (if enabled).
+4) Sprites overlay.
+
+## UI conventions implemented so far
+- Masks and dynamic masks show semi-transparent overlays + outlines on grid.
+- Background mask is independent from mask/dynamic mask toggles.
+- HD frames render as same absolute size with higher internal resolution.
+
+## Open tasks
+- Add libserum editor API and wire canvas + preview rendering.
+- Use libserum sprite matching to fix sprite placement in editor.
+- Add on-demand rotation preview button.

app/CMakeLists.txt

@@ -8,6 +8,7 @@ target_sources(ColorizingDMD PRIVATE
     CanvasWidget.h
     GLCanvasWidget.cpp
     GLCanvasWidget.h
+    ../libserum/src/serum-editor.cpp
     ProjectState.cpp
     ProjectState.h
     ProjectIO.cpp
@@ -47,4 +48,5 @@ target_link_libraries(ColorizingDMD PRIVATE
 
 target_include_directories(ColorizingDMD PRIVATE
     ${CMAKE_SOURCE_DIR}
+    ${CMAKE_SOURCE_DIR}/libserum/src
 )

app/CanvasWidget.cpp

@@ -21,6 +21,7 @@ CanvasWidget::CanvasWidget(const QString& title, QWidget* parent)
     , m_backgroundMaskButton(new QToolButton(m_canvas))
     , m_zoneButton(new QToolButton(m_canvas))
     , m_hdButton(new QToolButton(m_canvas))
+    , m_rotateButton(new QToolButton(m_canvas))
 {
     auto* layout = new QVBoxLayout(this);
     m_canvas->setOverlayText(title);
@@ -93,6 +94,13 @@ CanvasWidget::CanvasWidget(const QString& title, QWidget* parent)
     m_hdButton->setToolTip("Toggle HD frame");
     m_hdButton->setCursor(Qt::PointingHandCursor);
     connect(m_hdButton, &QToolButton::toggled, this, &CanvasWidget::hdToggled);
+
+    m_rotateButton->setText("Rotate");
+    m_rotateButton->setCheckable(true);
+    m_rotateButton->setAutoRaise(true);
+    m_rotateButton->setToolTip("Preview color rotations");
+    m_rotateButton->setCursor(Qt::PointingHandCursor);
+    connect(m_rotateButton, &QToolButton::toggled, this, &CanvasWidget::rotateToggled);
 }
 
 void CanvasWidget::setTitle(const QString& title)
@@ -236,6 +244,23 @@ void CanvasWidget::setZoneButtonVisible(bool visible)
     m_zoneButton->setVisible(visible);
 }
 
+void CanvasWidget::setRotateChecked(bool enabled)
+{
+    if (!m_rotateButton) {
+        return;
+    }
+    QSignalBlocker blocker(m_rotateButton);
+    m_rotateButton->setChecked(enabled);
+}
+
+void CanvasWidget::setRotateEnabled(bool enabled)
+{
+    if (!m_rotateButton) {
+        return;
+    }
+    m_rotateButton->setEnabled(enabled);
+}
+
 void CanvasWidget::setBackgroundChecked(bool enabled)
 {
     if (!m_backgroundButton) {
@@ -282,7 +307,8 @@ void CanvasWidget::resizeEvent(QResizeEvent* event)
 {
     QWidget::resizeEvent(event);
     if (!m_fitButton || !m_gridButton || !m_originalButton || !m_backgroundButton ||
-        !m_maskButton || !m_dynamicButton || !m_backgroundMaskButton || !m_zoneButton || !m_hdButton) {
+        !m_maskButton || !m_dynamicButton || !m_backgroundMaskButton || !m_zoneButton || !m_hdButton ||
+        !m_rotateButton) {
         return;
     }
     const int margin = 8;
@@ -331,4 +357,9 @@ void CanvasWidget::resizeEvent(QResizeEvent* event)
     const int hdX = bgX - hdSize.width() - 6;
     m_hdButton->move(hdX, y);
     m_hdButton->raise();
+
+    const QSize rotateSize = m_rotateButton->sizeHint();
+    const int rotateX = hdX - rotateSize.width() - 6;
+    m_rotateButton->move(rotateX, y);
+    m_rotateButton->raise();
 }

app/CanvasWidget.h

@@ -37,6 +37,8 @@ class CanvasWidget : public QWidget
     void setZoneButtonChecked(bool enabled);
     void setZoneButtonEnabled(bool enabled);
     void setZoneButtonVisible(bool visible);
+    void setRotateChecked(bool enabled);
+    void setRotateEnabled(bool enabled);
 
 signals:
     void fitRequested();
@@ -48,6 +50,7 @@ class CanvasWidget : public QWidget
     void backgroundMaskToggled(bool enabled);
     void zoneToggled(bool enabled);
     void hdToggled(bool enabled);
+    void rotateToggled(bool enabled);
 
 protected:
     void resizeEvent(QResizeEvent* event) override;
@@ -64,4 +67,5 @@ class CanvasWidget : public QWidget
     QToolButton* m_backgroundMaskButton;
     QToolButton* m_zoneButton;
     QToolButton* m_hdButton;
+    QToolButton* m_rotateButton;
 };