Adding plugin DupFileManager, and updating RenameFile and FileMonitor. (#422)

Author: David-Maisonave

plugins/DupFileManager/DupFileManager.py

@@ -0,0 +1,70 @@
+name: DupFileManager
+description: Manages duplicate files.
+version: 0.1.2
+url: https://github.com/David-Maisonave/Axter-Stash/tree/main/plugins/DupFileManager
+settings:
+  mergeDupFilename:
+    displayName: Merge Duplicate Tags
+    description: Before deletion, merge metadata from duplicate. E.g. Tag names, performers, studios, title, galleries, rating, details, etc...
+    type: BOOLEAN
+  permanentlyDelete:
+    displayName: Permanent Delete
+    description: Enable to permanently delete files, instead of moving files to trash can.
+    type: BOOLEAN
+  whitelistDelDupInSameFolder:
+    displayName: Whitelist Delete In Same Folder
+    description: Allow whitelist deletion of duplicates within the same whitelist folder.
+    type: BOOLEAN
+  whitelistDoTagLowResDup:
+    displayName: Whitelist Duplicate Tagging
+    description: Enable to tag whitelist duplicates of lower resolution or duration or same folder.
+    type: BOOLEAN
+  zCleanAfterDel:
+    displayName: Run Clean After Delete
+    description: After running a 'Delete Duplicates' task, run Clean, Clean-Generated, and Optimize-Database.
+    type: BOOLEAN
+  zSwapHighRes:
+    displayName: Swap High Resolution
+    description: If enabled, swap higher resolution duplicate files to preferred path.
+    type: BOOLEAN
+  zSwapLongLength:
+    displayName: Swap Longer Duration
+    description: If enabled, swap longer duration media files to preferred path. Longer is determine by significantLongerTime field.
+    type: BOOLEAN
+  zWhitelist:
+    displayName: White List
+    description: A comma seperated list of paths NOT to be deleted. E.g. C:\Favorite\,E:\MustKeep\
+    type: STRING
+  zxGraylist:
+    displayName: Gray List
+    description: List of preferential paths to determine which duplicate should be the primary. E.g. C:\2nd_Favorite\,H:\ShouldKeep\
+    type: STRING
+  zyBlacklist:
+    displayName: Black List
+    description: List of LEAST preferential paths to determine primary candidates for deletion. E.g. C:\Downloads\,F:\DeleteMeFirst\
+    type: STRING
+  zyMaxDupToProcess:
+    displayName: Max Dup Process
+    description: Maximum number of duplicates to process. If 0, infinity
+    type: NUMBER
+  zzdebugTracing:
+    displayName: Debug Tracing
+    description: (Default=false) [***For Advanced Users***] Enable debug tracing. When enabled, additional tracing logging is added to Stash\plugins\DupFileManager\DupFileManager.log
+    type: BOOLEAN
+exec:
+  - python
+  - "{pluginDir}/DupFileManager.py"
+interface: raw
+tasks:
+  - name: Tag Duplicates
+    description: Set tag DuplicateMarkForDeletion to the duplicates with lower resolution, duration, file name length, or black list path.
+    defaultArgs:
+      mode: tag_duplicates_task
+  - name: Delete Tagged Duplicates
+    description: Only delete scenes having DuplicateMarkForDeletion tag.
+    defaultArgs:
+      mode: delete_tagged_duplicates_task
+  - name: Delete Duplicates
+    description: Delete duplicate scenes. Performs deletion without first tagging.
+    defaultArgs:
+      mode: delete_duplicates_task

plugins/DupFileManager/DupFileManager.yml

@@ -0,0 +1,26 @@
+# Description: This is a Stash plugin which manages duplicate files.
+# By David Maisonave (aka Axter) Jul-2024 (https://www.axter.com/)
+# Get the latest developers version from following link: https://github.com/David-Maisonave/Axter-Stash/tree/main/plugins/DupFileManager
+config = {
+    # If enabled, adds the primary duplicate path to the scene detail.
+    "addPrimaryDupPathToDetails" : True,
+    # Alternative path to move duplicate files.
+    "dup_path": "", #Example: "C:\\TempDeleteFolder"
+    # The threshold as to what percentage is consider a significant shorter time.
+    "significantTimeDiff" : .90, # 90% threshold
+    # Valued passed to stash API function FindDuplicateScenes.
+    "duration_diff" : 10, # (default=10) A value from 1 to 10.
+    # If enabled, moves destination file to recycle bin before swapping Hi-Res file.
+    "toRecycleBeforeSwap" : True,
+    # Character used to seperate items on the whitelist, blacklist, and graylist
+    "listSeparator" : ",",
+    # Tag used to tag duplicates with lower resolution, duration, and file name length.
+    "DupFileTag" : "DuplicateMarkForDeletion",
+    # Tag name used to tag duplicates in the whitelist. E.g. DuplicateWhitelistFile
+    "DupWhiteListTag" : "DuplicateWhitelistFile",
+    
+    # The following fields are ONLY used when running DupFileManager in script mode
+    "endpoint_Scheme" : "http", # Define endpoint to use when contacting the Stash server
+    "endpoint_Host" : "0.0.0.0", # Define endpoint to use when contacting the Stash server
+    "endpoint_Port" : 9999, # Define endpoint to use when contacting the Stash server
+}

plugins/DupFileManager/DupFileManager_config.py

@@ -0,0 +1,50 @@
+# DupFileManager: Ver 0.1.2 (By David Maisonave)
+
+DupFileManager is a [Stash](https://github.com/stashapp/stash) plugin which manages duplicate file in the Stash system.
+
+### Features
+
+- Can merge potential source in the duplicate file names for tag names, performers, and studios.
+  - Normally when Stash searches the file name for tag names, performers, and studios, it only does so using the primary file.
+- Delete duplicate file task with the following options:
+  - Tasks (Settings->Task->[Plugin Tasks]->DupFileManager)
+    - **Tag Duplicates** - Set tag DuplicateMarkForDeletion to the duplicates with lower resolution, duration, file name length, and/or black list path.
+    - **Delete Tagged Duplicates** - Delete scenes having DuplicateMarkForDeletion tag.
+    - **Delete Duplicates** - Deletes duplicate files. Performs deletion without first tagging.
+  - Plugin UI options (Settings->Plugins->Plugins->[DupFileManager])
+    - Has a 3 tier path selection to determine which duplicates to keep, and which should be candidates for deletions.
+      - **Whitelist** - List of paths NOT to be deleted. 
+        - E.g. C:\Favorite\,E:\MustKeep\
+      - **Gray-List** - List of preferential paths to determine which duplicate should be the primary. 
+        - E.g. C:\2nd_Favorite\,H:\ShouldKeep\
+      - **Blacklist** - List of LEAST preferential paths to determine primary candidates for deletion. 
+        - E.g. C:\Downloads\,F:\DeleteMeFirst\
+    - **Permanent Delete** - Enable to permanently delete files, instead of moving files to trash can.
+    - **Max Dup Process** - Use to limit the maximum files to process. Can be used to do a limited test run.
+    - **Merge Duplicate Tags** - Before deletion, merge metadata from duplicate. E.g. Tag names, performers, studios, title, galleries, rating, details, etc...
+    - **Swap High Resolution** - When enabled, swaps higher resolution files between whitelist and blacklist/graylist files.
+    - **Swap Longer Duration** - When enabled, swaps scene with longer duration.
+  - Options available via DupFileManager_config.py
+    - **dup_path** - Alternate path to move deleted files to. Example: "C:\TempDeleteFolder"
+    - **toRecycleBeforeSwap** - When enabled, moves destination file to recycle bin before swapping files.
+    - **addPrimaryDupPathToDetails** - If enabled, adds the primary duplicate path to the scene detail.
+
+### Requirements
+
+`pip install --upgrade stashapp-tools`
+`pip install pyYAML`
+`pip install Send2Trash`
+
+### Installation
+
+- Follow **Requirements** instructions.
+- In the stash plugin directory (C:\Users\MyUserName\.stash\plugins), create a folder named **DupFileManager**.
+- Copy all the plugin files to this folder.(**C:\Users\MyUserName\\.stash\plugins\DupFileManager**).
+- Click the **[Reload Plugins]** button in Stash->Settings->Plugins->Plugins.
+
+That's it!!!
+
+### Options
+
+- Options are accessible in the GUI via Settings->Plugins->Plugins->[DupFileManager].
+- More options available in DupFileManager_config.py.