ADD: new 0.3.0 with rules and maid.json support

Author: fsoft72

CHANGELOG.md

@@ -2,6 +2,13 @@
 
 All notable changes to this project will be documented in this file.
 
+## [0.3.0] - 2024-10-07
+
+- Breaking: removed support for '.maidignore' files, now use 'maid.json' instead.
+- Added support for reading 'maid.json' file.
+- Added support for rules in 'maid.json' file.
+- Updated README.md with new usage instructions.
+
 ## [0.2.1] - 2024-10-07
 
 - Enhanced markdown output with better formatting for filenames.

README.md

@@ -1,6 +1,13 @@
 # MAID - `M`arkdown `AI` `D`oc creator
 
-`maid` is a Python script that aggregates content from directories and files into a single Markdown file. It supports various options for logging, blacklisting files, and reading local `.maidignore` files.
+`maid` is a Python script that aggregates content from directories and files into a single Markdown file.
+It is very powerful and flexible, with support for:
+
+- blacklisting files and directories
+- filtering text files using special `rules` (see below)
+- logging
+
+Configuration can be keept in a `maid.json` file in the root directory of the project and even modified in subdirectories using a `maid.json` file that is only applied to that directory and its subdirectories.
 
 ## Usage
 
@@ -13,7 +20,7 @@
 - `-o`, `--output`: Specify the output Markdown file (default: `_content.md` in the current directory).
 - `--log`: Enable logging to stdout.
 - `--blacklist`: Glob patterns for files or directories to skip (matched against filename only). This option can be used multiple times.
-- `--blacklist-file`: File containing blacklist glob patterns.
+- `--maid-file`: File containing `maid` configuration (default: `maid.json` in the current directory).
 - `--version`: Display the version.
 
 ### Arguments
@@ -46,53 +53,112 @@ To skip certain files or directories:
 ./maid.py -o output.md --blacklist "*.log" --blacklist "__pycache__" src
 ```
 
-### Using a Blacklist File
+### Using a Maid configuration File
 
 To use a blacklist file:
 
 ```bash
-./maid.py -o output.md --blacklist-file blacklist.txt src
+./maid.py -o output.md --maid-file maid-special.json src
 ```
 
-### Reading Global and Local .maidignore Files
+### Reading Global and Local maid.json Files
 
-Every directory scanned by `maid` will be checked for a `.maidignore` file. If found, the patterns in the file will be used to skip files or directories.
+Every directory scanned by `maid` will be checked for a `maid.json` file. If found, the patterns and rules in the file will be used to skip files or directories.
 Patterns are added to the current blacklist, so they can be combined with other blacklist patterns.
 
-At startup, `maid` will look for a global `.maidignore` file in the following locations:
+At startup, `maid` will look for a global `maid.json` file in the following locations:
 
+- current directory.
 - Home directory.
 - `.maid` directory in the home directory.
 - `.local/share/maid` directory in the home directory.
 - `.config/maid` directory in the home directory.
 
 ### Blacklist Patterns
 
-Blacklist patterns can be specified directly via the `--blacklist` option or through a file using the `--blacklist-file` option. Patterns are matched against filenames only.
+Blacklist patterns can be specified directly via the `--blacklist` option or through a file using the `patterns` section in `--maid-file` option. Patterns are matched against filenames only.
 
-### Example Blacklist File
+### Example `maid.json` File
 
-```bash
-*.log
-__pycache__
-.DS_Store
+```json
+{
+  "patterns": ["*.log", "__pycache__", ".DS_Store"],
+  "rules": []
+}
 ```
 
-## Logging
+## Rules definition
 
-If the `--log` option is enabled, `maid` will log its actions to stdout, including which files are being processed and which are being skipped due to blacklist patterns.
+Rules are a powerful way to filter text files. They are defined in the `rules` section of the `maid.json` file and are applied to text files only.
+
+A rule is some special text manipulation that can be applied to a range of lines in a text file.
+The range is defined by a start pattern (the `start` key in the `rule` definition).
+At the moment, the only supported end of the range is the rule inside the `delete` key.
 
-### Local `.maidignore` Files
+Let's see an example of a rule that deletes all lines in a Godot `.tscn` file
 
-If the `--local-maidignore` option is enabled, `maid` will look for a `.maidignore` file in each directory it scans. The `.maidignore` file should contain glob patterns, one per line, for files or directories to skip.
+Original godot `.tscn` file section:
 
-### Example `.maidignore` File
+```ini
+[gd_scene load_steps=98 format=3 uid="uid://clmi4pv7vlgn1"]
+
+[ext_resource type="Script" path="res://player/player_graphics.gd" id="1_ayi7m"]
+[ext_resource type="Texture2D" uid="uid://dhq5k7y6mo74e" path="res://player/assets/spritesheet.png" id="2_bmcav"]
+
+[sub_resource type="AtlasTexture" id="AtlasTexture_nbjsh"]
+atlas = ExtResource("2_bmcav")
+region = Rect2(48, 128, 48, 64)
+
+[sub_resource type="AtlasTexture" id="AtlasTexture_ktvfe"]
+atlas = ExtResource("2_bmcav")
+region = Rect2(48, 128, 48, 64)
+
+[node name="torso" type="AnimatedSprite2D" parent="."]
+position = Vector2(1, -30)
+sprite_frames = SubResource("SpriteFrames_w0qkt")
+animation = &"run"
+frame_progress = 0.410062
+```
 
-```text
-*.tmp
-*.bak
+Suppose you want to remove all sub resources of type `AtlasTexture`, you can write a rule similar to this:
+
+```json
+"rules": [
+        {
+            "pattern": "*.tscn",
+            "name": "subres AtlasTexture",
+            "start": ".sub_resource.*type=.AtlasTexture",
+            "delete": "::empty::",
+            "keep_start": false
+        }
+]
 ```
 
+This rule will delete all lines, starting from the line containing the `sub_resource` declaration with `type="AtlasTexture"` until the first empty line in the file.
+
+### Rule definition
+
+A rule is defined by a dictionary with the following keys:
+
+- `pattern`: A glob pattern to match the file name.
+- `name`: A name for the rule.
+- `start`: A Regular Expression pattern to match the start of the range.
+- `delete`: A Regular Expression pattern to match the end of the range or the special values:
+  - `::empty::`: The first empty line after the start of the range.
+  - `::line::`: the same line as the start of the range.
+- `keep_start`: A boolean value to keep the start line matched in the output.
+
+Rules are applied only if the pattern matches the file name and they are applied in the order they are defined in the `rules` section.
+
+## Logging
+
+If the `--log` option is enabled, `maid` will log its actions to stdout, including which files are being processed and which are being skipped due to blacklist patterns.
+
+### Local `maid.json` Files
+
+`maid` will look for a `maid.json` file in each directory it scans. The `maid.json` file should contain glob patterns in the `patterns` section and text manipultation rules in the `rules` section.
+Patterns and rules in local `maid.json` files are only applied to the directory and its subdirectories.
+
 ## License
 
 This project is licensed under the MIT License. See the LICENSE file for details.