|
| 1 | +``` |
| 2 | + ____ __ __ |
| 3 | +| _ \ ___ _ __ ___ | \/ | __ _ _ __ |
| 4 | +| |_) / _ \ '_ \ / _ \| |\/| |/ _` | '_ \ |
| 5 | +| _ < __/ |_) | (_) | | | | (_| | |_) | |
| 6 | +|_| \_\___| .__/ \___/|_| |_|\__,_| .__/ |
| 7 | + |_| |_| |
| 8 | +``` |
| 9 | + |
| 10 | +A Python CLI and library that scans repositories for cross-file references in YAML, JSON, and other configuration files, building a dependency graph that can be exported in multiple formats. |
| 11 | + |
| 12 | +## Features |
| 13 | + |
| 14 | +- **File Discovery**: Recursively scan directories for configuration files (YAML, JSON, TOML) |
| 15 | +- **Reference Detection**: Automatically detect file path references in configuration files |
| 16 | +- **Multiple Export Formats**: |
| 17 | + - **ASCII** (default): Tree-style output for terminal display, scripting, and CI pipelines |
| 18 | + - **Mermaid**: Flowchart syntax for visualization and documentation |
| 19 | + - **JSON**: Machine-readable format for automation and integration |
| 20 | + |
| 21 | +## Installation |
| 22 | + |
| 23 | +```bash |
| 24 | +# Clone the repository |
| 25 | +git clone https://github.com/SamPlaysKeys/Repository-Mapper.git |
| 26 | +cd Repository-Mapper |
| 27 | + |
| 28 | +# Option 1: Install as a package (recommended) |
| 29 | +pip install -e . |
| 30 | + |
| 31 | +# Option 2: Install dependencies only (run via python cli.py) |
| 32 | +pip install -r requirements.txt |
| 33 | +``` |
| 34 | + |
| 35 | +## Usage |
| 36 | + |
| 37 | +After installing as a package (`pip install -e .`), use the `repomap` command. If you only installed dependencies, use `python cli.py` instead. |
| 38 | + |
| 39 | +### Basic Usage |
| 40 | + |
| 41 | +```bash |
| 42 | +# Scan current directory, output ASCII tree |
| 43 | +repomap . |
| 44 | +# or: python cli.py . |
| 45 | + |
| 46 | +# Scan a specific directory |
| 47 | +repomap ./config |
| 48 | + |
| 49 | +# Output to a file |
| 50 | +repomap . -o graph.txt |
| 51 | +``` |
| 52 | + |
| 53 | +### Output Formats |
| 54 | + |
| 55 | +#### ASCII Tree (Default) |
| 56 | + |
| 57 | +```bash |
| 58 | +repomap . -f ascii |
| 59 | +``` |
| 60 | + |
| 61 | +Output: |
| 62 | +``` |
| 63 | +config/app.yaml |
| 64 | +├── data/users.json |
| 65 | +│ └── schemas/user.json |
| 66 | +└── config/local.yaml |
| 67 | + └── data/users.json [*] |
| 68 | +``` |
| 69 | + |
| 70 | +The `[*]` marker indicates a node that was already visited (cycle detection). |
| 71 | + |
| 72 | +Use `--ascii-style=ascii` for pure ASCII output (no Unicode characters): |
| 73 | + |
| 74 | +```bash |
| 75 | +repomap . -f ascii --ascii-style=ascii |
| 76 | +``` |
| 77 | + |
| 78 | +Output: |
| 79 | +``` |
| 80 | +config/app.yaml |
| 81 | +|-- data/users.json |
| 82 | +| \-- schemas/user.json |
| 83 | +\-- config/local.yaml |
| 84 | + \-- data/users.json [*] |
| 85 | +``` |
| 86 | + |
| 87 | +#### Mermaid Flowchart |
| 88 | + |
| 89 | +```bash |
| 90 | +repomap . -f mermaid |
| 91 | +``` |
| 92 | + |
| 93 | +Output: |
| 94 | +```mermaid |
| 95 | +flowchart LR |
| 96 | + config_app_yaml["config/app.yaml"] |
| 97 | + data_users_json["data/users.json"] |
| 98 | + schemas_user_json["schemas/user.json"] |
| 99 | +
|
| 100 | + config_app_yaml --> data_users_json |
| 101 | + data_users_json --> schemas_user_json |
| 102 | +``` |
| 103 | + |
| 104 | +Options: |
| 105 | +- `--orientation`: Set flowchart direction (LR, TD, TB, RL, BT) |
| 106 | +- `--group-by-dir`: Group nodes by top-level directory using subgraphs |
| 107 | + |
| 108 | +#### JSON |
| 109 | + |
| 110 | +```bash |
| 111 | +repomap . -f json |
| 112 | +``` |
| 113 | + |
| 114 | +Output: |
| 115 | +```json |
| 116 | +{ |
| 117 | + "nodes": ["config/app.yaml", "data/users.json", "schemas/user.json"], |
| 118 | + "edges": [["config/app.yaml", "data/users.json"], ["data/users.json", "schemas/user.json"]] |
| 119 | +} |
| 120 | +``` |
| 121 | + |
| 122 | +### Scanning Options |
| 123 | + |
| 124 | +```bash |
| 125 | +# Only scan YAML files |
| 126 | +repomap . --include-ext .yaml .yml |
| 127 | + |
| 128 | +# Exclude additional directories |
| 129 | +repomap . --exclude-dir vendor third_party |
| 130 | + |
| 131 | +# Limit scan depth |
| 132 | +repomap . --max-depth 3 |
| 133 | + |
| 134 | +# Change base path for relative path display |
| 135 | +repomap ./project/config --relative-to ./project |
| 136 | + |
| 137 | +# Hide unresolved file references from output |
| 138 | +repomap . --ignore-missing |
| 139 | + |
| 140 | +# Hide remote (URL) references from output |
| 141 | +repomap . --ignore-remote |
| 142 | +``` |
| 143 | + |
| 144 | +### Full CLI Reference |
| 145 | + |
| 146 | +``` |
| 147 | +usage: repomap [-h] [-o OUTPUT] [-f {ascii,mermaid,json}] |
| 148 | + [--orientation {LR,TD,TB,RL,BT}] [--group-by-dir] |
| 149 | + [--ascii-style {tree,ascii}] [--include-ext INCLUDE_EXT [INCLUDE_EXT ...]] |
| 150 | + [--exclude-dir EXCLUDE_DIR [EXCLUDE_DIR ...]] [--max-depth MAX_DEPTH] |
| 151 | + [--relative-to RELATIVE_TO] [--ignore-missing] [--ignore-remote] |
| 152 | + [root] |
| 153 | +
|
| 154 | +Scan a repository for cross-file references and generate dependency graphs. |
| 155 | +
|
| 156 | +positional arguments: |
| 157 | + root Repository root directory (default: current directory) |
| 158 | +
|
| 159 | +options: |
| 160 | + -h, --help show this help message and exit |
| 161 | + -o OUTPUT, --output OUTPUT |
| 162 | + Output file (default: stdout) |
| 163 | + -f {ascii,mermaid,json}, --format {ascii,mermaid,json} |
| 164 | + Output format (default: ascii) |
| 165 | + --orientation {LR,TD,TB,RL,BT} |
| 166 | + Mermaid flowchart orientation (default: LR) |
| 167 | + --group-by-dir Group nodes by top-level directory in Mermaid output |
| 168 | + --ascii-style {tree,ascii} |
| 169 | + ASCII output style: 'tree' (Unicode) or 'ascii' (pure ASCII) |
| 170 | + --include-ext INCLUDE_EXT [INCLUDE_EXT ...] |
| 171 | + File extensions to include (e.g., .yaml .json) |
| 172 | + --exclude-dir EXCLUDE_DIR [EXCLUDE_DIR ...] |
| 173 | + Directory names to exclude |
| 174 | + --max-depth MAX_DEPTH |
| 175 | + Maximum directory depth to scan |
| 176 | + --relative-to RELATIVE_TO |
| 177 | + Base path for relative path display |
| 178 | + --ignore-missing Hide missing (unresolved) file references from output |
| 179 | + --ignore-remote Hide remote (URL) references from output |
| 180 | +``` |
| 181 | + |
| 182 | +## Library Usage |
| 183 | + |
| 184 | +```python |
| 185 | +from pathlib import Path |
| 186 | +from scanner import build_graph |
| 187 | +from exporters import to_ascii, to_mermaid, to_json |
| 188 | + |
| 189 | +# Build the reference graph |
| 190 | +graph = build_graph( |
| 191 | + root=Path("./my-repo"), |
| 192 | + include_ext={".yaml", ".json"}, |
| 193 | + max_depth=5, |
| 194 | +) |
| 195 | + |
| 196 | +# Export to different formats |
| 197 | +ascii_output = to_ascii(graph, root=Path("./my-repo")) |
| 198 | +mermaid_output = to_mermaid(graph, root=Path("./my-repo"), orientation="TD") |
| 199 | +json_output = to_json(graph, root=Path("./my-repo")) |
| 200 | + |
| 201 | +# Work with the graph directly |
| 202 | +print(f"Found {len(graph)} files") |
| 203 | +print(f"Root files: {graph.get_roots()}") |
| 204 | + |
| 205 | +for source, target in graph.iter_edges(): |
| 206 | + print(f"{source} -> {target}") |
| 207 | +``` |
| 208 | + |
| 209 | +## UX Expectations for Automation |
| 210 | + |
| 211 | +### ASCII Format (`--format=ascii`) |
| 212 | + |
| 213 | +- **Deterministic**: Same input produces same output order |
| 214 | +- **Stable**: Suitable for diffing in CI pipelines |
| 215 | +- **Human-readable**: Easy to scan visually |
| 216 | + |
| 217 | +### Mermaid Format (`--format=mermaid`) |
| 218 | + |
| 219 | +- **Visualization**: Copy output into Mermaid-compatible tools |
| 220 | +- **Documentation**: Embed in Markdown files with ```mermaid code blocks |
| 221 | +- **Interactive**: Use with Mermaid Live Editor for exploration |
| 222 | + |
| 223 | +### JSON Format (`--format=json`) |
| 224 | + |
| 225 | +- **Machine-readable**: Easy to parse programmatically |
| 226 | +- **Integration**: Feed into other tools or scripts |
| 227 | +- **Automation**: Use in CI/CD pipelines for analysis |
| 228 | + |
| 229 | +## Development |
| 230 | + |
| 231 | +### Running Tests |
| 232 | + |
| 233 | +```bash |
| 234 | +# Install with dev dependencies |
| 235 | +pip install -e ".[dev]" |
| 236 | + |
| 237 | +# Run tests |
| 238 | +pytest tests/ -v |
| 239 | + |
| 240 | +# Run with coverage |
| 241 | +pytest tests/ --cov=. --cov-report=html |
| 242 | +``` |
| 243 | + |
| 244 | +### Project Structure |
| 245 | + |
| 246 | +``` |
| 247 | +repository-mapper/ |
| 248 | +├── cli.py # CLI entry point |
| 249 | +├── graph/ |
| 250 | +│ ├── __init__.py |
| 251 | +│ └── model.py # ReferenceGraph data model |
| 252 | +├── scanner/ |
| 253 | +│ ├── __init__.py |
| 254 | +│ ├── discovery.py # File discovery |
| 255 | +│ ├── parser.py # File parsing and path extraction |
| 256 | +│ ├── resolver.py # Path resolution |
| 257 | +│ └── builder.py # Graph builder |
| 258 | +├── exporters/ |
| 259 | +│ ├── __init__.py |
| 260 | +│ ├── ascii_exporter.py |
| 261 | +│ ├── mermaid_exporter.py |
| 262 | +│ └── json_exporter.py |
| 263 | +├── tests/ |
| 264 | +│ ├── test_graph.py |
| 265 | +│ ├── test_exporters.py |
| 266 | +│ └── test_scanner.py |
| 267 | +├── requirements.txt |
| 268 | +└── README.md |
| 269 | +``` |
| 270 | + |
| 271 | +## Acknowledgments |
| 272 | + |
| 273 | +This project was developed with the assistance of AI coding tools. AI was used to help write code, add references, and generate documentation comments throughout the project. |
| 274 | + |
| 275 | +## License |
| 276 | + |
| 277 | +This file is part of Repository Mapper. |
| 278 | + |
| 279 | +Repository Mapper is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. |
| 280 | + |
| 281 | +Repository Mapper is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. |
| 282 | + |
| 283 | +You should have received a copy of the GNU General Public License along with Repository Mapper. If not, see <https://www.gnu.org/licenses/>. |
0 commit comments