Skip to content

Commit 12305a2

Browse files
committed
refactoring, switching to integratin tests
1 parent 03fa0ad commit 12305a2

19 files changed

+720
-640
lines changed

.github/workflows/cd.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,8 @@ on:
1515
default: 'false'
1616
type: choice
1717
options:
18-
- 'true'
19-
- 'false'
18+
- 'true'
19+
- 'false'
2020

2121
jobs:
2222
create-release:
@@ -143,7 +143,7 @@ jobs:
143143

144144
publish-to-pypi:
145145
name: Publish to PyPI
146-
needs: [create-release, build-and-upload]
146+
needs: [ create-release, build-and-upload ]
147147
if: github.event.inputs.publish_to_pypi == 'true'
148148
runs-on: ubuntu-latest
149149
steps:
@@ -192,7 +192,7 @@ jobs:
192192
193193
update-main-branch:
194194
name: Update main branch
195-
needs: [publish-to-pypi]
195+
needs: [ publish-to-pypi ]
196196
runs-on: ubuntu-latest
197197
steps:
198198
- name: Checkout main branch

.github/workflows/ci.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,8 @@ jobs:
1111
test:
1212
strategy:
1313
matrix:
14-
os: [ubuntu-latest, ubuntu-20.04, ubuntu-22.04, macos-latest, windows-latest]
15-
python-version: [3.9, '3.10', '3.11']
14+
os: [ ubuntu-latest, ubuntu-20.04, ubuntu-22.04, macos-latest, windows-latest ]
15+
python-version: [ 3.9, '3.10', '3.11' ]
1616

1717
runs-on: ${{ matrix.os }}
1818

@@ -55,7 +55,7 @@ jobs:
5555
runs-on: ubuntu-latest
5656
strategy:
5757
matrix:
58-
python-version: [pypy-3.9]
58+
python-version: [ pypy-3.9 ]
5959

6060
steps:
6161
- name: Checkout Code

README.md

Lines changed: 53 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -1,97 +1,91 @@
11
# TreeMapper
22

3+
A tool for converting directory structures to YAML format, designed for use with Large Language Models (LLMs).
4+
TreeMapper maps your entire codebase into a structured YAML file, making it easy to analyze code, document projects, and
5+
work with AI tools.
6+
37
[![Build Status](https://img.shields.io/github/actions/workflow/status/nikolay-e/TreeMapper/ci.yml)](https://github.com/nikolay-e/TreeMapper/actions)
48
[![PyPI](https://img.shields.io/pypi/v/treemapper)](https://pypi.org/project/treemapper)
59
[![License](https://img.shields.io/github/license/nikolay-e/TreeMapper)](https://github.com/nikolay-e/TreeMapper/blob/main/LICENSE)
610

7-
TreeMapper is a Python tool designed to convert directory structures and file contents into a YAML format, primarily for use with Large Language Models (LLMs). It helps in codebase analysis, project documentation, and interacting with AI tools by providing a readable representation of your project’s structure.
8-
9-
## Key Features
10-
11-
- Converts directory structures into a **YAML format**
12-
- Captures **file contents** and includes them in the output
13-
- Supports `.gitignore` and custom ignore files (`.treemapperignore`)
14-
- Easy-to-use **CLI tool** with flexible options for input and output paths
15-
- Works cross-platform (Linux, macOS, and Windows)
16-
1711
## Installation
1812

19-
TreeMapper requires **Python 3.9 or higher**.
20-
21-
You can install TreeMapper using pip:
13+
Requires Python 3.9+:
2214

2315
```bash
2416
pip install treemapper
2517
```
2618

27-
## Quick Start
19+
## Usage
2820

29-
To quickly generate a YAML representation of the current directory and save it to `output.yaml`, run:
21+
Generate a YAML tree of a directory:
3022

3123
```bash
32-
treemapper . -o output.yaml
33-
```
24+
# Map current directory
25+
treemapper .
3426

35-
## Usage
27+
# Map specific directory
28+
treemapper /path/to/dir
3629

37-
TreeMapper can be run from the command line with the following options:
30+
# Custom output file
31+
treemapper . -o my-tree.yaml
3832

39-
```bash
40-
treemapper [-i IGNORE_FILE] [-o OUTPUT_FILE] [--no-git-ignore] [-v VERBOSITY] [directory_path]
41-
```
33+
# Custom ignore patterns
34+
treemapper . -i ignore.txt
4235

43-
| Option | Description |
44-
|-----------------------|------------------------------------------------------------------|
45-
| `directory_path` | The directory to analyze (default: current directory) |
46-
| `-i, --ignore-file` | Path to a custom ignore file |
47-
| `-o, --output-file` | Path for the output YAML file (default: `./directory_tree.yaml`) |
48-
| `--no-git-ignore` | Disable git-related default ignores |
49-
| `-v, --verbosity` | Set verbosity level (0: ERROR, 1: WARNING, 2: INFO, 3: DEBUG) |
36+
# Disable all default ignores
37+
treemapper . --no-default-ignores
38+
```
5039

51-
### Example Commands
40+
### Options
5241

53-
1. Analyze the current directory, respecting `.gitignore` and `.treemapperignore`:
54-
```bash
55-
python -m treemapper .
5642
```
57-
58-
2. Analyze a specific directory with a custom ignore file:
59-
```bash
60-
python -m treemapper ./my_project -i custom_ignore.txt
43+
treemapper [OPTIONS] [DIRECTORY]
44+
45+
Arguments:
46+
DIRECTORY Directory to analyze (default: current directory)
47+
48+
Options:
49+
-o, --output-file FILE Output YAML file (default: directory_tree.yaml)
50+
-i, --ignore-file FILE Custom ignore patterns file
51+
--no-default-ignores Disable all default ignores
52+
-v, --verbosity [0-3] Logging verbosity (default: 2)
53+
0=ERROR, 1=WARNING, 2=INFO, 3=DEBUG
54+
-h, --help Show this help
6155
```
6256

63-
3. Output the YAML representation to a different file:
64-
```bash
65-
python -m treemapper ./my_project -o project_structure.yaml
66-
```
57+
### Ignore Patterns
58+
59+
By default, TreeMapper ignores:
60+
61+
- The output file itself
62+
- All `.git` directories
63+
- Patterns from `.gitignore` files
64+
- Patterns from `.treemapperignore` file
6765

68-
## Example Output
66+
Use `--no-default-ignores` to disable all default ignores and only use patterns from `-i/--ignore-file`.
67+
68+
### Example Output
6969

7070
```yaml
71-
name: example_directory
71+
name: my-project
7272
type: directory
7373
children:
74-
- name: file1.txt
75-
type: file
76-
content: |
77-
This is the content of file1.txt
78-
- name: subdirectory
74+
- name: src
7975
type: directory
8076
children:
81-
- name: file2.py
77+
- name: main.py
8278
type: file
8379
content: |
84-
def hello_world():
85-
print("Hello, World!")
80+
def main():
81+
print("Hello World")
82+
- name: README.md
83+
type: file
84+
content: |
85+
# My Project
86+
Documentation here...
8687
```
8788
88-
## Configuration
89-
90-
You can use a `.treemapperignore` file in your project directory to exclude specific files or directories from the YAML output. The format is similar to `.gitignore`.
91-
9289
## License
9390
94-
This project is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details.
95-
96-
---
97-
91+
Apache License 2.0 - see [LICENSE](LICENSE) for details.

src/treemapper/cli.py

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
import argparse
2+
import sys
3+
from pathlib import Path
4+
from typing import Tuple
5+
6+
7+
def parse_args() -> Tuple[Path, Path, Path, bool, int]:
8+
"""Parse command line arguments."""
9+
parser = argparse.ArgumentParser(
10+
description="Generate a YAML representation of a directory structure.",
11+
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
12+
13+
parser.add_argument(
14+
"directory",
15+
nargs="?",
16+
default=".",
17+
help="The directory to analyze")
18+
19+
parser.add_argument(
20+
"-i", "--ignore-file",
21+
default=None,
22+
help="Path to the custom ignore file (optional)")
23+
24+
parser.add_argument(
25+
"-o", "--output-file",
26+
default="./directory_tree.yaml",
27+
help="Path to the output YAML file")
28+
29+
parser.add_argument(
30+
"--no-default-ignores",
31+
action="store_true",
32+
help="Disable all default ignores (including .gitignore and .treemapperignore)")
33+
34+
parser.add_argument(
35+
"-v", "--verbosity",
36+
type=int,
37+
choices=range(0, 4),
38+
default=2,
39+
metavar="[0-3]",
40+
help="Set verbosity level (0: ERROR, 1: WARNING, 2: INFO, 3: DEBUG)")
41+
42+
args = parser.parse_args()
43+
44+
root_dir = Path(args.directory).resolve()
45+
if not root_dir.is_dir():
46+
print(f"Error: The path '{root_dir}' is not a valid directory.")
47+
sys.exit(1)
48+
49+
output_file = Path(args.output_file)
50+
if not output_file.is_absolute():
51+
output_file = Path.cwd() / output_file
52+
53+
ignore_file = Path(args.ignore_file) if args.ignore_file else None
54+
55+
return root_dir, ignore_file, output_file, args.no_default_ignores, args.verbosity

src/treemapper/ignore.py

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
import logging
2+
import os
3+
from pathlib import Path
4+
from typing import List, Dict, Tuple
5+
6+
import pathspec
7+
8+
9+
def read_ignore_file(file_path: Path) -> List[str]:
10+
"""Read the ignore patterns from the specified ignore file."""
11+
ignore_patterns = []
12+
if file_path.is_file():
13+
with file_path.open('r') as f:
14+
ignore_patterns = [line.strip() for line in f
15+
if line.strip() and not line.startswith('#')]
16+
logging.info(f"Using ignore patterns from {file_path}")
17+
logging.debug(f"Read ignore patterns from {file_path}: {ignore_patterns}")
18+
return ignore_patterns
19+
20+
21+
def load_pathspec(patterns: List[str], syntax='gitwildmatch') -> pathspec.PathSpec:
22+
"""Load pathspec from a list of patterns."""
23+
spec = pathspec.PathSpec.from_lines(syntax, patterns)
24+
logging.debug(f"Loaded pathspec with patterns: {patterns}")
25+
return spec
26+
27+
28+
def get_ignore_specs(
29+
root_dir: Path,
30+
custom_ignore_file: Path = None,
31+
no_default_ignores: bool = False,
32+
output_file: Path = None
33+
) -> Tuple[pathspec.PathSpec, Dict[Path, pathspec.PathSpec]]:
34+
"""Get combined ignore specs and git ignore specs."""
35+
default_patterns = get_default_patterns(root_dir, no_default_ignores, output_file)
36+
custom_patterns = get_custom_patterns(root_dir, custom_ignore_file)
37+
combined_patterns = custom_patterns if no_default_ignores else default_patterns + custom_patterns
38+
combined_spec = load_pathspec(combined_patterns)
39+
gitignore_specs = get_gitignore_specs(root_dir, no_default_ignores)
40+
41+
return combined_spec, gitignore_specs
42+
43+
def get_default_patterns(root_dir: Path, no_default_ignores: bool, output_file: Path) -> List[str]:
44+
"""Retrieve default ignore patterns."""
45+
if no_default_ignores:
46+
return []
47+
48+
patterns = []
49+
# Add .treemapperignore patterns
50+
treemapper_ignore_file = root_dir / ".treemapperignore"
51+
patterns.extend(read_ignore_file(treemapper_ignore_file))
52+
53+
# Add default git patterns
54+
patterns.extend([".git/", ".git/**"])
55+
56+
# Add the output file to ignore patterns
57+
if output_file:
58+
try:
59+
relative_output = output_file.resolve().relative_to(root_dir.resolve())
60+
patterns.append(str(relative_output))
61+
if str(relative_output.parent) != ".":
62+
patterns.append(str(relative_output.parent) + "/")
63+
except ValueError:
64+
pass # Output file is outside root_dir; no need to add to ignores
65+
66+
return patterns
67+
68+
def get_custom_patterns(root_dir: Path, custom_ignore_file: Path) -> List[str]:
69+
"""Retrieve custom ignore patterns."""
70+
if not custom_ignore_file:
71+
return []
72+
73+
custom_ignore_file = custom_ignore_file if custom_ignore_file.is_absolute() else root_dir / custom_ignore_file
74+
if custom_ignore_file.is_file():
75+
return read_ignore_file(custom_ignore_file)
76+
77+
logging.warning(f"Custom ignore file '{custom_ignore_file}' not found.")
78+
return []
79+
80+
def get_gitignore_specs(root_dir: Path, no_default_ignores: bool) -> Dict[Path, pathspec.PathSpec]:
81+
"""Retrieve gitignore specs for all .gitignore files in the directory."""
82+
if no_default_ignores:
83+
return {}
84+
85+
gitignore_specs = {}
86+
for dirpath, _, filenames in os.walk(root_dir):
87+
if ".gitignore" in filenames:
88+
gitignore_path = Path(dirpath) / ".gitignore"
89+
patterns = read_ignore_file(gitignore_path)
90+
gitignore_specs[Path(dirpath)] = load_pathspec(patterns)
91+
92+
return gitignore_specs
93+
94+
95+
96+
def should_ignore(file_path: str, combined_spec: pathspec.PathSpec) -> bool:
97+
"""Check if a file or directory should be ignored based on combined pathspec."""
98+
paths_to_check = [file_path]
99+
100+
# Add path variations for checking
101+
if file_path.endswith('/'):
102+
paths_to_check.append(file_path)
103+
104+
# Add parent directories with trailing slash
105+
for part in Path(file_path).parents:
106+
if part != Path('.'):
107+
paths_to_check.append(part.as_posix() + '/')
108+
109+
result = any(combined_spec.match_file(path) for path in paths_to_check)
110+
logging.debug(
111+
f"Should ignore '{file_path}': {result} (checking paths: {paths_to_check})")
112+
return result

src/treemapper/logger.py

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
import logging
2+
3+
4+
def setup_logging(verbosity: int) -> None:
5+
"""Configure the logging level based on verbosity."""
6+
level_map = {
7+
0: logging.ERROR,
8+
1: logging.WARNING,
9+
2: logging.INFO,
10+
3: logging.DEBUG
11+
}
12+
level = level_map.get(verbosity, logging.INFO)
13+
14+
logging.basicConfig(
15+
level=level,
16+
format='%(levelname)s: %(message)s'
17+
)

0 commit comments

Comments
 (0)