Skip to content

Commit bf43de5

Browse files
authored
docs add Codebase Overview (#1129)
1 parent e934f74 commit bf43de5

File tree

2 files changed

+82
-3
lines changed

2 files changed

+82
-3
lines changed

docs/CodebaseOverview.md

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
## OWASP Nettacker Codebase Overview
2+
OWASP Nettacker is an open‑source, Python‑based framework for automated penetration testing and information gathering. It supports modular tasks such as port scanning, service detection, subdomain enumeration, vulnerability scans, and credential brute forcing, all driven by a unified CLI, REST API, and Web UI.
3+
4+
5+
## Project layout
6+
7+
```
8+
.
9+
├── docs
10+
├── nettacker
11+
│   ├── api
12+
│   ├── core
13+
│   │   ├── lib
14+
│   │   └── utils
15+
│   ├── database
16+
│   ├── lib
17+
│   │   ├── compare_report
18+
│   │   ├── graph
19+
│   │   │   ├── d3_tree_v1
20+
│   │   │   └── d3_tree_v2
21+
│   │   ├── html_log
22+
│   │   ├── icmp
23+
│   │   └── payloads
24+
│   │   ├── User-Agents
25+
│   │   ├── passwords
26+
│   │   └── wordlists
27+
│   ├── locale
28+
│   ├── modules
29+
│   │   ├── brute
30+
│   │   ├── scan
31+
│   │   └── vuln
32+
│   └── web
33+
│   └── static
34+
│   ├── css
35+
│   ├── fonts
36+
│   ├── img
37+
│   │   └── flags
38+
│   │   ├── 1x1
39+
│   │   └── 4x3
40+
│   ├── js
41+
│   └── report
42+
└── tests
43+
├── api
44+
├── core
45+
│   ├── lib
46+
│   └── utils
47+
├── database
48+
└── lib
49+
└── payloads
50+
51+
```
52+
53+
- **Entry point**`nettacker/main.py` creates a `Nettacker` application instance and runs it when invoked via the provided script or CLI
54+
- **Core engine (`nettacker/core`)**
55+
- `app.py` orchestrates scans: parsing arguments, expanding targets (including IP ranges and subdomains), launching multiprocess/multithread modules, and generating reports
56+
- `module.py` loads YAML-defined modules, applies service discovery results, expands payload loops, and dispatches protocol-specific engines in threaded fashion
57+
- `arg_parser.py`, `ip.py`, `messages.py`, and `utils` provide CLI parsing, IP range handling, internationalized messages, and common helpers
58+
- Protocol engines reside in `core/lib` (e.g., HTTP, FTP, SSH, SMTP, socket) and are invoked by modules
59+
- **Modules (`nettacker/modules`)** – Scanning logic is defined declaratively in YAML under three categories (`brute`, `scan`, `vuln`). Each module contains an `info` block and a list of `payloads` that specify library, request parameters, fuzzing rules, and response conditions. Example: `dir_scan` performs directory discovery over HTTP using wordlists and response conditions
60+
- **Database layer (`nettacker/database`)** – Uses SQLAlchemy to interface with SQLite, MySQL, or PostgreSQL for persisting events and reports
61+
- **API & Web UI (`nettacker/api`, `nettacker/web`)** – Flask-based REST API plus static assets enabling web‑based scan management
62+
- **Supporting libraries (`nettacker/lib`)** – Reporting helpers, ICMP tools, graph generation, and payload wordlists
63+
- **Configuration**`config.py` defines default paths, database settings, and runtime options
64+
- **Tests** – The `tests` directory includes unit tests and validation checks; for example, `test_yaml_regexes.py` ensures regex definitions in YAML modules compile correctly
65+
- **Build & dependencies**`pyproject.toml` defines the project as a Poetry package and lists dependencies such as `aiohttp`, `multiprocess`, `paramiko`, and `sqlalchemy`
66+
67+
## Important concepts
68+
- **Modular architecture:** Modules are YAML files; the engine interprets them and runs protocol-specific steps.
69+
- **Target expansion:** Before scanning, the engine normalizes URLs, enumerates IP ranges, resolves subdomains, and runs preliminary checks like ICMP and port scans
70+
- **Service discovery:** Results from `port_scan` feed into subsequent modules, allowing conditional execution based on discovered services. Service discovery can be turned off during scans using `-d` or `--skip-service-discovery` run-time option.
71+
- **Concurrency:** Scans are distributed across processes and threads for performance, with configurable limits per host and module using the `-t` and `-M` runtime options. The requests can be rate-limited using the `-w` option.
72+
73+
## Where to go next
74+
- **Documentation:** Review `docs/Installation.md` and `docs/Usage.md` for setup and basic usage; `docs/Modules.md` explains module types and parameters; `docs/Developers.md` covers contribution guidelines and how to add languages or modules
75+
- **Explore modules:** Study YAML files under `nettacker/modules/*` to see how scans, brute-force checks, and vulnerability tests are structured.
76+
- **Understand protocol engines:** Read files in `nettacker/core/lib/` to learn how HTTP, socket, and other protocol interactions are implemented.
77+
- **Run locally:** Use the CLI (`nettacker`) or Docker instructions in [Installation](Installation.md) and [Usage](Usage.md)
78+
- **Contribute:** Follow the guidelines in `docs/Developers.md` and run `make pre-commit` and `make test` before submitting changes.

docs/Developers.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ ________
1919

2020
These are the guidelines you need to keep in mind while contributing:
2121

22+
* Start by familiarising yourself with the Nettacker Codebase: [Codebase Overview](CodebaseOverview.md)
2223
* Use the automated checks: run `make pre-commit` and `make test`
2324
* Thoroughly test your code locally.
2425
* Be sure to add/update related documentation.
@@ -44,8 +45,8 @@ OWASP Nettacker is using multi-language libraries (default English) to create a
4445

4546
In some cases language library does not exist, you can create a new file and add it to the framework.
4647

47-
* 1- Goto `lib/messages`
48-
* 2- Name your message library e.g. `fa.yaml`
48+
* 1- Goto `nettacker/locale`
49+
* 2- Name your message library in accordance with the ISO two-letter code e.g. `fa.yaml`
4950
* 3- Copy the default language lib (`en.yaml`) and start your translation.
5051
* 4- **Please notice that you should not change the key-value like `scan_started`, `options` and etc. you just need to modify the Values.**
5152

@@ -59,7 +60,7 @@ To contribute to the existing libraries, You may go to `lib/messages` select the
5960

6061
# Contribute to Modules
6162

62-
Modules exist in path `/modules/module_category`. Currently, we have three categories (scan, brute, vuln). if you need to add more just create a directory with a name! To start a new module you should understand what kind of protocol you want to use. The list of protocols and module functionalities are in `core/module_protocols`. To understand how they work read the below example.
63+
Modules exist in path `nettacker/modules/module_category`. Currently, we have three categories (scan, brute, vuln). if you need to add more just create a directory with a name! To start a new module you should understand what kind of protocol you want to use. The list of protocols and module functionalities are in `core/module_protocols`. To understand how they work read the below example.
6364

6465
```yaml
6566
info: # this section is to store information about module

0 commit comments

Comments
 (0)