Skip to content

Commit dacd450

Browse files
thientcthientc
committed
feat: Add custom exceptions for structured error handling in Futag
- Introduced a hierarchy of exceptions in `exceptions.py` to replace `sys.exit()` calls. - Implemented specific exceptions: `InvalidPathError`, `InvalidConfigError`, `BuildError`, `GenerationError`, and `AnalysisError`. docs: Enhance documentation for contributing and architecture - Created `CONTRIBUTING.md` with development setup, code style guidelines, and pull request process. - Added `architecture.md` detailing the three-layer architecture of Futag and data flow. feat: Implement GitHub Actions for testing and syntax checking - Added `python-tests.yml` for running tests on multiple Python versions. - Created `syntax-check.yml` for checking Python syntax across the codebase. refactor: Update template script with improved comments and structure - Enhanced comments in `template-script.py` to clarify usage patterns and parameters. test: Add unit tests for exceptions and generator functionality - Created `test_exceptions.py` to validate the exception hierarchy. - Added tests for `FuzzDataProviderGenerator` and `Fuzzer` classes in their respective test files. - Implemented tests for the `Generator` class to ensure correct type generation methods. chore: Add architecture and Python API documentation - Introduced `python-api.md` for comprehensive reference on the Futag Python API.
1 parent a383cca commit dacd450

34 files changed

+2179
-413
lines changed

‎.github/workflows/python-tests.yml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
name: Python Tests
2+
3+
on:
4+
push:
5+
paths:
6+
- 'src/python/**'
7+
- '.github/workflows/python-tests.yml'
8+
pull_request:
9+
paths:
10+
- 'src/python/**'
11+
12+
jobs:
13+
test:
14+
runs-on: ubuntu-latest
15+
strategy:
16+
matrix:
17+
python-version: ['3.8', '3.10', '3.12']
18+
steps:
19+
- uses: actions/checkout@v4
20+
- uses: actions/setup-python@v5
21+
with:
22+
python-version: ${{ matrix.python-version }}
23+
- name: Install package with test dependencies
24+
run: |
25+
cd src/python/futag-package
26+
pip install -e ".[test]"
27+
- name: Run tests
28+
run: |
29+
cd src/python/futag-package
30+
python -m pytest tests/ -v --tb=short

‎.github/workflows/syntax-check.yml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
name: Syntax Check
2+
3+
on: [push, pull_request]
4+
5+
jobs:
6+
python-syntax:
7+
runs-on: ubuntu-latest
8+
steps:
9+
- uses: actions/checkout@v4
10+
- uses: actions/setup-python@v5
11+
with:
12+
python-version: '3.12'
13+
- name: Check Python syntax
14+
run: |
15+
python -c "
16+
import ast, sys, pathlib
17+
errors = []
18+
for f in pathlib.Path('src/python/futag-package/src/futag').glob('*.py'):
19+
try:
20+
ast.parse(f.read_text())
21+
except SyntaxError as e:
22+
errors.append(f'{f}: {e}')
23+
if errors:
24+
for e in errors: print(e)
25+
sys.exit(1)
26+
print('All Python files: syntax OK')
27+
"

‎CHANGELOG.md‎

Lines changed: 99 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -1,48 +1,90 @@
11

22
# Change Log
33
All notable changes to this project will be documented in this file.
4-
4+
55
The format is based on [Keep a Changelog](http://keepachangelog.com/)
66
and this project adheres to [Semantic Versioning](http://semver.org/).
77

8+
## [3.0.1] - 2025
9+
10+
### Major Refactoring
11+
- Extracted `BaseGenerator` ABC from 5 duplicated generator classes (12,924 → 4,767 lines)
12+
- Created `GeneratorState` dataclass replacing 13 mutable instance variables
13+
- Extracted `BaseFuzzer` from duplicated Fuzzer/NatchFuzzer (1,602 → 891 lines)
14+
- Moved `ContextGenerator` and `NatchGenerator` to separate modules
15+
- `FuzzDataProviderGenerator` reduced from 2,715 to 222 lines
16+
- `BlobStamperGenerator` reduced from 2,693 to 37 lines
17+
18+
### New Features
19+
- Custom exception hierarchy (`futag.exceptions`: FutagError, InvalidPathError, etc.)
20+
- Python `logging` module integration (replaces print statements)
21+
- `GeneratorState.save()`/`restore_from()` for clean recursive backtracking
22+
- GitHub Actions CI (python-tests.yml, syntax-check.yml)
23+
24+
### Bug Fixes
25+
- Fixed null-pointer dereference in FutagConsumerAnalyzer (cfg->size() before null check)
26+
- Fixed memory leak in FutagConsumerAnalyzer (new int() instead of new int[])
27+
- Fixed `param_list` duplication bug in `__save_old_values`
28+
- Fixed 14+ file handle leaks (bare open() → with statements)
29+
- Fixed `_build_ovearall_coverage` typo → `_build_overall_coverage`
30+
- Replaced bubble sort in `sort_callexprs` with `sorted()`
31+
32+
### Documentation
33+
- Created docs/architecture.md, docs/generators.md, docs/checkers.md, docs/python-api.md
34+
- Created CONTRIBUTING.md
35+
- Added comprehensive docstrings and return type hints to all methods
36+
- Added GPL v3 license headers to all Python source files
37+
- Translated template-script.py comments from Russian to English
38+
39+
### C++ Checker Improvements
40+
- Added `MAX_CFG_BLOCKS` and `REPORT_FILENAME_RAND_LEN` constants
41+
- Changed `SmallString<0>` to `SmallString<256>`
42+
- Added Doxygen comments to all checker methods
43+
- Synchronized base files with LLVM 18 variants
44+
845
## 20250824
946
- Add support for Fuzzed Data Provider
1047

11-
## 20220716
12-
- Add modules preprocessor to Futag python-package
13-
- Fix README of Futag python-package
48+
## 20230807
49+
- Optimize ConsumerBuilder
50+
- Add example for context-generation https://github.com/thientc/Futag-tests/tree/main/json-c-contexts
1451

15-
## 20220727
16-
- Add custom-llvm: download and build llvm, clang, compiler-rt
17-
- Fix document
52+
## 20230711
53+
- Support generation fuzz driver for Natch data: https://github.com/thientc/Futag-tests/tree/main/Natch
1854

19-
## 20220801
20-
- Add multi-processing support for compiling
21-
- TODO: Check analysis result befor generating fuzz-driver
55+
## 20230522
56+
- Fix error in generator
57+
- Add generation for pugi::char_t *&
2258

23-
## 20220808
24-
- Fix bug in generator
25-
- Fix for svace analysing
26-
- add first version of fuzzer and result of Fuzzing for Svace
59+
## 20230417
60+
- Add generation for anonymous function
61+
- Fix error in Builder
2762

28-
## 20220811
29-
- Fix bug in generator
30-
- Add pre release package
31-
- Fix document
63+
## 20230320
64+
- Support for context generation
3265

33-
## 20220821
34-
- Fix bug in generator
35-
- Add release package
36-
- Fix document
66+
## 20230305
67+
- Fix error python in Builder
68+
- Fix error python in Generator for wchar_t string
3769

38-
## 20220911
39-
- Add support for fuzz-introspector
40-
- Migrate to llvm-14.0.6
70+
## 20230214
71+
- Add is_simple for 4consummer_analysis_db
72+
- Add CFG and DFC analysis
73+
- Add Fuzzer extra params support
4174

42-
## 20220921
43-
- Add support for Makefile
44-
- Generation for global function of C++ libraries
45-
- Add testing repository: https://github.com/thientc/Futag-tests
75+
## 20221220
76+
- Fix errors while compiling AFL++, return coverage parameters
77+
- Fix Readme
78+
- change LLVM_ENABLE_ZLIB to ON
79+
80+
## 20221107
81+
- And generation for anonymous functions
82+
- Reformat Python classes
83+
- Fix included paths of compiling command
84+
85+
## 20221018
86+
- Add support for C++, generate for constructors and for method of class, which has default constructors
87+
- Tested on FreeImage and Pugixml
4688

4789
## 20221012
4890
- Add support for AFLplusplus
@@ -51,43 +93,38 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
5193
- Add compilition database of building
5294
- Add analysis of headers
5395

54-
## 20221018
55-
- Add support for C++, generate for constructors and for method of class, which has default constructors
56-
- Tested on FreeImage and Pugixml
57-
58-
## 20221107
59-
- And generation for anonymous functions
60-
- Reformat Python classes
61-
- Fix included paths of compiling command
62-
63-
## 20221220
64-
- Fix errors while compiling AFL++, return coverage parameters
65-
- Fix Readme
66-
- change LLVM_ENABLE_ZLIB to ON
96+
## 20220921
97+
- Add support for Makefile
98+
- Generation for global function of C++ libraries
99+
- Add testing repository: https://github.com/thientc/Futag-tests
67100

68-
## 20230214
69-
- Add is_simple for 4consummer_analysis_db
70-
- Add CFG and DFC analysis
71-
- Add Fuzzer extra params support
101+
## 20220911
102+
- Add support for fuzz-introspector
103+
- Migrate to llvm-14.0.6
72104

73-
## 20230320
74-
- Support for context generation
105+
## 20220821
106+
- Fix bug in generator
107+
- Add release package
108+
- Fix document
75109

76-
## 20230305
77-
- Fix error python in Builder
78-
- Fix error python in Generator for wchar_t string
110+
## 20220811
111+
- Fix bug in generator
112+
- Add pre release package
113+
- Fix document
79114

80-
## 20230417
81-
- Add generation for anonymous function
82-
- Fix error in Builder
115+
## 20220808
116+
- Fix bug in generator
117+
- Fix for svace analysing
118+
- add first version of fuzzer and result of Fuzzing for Svace
83119

84-
## 20230522
85-
- Fix error in generator
86-
- Add generation for pugi::char_t *&
120+
## 20220801
121+
- Add multi-processing support for compiling
122+
- TODO: Check analysis result befor generating fuzz-driver
87123

88-
## 20230711
89-
- Support generation fuzz driver for Natch data: https://github.com/thientc/Futag-tests/tree/main/Natch
124+
## 20220727
125+
- Add custom-llvm: download and build llvm, clang, compiler-rt
126+
- Fix document
90127

91-
## 20230807
92-
- Optimize ConsumerBuilder
93-
- Add example for context-generation https://github.com/thientc/Futag-tests/tree/main/json-c-contexts
128+
## 20220716
129+
- Add modules preprocessor to Futag python-package
130+
- Fix README of Futag python-package

‎CONTRIBUTING.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Contributing to Futag
2+
3+
## Development Setup
4+
5+
### Prerequisites
6+
- CMake >= 3.13.4
7+
- GCC >= 7.1.0
8+
- Python >= 3.8
9+
- pip >= 22.1.1
10+
11+
### Building the LLVM toolchain
12+
13+
```bash
14+
cd custom-llvm && ./prepare.sh
15+
cd ../build && ./build.sh
16+
```
17+
18+
### Installing the Python package (development mode)
19+
20+
```bash
21+
cd src/python/futag-package
22+
pip install -e ".[test]"
23+
```
24+
25+
### Running tests
26+
27+
```bash
28+
cd src/python/futag-package
29+
python -m pytest tests/ -v
30+
```
31+
32+
## Code Style
33+
34+
### Python
35+
- Follow PEP 8
36+
- Use Google-style docstrings
37+
- All new methods must have type hints and docstrings
38+
- Use `logging` module instead of `print()`
39+
- Use `with` statements for file I/O
40+
- Raise exceptions from `futag.exceptions` instead of `sys.exit()`
41+
42+
### C++
43+
- Follow LLVM coding style (configured in .clang-format)
44+
- Column limit: 80 characters
45+
- Indent: 4 spaces
46+
47+
## Pull Request Process
48+
49+
1. Create a feature branch from `main`
50+
2. Make your changes with clear commit messages
51+
3. Ensure all Python tests pass
52+
4. Update documentation if APIs change
53+
5. Submit PR with description of changes
54+
55+
## Adding a New Generator Backend
56+
57+
See docs/generators.md for the BaseGenerator pattern. Implement 10 abstract `_gen_*` methods.
58+
59+
## LLVM Version Support
60+
61+
When adding support for a new LLVM version:
62+
1. Create version-specific source files (e.g., FutagAnalyzer19.cpp)
63+
2. Create version-specific CMakeLists (e.g., CMakeLists19.txt)
64+
3. Update Checkers.td with version-specific copy
65+
4. Update build/build.sh version detection
66+
5. The base file should always match the latest supported LLVM version

‎How-to-work-with-Futag.md‎

Lines changed: 56 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -164,4 +164,59 @@ json_file="/path/to/analysis/folder/futag-analysis-result.json" #Путь к ф
164164
g.gen_targets() # генерация фаззинг-оберток
165165
g.compile_targets( # компиляция фаззинг-оберток
166166
True, # генерация Makefile
167-
16) # количество потоков при компиляции
167+
16) # количество потоков при компиляции
168+
169+
## Генераторы фаззинг-оберток
170+
171+
Futag поддерживает несколько вариантов генерации фаззинг-оберток. Все генераторы наследуют от базового класса `BaseGenerator`.
172+
173+
### Generator (стандартный)
174+
Использует `memcpy()` для чтения данных из буфера. Поддерживает C и C++.
175+
176+
```python
177+
from futag.generator import Generator
178+
generator = Generator(futag_llvm_path, library_root)
179+
generator.gen_targets(max_wrappers=10)
180+
generator.compile_targets(workers=4)
181+
```
182+
183+
### FuzzDataProviderGenerator
184+
Использует API `FuzzedDataProvider` из libFuzzer для типобезопасного чтения данных. Только C++.
185+
186+
```python
187+
from futag.fdp_generator import FuzzDataProviderGenerator
188+
generator = FuzzDataProviderGenerator(futag_llvm_path, library_root)
189+
generator.gen_targets(max_wrappers=100)
190+
generator.compile_targets(workers=4, keep_failed=True)
191+
```
192+
193+
### ContextGenerator
194+
Генерирует обёртки на основе контекстов использования библиотеки в потребительских программах.
195+
196+
```python
197+
from futag.generator import ContextGenerator
198+
ctx_gen = ContextGenerator(futag_llvm_path, library_root)
199+
ctx_gen.gen_context()
200+
ctx_gen.compile_targets(keep_failed=True)
201+
```
202+
203+
## Управление логированием
204+
205+
Futag использует модуль `logging` для вывода информации. Настройка уровня логирования:
206+
207+
```python
208+
import logging
209+
logging.basicConfig(level=logging.INFO) # Стандартный вывод
210+
logging.basicConfig(level=logging.DEBUG) # Подробный вывод
211+
logging.basicConfig(level=logging.WARNING) # Тихий режим
212+
```
213+
214+
## Обработка ошибок
215+
216+
Модуль `futag.exceptions` предоставляет иерархию исключений:
217+
- `FutagError` — базовое исключение
218+
- `InvalidPathError` — неверный путь к файлу или директории
219+
- `InvalidConfigError` — неверная конфигурация
220+
- `BuildError` — ошибка сборки библиотеки
221+
- `GenerationError` — ошибка генерации фаззинг-оберток
222+
- `AnalysisError` — ошибка анализа результатов

0 commit comments

Comments
 (0)