Skip to content

Commit 030776a

Browse files
✨ Add initial support for Docker operations and enhance project structure
Other improvements: - Updated utility functions in `utils.py` for better command execution and date handling. - Improved main menu navigation and service checks in `velez.py`. - Replaced `setup.py` with `pyproject.toml` for better build and installation processes. - Updated constant strings to capital case. - Added `.pylintrc` to suppress some warning from external libs. - Added `Makefile` for most important operations. - Added utils function for converting data sizes, printing tables or working with dates.
1 parent 6c7b002 commit 030776a

17 files changed

+1652
-272
lines changed

.pylintrc

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[MASTER]
2+
disable=
3+
C0114, # missing-module-docstring

Makefile

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
.PHONY: clean install install-dev reinstall test
2+
3+
clean:
4+
@rm -rf dist build *.egg-info
5+
6+
install: clean
7+
@python3 -m build --wheel
8+
@pip3 install dist/*.whl
9+
10+
install-dev: clean
11+
@pip3 install -r requirements.txt
12+
13+
reinstall: clean
14+
@python3 -m build --wheel
15+
@pip3 install dist/*.whl --force-reinstall
16+
17+
test:
18+
@pytest -v --tb=short

README.md

Lines changed: 46 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -14,22 +14,33 @@ files before committing!
1414
/>
1515
</a>
1616

17+
18+
## Advantages
19+
20+
- Terragrunt **backend configuration** is read by invoking `terragrunt` command, so it is **always up to date** and resembles the
21+
**real state of the infrastructure**, not just hardcoded values in the code.
22+
- All operations are performed in the **context of the current directory**, so you don't have to worry about running
23+
commands in the wrong environment.
24+
25+
1726
## Disclaimer
1827

19-
This project is in the early development stage and is not ready for production use. It is a work in progress and may
28+
**This project is in the early development stage and is not ready for production use. It is a work in progress and may
2029
contain bugs, incomplete features, incorrect documentation, backward incompatible changes or other issues.
21-
Use it at your own risk.
30+
Use it at your own risk before it reaches 1.x.x version. Read the documentation carefully and test it in a
31+
safe environment before using it.**
2232

2333
Since it operates on the infrastructure, user is responsible for the consequences of the actions taken by the tool and
2434
should review the code before using it.
2535

2636
![Velez](img/velez.jpg)
2737

38+
2839
## Features
2940

3041
Supporting following services/tools and operations on them:
3142

32-
- Terragrunt operations `-tg`:
43+
- Terragrunt operations `-tg` or `--terragrunt`:
3344
- Walk directory structure containing Terragrunt modules.
3445
- Run Plan, Apply, Destroy and Output on a selected module or a specific target.
3546
- Taint and Untaint a resource.
@@ -41,15 +52,19 @@ Supporting following services/tools and operations on them:
4152
- Move a module to a new directory, including moving remote state.
4253
- Destroy resources and backend of the module.
4354
- Destroy backend of the module.
44-
- File operations `-f`:
55+
- File operations `-f` or :
4556
- Formatting all HCL files in the project.
4657
- Cleaning up temporary files in the project or a selected module.
47-
- GitHub operations `-gh`:
58+
- GitHub operations `-gh` or `--github`:
4859
- Source operations, like commit, amend, push, pull or rebase.
4960
- Branch operations, like create, change local or remote, delete local or remote.
5061
- Manage pull requests, like create, list in the repository or the whole organization.
5162
- Manage issues, like create, list in the repository or the whole organization.
5263
- Easily remove stale branches.
64+
- Docker operations `-d` or `--docker`:
65+
- List images in the registry or save them to a file for later use.
66+
- Manage organization members, like invites or removes.
67+
- Manage organization groups.
5368

5469
## Installation
5570

@@ -84,6 +99,14 @@ Framework is written in Python and can be installed as a package.
8499

85100
## Usage
86101

102+
Tool is designed to speed up the work, so it expects some conditions:
103+
* It should be run in the project directory.
104+
* It should be run with the `direnv` or similar solution to load environment variables automatically.
105+
* It expects the project directory to be a Git repository.
106+
* It expects the project directory to be a Terragrunt project with a `root.hcl` (or other defined by `VELEZ_TG_ROOT_HCL`) file.
107+
* It expects the project directory to be a Docker project, with a project directory to be repository name and parent directory to be
108+
organization name or personal account name (excluding `-` characters from the folder name).
109+
87110
### Help
88111

89112
Run the CLI with the `--help` argument to see the available commands:
@@ -107,7 +130,7 @@ velez
107130
#### Terragrunt operations (`-tg` or `--terragrunt`)
108131

109132
```sh
110-
velez --terragrunt <operation> <module> <other-arguments>
133+
velez --terragrunt
111134
```
112135

113136
#### File operations (`-f` or `--file`)
@@ -167,18 +190,23 @@ velez -tg plan aws/dev-account
167190

168191
Velez expects following environment variables to be set:
169192

170-
| Variable | Description | Required for operations | Default |
171-
|-----------------------------------|-------------------------------------------------------------------------------|-------------------------|-----------------------|
172-
| `VELEZ_TG_ROOT_HCL` | Relative path to the Terragrunt configuration file. | Terragrunt | `root.hcl` |
173-
| `VELEZ_TG_TEMP_CONFIG` | Absolute path to a temporary file created to render Terragrunt configuration. | Terragrunt | `/tmp/terragrunt.hcl` |
174-
| `GITHUB_TOKEN` | GitHub token for accessing the GitHub API. | GitHub | `N/A` |
175-
| `VELEZ_GH_STALE_BRANCHES_DAYS` | Number of days after which branches are considered stale. | GitHub | `45` |
176-
| `VELEZ_GH_STALE_BRANCHES_COMMITS` | Number of commits after which branches are considered stale. | GitHub | `30` |
177-
178-
For the convenience, these variables can be set in a `.env` file in the project directory and use the `direnv` (
179-
mentioned above) to load them automatically for every project separately.
180-
181-
Veles will read Terragrunt configuration and expand any dynamic config available statically.
193+
| Variable | Description | Required for operations | Default |
194+
|---------------------------------|---------------------------------------------------------------------------------------------------------------------------------------|-------------------------|-----------------------|
195+
| `VELEZ_TG_ROOT_HCL` | Relative path to the Terragrunt configuration file. | Terragrunt | `root.hcl` |
196+
| `VELEZ_TG_TEMP_CONFIG` | Absolute path to a temporary file created to render Terragrunt configuration. | Terragrunt | `/tmp/terragrunt.hcl` |
197+
| `GITHUB_TOKEN` | GitHub token for accessing the GitHub API. | GitHub | `N/A` |
198+
| `GITHUB_STALE_BRANCHES_DAYS` | Number of days after which branches are considered stale. | GitHub | `45` |
199+
| `GITHUB_STALE_BRANCHES_COMMITS` | Number of commits after which branches are considered stale. | GitHub | `30` |
200+
| `DOCKER_USERNAME` | Docker username for logging in to the Docker registry. | Docker | `N/A` |
201+
| `DOCKER_TOKEN` | Docker personal access token or organization access token for logging in to the Docker registry. | Docker | `N/A` |
202+
| `DOCKER_REPOSITORY` | Default Docker repository for operations. | Docker | current directory |
203+
| `DOCKER_OWNER` | Default Docker owner of repositories.<br>Can be not set or set to Docker username for personal repositories, or to organization name. | Docker | parent directory |
204+
| `DOCKER_STALE_IMAGES_DAYS` | Number of days after which images are considered stale. | Docker | `30` |
205+
206+
For the convenience, these variables can be set in a `.envrc` or similar file in the project directory and use with the `direnv` (
207+
mentioned above; or other similar software) to load them automatically for every project separately and use inheritance for the convenience.
208+
209+
Velez will read Terragrunt configuration to expand any dynamic load available config remotely.
182210
For example for each selected Terragrunt module backed configuration will be read to determine exact values of the S3
183211
bucket and DynamoDB table and key used for locking the state.
184212

pyproject.toml

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
[build-system]
2+
requires = ["setuptools", "wheel"]
3+
build-backend = "setuptools.build_meta"
4+
5+
[project]
6+
name = "velez"
7+
version = "0.3.0"
8+
description = "∀elez is a tool to easily manage complex Terragrunt configurations."
9+
license = {text = "MIT"}
10+
readme = "README.md"
11+
requires-python = ">=3.12"
12+
dependencies = [
13+
"pick",
14+
"boto3",
15+
"python-hcl2",
16+
"PyGithub",
17+
]
18+
authors = [
19+
{name = "Krzysztof Szyper", email = "christoph@shyper.pro"}
20+
]
21+
classifiers = [
22+
"Programming Language :: Python :: 3",
23+
"License :: OSI Approved :: MIT License",
24+
"Operating System :: OS Independent",
25+
"Development Status :: 3 - Alpha",
26+
"Intended Audience :: System Administrators",
27+
"Topic :: Utilities",
28+
]
29+
30+
[project.scripts]
31+
velez = "velez.velez:main"
32+
33+
[project.urls]
34+
"Homepage" = "https://github.com/devops-infra/velez"
35+
"Issues" = "https://github.com/devops-infra/velez/issues"
36+
37+
[tool.setuptools]
38+
packages = ["velez"]

requirements.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,5 @@ setuptools==75.8.2
33
boto3==1.37.7
44
python-hcl2==6.1.1
55
PyGithub==2.6.1
6+
pytest==8.3.4
7+
build==1.2.2

setup.py

Lines changed: 0 additions & 18 deletions
This file was deleted.

tests/test_docker_ops.py

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
import unittest
2+
from velez.docker_ops import DockerOperations
3+
from unittest.mock import patch, MagicMock
4+
5+
class TestDockerOperations(unittest.TestCase):
6+
7+
@patch('docker.from_env')
8+
def setUp(self, mock_docker):
9+
self.mock_velez = MagicMock()
10+
self.docker_ops = DockerOperations(self.mock_velez)
11+
12+
@patch('builtins.input', return_value='username')
13+
@patch('builtins.input', return_value='password')
14+
def test_login(self, mock_input_username, mock_input_password):
15+
self.docker_ops.login()
16+
self.docker_ops.client.login.assert_called_with(username='username', password='password')
17+
18+
def test_list_repositories_menu(self):
19+
with patch('velez.docker_ops.pick', return_value=("List last pushed tags", 0)):
20+
self.docker_ops.list_repositories_menu()
21+
# Add assertions for list_last_pushed_tags
22+
23+
if __name__ == '__main__':
24+
unittest.main()

tests/test_file_ops.py

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
import pytest
2+
from unittest.mock import patch, MagicMock, mock_open
3+
from velez.file_ops import FileOperations
4+
from velez.velez import Velez
5+
6+
@pytest.fixture
7+
def file_ops():
8+
velez = Velez()
9+
return FileOperations(velez)
10+
11+
@patch('velez.file_ops.run_command')
12+
def test_format_hcl_files_tf(mock_run_command, file_ops):
13+
"""Test format_hcl_files with Terraform."""
14+
file_ops.velez.check_terragrunt = MagicMock(return_value=True)
15+
file_ops.velez.get_tf_ot = MagicMock(return_value='terraform')
16+
file_ops.format_hcl_files()
17+
mock_run_command.assert_any_call(['terragrunt', 'hclfmt'])
18+
mock_run_command.assert_any_call(['terraform', 'fmt', '-recursive', '.'])
19+
20+
@patch('velez.file_ops.run_command')
21+
def test_format_hcl_files_ot(mock_run_command, file_ops):
22+
"""Test format_hcl_files with OpenTofu."""
23+
file_ops.velez.check_terragrunt = MagicMock(return_value=True)
24+
file_ops.velez.get_tf_ot = MagicMock(return_value='tofu')
25+
file_ops.format_hcl_files()
26+
mock_run_command.assert_any_call(['terragrunt', 'hclfmt'])
27+
mock_run_command.assert_any_call(['tofu', 'fmt', '-recursive', '.'])
28+
29+
@patch('builtins.input', return_value='') # Mock input to return an empty string
30+
@patch('os.remove')
31+
@patch('shutil.rmtree')
32+
def test_clean_files(mock_rmtree, mock_remove, mock_input, file_ops):
33+
"""Test clean_files."""
34+
with patch('os.walk') as mock_walk:
35+
mock_walk.return_value = [
36+
('/path', ['.terraform'], ['.terraform.lock.hcl', 'tfplan']),
37+
('/path/.terraform', [], []),
38+
]
39+
file_ops.clean_files('/path')
40+
mock_rmtree.assert_any_call('/path/.terraform')
41+
mock_remove.assert_any_call('/path/.terraform.lock.hcl')
42+
mock_remove.assert_any_call('/path/tfplan')
43+
44+
@patch('builtins.open', new_callable=mock_open, read_data='key = "value"')
45+
@patch('velez.file_ops.hcl2.load', return_value={"key": "value"})
46+
def test_load_hcl_file(mock_hcl_load, mock_file):
47+
"""Test load_hcl_file."""
48+
result = FileOperations.load_hcl_file('dummy.hcl')
49+
assert result == {"key": "value"}
50+
mock_file.assert_called_once_with('dummy.hcl', 'r')
51+
mock_hcl_load.assert_called_once()
52+
53+
@patch('builtins.open', new_callable=mock_open, read_data='{"key": "value"}')
54+
def test_load_json_file(mock_file):
55+
"""Test load_json_file."""
56+
result = FileOperations.load_json_file('dummy.json')
57+
assert result == {"key": "value"}
58+
mock_file.assert_called_once_with('dummy.json', 'r')

0 commit comments

Comments
 (0)