Skip to content

Commit 5f085d0

Browse files
CopilotMarenz
Copilot
and
Marenz
committed
Add comprehensive GitHub Copilot instructions for frequenz-api-dispatch
Co-authored-by: Marenz <[email protected]>
1 parent 8c7cfaf commit 5f085d0

File tree

1 file changed

+145
-0
lines changed

1 file changed

+145
-0
lines changed

‎.github/copilot-instructions.md‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
# Frequenz Dispatch API
2+
3+
**ALWAYS FOLLOW THESE INSTRUCTIONS FIRST. Only fallback to additional search and context gathering if the information in these instructions is incomplete or found to be in error.**
4+
5+
Frequenz Dispatch API is a Python gRPC API package that generates Python code from Protocol Buffer (.proto) files. It provides operations for dispatching microgrid components and automation of electricity management.
6+
7+
Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.
8+
9+
## Working Effectively
10+
11+
### Repository Setup and Build Process
12+
- **CRITICAL**: Always update git submodules first:
13+
- `git submodule update --init` -- takes ~5 seconds. NEVER CANCEL.
14+
- Install system dependencies:
15+
- `sudo apt update && sudo apt install -y protobuf-compiler python3-protobuf python3-grpcio python3-pytest` -- takes 2-3 minutes. NEVER CANCEL.
16+
- Generate protobuf Python files (REQUIRED for functionality):
17+
- `protoc --python_out=py/ --proto_path=proto:submodules/frequenz-api-common/proto:submodules/api-common-protos proto/frequenz/api/dispatch/v1/dispatch.proto` -- takes <1 second
18+
- `protoc --python_out=py/ --proto_path=proto:submodules/frequenz-api-common/proto:submodules/api-common-protos $(find submodules/frequenz-api-common/proto -name "*.proto")` -- takes <1 second
19+
- **When grpcio-tools is available**: Generate gRPC stubs:
20+
- `python -m grpc_tools.protoc --grpc_python_out=py/ --proto_path=proto:submodules/frequenz-api-common/proto:submodules/api-common-protos proto/frequenz/api/dispatch/v1/dispatch.proto`
21+
- **Network Limitation**: Package installation via pip often fails due to network timeouts. Use system packages where possible.
22+
23+
### Testing
24+
- Run tests: `PYTHONPATH=py python3 -m pytest pytests/ -v` -- takes <1 second. NEVER CANCEL.
25+
- **LIMITATION**: The dispatch_pb2_grpc module requires grpcio-tools which may not be available due to network limitations
26+
- Basic protobuf functionality works with system packages
27+
- Tests location: `pytests/test_dispatch.py`
28+
- **SUCCESS CRITERIA**: The test_package_import should always pass; test_module_import_components requires gRPC tools
29+
30+
### Build Validation
31+
- **CANNOT BUILD**: Full `python -m build` fails due to network connectivity issues with PyPI
32+
- **Alternative**: Manual protobuf generation works as shown above
33+
- **CANNOT INSTALL**: `pip install -e .` and `pip install .[dev]` fail due to network timeouts
34+
- **WORKAROUND**: Use `PYTHONPATH=py` to add the generated code to Python path
35+
36+
### Development Tools Status
37+
- **AVAILABLE**: protoc (libprotoc 3.21.12), python3-protobuf, python3-grpcio, python3-pytest
38+
- **NOT AVAILABLE due to network issues**: nox, black, isort, pylint, mypy, flake8, mkdocs, grpcio-tools
39+
- **TIMING**: All basic operations (submodule update, protobuf generation, tests) complete in seconds
40+
41+
## Validation
42+
43+
- **CRITICAL LIMITATION**: Full validation cannot be performed due to network connectivity preventing package installation
44+
- **MANUAL VALIDATION**: Test protobuf import with: `PYTHONPATH=py python3 -c "from frequenz.api.dispatch import v1; print('Import successful')"`
45+
- **BASIC TESTING**: Run `PYTHONPATH=py python3 -m pytest pytests/test_dispatch.py::test_package_import -v` to verify package import works
46+
- **CANNOT TEST**: gRPC functionality due to missing grpcio-tools
47+
- **CANNOT VALIDATE**: Full CI pipeline locally due to missing nox and other tools
48+
49+
### Validation Scenarios (when full environment is available)
50+
After making any changes to the codebase, ALWAYS run these validation steps:
51+
1. **Protocol Buffer Validation**: `protolint lint proto` -- ensures proto files follow style guidelines
52+
2. **Python Import Test**: `python -c "from frequenz.api.dispatch.v1 import dispatch_pb2, dispatch_pb2_grpc; print('Both imports successful')"` -- verifies generated code works
53+
3. **Full Test Suite**: `nox -s pytest_min` -- runs all unit tests. Takes 1-2 minutes. NEVER CANCEL.
54+
4. **Linting Validation**: `nox -s ci_checks_max` -- runs black, isort, pylint, mypy. Takes 3-5 minutes. NEVER CANCEL.
55+
5. **Documentation Build**: `mkdocs build` -- ensures documentation can be generated without errors
56+
6. **Package Build**: `python -m build` -- creates source and wheel distributions. Takes 1-3 minutes. NEVER CANCEL.
57+
58+
### Manual Testing Scenarios
59+
Test actual functionality by exercising the API:
60+
- Import and inspect the generated protobuf messages: `python -c "from frequenz.api.dispatch.v1.dispatch_pb2 import *; print(dir())"`
61+
- Verify protobuf serialization works: Create a dispatch message, serialize it, and deserialize it
62+
- **ALWAYS** test import after regenerating protobuf files
63+
64+
## Common Tasks
65+
66+
### Repository Structure
67+
```
68+
/home/runner/work/frequenz-api-dispatch/frequenz-api-dispatch/
69+
├── .github/ # GitHub workflows and templates
70+
│ └── workflows/ci.yaml # Main CI pipeline
71+
├── proto/ # Protocol buffer source files
72+
│ └── frequenz/api/dispatch/v1/dispatch.proto
73+
├── py/ # Generated Python code location
74+
│ └── frequenz/api/dispatch/ # Generated modules
75+
├── submodules/ # Git submodules
76+
│ ├── api-common-protos/ # Google common protobuf types
77+
│ └── frequenz-api-common/ # Frequenz common API definitions
78+
├── pytests/ # Test files
79+
├── docs/ # Documentation source
80+
├── pyproject.toml # Project configuration
81+
├── noxfile.py # Nox session configuration (needs network)
82+
└── mkdocs.yml # Documentation configuration
83+
```
84+
85+
### Key Files Content
86+
87+
#### pyproject.toml (main project config)
88+
- Python 3.11+ required
89+
- Uses setuptools, setuptools_scm, frequenz-repo-config for build
90+
- Dependencies: frequenz-api-common, googleapis-common-protos, protobuf, grpcio
91+
- Development tools: black, isort, pylint, mypy, pytest, mkdocs
92+
93+
#### noxfile.py
94+
```python
95+
from frequenz.repo.config import RepositoryType, nox
96+
nox.configure(RepositoryType.API)
97+
```
98+
99+
#### .github/workflows/ci.yaml
100+
- Runs protolint on proto files
101+
- Uses nox for testing with Python 3.11 and 3.12
102+
- Tests on amd64 and arm architectures
103+
- Builds and tests package installation
104+
- Publishes documentation to GitHub Pages
105+
- Creates GitHub releases for tags
106+
107+
### Expected Workflow (when network is available)
108+
1. `git submodule update --init` -- 5 seconds. NEVER CANCEL.
109+
2. `python -m pip install .[dev]` -- includes all dev dependencies. Can take 2-5 minutes due to package compilation. NEVER CANCEL.
110+
3. `python -m build` -- build distribution packages. Takes 1-3 minutes. NEVER CANCEL.
111+
4. `nox` -- run all tests and checks. Takes 10-20 minutes total. NEVER CANCEL.
112+
- `nox -s ci_checks_max` -- linting and type checking. Takes 3-5 minutes.
113+
- `nox -s pytest_min` -- run tests. Takes 1-2 minutes.
114+
5. `mkdocs serve` -- preview documentation. Starts in 30-60 seconds.
115+
6. `protolint lint proto` -- lint protocol buffer files. Takes <10 seconds.
116+
- Install: `curl -LO https://github.com/yoheimuta/protolint/releases/download/v0.53.0/protolint_0.53.0_Linux_x86_64.tar.gz && tar -xzf protolint_0.53.0_Linux_x86_64.tar.gz && sudo mv protolint /usr/local/bin/`
117+
118+
### Network-Limited Workaround Workflow
119+
1. `git submodule update --init` -- 5 seconds
120+
2. `sudo apt install -y protobuf-compiler python3-protobuf python3-grpcio python3-pytest`
121+
3. Generate protobuf files manually (see commands above)
122+
4. `PYTHONPATH=py python3 -m pytest pytests/ -v` -- <1 second
123+
5. **NOTE**: Many development tools unavailable, use basic Python syntax checking
124+
125+
### Important Notes
126+
- **NEVER CANCEL**: In normal circumstances, `nox` can take 15+ minutes, `python -m build` can take 2+ minutes
127+
- **TIMING ESTIMATES**: With network access, expect 15-30 minutes for full build and test cycle
128+
- **FIREWALL LIMITATION**: Current environment has PyPI connectivity issues preventing normal pip operations
129+
- **CRITICAL**: Always run `git submodule update --init` before any other operations
130+
- **GENERATED CODE**: The py/ directory contains generated Python code from protobuf files
131+
- **TEST DEPENDENCIES**: Basic tests work with system packages, full test suite requires pip-installed dependencies
132+
133+
### Proto File Workflow
134+
- Protocol buffer definitions are in `proto/frequenz/api/dispatch/v1/dispatch.proto`
135+
- Common types come from submodules (frequenz-api-common, api-common-protos)
136+
- Use `protolint lint proto` to validate proto files (when protolint is available)
137+
- Generated Python files go in `py/frequenz/api/dispatch/v1/`
138+
139+
### Development Limitations in Current Environment
140+
- Cannot run full linting (black, isort, pylint, mypy)
141+
- Cannot build documentation (mkdocs)
142+
- Cannot run nox sessions
143+
- Cannot install packages via pip due to network timeouts
144+
- Cannot generate gRPC stubs (requires grpcio-tools)
145+
- **WORKAROUND**: Use system packages and manual protobuf generation for basic functionality

0 commit comments

Comments
 (0)