Feature/pylint refactoring (#98)

Author: simonkurtz-MSFT

.github/copilot-instructions.md

@@ -6,7 +6,7 @@ applyTo: "**"
 
 ## Purpose
 
-This instructions file is designed to guide GitHub Copilot's behavior specifically for this repository. It is intended to provide clear, general, and maintainable guidelines for code generation, style, and collaboration. 
+This instructions file is designed to guide GitHub Copilot's behavior specifically for this repository. It is intended to provide clear, general, and maintainable guidelines for code generation, style, and collaboration.
 
 **In case of any conflict, instructions from other individualized or project-specific files (such as `my-copilot.instructions.md`) take precedence over this file.**
 
@@ -24,7 +24,7 @@ In case of any conflicting instructions, the following hierarchy shall apply. If
   1. Individualized instructions (e.g. a developer's or an organization's instruction file(s)), if present
   2. This repository's `.github/.copilot-instructions.md`
   3. General best practices and guidelines from sources such as [Microsoft Learn](https://learn.microsoft.com/docs/)
-    This includes the [Microsoft Cloud Adoption Framework](https://learn.microsoft.com/azure/cloud-adoption-framework/). 
+    This includes the [Microsoft Cloud Adoption Framework](https://learn.microsoft.com/azure/cloud-adoption-framework/).
   4. Official [GitHub Copilot best practices documentation](https://docs.github.com/enterprise-cloud@latest/copilot/using-github-copilot/coding-agent/best-practices-for-using-copilot-to-work-on-tasks)
 
 ## Copilot Personality Behavior
@@ -58,11 +58,11 @@ In case of any conflicting instructions, the following hierarchy shall apply. If
 - `/`: Root directory containing the main files and folders. Bicep configuration is stored in `bicepconfig.json`.
 - The following folders are all at the root level:
     - `assets/`: PlantUML diagrams and images. Static assets such as these should be placed here. Any diagrams should be placed in the /diagrams/src subfolder.
-    - `infrastructure/`: Contains Jupyter notebooks for setting up various API Management infrastructures. When modifying samples, these notebooks should not need to be modified. 
+    - `infrastructure/`: Contains Jupyter notebooks for setting up various API Management infrastructures. When modifying samples, these notebooks should not need to be modified.
     - `samples/`: Various policy and scenario samples that can be applied to the infrastructures.
     - `setup/`: General setup scripts and configurations for the repository and dev environment setup.
     - `shared/`: Shared resources, such as Bicep modules, Python libraries, and other reusable components.
-    - `tests/`: Contains unit tests for Python code and Bicep modules. This folder should contain all tests for all code in the repository. 
+    - `tests/`: Contains unit tests for Python code and Bicep modules. This folder should contain all tests for all code in the repository.
 
 ## Formatting and Style
 
@@ -109,14 +109,14 @@ param resourceSuffix string = uniqueString(subscription().id, resourceGroup().id
 
 - Overall layout of a Bicep file should be:
     - Visible sections of code with the following format should be used:
-        
+
         ```bicep
         // ------------------------------
         //    <SECTION HEADER>
         // ------------------------------
         ```
 
-    - <SECTION HEADER> should be indented three spaces and be in all caps. 
+    - <SECTION HEADER> should be indented three spaces and be in all caps.
     - Section headers should have only two blank lines before and only one blank line after.
     - Top-to-bottom, the following comma-separated section headers should be inserted unless the section is empty:
         - Parameters
@@ -128,19 +128,20 @@ param resourceSuffix string = uniqueString(subscription().id, resourceGroup().id
 ### Python Instructions
 
 - Prefer Python 3.12+ syntax and features unless otherwise specified.
+- Respect the repository's `.pylintrc` file for linting rules. The file is found in the `tests/python/` folder.
 - When inserting a comment to describe a method, insert a blank line after the comment section.
 - Never leave a blank line at the very top of a Python file. The file must start immediately with the module docstring or code. Always remove any leading blank line at the top.
 - Do not have imports such as `from shared.python import Foo`. The /shared/python directory is covered by a root `.env` file. Just use `import Foo` or `from Foo import Bar` as appropriate.
 - After the module docstring, all import statements must come before any section headers (e.g., CONSTANTS, VARIABLES, etc.). Section headers should only appear after the imports. Here is a more explicit example:
-    
+
     ```python
     """
     Module docstring.
     """
 
     import ...
     ...
-    
+
 
     # ------------------------------
     #    CONSTANTS
@@ -150,14 +151,14 @@ param resourceSuffix string = uniqueString(subscription().id, resourceGroup().id
 
 - Overall layout of a Python file should be:
     - Visible sections of code with the following format should be used:
-        
+
         ```python
         # ------------------------------
         #    <SECTION HEADER>
         # ------------------------------
         ```
 
-    - <SECTION HEADER> should be indented three spaces and be in all caps. 
+    - <SECTION HEADER> should be indented three spaces and be in all caps.
     - Section headers should have only two blank lines before and only one blank line after.
     - Top-to-bottom, the following comma-separated section headers should be inserted unless the section is empty:
         - Constants
@@ -173,7 +174,7 @@ param resourceSuffix string = uniqueString(subscription().id, resourceGroup().id
             - Private Methods
             - Public Methods
 
-- Python Docstring/Class Formatting Rule: 
+- Python Docstring/Class Formatting Rule:
     - Always insert a single blank line after a class docstring and before any class attributes or methods.
     - Never place class attributes or decorators on the same line as the docstring. Example:
 
@@ -186,7 +187,7 @@ param resourceSuffix string = uniqueString(subscription().id, resourceGroup().id
             attribute: str
             ...
         ```
-        
+
 ### Jupyter Notebook Instructions
 
 - Use these [configuration settings](https://github.com/microsoft/vscode-jupyter/blob/dd568fde/package.nls.json) as a reference for the VS Code Jupyter extension configuration.
@@ -195,10 +196,10 @@ param resourceSuffix string = uniqueString(subscription().id, resourceGroup().id
 
 - Ensure you verify that all include links are correct and up to date. This link provides a starting point: https://github.com/plantuml-stdlib/Azure-PlantUML/blob/master/AzureSymbols.md
 - Keep diagrams simple. For Azure, include major components, not individual aspects of components. For example, there is no need for individual policies in WAFs or APIs in API Management, Smart Detector Alert Rules, etc.
-- Less is more. Don't be too verbose in the diagrams. 
+- Less is more. Don't be too verbose in the diagrams.
 - Never include subscription IDs, resource group names, or any other sensitive information in the diagrams. That data is not relevant.
 - Don't use the "legend" command if the information is relatively obvious.
 
 ### API Management Policy XML Instructions
 
-- Policies should use camelCase for all variable names.
+- Policies should use camelCase for all variable names.

.gitignore

@@ -28,6 +28,11 @@ labs-in-progress/
 .coverage
 tests/python/htmlcov/
 
+# Pylint reports
+tests/python/pylint/reports/
+tests/python/$JsonReport
+tests/python/$TextReport
+
 shared/bicep/modules/**/*.json
 main.json
 

.vscode/settings.json

@@ -1,4 +1,11 @@
 {
+    // Whitespace and formatting
+    "files.trimTrailingWhitespace": true,
+    "files.insertFinalNewline": true,
+    "files.trimFinalNewlines": true,
+    "editor.renderWhitespace": "trailing",
+
+    // PlantUML
     "plantuml.diagramsRoot": "assets/diagrams/src",
     "plantuml.exportFormat": "svg",
     "plantuml.exportOutDir": "assets/diagrams/out",

README.md

@@ -217,7 +217,35 @@ As you work with this repo, you will likely want to make your own customizations
 
 The repo uses the bicep linter and has rules defined in `bicepconfig.json`. See the [bicep linter documentation][bicep-linter-docs] for details.
 
-**We welcome contributions!** Please consider forking the repo and creating issues and pull requests to share your samples. Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details. Thank you! 
+**We welcome contributions!** Please consider forking the repo and creating issues and pull requests to share your samples. Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details. Thank you!
+
+### 🔍 Code Quality & Linting
+
+The repository uses [pylint][pylint-docs] to maintain Python code quality standards. Configuration is located in `tests/python/.pylintrc`.
+
+#### Running Pylint
+
+**Using the convenience script (recommended):**
+```powershell
+# From tests/python directory
+.\run_pylint.ps1                          # Run with default settings
+.\run_pylint.ps1 -ShowReport              # Include full detailed report
+.\run_pylint.ps1 -Target "../../samples"  # Analyze a different directory
+```
+
+**Manual execution:**
+```powershell
+pylint --rcfile tests/python/.pylintrc shared/python
+```
+
+#### Pylint Reports
+
+All pylint runs generate timestamped reports in `tests/python/pylint/reports/`:
+- **JSON format**: Machine-readable for CI/CD integration
+- **Text format**: Human-readable detailed analysis
+- **Latest symlinks**: `latest.json` and `latest.txt` always point to the most recent run
+
+The script automatically displays a **Top 10 Issues Summary** showing the most frequent code quality issues to help prioritize improvements. 
 
 ### ➕ Adding a Sample
 
@@ -310,6 +338,7 @@ The original author of this project is [Simon Kurtz][simon-kurtz].
 [badge-python-tests]: https://github.com/Azure-Samples/Apim-Samples/actions/workflows/python-tests.yml/badge.svg?branch=main
 [bicep-linter-docs]: https://learn.microsoft.com/azure/azure-resource-manager/bicep/bicep-config-linter
 [houssem-dellai]: https://github.com/HoussemDellai
+[pylint-docs]: https://pylint.pycqa.org/
 [import-troubleshooting]: .devcontainer/IMPORT-TROUBLESHOOTING.md
 [infra-afd-apim-pe]: ./infrastructure/afd-apim-pe
 [infra-apim-aca]: ./infrastructure/apim-aca

infrastructure/afd-apim-pe/create_infrastructure.py

@@ -13,15 +13,15 @@ def create_infrastructure(location: str, index: int, apim_sku: APIM_SKU, no_aca:
     try:
         # Check if infrastructure already exists to determine messaging
         infrastructure_exists = utils.does_resource_group_exist(utils.get_infra_rg_name(utils.INFRASTRUCTURE.AFD_APIM_PE, index))
-        
+
         # Create custom APIs for AFD-APIM-PE with optional Container Apps backends
         custom_apis = _create_afd_specific_apis(not no_aca)
-        
+
         infra = AfdApimAcaInfrastructure(location, index, apim_sku, infra_apis = custom_apis)
         result = infra.deploy_infrastructure(infrastructure_exists)
-        
+
         sys.exit(0 if result.success else 1)
-            
+
     except Exception as e:
         print(f'\n💥 Error: {str(e)}')
         sys.exit(1)
@@ -30,14 +30,14 @@ def create_infrastructure(location: str, index: int, apim_sku: APIM_SKU, no_aca:
 def _create_afd_specific_apis(use_aca: bool = True) -> list[API]:
     """
     Create AFD-APIM-PE specific APIs with optional Container Apps backends.
-    
+
     Args:
         use_aca (bool): Whether to include Azure Container Apps backends. Defaults to true.
-        
+
     Returns:
         list[API]: List of AFD-specific APIs.
     """
-    
+
     # If Container Apps is enabled, create the ACA APIs in APIM
     if use_aca:
         pol_backend          = utils.read_policy_xml(BACKEND_XML_POLICY_PATH)
@@ -58,13 +58,13 @@ def _create_afd_specific_apis(use_aca: bool = True) -> list[API]:
         api_hwaca_pool       = API('hello-world-aca-pool', 'Hello World (ACA Pool)', '/aca-pool', 'This is the ACA API for Backend Pool', pol_aca_backend_pool, [api_hwaca_pool_get])
 
         return [api_hwaca_1, api_hwaca_2, api_hwaca_pool]
-    
+
     return []
 def main():
     """
     Main entry point for command-line usage.
     """
-        
+
     parser = argparse.ArgumentParser(description = 'Create AFD-APIM-PE infrastructure')
     parser.add_argument('--location', default = 'eastus2', help = 'Azure region (default: eastus2)')
     parser.add_argument('--index', type = int, help = 'Infrastructure index')

infrastructure/apim-aca/create_infrastructure.py

@@ -13,15 +13,15 @@ def create_infrastructure(location: str, index: int, apim_sku: APIM_SKU) -> None
     try:
         # Check if infrastructure already exists to determine messaging
         infrastructure_exists = utils.does_resource_group_exist(utils.get_infra_rg_name(utils.INFRASTRUCTURE.APIM_ACA, index))
-        
+
         # Create custom APIs for APIM-ACA with Container Apps backends
         custom_apis = _create_aca_specific_apis()
-        
+
         infra = ApimAcaInfrastructure(location, index, apim_sku, infra_apis = custom_apis)
         result = infra.deploy_infrastructure(infrastructure_exists)
-        
+
         sys.exit(0 if result.success else 1)
-            
+
     except Exception as e:
         print(f'\n💥 Error: {str(e)}')
         sys.exit(1)
@@ -30,11 +30,11 @@ def create_infrastructure(location: str, index: int, apim_sku: APIM_SKU) -> None
 def _create_aca_specific_apis() -> list[API]:
     """
     Create APIM-ACA specific APIs with Container Apps backends.
-    
+
     Returns:
         list[API]: List of ACA-specific APIs.
     """
-    
+
     # Define the APIs with Container Apps backends
     pol_backend          = utils.read_policy_xml(BACKEND_XML_POLICY_PATH)
     pol_aca_backend_1    = pol_backend.format(backend_id = 'aca-backend-1')
@@ -52,18 +52,18 @@ def _create_aca_specific_apis() -> list[API]:
     # API 3: Hello World (ACA Backend Pool)
     api_hwaca_pool_get   = GET_APIOperation('This is a GET for Hello World on ACA Backend Pool')
     api_hwaca_pool       = API('hello-world-aca-pool', 'Hello World (ACA Pool)', '/aca-pool', 'This is the ACA API for Backend Pool', pol_aca_backend_pool, [api_hwaca_pool_get])
-    
+
     return [api_hwaca_1, api_hwaca_2, api_hwaca_pool]
 
 def main():
     """
     Main entry point for command-line usage.
     """
-        
+
     parser = argparse.ArgumentParser(description = 'Create APIM-ACA infrastructure')
     parser.add_argument('--location', default = 'eastus2', help = 'Azure region (default: eastus2)')
     parser.add_argument('--index', type = int, help = 'Infrastructure index')
-    parser.add_argument('--sku', choices = ['Basicv2', 'Standardv2', 'Premiumv2'], default = 'Basicv2', help = 'APIM SKU (default: Basicv2)')    
+    parser.add_argument('--sku', choices = ['Basicv2', 'Standardv2', 'Premiumv2'], default = 'Basicv2', help = 'APIM SKU (default: Basicv2)')
     args = parser.parse_args()
 
     # Convert SKU string to enum using the enum's built-in functionality

infrastructure/simple-apim/create_infrastructure.py

@@ -9,14 +9,14 @@
 import utils
 
 
-def create_infrastructure(location: str, index: int, apim_sku: APIM_SKU) -> None:    
+def create_infrastructure(location: str, index: int, apim_sku: APIM_SKU) -> None:
     try:
         # Check if infrastructure already exists to determine messaging
         infrastructure_exists = utils.does_resource_group_exist(utils.get_infra_rg_name(utils.INFRASTRUCTURE.SIMPLE_APIM, index))
-        
+
         result = SimpleApimInfrastructure(location, index, apim_sku).deploy_infrastructure(infrastructure_exists)
         sys.exit(0 if result.success else 1)
-            
+
     except Exception as e:
         print(f'\n💥 Error: {str(e)}')
         sys.exit(1)
@@ -25,11 +25,11 @@ def main():
     """
     Main entry point for command-line usage.
     """
-        
+
     parser = argparse.ArgumentParser(description = 'Create Simple APIM infrastructure')
     parser.add_argument('--location', default = 'eastus2', help = 'Azure region (default: eastus2)')
     parser.add_argument('--index', type = int, help = 'Infrastructure index')
-    parser.add_argument('--sku', choices = ['Basicv2', 'Standardv2', 'Premiumv2'], default = 'Basicv2', help = 'APIM SKU (default: Basicv2)')    
+    parser.add_argument('--sku', choices = ['Basicv2', 'Standardv2', 'Premiumv2'], default = 'Basicv2', help = 'APIM SKU (default: Basicv2)')
     args = parser.parse_args()
 
     # Convert SKU string to enum using the enum's built-in functionality
@@ -42,4 +42,4 @@ def main():
     create_infrastructure(args.location, args.index, apim_sku)
 
 if __name__ == '__main__':
-    main()
+    main()

requirements.txt

@@ -1,15 +1,19 @@
 # This is a comprehensive requirements.txt file to ensure that one installation covers everything.
 
+# Core dependencies
 requests
 setuptools
 pandas
 matplotlib
 pyjwt
-pytest
-pytest-cov
 azure.storage.blob
 azure.identity
 jupyter
 ipykernel
 notebook
-python-dotenv
+python-dotenv
+
+# Dev tools for linting, formatting, testing, etc.
+pylint
+pytest
+pytest-cov