|
| 1 | +# Azure PowerShell Repository |
| 2 | + |
| 3 | +Azure PowerShell is a collection of 200+ PowerShell modules for managing Azure resources. The repository contains modules with two types of projects: SDK-based projects and AutoRest-generated projects, all built using .NET and MSBuild. |
| 4 | + |
| 5 | +## Architecture Overview |
| 6 | + |
| 7 | +Azure PowerShell consists of **two main development approaches for projects**: |
| 8 | + |
| 9 | +1. **SDK-based projects** - Hand-written C# cmdlets with custom business logic |
| 10 | +2. **AutoRest-based projects** - Auto-generated from OpenAPI specs via AutoRest PowerShell (mostly newer Azure services) |
| 11 | + |
| 12 | +Always check project type before making changes - SDK vs AutoRest projects have different development patterns. |
| 13 | + |
| 14 | +### Modules vs Projects |
| 15 | +- **Module**: A complete PowerShell module (e.g., `Az.Compute`) that can consist of multiple projects |
| 16 | +- **Project**: Individual C# project within a module, developed with one approach (SDK-based OR AutoRest) |
| 17 | +- **Hybrid Module**: Contains both SDK-based and AutoRest projects (e.g., `Az.Resources` with both approaches) |
| 18 | + |
| 19 | +Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here. |
| 20 | + |
| 21 | +## Working Effectively |
| 22 | + |
| 23 | +### Prerequisites and Setup |
| 24 | +First-time setup requires these exact steps: |
| 25 | +- **CRITICAL**: Ensure you have network connectivity to Azure DevOps package feeds. Build failures with "Name or service not known" errors indicate firewall/connectivity issues that must be resolved before building. |
| 26 | +- Install .NET SDK 8.0+ and .NET Framework Dev Pack 4.7.2+: `dotnet --version` should show 8.0+ |
| 27 | +- Install PowerShell 7+: `pwsh --version` should show 7.0+ |
| 28 | +- Install platyPS module: `pwsh -c "Install-Module -Name platyPS -Force -Scope CurrentUser"` |
| 29 | +- Install PSScriptAnalyzer: `pwsh -c "Install-Module -Name PSScriptAnalyzer -Force -Scope CurrentUser"` |
| 30 | +- Set PowerShell execution policy: `pwsh -c "Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope CurrentUser"` |
| 31 | + |
| 32 | +### Building the Repository |
| 33 | +**NEVER CANCEL** any build commands - they may take 45-60 minutes to complete. |
| 34 | + |
| 35 | +Core build commands (run from repository root): |
| 36 | +- Clean: `dotnet msbuild build.proj /t:Clean` -- takes ~15 seconds |
| 37 | +- Full build: `dotnet msbuild build.proj` -- takes 45-60 minutes. NEVER CANCEL. Set timeout to 90+ minutes. |
| 38 | +- Build specific module: `dotnet msbuild build.proj /p:Scope=Accounts` -- takes 15-30 minutes |
| 39 | +- Generate help: `dotnet msbuild build.proj /t:GenerateHelp` -- takes 10-15 minutes. NEVER CANCEL. Set timeout to 30+ minutes. |
| 40 | +- Static analysis: `dotnet msbuild build.proj /t:StaticAnalysis` -- takes 10-15 minutes. NEVER CANCEL. Set timeout to 30+ minutes. |
| 41 | +- Run tests: `dotnet msbuild build.proj /t:Test` -- takes 15+ minutes. NEVER CANCEL. Set timeout to 45+ minutes. |
| 42 | + |
| 43 | +Alternative PowerShell build commands: |
| 44 | +- Build single module: `pwsh -c "./tools/BuildScripts/BuildModules.ps1 -RepoRoot $(pwd) -Configuration Debug -TargetModule Accounts"` -- takes 5-15 minutes depending on module size |
| 45 | + |
| 46 | +### Testing |
| 47 | +**NEVER CANCEL** test commands - they can take 15+ minutes. |
| 48 | + |
| 49 | +Test execution patterns: |
| 50 | +- All tests: `dotnet msbuild build.proj /t:Test` -- takes 15+ minutes. NEVER CANCEL. |
| 51 | +- Core tests only: `dotnet msbuild build.proj /t:Test /p:TestsToRun=Core` -- tests Compute, Network, Resources, Sql, Websites modules |
| 52 | +- Individual module tests: Navigate to module test directory and run `dotnet test` or use Pester |
| 53 | + |
| 54 | +Test framework uses .NET 6 target framework and Azure Test Framework for recording/playback of HTTP requests. |
| 55 | + |
| 56 | +### Manual Validation Scenarios |
| 57 | +**CRITICAL**: Always perform manual validation after making changes. Simply building is not sufficient. |
| 58 | + |
| 59 | +**Core validation steps:** |
| 60 | +1. Build succeeds: `dotnet msbuild build.proj /p:Scope=YourModule` |
| 61 | +2. Static analysis passes: `dotnet msbuild build.proj /t:StaticAnalysis` |
| 62 | +3. Help generation succeeds: `dotnet msbuild build.proj /t:GenerateHelp` |
| 63 | +4. Tests pass: Navigate to test directory and run tests for your module |
| 64 | + |
| 65 | +**Manual testing scenarios:** |
| 66 | +After building, always test real functionality: |
| 67 | +```powershell |
| 68 | +# Import the built module |
| 69 | +Import-Module ./artifacts/Debug/Az.YourModule/Az.YourModule.psd1 |
| 70 | +
|
| 71 | +# Verify cmdlets are available |
| 72 | +Get-Command -Module Az.YourModule |
| 73 | +
|
| 74 | +# Test a basic cmdlet workflow (example for Compute module) |
| 75 | +# Get-AzVM -ResourceGroupName "test-rg" |
| 76 | +# New-AzVM -Name "test-vm" -ResourceGroupName "test-rg" -Location "East US" -Image "Win2019Datacenter" -Credential (Get-Credential) |
| 77 | +
|
| 78 | +# For AutoRest projects, test generated cmdlets |
| 79 | +# Get-AzQuota -SubscriptionId "your-subscription-id" |
| 80 | +
|
| 81 | +# Verify help is working |
| 82 | +Get-Help Your-AzCmdlet -Examples |
| 83 | +
|
| 84 | +# Test parameter validation and error handling |
| 85 | +Get-AzVM -InvalidParameterName "test" |
| 86 | +``` |
| 87 | + |
| 88 | +**Required pre-commit validation workflow:** |
| 89 | +1. Clean and build your module: `dotnet msbuild build.proj /t:Clean; dotnet msbuild build.proj /p:Scope=YourModule` |
| 90 | +2. Run static analysis: `dotnet msbuild build.proj /t:StaticAnalysis` |
| 91 | +3. Import and manually test your cmdlets using realistic scenarios |
| 92 | +4. Run your module's specific tests: `cd src/YourModule/YourModule.Test; dotnet test` |
| 93 | +5. Update ChangeLog.md files with your changes |
| 94 | + |
| 95 | +## Common Build Issues and Workarounds |
| 96 | + |
| 97 | +### Network Connectivity Issues |
| 98 | +**CRITICAL**: If you see "Failed to download package" or "Name or service not known" errors: |
| 99 | +- This indicates firewall/proxy issues blocking access to Azure DevOps package feeds |
| 100 | +- Affected URLs: `*.vsblob.vsassets.io`, `pkgs.dev.azure.com` |
| 101 | +- Workaround 1: Configure corporate proxy/firewall to allow these domains |
| 102 | +- Workaround 2: Use VPN or different network environment |
| 103 | +- **DO NOT** attempt to modify NuGet.Config - this will break the build |
| 104 | + |
| 105 | +### Incomplete Builds |
| 106 | +If build appears to hang or shows no progress: |
| 107 | +- **DO NOT CANCEL** - builds can take 45+ minutes with periods of no visible progress |
| 108 | +- Monitor system resources - builds are CPU and network intensive |
| 109 | +- Check network connectivity if stuck on package restore |
| 110 | +- Builds may pause during package downloads or complex compilation steps |
| 111 | + |
| 112 | +### Module-Specific Issues |
| 113 | +- Help generation requires platyPS module to be installed and functioning |
| 114 | +- Static analysis requires PSScriptAnalyzer module |
| 115 | +- Some projects depend on specific Azure SDK versions from Azure DevOps feeds |
| 116 | +- Missing ChangeLog.md updates will cause PR validation to fail |
| 117 | + |
| 118 | +### Build Artifacts Issues |
| 119 | +If you see errors like "Cannot find path '/artifacts/Debug'": |
| 120 | +- This means the build didn't complete successfully |
| 121 | +- Run a clean build: `dotnet msbuild build.proj /t:Clean; dotnet msbuild build.proj` |
| 122 | +- Check for network connectivity issues during package restore |
| 123 | + |
| 124 | +### PowerShell Module Import Issues |
| 125 | +If `Import-Module` fails after build: |
| 126 | +- Ensure the build completed successfully and artifacts exist in `/artifacts/Debug/` |
| 127 | +- Try restarting PowerShell session to clear any cached modules |
| 128 | +- Use `-Force` parameter: `Import-Module ./artifacts/Debug/Az.YourModule/Az.YourModule.psd1 -Force` |
| 129 | + |
| 130 | +## Repository Structure and Navigation |
| 131 | + |
| 132 | +### Key Directories |
| 133 | +- `/src/` - All modules, containing both SDK-based projects and AutoRest-based projects |
| 134 | +- `/generated/` - Pure generated code for AutoRest-based projects |
| 135 | +- `/tools/` - Build scripts, static analysis, testing utilities |
| 136 | +- `/documentation/` - Developer guides, design guidelines, testing docs |
| 137 | +- `/artifacts/` - Build outputs (created during build process) |
| 138 | + |
| 139 | +### Project Types |
| 140 | +Two types of projects with different development approaches: |
| 141 | + |
| 142 | +**SDK-based projects** (in `/src/`): |
| 143 | +- Hand-written C# cmdlets using Azure .NET SDKs |
| 144 | +- Example: `/src/Accounts/`, `/src/Compute/` |
| 145 | +- Build using main repository build system |
| 146 | +- Follow patterns in `/documentation/development-docs/azure-powershell-developer-guide.md` |
| 147 | + |
| 148 | +**AutoRest-generated projects** (in `/generated/` and some `/src/`): |
| 149 | +- Generated from REST API specifications |
| 150 | +- Example: `/generated/Cdn/Cdn.Autorest/` |
| 151 | +- Have individual build scripts: `build-module.ps1`, `test-module.ps1`, `pack-module.ps1` |
| 152 | +- Follow patterns in individual module `how-to.md` files |
| 153 | + |
| 154 | +**Hybrid modules**: Some modules contain both SDK-based and AutoRest-based projects, requiring understanding of both approaches. |
| 155 | + |
| 156 | +### Important Files |
| 157 | +- `build.proj` - Main MSBuild file for entire repository |
| 158 | +- `NuGet.Config` - Package source configuration (DO NOT MODIFY) |
| 159 | +- `CONTRIBUTING.md` - Contribution guidelines and PR requirements |
| 160 | +- `ChangeLog.md` - Release notes (must be updated for changes) |
| 161 | + |
| 162 | +### Development Workflow Files |
| 163 | +- `/tools/BuildScripts/BuildModules.ps1` - Core module build automation |
| 164 | +- `/tools/GenerateHelp.ps1` - Help documentation generation |
| 165 | +- `/tools/StaticAnalysis/` - Code analysis and validation tools |
| 166 | +- `/tools/Test/` - Testing infrastructure and utilities |
| 167 | + |
| 168 | +## Typical Development Tasks |
| 169 | + |
| 170 | +### Adding a New Cmdlet |
| 171 | +1. Navigate to appropriate module directory in `/src/` |
| 172 | +2. Add cmdlet class following existing patterns |
| 173 | +3. Update module manifest (`.psd1`) if needed |
| 174 | +4. Build module: `dotnet msbuild build.proj /p:Scope=YourModule` |
| 175 | +5. Generate help: `dotnet msbuild build.proj /t:GenerateHelp` |
| 176 | +6. Add tests following patterns in `ModuleName.Test` directory |
| 177 | +7. Update `ChangeLog.md` with your changes |
| 178 | + |
| 179 | +### Individual Module Development |
| 180 | +For individual module work when you don't need to build the entire repository: |
| 181 | + |
| 182 | +**SDK-based projects** (in `/src/`): |
| 183 | +```bash |
| 184 | +# Build just your module (much faster than full repository build) |
| 185 | +dotnet msbuild build.proj /p:Scope=YourModule |
| 186 | + |
| 187 | +# Build with dependencies (recommended) |
| 188 | +dotnet msbuild build.proj /p:TargetModule=YourModule |
| 189 | +``` |
| 190 | + |
| 191 | +**AutoRest projects** (in `/src/` and `/generated/`): |
| 192 | +```bash |
| 193 | +# Most AutoRest projects are built as part of the main build system |
| 194 | +dotnet msbuild build.proj /p:Scope=YourModule |
| 195 | + |
| 196 | +# Some generated projects have individual test scripts |
| 197 | +cd generated/YourModule/YourModule.Autorest |
| 198 | +./test-module.ps1 -Playback # Run tests in playback mode |
| 199 | +./test-module.ps1 -Record # Record new tests (requires Azure connection) |
| 200 | +``` |
| 201 | + |
| 202 | +**Module testing patterns:** |
| 203 | +```bash |
| 204 | +# Run tests for specific module |
| 205 | +cd src/YourModule/YourModule.Test |
| 206 | +dotnet test |
| 207 | + |
| 208 | +# For Pester-based tests |
| 209 | +cd src/YourModule/YourModule.Test |
| 210 | +pwsh -c "Invoke-Pester" |
| 211 | +``` |
| 212 | + |
| 213 | +### Running Specific Module Tests |
| 214 | +1. Navigate to module test directory: `cd src/YourModule/YourModule.Test` |
| 215 | +2. Set up test environment (see `/documentation/testing-docs/using-azure-test-framework.md`) |
| 216 | +3. Run tests: `dotnet test` or use PowerShell/Pester patterns |
| 217 | +4. Record new tests in "Record" mode, run existing tests in "Playback" mode |
| 218 | + |
| 219 | +### Pre-commit Validation |
| 220 | +Always run before submitting PR: |
| 221 | +1. `dotnet msbuild build.proj /t:Clean` |
| 222 | +2. `dotnet msbuild build.proj /p:Scope=YourModule` -- wait for completion, 15-30 minutes |
| 223 | +3. `dotnet msbuild build.proj /t:StaticAnalysis` -- wait for completion, 10-15 minutes |
| 224 | +4. Test your specific changes manually by importing and using the cmdlets |
| 225 | +5. Update ChangeLog.md with your changes |
| 226 | + |
| 227 | +## Timing Expectations |
| 228 | +**CRITICAL**: All timing estimates include buffers. NEVER CANCEL commands before these timeouts: |
| 229 | + |
| 230 | +| Command | Expected Time | Timeout Setting | |
| 231 | +|---------|---------------|-----------------| |
| 232 | +| Clean build | 15 seconds | 60 seconds | |
| 233 | +| Full repository build | 45-60 minutes | 90 minutes | |
| 234 | +| Single module build | 15-30 minutes | 45 minutes | |
| 235 | +| Help generation | 10-15 minutes | 30 minutes | |
| 236 | +| Static analysis | 10-15 minutes | 30 minutes | |
| 237 | +| Test execution | 15+ minutes | 45 minutes | |
| 238 | + |
| 239 | +**NEVER CANCEL** any build or test command before the timeout period. Builds may show no progress for extended periods while downloading packages or compiling. |
| 240 | + |
| 241 | +## Files to Always Update |
| 242 | +- **ChangeLog.md** - Add entry under "## Upcoming Release" section |
| 243 | +- **Module-specific ChangeLog.md** - Located at `src/YourModule/YourModule/ChangeLog.md` |
| 244 | +- **Help documentation** - Regenerate using help generation commands |
| 245 | +- **Tests** - Add or update tests for new functionality |
| 246 | + |
| 247 | +## Common Command Reference |
| 248 | +```bash |
| 249 | +# Repository setup |
| 250 | +dotnet --version # Check .NET version (need 8.0+) |
| 251 | +pwsh --version # Check PowerShell version (need 7.0+) |
| 252 | + |
| 253 | +# Build commands |
| 254 | +dotnet msbuild build.proj /t:Clean # Clean build |
| 255 | +dotnet msbuild build.proj # Full build (45-60 min) |
| 256 | +dotnet msbuild build.proj /p:Scope=ModuleName # Build specific module |
| 257 | +dotnet msbuild build.proj /t:GenerateHelp # Generate help (10-15 min) |
| 258 | +dotnet msbuild build.proj /t:StaticAnalysis # Run static analysis (10-15 min) |
| 259 | +dotnet msbuild build.proj /t:Test # Run tests (15+ min) |
| 260 | + |
| 261 | +# Module development |
| 262 | +pwsh -c "Import-Module ./artifacts/Debug/Az.ModuleName/Az.ModuleName.psd1" |
| 263 | +pwsh -c "Get-Command -Module Az.ModuleName" |
| 264 | + |
| 265 | +# Individual module builds (for AutoRest projects) |
| 266 | +cd generated/ModuleName/ModuleName.Autorest |
| 267 | +./build-module.ps1 # If available |
| 268 | +./test-module.ps1 # If available |
| 269 | +``` |
| 270 | + |
| 271 | +Remember: This is a large, complex repository with extensive build times. Plan accordingly and never cancel long-running operations. |
0 commit comments