Skip to content

Enhanced GitHub Copilot instructions with comprehensive validation, timeouts, and user scenarios #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aurelianware merged 2 commits into from
Sep 22, 2025
Merged

Enhanced GitHub Copilot instructions with comprehensive validation, timeouts, and user scenarios #8

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bf4dcb1
Initial plan
Copilot Sep 22, 2025
3f34fe6
Enhanced GitHub Copilot instructions with NEVER CANCEL warnings, time…
Copilot Sep 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
227 changes: 205 additions & 22 deletions .github/copilot-instructions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Copilot Instructions for HIPAA Attachments Repository

**ALWAYS follow these instructions first and only fallback to search or additional context gathering if the information here is incomplete or found to be in error.**

## Repository Overview

This repository implements a **HIPAA 275 Attachments Processing System** using Azure Logic Apps Standard and Infrastructure-as-Code. The solution processes electronic attachments from Availity SFTP, archives them in Azure Data Lake Storage Gen2 with date-based partitioning, decodes X12 275 messages, and integrates with QNXT claims system via APIs.
Expand All @@ -23,6 +25,49 @@ This repository implements a **HIPAA 275 Attachments Processing System** using A

Both workflows use API connections for SFTP, Blob Storage, Service Bus, and Integration Account.

## Working Effectively

**Bootstrap, validate, and deploy the repository - NEVER CANCEL these operations:**

### Essential Commands (Run in Order)
```bash
# 1. Validate prerequisites - TIMEOUT: 60 seconds
az --version # Azure CLI 2.77.0+
az bicep version # Bicep CLI 0.37.4+
jq --version # jq 1.7+
pwsh --version # PowerShell 7.4+

# 2. Run complete validation pipeline - TIMEOUT: 300 seconds (5 minutes)
# This is your primary build command - validates all components
find "logicapps/workflows" -type f -name "workflow.json" -exec jq . {} \; >/dev/null
az bicep build --file "infra/main.bicep" --outfile /tmp/arm.json
cd logicapps && zip -r ../workflows.zip workflows/ && cd ..

# 3. Test individual components - TIMEOUT: 180 seconds each
az bicep build --file "infra/main.bicep" --outfile /tmp/test.json # Bicep compilation test
find logicapps/workflows -name "workflow.json" -exec jq . {} \; # JSON validation test

# 4. PowerShell script execution - TIMEOUT: 60 seconds
pwsh -ExecutionPolicy Bypass -File fix_repo_structure.ps1 # Structure normalization
pwsh -ExecutionPolicy Bypass -File bootstrap_repo.ps1 <params> # Repository scaffolding (requires parameters)
```

**Expected Times (NEVER CANCEL before these minimums):**
- Prerequisites check: 5 seconds
- Complete validation: 2-6 minutes
- Bicep compilation: 4-15 seconds
- JSON validation: 1-5 seconds
- PowerShell scripts: 5-30 seconds

### Deployment via GitHub Actions
```bash
# Deploy using workflow dispatch (manual trigger)
# Go to: GitHub repo → Actions → "Deploy HIPAA Attachments (Matrix: DEV/UAT/PROD + DEV Lint)"
# NEVER CANCEL: Total deployment time 15-30 minutes for all environments
# Per environment: 5-8 minutes
# Lint stage: 30 seconds
```

## Build and Validation Commands

### Prerequisites
Expand All @@ -36,9 +81,9 @@ pwsh --version # PowerShell 7.4+
```

### JSON Workflow Validation
**Always run before making workflow changes:**
**Always run before making workflow changes - NEVER CANCEL:**
```bash
# Validate JSON syntax and structure
# Validate JSON syntax and structure - TIMEOUT: 60 seconds
find "logicapps/workflows" -type f -name "workflow.json" -print0 | \
while IFS= read -r -d '' f; do
echo "Checking $f"
Expand All @@ -53,24 +98,24 @@ while IFS= read -r -d '' f; do
echo "✓ Valid workflow JSON: $f"
done
```
**Expected time:** < 5 seconds
**Expected time:** < 5 seconds - NEVER CANCEL, set timeout to 60+ seconds
**Must pass:** This validation is mandatory before any deployment

### Bicep Infrastructure Validation
**Always run before infrastructure changes:**
**Always run before infrastructure changes - NEVER CANCEL:**
```bash
# Validate and compile Bicep template
# Validate and compile Bicep template - TIMEOUT: 180 seconds
az bicep build --file "infra/main.bicep" --outfile /tmp/arm.json
test -s /tmp/arm.json && echo "✓ Bicep compilation successful"
```
**Expected time:** 10-15 seconds
**Expected time:** 4-15 seconds - NEVER CANCEL, set timeout to 180+ seconds
**Warnings:** May show linter warnings about parent properties (safe to ignore)
**Must pass:** Compilation failure will block deployment

### Workflow Packaging
**Required for Logic App deployment:**
**Required for Logic App deployment - NEVER CANCEL:**
```bash
# Create ZIP package for workflows (preserves folder structure)
# Create ZIP package for workflows (preserves folder structure) - TIMEOUT: 60 seconds
WF_PATH="logicapps/workflows"
PARENT_DIR="$(dirname "$WF_PATH")"
BASE_NAME="$(basename "$WF_PATH")"
Expand All @@ -80,7 +125,7 @@ cd ..
echo "Workflow package created: workflows.zip"
unzip -l workflows.zip | head -10 # Verify contents
```
**Expected time:** < 5 seconds
**Expected time:** < 5 seconds - NEVER CANCEL, set timeout to 60+ seconds
**Critical:** ZIP must contain `workflows/` as top-level directory for Logic App Standard deployment

## Project Layout and Architecture
Expand Down Expand Up @@ -126,7 +171,7 @@ unzip -l workflows.zip | head -10 # Verify contents
1. **Lint (DEV environment)**:
- JSON validation of all `workflow.json` files
- Structure validation (definition/kind/parameters keys)
- **Time:** ~30 seconds
- **Time:** ~30 seconds - NEVER CANCEL, set timeout to 120+ seconds

2. **Matrix Deploy (DEV/UAT/PROD)**:
- Azure OIDC authentication per environment
Expand All @@ -135,7 +180,7 @@ unzip -l workflows.zip | head -10 # Verify contents
- Infrastructure deployment via `azure/arm-deploy@v2`
- Workflow ZIP packaging and deployment via `az webapp deploy`
- Logic App restart (optional)
- **Time per environment:** ~5-8 minutes
- **Time per environment:** ~5-8 minutes - NEVER CANCEL, set timeout to 20+ minutes

### Environment Configuration
Each environment requires these GitHub secrets:
Expand All @@ -144,25 +189,103 @@ Each environment requires these GitHub secrets:
- `AZURE_SUBSCRIPTION_ID_{ENV}`: Target subscription ID

### Manual Validation Steps
For manual testing of changes:
For manual testing of changes - NEVER CANCEL these operations:
```bash
# 1. Validate all workflows
# 1. Validate all workflows - TIMEOUT: 60 seconds
find logicapps/workflows -name "workflow.json" -exec jq . {} \;

# 2. Test Bicep compilation
# 2. Test Bicep compilation - TIMEOUT: 180 seconds
az bicep build --file infra/main.bicep --outfile /tmp/test.json

# 3. Preview infrastructure changes (requires Azure login)
# 3. Preview infrastructure changes (requires Azure login) - TIMEOUT: 300 seconds
az deployment group what-if \
--resource-group "test-rg" \
--template-file "infra/main.bicep" \
--parameters baseName="test-hipaa" \
--no-pretty-print

# 4. Package workflows for deployment
# 4. Package workflows for deployment - TIMEOUT: 60 seconds
cd logicapps && zip -r ../workflows.zip workflows/
```

## Validation Scenarios

**MANDATORY**: After making changes, always validate functionality through these complete end-to-end scenarios:

### Scenario 1: Complete Build and Validation Pipeline
**NEVER CANCEL - Expected time: 2-3 minutes**
```bash
# TIMEOUT: 300 seconds - Run complete validation pipeline
cd /path/to/hipaa-attachments

# Step 1: Validate prerequisites (5 seconds)
az --version && az bicep version && jq --version && pwsh --version

# Step 2: JSON workflow validation (5 seconds)
find "logicapps/workflows" -type f -name "workflow.json" -print0 | \
while IFS= read -r -d '' f; do
echo "Checking $f"
jq . "$f" >/dev/null && \
jq -e 'has("definition") and has("kind") and has("parameters")' "$f" >/dev/null && \
echo "✓ Valid: $f"
done

# Step 3: Bicep validation (15 seconds)
az bicep build --file "infra/main.bicep" --outfile /tmp/arm.json
test -s /tmp/arm.json && echo "✓ Bicep compilation successful"

# Step 4: Workflow packaging (5 seconds)
WF_PATH="logicapps/workflows"
PARENT_DIR="$(dirname "$WF_PATH")"
BASE_NAME="$(basename "$WF_PATH")"
cd "$PARENT_DIR"
zip -r ../workflows.zip "$BASE_NAME"
cd ..
unzip -l workflows.zip | head -5
echo "✓ Complete validation pipeline passed"
```

### Scenario 2: PowerShell Script Validation
**NEVER CANCEL - Expected time: 10-15 seconds**
```bash
# TIMEOUT: 60 seconds - Validate PowerShell capabilities
pwsh -ExecutionPolicy Bypass -Command "
Write-Host '✓ PowerShell execution working';
Get-ChildItem ./infra/*.bicep;
Get-ChildItem ./logicapps/workflows/*/workflow.json
"
```

### Scenario 3: GitHub Actions Lint Simulation
**NEVER CANCEL - Expected time: 30-45 seconds**
```bash
# TIMEOUT: 120 seconds - Simulate GitHub Actions lint workflow
WF_PATH="logicapps/workflows"

# Validate workflow folder exists
[ -d "$WF_PATH" ] && echo "✓ Workflows folder exists" || exit 1

# Find workflow files
find "$WF_PATH" -maxdepth 2 -type f -name "workflow.json" -print

# Lint JSON structure (matching GitHub Actions logic)
failed=0
while IFS= read -r -d '' f; do
echo "Linting $f"
jq . "$f" >/dev/null 2>&1 || { echo "::error file=$f::Invalid JSON"; failed=1; }
jq -e 'has("definition") and has("kind") and has("parameters")' "$f" >/dev/null || { echo "::error file=$f::Missing required keys"; failed=1; }
jq -e '.kind=="Stateful" or .kind=="Stateless"' "$f" >/dev/null || echo "::warning file=$f::kind verification"
done < <(find "$WF_PATH" -type f -name "workflow.json" -print0)

[ $failed -eq 0 ] && echo "✓ GitHub Actions lint simulation passed"
```

**Critical Validation Rules:**
- **NEVER CANCEL** any of these scenarios - they may take up to 5 minutes total
- **MUST PASS**: All three scenarios must complete successfully before deployment
- **Manual verification**: Check that all ✓ success messages appear
- **Error handling**: Any scenario that fails indicates changes broke the build

## Data Lake Storage Structure

Files are organized in Azure Data Lake Storage Gen2 with this partitioning:
Expand Down Expand Up @@ -201,11 +324,20 @@ This structure supports efficient querying and lifecycle management.

## Development Workflow

1. **Before making changes**: Run JSON and Bicep validation commands
2. **Test locally**: Use manual validation steps above
3. **Deploy via GitHub Actions**: Use workflow dispatch with appropriate environment
4. **Monitor deployment**: Check GitHub Actions logs and Azure portal
5. **Verify functionality**: Test Logic App workflows in Azure portal
**ALWAYS follow this exact sequence - NEVER SKIP validation steps:**

1. **Before making changes**: Run JSON and Bicep validation commands (TIMEOUT: 300 seconds)
2. **After making changes**: Run complete validation scenarios 1-3 above (TIMEOUT: 600 seconds)
3. **Test locally**: Use manual validation steps with proper timeouts
4. **Deploy via GitHub Actions**: Use workflow dispatch with appropriate environment
5. **Monitor deployment**: Check GitHub Actions logs and Azure portal (deployments may take 5-8 minutes per environment)
6. **Verify functionality**: Test Logic App workflows in Azure portal

**CRITICAL TIMING REQUIREMENTS:**
- Build operations: NEVER CANCEL before 5 minutes minimum
- Validation scenarios: NEVER CANCEL before 10 minutes minimum
- GitHub Actions deployment: NEVER CANCEL before 20 minutes minimum
- Always set timeouts to 2x expected time to account for Azure service delays

## Trust These Instructions

Expand All @@ -214,4 +346,55 @@ These instructions are comprehensive and tested. Only search for additional info
- You need to add new Azure resources beyond the current scope
- You need to modify the CI/CD pipeline structure

Always follow the validation steps before making changes to prevent deployment failures.
Always follow the validation steps before making changes to prevent deployment failures.

## Common Tasks and Quick Reference

The following are validated outputs from frequently run commands. Reference these instead of running bash commands to save time.

### Repository Structure
```
/home/runner/work/hipaa-attachments/hipaa-attachments/
├── .github/workflows/ # CI/CD pipelines
│ ├── deploy_logicapps_workflows_matrix_with_lint.yml # Main deployment (15-30 min)
│ └── sanity.yml # Basic health check (30 sec)
├── infra/
│ └── main.bicep # Azure infrastructure template (4-15 sec to compile)
├── logicapps/workflows/ # Logic App workflows (5 sec to package)
│ ├── ingest275/workflow.json # 275 ingestion workflow (~8KB)
│ └── rfai277/workflow.json # 277 RFAI workflow (~5KB)
├── bootstrap_repo.ps1 # Repository scaffolding script (requires parameters)
├── fix_repo_structure.ps1 # Structure normalization script (5-30 sec)
└── README.md # Architecture and deployment guide
```

### Workflow File Structure
```json
// Both workflow.json files contain:
{
"definition": { /* Logic App definition */ },
"kind": "Stateful", // or "Stateless"
"parameters": { /* Connection and configuration parameters */ }
}
```

### Prerequisites Versions (Validated)
```
azure-cli: 2.77.0
Bicep CLI: 0.37.4 (27cc8db2ed)
jq: 1.7
PowerShell: 7.4.11
```

### GitHub Actions Environments
- **DEV**: `pchp-attachments-dev-rg` in eastus
- **UAT**: `pchp-attachments-uat-rg` in eastus
- **PROD**: `pchp-attachments-prod-rg` in eastus

Each requires: `AZURE_CLIENT_ID_{ENV}`, `AZURE_TENANT_ID_{ENV}`, `AZURE_SUBSCRIPTION_ID_{ENV}` secrets

### Key File Locations for Changes
- **Infrastructure changes**: `infra/main.bicep`
- **Workflow logic**: `logicapps/workflows/*/workflow.json`
- **CI/CD pipeline**: `.github/workflows/deploy_logicapps_workflows_matrix_with_lint.yml`
- **Repository structure**: Use `fix_repo_structure.ps1` after changes