updated doc and tool

Author: naman-msft

tools/Deploy Linux VM with SSH Access_ai_generated.md

@@ -0,0 +1,108 @@
+---
+title: 'Quickstart: Create a Linux VM and SSH into it'
+description: Learn how to create a Linux virtual machine in Azure using Azure CLI and then SSH into it.
+ms.topic: quickstart
+ms.date: 10/12/2023
+author: yourgithubusername
+ms.author: yourgithubusername
+ms.custom: innovation-engine, azurecli, linux-related-content
+---
+
+# Quickstart: Create a Linux VM and SSH into it
+
+This Exec Doc demonstrates how to create a resource group, deploy a Linux VM using a supported Ubuntu image, retrieve its public IP address, and then SSH into the VM. The process uses environment variables to manage configuration details and appends a random suffix to resource names to ensure uniqueness.
+
+The following sections walk through each step with code blocks. Remember that you must already be logged in to Azure and have your subscription set.
+
+## Step 1: Create a Resource Group
+
+In this section, we declare environment variables necessary for the deployment and create a resource group in the "centralindia" region. A random suffix is appended to the resource group name to guarantee uniqueness.
+
+```bash
+export RANDOM_SUFFIX=$(openssl rand -hex 3)
+export REGION="centralindia"
+export RG_NAME="LinuxRG$RANDOM_SUFFIX"
+az group create --name $RG_NAME --location $REGION
+```
+
+Results:
+
+<!-- expected_similarity=0.3 -->
+```JSON
+{
+  "id": "/subscriptions/xxxxx/resourceGroups/LinuxRGabc123",
+  "location": "centralindia",
+  "managedBy": null,
+  "name": "LinuxRGabc123",
+  "properties": {
+    "provisioningState": "Succeeded"
+  },
+  "tags": null,
+  "type": "Microsoft.Resources/resourceGroups"
+}
+```
+
+## Step 2: Create a Linux Virtual Machine
+
+Now we create a Linux VM using a supported Ubuntu image ('Ubuntu2204'). In this example, we use a Standard_B1s VM size. We also set an administrator username and let Azure generate SSH key pairs automatically. A random suffix is used in the VM name for uniqueness.
+
+```bash
+export VM_NAME="LinuxVM$RANDOM_SUFFIX"
+export ADMIN_USERNAME="azureuser"
+az vm create \
+  --resource-group $RG_NAME \
+  --name $VM_NAME \
+  --image Ubuntu2204 \
+  --size Standard_B1s \
+  --admin-username $ADMIN_USERNAME \
+  --generate-ssh-keys
+```
+
+Results:
+
+<!-- expected_similarity=0.3 -->
+```JSON
+{
+  "fqdns": "",
+  "id": "/subscriptions/xxxxx/resourceGroups/LinuxRGabc123/providers/Microsoft.Compute/virtualMachines/LinuxVMabc123",
+  "location": "centralindia",
+  "macAddress": "00-0X-0X-0X-0X-0X",
+  "machineId": "xxxxx",
+  "name": "LinuxVMabc123",
+  "powerState": "VM running",
+  "privateIpAddress": "10.0.0.4",
+  "publicIpAddress": "13.92.xxx.xxx",
+  "resourceGroup": "LinuxRGabc123",
+  "zones": ""
+}
+```
+
+## Step 3: Retrieve the VM Public IP Address
+
+This step retrieves the public IP address of the newly created VM. The public IP is stored in an environment variable to be used in the SSH step.
+
+```bash
+export VM_PUBLIC_IP=$(az vm list-ip-addresses --resource-group $RG_NAME --name $VM_NAME --query "[].virtualMachine.network.publicIpAddresses[0].ipAddress" --output tsv)
+echo "The public IP address of the VM is: $VM_PUBLIC_IP"
+```
+
+Results:
+
+<!-- expected_similarity=0.3 -->
+```text
+The public IP address of the VM is: 13.92.xxx.xxx
+```
+
+## Step 4: SSH into the Linux VM
+
+Finally, once you have retrieved the public IP address, you can SSH into your Linux VM using the generated SSH key pair. This command establishes an SSH connection without prompting for host key verification.
+
+```bash
+ssh -o StrictHostKeyChecking=no $ADMIN_USERNAME@$VM_PUBLIC_IP
+```
+
+When executed, this command initiates an SSH session with your Linux VM. After connecting, you will have full access to manage and configure the VM as needed.
+
+---
+
+This Exec Doc has successfully deployed a Linux VM in Azure using a supported Ubuntu image and shown how to connect to it using SSH, all accomplished with a series of Azure CLI commands executed via the Innovation Engine.

tools/ada.py

@@ -438,6 +438,31 @@ def generate_dependency_files(doc_path):
 
     print("\nAnalyzing document for dependencies...")
 
+    # First, detect file creation patterns in the document to avoid conflicts
+    file_creation_patterns = [
+        # Cat heredoc to a file
+        (r'cat\s*<<\s*[\'"]?(EOF|END)[\'"]?\s*>\s*([^\s;]+)', 1),
+        # Echo content to a file
+        (r'echo\s+.*?>\s*([^\s;]+)', 0),
+        # Tee command
+        (r'tee\s+([^\s;]+)', 0)
+    ]
+    
+    doc_created_files = []
+    for pattern, group_idx in file_creation_patterns:
+        matches = re.findall(pattern, doc_content, re.DOTALL)
+        for match in matches:
+            if isinstance(match, tuple):
+                filename = match[group_idx]
+            else:
+                filename = match
+            doc_created_files.append(filename)
+    
+    if doc_created_files:
+        print("\nDetected file creation commands in document:")
+        for file in doc_created_files:
+            print(f"  - {file}")
+    
     # Enhanced prompt for better dependency file identification
     dependency_prompt = """Analyze this Exec Doc and identify ANY files that the user is instructed to create.
     
@@ -507,9 +532,23 @@ def generate_dependency_files(doc_path):
             print("\nNo dependency files identified.")
             return True, []
 
+        # Filter out dependency files that have inline creation commands in the document
+        filtered_deps = []
+        for dep in dependency_list:
+            filename = dep.get("filename")
+            if not filename:
+                continue
+                
+            if filename in doc_created_files:
+                print(f"\nWARNING: File '{filename}' is both created in document and identified as a dependency.")
+                print(f"  - Skipping dependency management for this file to avoid conflicts.")
+                continue
+            
+            filtered_deps.append(dep)
+        
         # Create each dependency file with type-specific handling
         created_files = []
-        for dep in dependency_list:
+        for dep in filtered_deps:
             filename = dep.get("filename")
             content = dep.get("content")
             file_type = dep.get("type", "").lower()
@@ -596,6 +635,60 @@ def generate_dependency_files(doc_path):
         print("\nResponse from model was not valid JSON. Raw response:")
         return False, []
 
+# Add this function after generate_dependency_files function (approximately line 609)
+
+def transform_document_for_dependencies(doc_path, dependency_files):
+    """Remove file creation commands from document when using dependency files."""
+    if not dependency_files:
+        return False
+    
+    try:
+        with open(doc_path, "r") as f:
+            doc_content = f.read()
+            
+        original_content = doc_content
+        modified = False
+        
+        for dep_file in dependency_files:
+            filename = dep_file["filename"]
+            
+            # Pattern to match cat/EOF blocks for file creation
+            cat_pattern = re.compile(
+                r'```(?:bash|azurecli|azure-cli-interactive|azurecli-interactive)\s*\n'
+                r'(.*?cat\s*<<\s*[\'"]?(EOF|END)[\'"]?\s*>\s*' + re.escape(filename) + r'.*?EOF.*?)'
+                r'\n```',
+                re.DOTALL
+            )
+            
+            # Replace with a reference to the external file
+            if cat_pattern.search(doc_content):
+                replacement = f"```bash\n# Using external file: {filename}\n```\n\n"
+                doc_content = cat_pattern.sub(replacement, doc_content)
+                modified = True
+                print(f"\nTransformed document to use external {filename} instead of inline creation")
+                
+            # Handle other file creation patterns (echo, tee)
+            echo_pattern = re.compile(
+                r'```(?:bash|azurecli|azure-cli-interactive|azurecli-interactive)\s*\n'
+                r'(.*?echo\s+.*?>\s*' + re.escape(filename) + r'.*?)'
+                r'\n```',
+                re.DOTALL
+            )
+            if echo_pattern.search(doc_content):
+                replacement = f"```bash\n# Using external file: {filename}\n```\n\n"
+                doc_content = echo_pattern.sub(replacement, doc_content)
+                modified = True
+                
+        if modified:
+            with open(doc_path, "w") as f:
+                f.write(doc_content)
+            print("\nDocument transformed to use external dependency files")
+            return True
+        return False
+    except Exception as e:
+        print(f"\nError transforming document: {e}")
+        return False
+    
 def update_dependency_file(file_info, error_message, main_doc_path):
     """Update a dependency file based on error message."""
     filename = file_info["filename"]
@@ -857,7 +950,7 @@ def main():
     print("  2. Describe workload to generate a new Exec Doc")
     print("  3. Add descriptions to a shell script as an Exec Doc")
     print("  4. Redact PII from an existing Exec Doc")
-    print("  5. Perform security vulnerability check on an Exec Doc")
+    print("  5. Generate a security analysis report for an Exec Doc")
     choice = input("\nEnter the number corresponding to your choice: ")
 
     if choice == "1":
@@ -949,6 +1042,15 @@ def main():
             if generate_deps and not dependency_files_generated:
                 _, dependency_files = generate_dependency_files(output_file)
                 dependency_files_generated = True
+
+                 # Generate dependency files after first creation
+            if generate_deps and not dependency_files_generated:
+                _, dependency_files = generate_dependency_files(output_file)
+                dependency_files_generated = True
+                
+                # Add this new line to transform the document after dependency generation
+                if dependency_files:
+                    transform_document_for_dependencies(output_file, dependency_files)
         else:
             print(f"\n{'='*40}\nAttempt {attempt}: Generating corrections based on error...\n{'='*40}")
 

tools/aks.sh

@@ -0,0 +1,45 @@
+#!/bin/bash
+# This script creates an AKS cluster using Azure CLI
+
+# Exit on error
+set -e
+
+# Configuration variables
+RESOURCE_GROUP="myAKSResourceGroup"
+LOCATION="eastus"
+CLUSTER_NAME="myAKSCluster"
+NODE_COUNT=3
+NODE_VM_SIZE="Standard_DS2_v2"
+KUBERNETES_VERSION="1.26.3"  # Check available versions with: az aks get-versions --location $LOCATION --output table
+
+# Login to Azure (uncomment if not already logged in)
+# az login
+
+# Create resource group
+echo "Creating resource group $RESOURCE_GROUP in $LOCATION..."
+az group create --name $RESOURCE_GROUP --location $LOCATION
+
+# Create AKS cluster
+echo "Creating AKS cluster $CLUSTER_NAME..."
+az aks create \
+    --resource-group $RESOURCE_GROUP \
+    --name $CLUSTER_NAME \
+    --node-count $NODE_COUNT \
+    --node-vm-size $NODE_VM_SIZE \
+    --kubernetes-version $KUBERNETES_VERSION \
+    --generate-ssh-keys \
+    --enable-managed-identity \
+    --enable-cluster-autoscaler \
+    --min-count 1 \
+    --max-count 5
+
+# Get credentials for the Kubernetes cluster
+echo "Getting credentials for cluster $CLUSTER_NAME..."
+az aks get-credentials --resource-group $RESOURCE_GROUP --name $CLUSTER_NAME
+
+echo "AKS cluster $CLUSTER_NAME has been created successfully!"
+echo "You can now use kubectl to manage your cluster"
+
+# Verify connection to the cluster
+echo "Verifying connection to the cluster..."
+kubectl get nodes