[DevSetting] Add install_proxy_url which will allow ParallelCluster to set Proxy environment for Build Image installation

* Adding a recipe to setup proxy env as part of cookbook to avoid Sizing issue in parallelcluster.yaml
* Add s3 global endpoint to overcome cfn_bootstrap script installation
* Add s3 enpoint for no proxy list so it goes to VPC endpoint
* addding old style s3 naming convention bucket-name.s3-eu-west-1.amazonaws.com to handle S3 redirecting (307 Temporary Redirect) when we use non-us-east-1 region
* Add cdn for amazonlinux in no_proxy list to re-direct to VPC endpoint

Author: Himani Anil Deshpande

cookbooks/aws-parallelcluster-entrypoints/recipes/install.rb

@@ -17,6 +17,7 @@
 return if node['conditions']['ami_bootstrapped']
 
 include_recipe "aws-parallelcluster-shared::setup_envars"
+include_recipe "aws-parallelcluster-shared::detect_proxy" if node['cluster']['install_proxy_url']
 
 include_recipe 'aws-parallelcluster-platform::install'
 include_recipe 'aws-parallelcluster-environment::install'

cookbooks/aws-parallelcluster-shared/recipes/detect_proxy.rb

@@ -0,0 +1,77 @@
+# frozen_string_literal: true
+
+#
+# Cookbook:: aws-parallelcluster
+# Recipe:: detect_proxy
+#
+# Copyright:: 2026 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the
+# License. A copy of the License is located at
+#
+# http://aws.amazon.com/apache2.0/
+#
+# or in the "LICENSE.txt" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
+# OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This recipe configures proxy environment variables for build-image in isolated networks.
+#
+# It reads the proxy URL from node['cluster']['install_proxy_url'] (set via ExtraChefAttributes)
+# and configures http_proxy/https_proxy ENV vars for the Chef run. This makes all subsequent
+# Chef resources (remote_file, bash, execute, etc.) use the explicit proxy for HTTPS traffic
+# instead of trying direct connections that would fail in an isolated network.
+#
+# The no_proxy list excludes S3 endpoints so downloads from S3 go through the VPC Gateway
+# Endpoint directly, not through the proxy.
+# S3 endpoints are excluded so cookbook/dependency downloads from S3 go through 
+# the S3 VPC Gateway Endpoint directly, not through the proxy.
+ # Both regional (s3.{region}.amazonaws.com) and global (s3.amazonaws.com) endpoints
+ # are included because some resources use the global endpoint (e.g., cloudformation-examples
+ # bucket uses https://s3.amazonaws.com/cloudformation-examples/...).
+ # Note: only the regional S3 endpoint is in no_proxy because the S3 VPC Gateway Endpoint
+ # handles regional endpoints correctly. The global s3.amazonaws.com endpoint does NOT work
+ # through the VPC Gateway Endpoint (SSL errors), so it is intentionally left out of no_proxy
+ # and instead goes through the proxy which has internet access. The proxy allowlist in
+ # proxy_stack.yaml must include s3.amazonaws.com for this to work.
+ # IMDS (169.254.169.254) and ECS task metadata (169.254.170.2) are also excluded.
+#
+# This recipe only runs when install_proxy_url is set — normal builds are unaffected.
+
+ruby_block 'configure proxy from install_proxy_url' do
+  block do
+    proxy_url = node['cluster']['install_proxy_url']
+
+    if proxy_url && !proxy_url.empty?
+      region = node['cluster']['region']
+
+      # S3 endpoints bypass the proxy and use the VPC Gateway Endpoint.
+      # Includes regional (s3.{region}), dash-style (s3-{region}), global (s3.amazonaws.com),
+      # and dualstack (s3.dualstack.{region}) variants used by different AWS services and repos.
+      no_proxy = [
+        "localhost",
+        "127.0.0.1",
+        "169.254.169.254",
+        "169.254.170.2",
+        ".s3.#{region}.amazonaws.com",
+        "s3.#{region}.amazonaws.com",
+        ".s3-#{region}.amazonaws.com",
+        "s3-#{region}.amazonaws.com",
+        ".s3.amazonaws.com",
+        ".s3.dualstack.#{region}.amazonaws.com",
+        "s3.dualstack.#{region}.amazonaws.com",
+      ].join(",")
+
+      Chef::Log.info("Configuring proxy: #{proxy_url}")
+
+      ENV['http_proxy'] = proxy_url
+      ENV['https_proxy'] = proxy_url
+      ENV['HTTP_PROXY'] = proxy_url
+      ENV['HTTPS_PROXY'] = proxy_url
+      ENV['no_proxy'] = no_proxy
+      ENV['NO_PROXY'] = no_proxy
+    else
+      Chef::Log.info("No install_proxy_url set, skipping proxy configuration")
+    end
+  end
+end