Import of AWS.Tools.Common makes unexpected TCP connections causing extreme slowdown in some situations

[boernsen-development]

Describe the bug

When I import AWS.Tools.Common, it seems to make a few TCP connections without notice, no matter whether they succeed or not. In some situations where the (cloud-)network doesn't allow these connections, they may timeout causing the import to take much longer than actually required (over 20 seconds im my case). This is hard to debug as there is no information about it.

Possible workaround: Add firewall rules actively blocking these connections. However, this required exact knowledge about all the IPs contacted.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

• No unexpected connections are made
• they are at least well documented
• they are printed by verbose/debug output

Current Behavior

• Connections are made in background without notice
• there is neither documentation
• nor any related verbose/debug output

Reproduction Steps

Open Windows' Resource Monitor while executing the following command in a PowerShell Core console:

Import-Module -Name AWS.Tools.Common

Observe the connections.

When importing other modules, there are no are way less connections.

Possible Solution

Some options:

• Remove connections completely
• Make them optional by a switch
• Document which connections are made and output them via verbose/debug output

Additional Information/Context

No response

AWS Tools for PowerShell version used

AWS.Tools.Common 5.0.147

PowerShell version used

Name Value

---

PSVersion 7.5.4
PSEdition Core
GitCommitId 7.5.4
OS Microsoft Windows 10.0.20348
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0

Operating System and version

Windows Server 2022 Datacenter

[afroz429]

Thanks for reporting the issue. We will review the issue and get back to you.

[afroz429]

@boernsen-development are you running or intending to run this code on EC2?

[boernsen-development]

Yes, we already run the code on EC2.

Regarding a workaround:
Blocking three of the contacted IPs (outgoing on port 80) shown in the resource monitor with Windows' built-in firewall reduces the runtime to about 1 second. Remaining caveat: I don't know whether I can rely on these IPs.

[afroz429]

Can you please share the IP addresses you have blocked?

[boernsen-development]

Sure, sorry for holding them back ;)

Yesterday I had:

2.17.190.73
172.66.2.5
162.159.142.9

Today I have:

184.30.131.245
146.75.122.172

All connections to these IP addresses are done on port 80.

Apparently, the IP addresses change over time, probably because of changing DNS resolutions? For a reliable workaround we therefore could block the DNS address itself via the hosts file, but we would need to know the actual DNS address :)

[boernsen-development]

nslookup:

PS C:\Windows\System32> nslookup 2.17.190.73
Server:  ip-10-106-44-2.eu-central-1.compute.internal
Address:  10.106.44.2

Name:    a2-17-190-73.deploy.static.akamaitechnologies.com
Address:  2.17.190.73

PS C:\Windows\System32> nslookup 172.66.2.5
Server:  ip-10-106-44-2.eu-central-1.compute.internal
Address:  10.106.44.2

*** ip-10-106-44-2.eu-central-1.compute.internal can't find 172.66.2.5: Non-existent domain
PS C:\Windows\System32> nslookup 162.159.142.9
Server:  ip-10-106-44-2.eu-central-1.compute.internal
Address:  10.106.44.2

*** ip-10-106-44-2.eu-central-1.compute.internal can't find 162.159.142.9: Non-existent domain
PS C:\Windows\System32> nslookup 184.30.131.245
Server:  ip-10-106-44-2.eu-central-1.compute.internal
Address:  10.106.44.2

Name:    a184-30-131-245.deploy.static.akamaitechnologies.com
Address:  184.30.131.245

PS C:\Windows\System32> nslookup 146.75.122.172
Server:  ip-10-106-44-2.eu-central-1.compute.internal
Address:  10.106.44.2

*** ip-10-106-44-2.eu-central-1.compute.internal can't find 146.75.122.172: Non-existent domain

[afroz429]

The 20+ second delays during Import-Module AWS.Tools.Common appear to be caused by Windows certificate validation attempting to verify the digital signatures on AWS SDK assemblies (AWSSDK.Core.dll, AWSSDK.SecurityToken.dll, etc.) which are signed by Amazon using DigiCert certificates. When your network blocks access to DigiCert's certificate revocation endpoints, Windows may wait for timeout periods before falling back to offline validation.

Ideally allowlist these domains in your network for: crl3.digicert.com, crl4.digicert.com, ocsp.digicert.com, cacerts.digicert.com (or use wildcard *.digicert.com), but you can also allowlist all the specific IP addresses from the complete list at DigiCert Certificate Status IP Addresses. The specific IP addresses you observed (2.17.190.73, 172.66.2.5, 162.159.142.9, 184.30.131.245) appear to match DigiCert's official certificate status infrastructure.

[afroz429]

I will close this ticket. Please re-open if this is not resolved. Please feel free to report if you observe any new external IPs that are not digicert. Thanks

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[boernsen-development]

@afroz429 We now white-listed "*.digicert.com" and successfully tested the connection to the mentioned DNS addresses (crl3.digicert.com, crl4.digicert.com, ocsp.digicert.com, cacerts.digicert.com) via Invoke-WebRequest.

However, we still get the same delay and it still can be mitigated by adding a firewall rule for the contacted IPs.

Do you have any idea what else we could do?