docker stopped working overnight

[danlwright]

I have been running the same docker container for a while without issue, but a couple of nights ago it stopped working. I had the container set to automatically restart. I'm not sure if NordVPN blocked me because of all the attempted requests.

Here is what my logs show when I attempt to start the container:

Starting container with revision: b33d0fe
Creating TUN device /dev/net/tun
Using OpenVPN provider: NORDVPN
Running with VPN_CONFIG_SOURCE auto
Provider NORDVPN has a bundled setup script. Defaulting to internal config
Executing setup script for NORDVPN
INFO: OVPN: Checking curl installation
INFO: OVPN: DNS resolution ok

Can someone get me pointed in the right direction to resolve the issue?

Thanks,

Dan

[edgd1er]

Nordvpn throttle requests to the api if that's occuring, you should have a log saying that html was found in api response.
You may check if your ip is throttled through a simple light request: https://api.nordvpn.com/v1/servers/groups

[
  {
    "id": 1,
    "created_at": "2017-06-13 13:41:00",
    "updated_at": "2017-06-13 13:41:00",
    "title": "Double VPN",
    "identifier": "legacy_double_vpn",
    "type": {
      "id": 3,
      "created_at": "2017-06-13 13:40:17",
      "updated_at": "2017-06-13 13:40:23",
      "title": "Legacy category",
      "identifier": "legacy_group_category"
    }
  },....

If you are blacklisted, stop the container, a few hours later the API would be available again.
I think there is a NORDVPN_SERVER that by pass all api request and get directly the ovpn file configuration. this could be a workaround during the api's throttle.

[danlwright]

I guess that I must not be blacklisted. I had the container stopped and the restart policy set to none, so that it wouldn't keep attempting to start. I just tried starting it again and got the same log results. Any other ideas?

Thanks!

[edgd1er]

DEBUG env var if set to TRUE will activate -x flag in bash, allowing verbose mode.
You may set the var and post the log. the previously posted log show no error and is incomplete.

[edgd1er]

you should check your host as it seems the container is killed. docker ps will show running containers.
There is a debug page in Haugene's documentation that may help you.

[danlwright]

Thanks so much. I was able to run through the troubleshooting steps and get it working.

[edgd1er]

@danlwright , if someone is having a similar problem, could you tell us how you fix it ?

[danlwright]

I will post the resolution once I get it figured out. It stopped running during the same night that I got it running. Haven't been able to get it working yet.

[danlwright]

Although I haven't found a solution, I have been collecting more information. This is what the log shows prior to the container exiting:

2022-12-05 00:00:20 VERIFY OK: depth=2, C=PA, O=NordVPN, CN=NordVPN Root CA
2022-12-05 00:00:20 VERIFY OK: depth=1, C=PA, O=NordVPN, CN=NordVPN CA7
2022-12-05 00:00:20 VERIFY KU OK
2022-12-05 00:00:20 Validating certificate extended key usage
2022-12-05 00:00:20 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
2022-12-05 00:00:20 VERIFY EKU OK
2022-12-05 00:00:20 VERIFY X509NAME OK: CN=us9949.nordvpn.com
2022-12-05 00:00:20 VERIFY OK: depth=0, CN=us9949.nordvpn.com
2022-12-05 00:00:20 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2022-12-05 00:00:20 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2022-12-05 00:00:20 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, peer certificate: 4096 bit RSA, signature: RSA-SHA512
2022-12-05 00:57:50 VERIFY OK: depth=2, C=PA, O=NordVPN, CN=NordVPN Root CA
2022-12-05 00:57:50 VERIFY OK: depth=1, C=PA, O=NordVPN, CN=NordVPN CA7
2022-12-05 00:57:50 VERIFY KU OK
2022-12-05 00:57:50 Validating certificate extended key usage
2022-12-05 00:57:50 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
2022-12-05 00:57:50 VERIFY EKU OK
2022-12-05 00:57:50 VERIFY X509NAME OK: CN=us9949.nordvpn.com
2022-12-05 00:57:50 VERIFY OK: depth=0, CN=us9949.nordvpn.com
2022-12-05 00:57:50 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2022-12-05 00:57:50 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2022-12-05 00:57:50 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, peer certificate: 4096 bit RSA, signature: RSA-SHA512
2022-12-05 01:53:13 [us9949.nordvpn.com] Inactivity timeout (--ping-restart), restarting
2022-12-05 01:53:13 SIGTERM received, sending exit notification to peer
2022-12-05 01:53:14 /etc/openvpn/tunnelDown.sh tun0 1500 1585 10.8.3.5 255.255.255.0 init
resolv.conf was restored
Sending kill signal to transmission-daemon
Waiting 5s for transmission-daemon to die
Successfuly closed transmission-daemon
2022-12-05 01:53:18 net_route_v4_del: 193.29.61.16/32 via 172.18.0.1 dev [NULL] table 0 metric -1
2022-12-05 01:53:18 net_route_v4_del: 0.0.0.0/1 via 10.8.3.1 dev [NULL] table 0 metric -1
2022-12-05 01:53:18 net_route_v4_del: 128.0.0.0/1 via 10.8.3.1 dev [NULL] table 0 metric -1
2022-12-05 01:53:18 Closing TUN/TAP interface
2022-12-05 01:53:18 net_addr_v4_del: 10.8.3.5 dev tun0
2022-12-05 01:53:18 SIGTERM[soft,exit-with-notification] received, process exiting

I do have an environment variable set for: OPENVPN_OPTS
--inactive 3600 --ping 10 --ping-exit 60, but that doesn't seem to be keeping the connection alive.

Once the container exits, it cannot be restarted as usual. Even deleting the container and recreating it via docker compose doesn't work. Restarting Docker also has no effect. The only thing that I have found that consistently allows the container to be restarted is rebooting the machine.