Failure to Run on Ubuntu 24.04 with Podman

[RobertWi]

Description

codegate container runs successfully with podman on macOS but fails on Ubuntu 24.04 when using podman. The issue appears related to certificate handling and connectivity to localhost services on port 8080. Callback url? GitHub Copilot functionality is also impacted.Nothing is listening on 8080.

Steps to Reproduce

1. Run the following command in Ubuntu 24.04 with Podman installed:

   podman run --name codegate -d -p 8989:8989 -p 9091:9090 \
     -p 8990:8990 \
     --mount type=volume,src=codegate_volume,dst=/app/codegate_volume \
     --restart unless-stopped ghcr.io/stacklok/codegate:latest

Used alternative port 9091 as 9090 was taken

2. cp cert and trust

podman  cp codegate:/app/codegate_volume/certs/ca.crt ./codegate.crt
sudo cp ./codegate.crt /usr/local/share/ca-certificates/codegate.crt
sudo update-ca-certificates

3. add the following to user settings,json

  "http.proxy": "https://localhost:8990",
  "http.proxyStrictSSL": true,
  "http.proxySupport": "on",
  "http.systemCertificates": true,
  "github.copilot.advanced": {
    "debug.useNodeFetcher": true,
    "debug.useElectronFetcher": true,
    "debug.testOverrideProxyUrl": "https://localhost:8990",
    "debug.overrideProxyUrl": "https://localhost:8990"
  }

4. restart vscode

5. Check the container logs using:

   podman logs codegate

6. Test GitHub Copilot functionality in VS Code (version details below).

Observed Behavior

see logs

Copilot diagnostics below

## Copilot

  

- Version: 1.257.0

- Build: prod

- Editor: vscode/1.96.4

  

## Environment

  

- http_proxy: n/a

- https_proxy: n/a

- no_proxy: n/a

- SSL_CERT_FILE: n/a

- SSL_CERT_DIR: n/a

- OPENSSL_CONF: n/a

  

## Feature Flags

  
  
  

## Node setup

  

- Number of root certificates: 147

- Operating system: Linux

- Operating system version: 6.8.0-51-generic

- Operating system architecture: x64

- NODE_OPTIONS: n/a

- NODE_EXTRA_CA_CERTS: n/a

- NODE_TLS_REJECT_UNAUTHORIZED: n/a

- tls default min version: TLSv1.2

- tls default max version: TLSv1.3

  

## Network Configuration

  

- Proxy host: n/a

- Proxy port: n/a

- Kerberos SPN: n/a

- Reject unauthorized: enabled

- Fetcher: ElectronFetcher

  

## Reachability

  

- github.com: Error: net::ERR_PROXY_CERTIFICATE_INVALID

- api.github.com: Error: net::ERR_PROXY_CERTIFICATE_INVALID

- localhost:8990: Error: net::ERR_CERT_AUTHORITY_INVALID

- api.githubcopilot.com: Error: net::ERR_PROXY_CERTIFICATE_INVALID

- copilot-telemetry.githubusercontent.com: Error: net::ERR_PROXY_CERTIFICATE_INVALID

  

## VS Code Configuration

  

- HTTP proxy: https://localhost:8990

- HTTP proxy authentication: n/a

- Proxy Strict SSL: true

- Extension HTTP proxy support: on

  

## Extensions

  

- Is `win-ca` installed?: false

- Is `mac-ca` installed?: false

  

## Authentication

  

- GitHub username: RobertWi

Additional Observations

• The entrypoint script includes --ollama-url, which seems unnecessary for this setup.
• Connectivity to localhost:8080 repeatedly fails in the container logs.
• Copilot fails with ERR_PROXY_CERTIFICATE_INVALID, suggesting certificate issues.

Operating System

Linux (Intel)

IDE and Version

VS code 1.96.4

Extension and Version

GitHub Copilot Extension Version: 1.257.0

Provider

GitHub Copilot

Model

gpt4

Codegate version

0.1.6

Logs

Container Logs

initializing entrypoint script...
Backup path or mode not provided. Skipping restore.
Generating certificates...
/usr/local/lib/python3.12/site-packages/pydantic/_internal/_config.py:345: UserWarning: Valid config keys have changed in V2:
* 'fields' has been removed
  warnings.warn(message, UserWarning)
Starting the dashboard...
Starting the application with args: --port 8989 --host 0.0.0.0 --model-base-path /app/codegate_volume/models --db-path /app/codegate_volume/db/codegate.db --vec-db-path /app/sqlite_data/vectordb.db --ollama-url http://host.docker.internal:11434 --log-level WARNING --log-format TEXT
/usr/local/lib/python3.12/site-packages/pydantic/_internal/_config.py:345: UserWarning: Valid config keys have changed in V2:
* 'fields' has been removed
  warnings.warn(message, UserWarning)
alembic
Existing Certificates are already present.
2025-01-21T14:58:23.678228Z [error    ] Failed to connect to target localhost:8080: [Errno 111] Connect call failed ('127.0.0.1', 8080) module=provider origin=copilot_proxy pathname=/app/src/codegate/providers/copilot/provider.py
2025-01-21T14:58:53.701419Z [error    ] Failed to connect to target localhost:8080: [Errno 111] Connect call failed ('127.0.0.1', 8080) module=provider origin=copilot_proxy pathname=/app/src/codegate/providers/copilot/provider.py
2025-01-21T14:59:23.695671Z [error    ] Failed to connect to target localhost:8080: [Errno 111] Connect call failed ('127.0.0.1', 8080) module=provider origin=copilot_proxy pathname=/app/src/codegate/providers/copilot/provider.py

GitHub Copilot Logs

2025-01-21 15:58:23.576 [error] [auth] auth: Error: net::ERR_PROXY_CERTIFICATE_INVALID 2025-01-21 15:58:38.476 [error] [default] Error sending telemetry Error: net::ERR_PROXY_CERTIFICATE_INVALID```

### Additional Context

_No response_

[jhrozek]

tagging @JAORMX who uses codegate with podman on Fedora and might have an idea.

[JAORMX]

@RobertWi did you run the command to trust the certificate generated by Codegate?

[RobertWi]

Yes i did @JAORMX Forgot to add that in steps to reproduce, added it now, apologies

[JAORMX]

@RobertWi can you try with a versioned container tag just to be sure?

podman run --name codegate -d -p 8989:8989 -p 9090:9090 -p 8990:8990 --mount type=volume,src=codegate_volume,dst=/app/codegate_volume --restart unless-stopped ghcr.io/stacklok/codegate:v0.1.10

[RobertWi]

@JAORMX
Initial issue was reported with
v0.1.10 34ce6fd1819d
same issue on
v0.1.9 057dbcb5d0d0
but also on latest
e766f2a7e74b

Spun up fresh ubuntu24.04 with libvirt, same issue with docker... Started of with docker to stay as close as per doc.

All had a clean install

rm /usr/local/share/ca-certificates/codegate.crt 
rm ./codegate.crt 
sudo docker stop codegate
sudo docker rm codegate
sudo docker volume rm codegate 

then as per doc install

Dashboard is not reporting version clearly which may be more user friendly then looking at the image id as with prompt codegate version does not apply

Will spin up some other distro and let you know.

[JAORMX]

FWIW, I'm running on Fedora

[RobertWi]

Alternatively

As node (electron) is complaining I tried this to no avail

echo 'export NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/codegate.crt' >> ~/.bashrc && source ~/.bashrc && code 

suprisingly using nss database then things are consumed by vscode

sudo apt install libnss3-tools
certutil -d sql:$HOME/.pki/nssdb -A -t "C,," -n codegate -i /usr/local/share/ca-certificates/codegate.crt && code 

Working fine.
Guess I am working my way around things..

[jhrozek]

@danbarr this might be something for us to document

[danbarr]

Thanks for the ping, and thanks @RobertWi for the investigation and sticking with it!

I'll do some testing to see if the nssdb approach might work consistently for both Fedora/RHEL-derived and Debian/Ubuntu-derived distros instead of the system certs and update the docs accordingly.