Skip to content

Commit b740196

Browse files
ewdurbinewdurbin
authored
Merge branch 'main' into add-starttls
2 parents f2ab9c1 + 6e9f437 commit b740196

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+742
-626
lines changed

‎docs/conf.py‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
# Add any Sphinx extension module names here, as strings. They can be
2323
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
2424
# ones.
25-
extensions = ['myst_parser', "sphinx_copybutton"]
25+
extensions = ['myst_parser', "sphinx_copybutton", 'sphinxcontrib.jquery', 'sphinx_datatables']
2626

2727
# Add any paths that contain templates here, relative to this directory.
2828
templates_path = ["_templates"]
@@ -185,3 +185,9 @@
185185

186186
# Output file base name for HTML help builder.
187187
htmlhelp_basename = "PythonInfrastructuredoc"
188+
189+
# Enable MyST heading anchors
190+
myst_heading_anchors = 2
191+
192+
# sphinx-datatables
193+
datatables_options = {"paging": False}

‎docs/guides/external-host-requirements.md‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# External Host Requirements for PSF Salt Management
2+
3+
This document outlines the requirements and processes for external hosts that will be managed by the
4+
Python Software Foundation's Salt infrastructure.
5+
6+
## Overview
7+
8+
When providing hardware for PSF services, your server will be managed through our Salt configuration management system.
9+
This document details the network, security, and access requirements for integration with our infrastructure.
10+
11+
## Network Requirements
12+
13+
### Required Ports
14+
15+
Your server MUST allow **outbound** connections to the following ports on our Salt master:
16+
17+
| Port | Protocol | Purpose | Salt Master |
18+
|----------|----------|------------------------------|--------------------|
19+
| **4505** | TCP | Salt Publisher (ZeroMQ) | salt-master.psf.io |
20+
| **4506** | TCP | Salt Request Server (ZeroMQ) | salt-master.psf.io |
21+
22+
### Inbound Access Requirements
23+
24+
Your server MUST allow **inbound** connections on:
25+
26+
| Port | Protocol | Purpose | Access |
27+
|--------|----------|----------------|---------------|
28+
| **22** | TCP | SSH Management | PSF Sysadmins |
29+
30+
### DNS Requirements
31+
32+
- Preferrably, the Server will be accessible via a stable DNS name
33+
- PSF Salt master is accessible at `salt-master.psf.io`
34+
- Static IP address preferred (IP changes require coordination)
35+
36+
## Security Configuration
37+
38+
### SSH Access
39+
40+
**Initial Setup:**
41+
- Root SSH access required for initial bootstrap
42+
- SSH key-based authentication only (no password authentication)
43+
- Source keys from GitHub profiles ([@JacobCoffee](https://github.com/JacobCoffee.keys), [@ewdurbin](https://github.com/ewdurbin.keys))
44+
45+
> **Note**: Root login will be disabled after user accounts are created
46+
47+
**Production Access:**
48+
- SSH access provided to PSF sysadmins and service managers (see [User Management](#user-management))
49+
- All access through SSH keys managed in Salt pillar data
50+
- No direct root access after initial setup
51+
52+
### System Hardening
53+
54+
Salt will automatically apply comprehensive security hardening (see [Salt harden state](https://github.com/python/psf-salt/blob/main/salt/base/harden)):
55+
56+
**SSH Hardening:**
57+
- Root login disabled after bootstrap
58+
- Password authentication disabled
59+
- Strong cryptographic algorithms only
60+
- Connection limits and timeout controls
61+
- X11 forwarding disabled
62+
63+
**System Security:**
64+
- Firewall rules (iptables) with default deny policy
65+
- File system permissions hardened
66+
- Core dumps disabled
67+
- Account lockout policies (5 failed attempts = 10 minute lockout)
68+
- System resource limits configured
69+
70+
**Network Security:**
71+
- Stateful firewall with connection tracking
72+
- IPv4 and IPv6 rules applied
73+
- Only explicitly allowed ports accessible
74+
- Internal network traffic restrictions
75+
76+
## User Management
77+
78+
### User Accounts
79+
80+
**User Management:**
81+
- Created from PSF pillar data (see [Salt users pillar data](https://github.com/python/psf-salt/blob/main/pillar/base/users))
82+
- The pillar data determines, per service, which users are created, their roles (root, etc.), and their SSH keys
83+
- Sudo access granted to `psf-admin` group (see [Salt sudo pillar data](../../pillar/base/sudoers/init.sls))
84+
85+
### SSH Key Management
86+
87+
**Key Sources:**
88+
- SSH keys stored in Salt pillar data
89+
- Automated key rotation capabilities via Salt highstate runs and Git repository updates
90+
91+
**Key Deployment:**
92+
- Keys automatically deployed during Salt runs
93+
- `authorized_keys` files managed by Salt
94+
- Revocation through pillar data updates
95+
96+
## Security Updates
97+
98+
### Automatic Updates
99+
100+
Salt configures Ubuntu's unattended upgrades:
101+
102+
**Update Sources:**
103+
- Ubuntu security updates
104+
- Ubuntu stable updates
105+
- Critical package updates
106+
107+
**Configuration:**
108+
- Automatic installation of security updates
109+
- Email notifications to `[email protected]` (see [Salt unattended-upgrades](../../salt/unattended-upgrades/config/50unattended-upgrades))
110+
111+
**Monitoring:**
112+
- Monitoring generally happens through Sentry or Datadog metric checks.

‎docs/guides/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,4 @@ Guides
99
migration-recipe.md
1010
haproxy-registration-guide.md
1111
port-guide.md
12+
external-host-requirements.md

‎docs/guides/migration-recipe.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,7 @@ index 68387c9..7a8ace1 100644
155155
sudo service nginx stop
156156
```
157157
```{note}
158-
Don't forget to pause service checks for both the old and new hosts in things like Dead Man's Snitch, Pingdom, etc.
158+
Don't forget to pause service checks for both the old and new hosts in things like Sentry monitors, Pingdom, etc.
159159
```
160160
4. Ensure that any additional volumes are mounted and in the correct location:
161161
- Check what disks are currently mounted and where: `df`

‎docs/overview.rst‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,11 @@ Pingdom
6363
`Pingdom <https://www.pingdom.com>`_ provides monitoring and complains to us
6464
when services are down.
6565

66+
Sentry
67+
`Sentry <https://sentry.io>`_ is used for error reporting and monitoring of
68+
many services. It also provides Salt highstate cron monitoring, which
69+
notifies us when runs fail over a certain threshold.
70+
6671
PagerDuty
6772
`PagerDuty <https://www.pagerduty.com>`_ is used for on-call rotation for PSF
6873
Infrastructure employees on the front-line, and volunteers as backup.

‎docs/requirements.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,4 @@ furo
22
sphinx
33
myst-parser
44
sphinx-copybutton
5+
sphinx-datatables

‎docs/salt-server-list.rst‎

Lines changed: 21 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -4,28 +4,29 @@
44

55

66
.. csv-table::
7-
:header: "Name", "Purpose", "Contact", "Distro", "Datacener"
7+
:header: "Name", "Purpose", "Contact", "Distro", "Datacenter", "Category"
8+
:class: sphinx-datatable display compact
89

910

10-
"backup.sfo1.psf.io", "Automated backup of infrastructure", "Infrastructure staff", "Ubuntu-24.04", "sfo1"
11-
"bugs.nyc1.psf.io", "Roundup hosting for CPython, Jython, and Roundup", "Infrastructure staff", "Ubuntu-22.04", "nyc1"
12-
"buildbot.nyc1.psf.io", "Hosting for CPython buildbot server", "zware, haypo, pablogsa", "Ubuntu-24.04", "nyc1"
13-
"cdn-logs.nyc1.psf.io", "Realtime log streaming from Fastly CDN for debug", "Infrastructure Staff", "Ubuntu-20.04", "nyc1"
14-
"codespeed.nyc1.psf.io", "Hosting for speed.python.org and speed.pypy.org", "", "Ubuntu-22.04", "nyc1"
15-
"consul-1.nyc1.psf.io", "Runs `Consul <https://www.consul.io/>`_ discovery service", "Infrastructure Staff", "Ubuntu-24.04", "nyc1"
16-
"consul-2.nyc1.psf.io", "Runs `Consul <https://www.consul.io/>`_ discovery service", "Infrastructure Staff", "Ubuntu-24.04", "nyc1"
17-
"consul-3.nyc1.psf.io", "Runs `Consul <https://www.consul.io/>`_ discovery service", "Infrastructure Staff", "Ubuntu-24.04", "nyc1"
18-
"docs.nyc1.psf.io", "Builds and serves CPython's documentation", "mdk", "Ubuntu-24.04", "nyc1"
19-
"downloads.nyc1.psf.io", "Serves python.org downloads", "CPython Release Managers", "Ubuntu-24.04", "nyc1"
20-
"gnumailman.nyc1.psf.io", "GNU Mailman Project wiki and lists", "Mark Sapiro", "Ubuntu-20.04", "nyc1"
21-
"hg.nyc1.psf.io", "Version Control Archives, serves hg.python.org and svn.python.org", "Infrastructure Staff", "Ubuntu-24.04", "nyc1"
22-
"lb-0.nyc1.psf.io", "Load balancer", "Infrastructure Staff", "Ubuntu-24.04", "nyc1"
23-
"lb-1.nyc1.psf.io", "Load balancer", "Infrastructure Staff", "Ubuntu-24.04", "nyc1"
24-
"mail.ams1.psf.io", "Mail and mailman server", "postmasters", "Ubuntu-14.04", "ams1"
25-
"moin.nyc1.psf.io", "Hosts moin sites for wiki.python.org, wiki.jython.org", "lemburg", "Ubuntu-20.04", "nyc1"
26-
"planet.nyc1.psf.io", "Planet Python", "benjamin", "Ubuntu-18.04", "nyc1"
27-
"pythontest.nyc3.psf.io", "Test resources for CPython's test suite.", "Infrastructure Staff", "Ubuntu-24.04", "nyc3"
28-
"salt.nyc1.psf.io", "Salt server", "Infrastructure Staff", "Ubuntu-20.04", "nyc1"
11+
"backup.sfo1.psf.io", "Automated backup of infrastructure", "Infrastructure staff", "Ubuntu-24.04", "sfo1", "infra-infra"
12+
"bugs.nyc1.psf.io", "Roundup hosting for CPython, Jython, and Roundup", "Infrastructure staff", "Ubuntu-22.04", "nyc1", "python-core"
13+
"buildbot.nyc1.psf.io", "Hosting for CPython buildbot server", "zware, haypo, pablogsa", "Ubuntu-24.04", "nyc1", "python-core"
14+
"cdn-logs.nyc1.psf.io", "Realtime log streaming from Fastly CDN for debug", "Infrastructure Staff", "Ubuntu-20.04", "nyc1", "infra-infra"
15+
"codespeed.nyc1.psf.io", "Hosting for speed.python.org and speed.pypy.org", "", "Ubuntu-22.04", "nyc1", "python-core"
16+
"consul-1.nyc1.psf.io", "Runs `Consul <https://www.consul.io/>`_ discovery service", "Infrastructure Staff", "Ubuntu-24.04", "nyc1", "infra-infra"
17+
"consul-2.nyc1.psf.io", "Runs `Consul <https://www.consul.io/>`_ discovery service", "Infrastructure Staff", "Ubuntu-24.04", "nyc1", "infra-infra"
18+
"consul-3.nyc1.psf.io", "Runs `Consul <https://www.consul.io/>`_ discovery service", "Infrastructure Staff", "Ubuntu-24.04", "nyc1", "infra-infra"
19+
"docs.nyc1.psf.io", "Builds and serves CPython's documentation", "mdk", "Ubuntu-24.04", "nyc1", "python-core"
20+
"downloads.nyc1.psf.io", "Serves python.org downloads", "CPython Release Managers", "Ubuntu-24.04", "nyc1", "python-core"
21+
"gnumailman.nyc1.psf.io", "GNU Mailman Project wiki and lists", "Mark Sapiro", "Ubuntu-20.04", "nyc1", "mail"
22+
"hg.nyc1.psf.io", "Version Control Archives, serves hg.python.org and svn.python.org", "Infrastructure Staff", "Ubuntu-24.04", "nyc1", "python-core"
23+
"lb-0.nyc1.psf.io", "Load balancer", "Infrastructure Staff", "Ubuntu-24.04", "nyc1", "infra-infra"
24+
"lb-1.nyc1.psf.io", "Load balancer", "Infrastructure Staff", "Ubuntu-24.04", "nyc1", "infra-infra"
25+
"mail.ams1.psf.io", "Mail and mailman server", "postmasters", "Ubuntu-14.04", "ams1", "mail"
26+
"moin.nyc1.psf.io", "Hosts moin sites for wiki.python.org, wiki.jython.org", "lemburg", "Ubuntu-20.04", "nyc1", "community"
27+
"planet.nyc1.psf.io", "Planet Python", "benjamin", "Ubuntu-24.04", "nyc1", "community"
28+
"pythontest.nyc3.psf.io", "Test resources for CPython's test suite.", "Infrastructure Staff", "Ubuntu-24.04", "nyc3", "python-core"
29+
"salt.nyc1.psf.io", "Salt server", "Infrastructure Staff", "Ubuntu-20.04", "nyc1", "infra-infra"
2930

3031
..
3132
END AUTOMATED SECTION **DO NOT DIRECTLY EDIT - Salt will blow away your changes!!!**

‎pillar/base/haproxy.sls‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -156,10 +156,6 @@ haproxy:
156156
hsts_preload: False
157157

158158
listens:
159-
hg_ssh:
160-
bind: :20100
161-
service: hg-ssh
162-
163159
buildbot_worker:
164160
bind: :20101
165161
service: buildbot-master-worker

‎pillar/base/planet.sls‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,5 +3,15 @@ planet:
33
- planetpython.org
44
- www.planetpython.org
55
sites:
6-
planetpython.org:
6+
planetpython:
7+
domain: planetpython.org
8+
cache: /srv/cache/
9+
output: /srv/planetpython.org/
10+
image: ghcr.io/python/planetpython:latest
11+
config: config.ini
12+
planetpython-3:
13+
domain: 3.planetpython.org
14+
cache: /srv/cache3/
15+
output: /srv/planetpython.org/3/
16+
image: ghcr.io/python/planetpython-3:latest
717
config: config.ini

‎pillar/base/users/_admin/coffee.sls‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,5 +3,5 @@ users:
33
admin: true
44
fullname: Jacob Coffee
55
ssh_keys:
6-
- ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAddLP9ByPv5ZzebYW5D0zVwmugZOlgkvTPOrj2FqpvPtA0lroLur8w606JV0DYQiAkud+/Q7+7fM8StnNld7oA=
7-
Main@secretive.EcheXMBP.local
6+
- ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM+eCfVgXZw89gO6PMbwh4S2Xjkk/NPsMOM/F5IfIpatcS8ghoKj0lEfcDH9c+yCbOyzRioax5v5fAKNxUz6Tv4=
7+
git@secretive.EcheXMBP.local

0 commit comments

Comments
 (0)