General PostgreSQL role for Debian- and RedHat-based hosts.

Author: natefoo

.gitignore

@@ -0,0 +1,4 @@
+syntax: glob
+
+*~
+*.*.swp

README.md

@@ -0,0 +1,187 @@
+PostgreSQL
+==========
+
+An [Ansible][ansible] role for installing and managing [PostgreSQL][postgresql]
+servers. This role works with both Debian and RedHat based systems, and
+provides backup scripts for [PostgreSQL Continuous Archiving and Point-in-Time
+Recovery][postgresql_pitr].
+
+On RedHat-based platforms, the [PostgreSQL Global Development Group (PGDG)
+packages][pgdg_yum] packages will be installed. On Debian-based platforms, you
+can choose from the distribution's packages (from APT) or the [PGDG
+packages][pgdg_apt].
+
+[ansible]: http://www.ansible.com/
+[postgresql]: http://www.postgresql.org/
+[postgresql_pitr]: http://www.postgresql.org/docs/9.4/static/continuous-archiving.html
+[pgdg_yum]: http://yum.postgresql.org/
+[pgdg_apt]: http://apt.postgresql.org/
+
+**Changes that require a restart will not be applied unless you manually
+restart PostgreSQL.** This role will reload the server for those configuration
+changes that can be updated with only a reload because reloading is a
+non-intrusive operation, but options that require a full restart will not cause
+the server to restart.
+
+Requirements
+------------
+
+This role requires Ansible 1.8+
+
+Role Variables
+--------------
+
+### All variables are optional ###
+
+- `postgresql_version`: PostgreSQL version to install. On Debian-based
+  platforms, the default is whatever version is pointed to by the `postgresql`
+  metapackage). On RedHat-based platforms, the default is `9.4`.
+
+- `postgresql_flavor`: On Debian-based platforms, this specifies whether you
+  want to use PostgreSQL packages from pgdg or the distribution's apt
+  repositories. Possible values: `apt`, `pgdg` (default: `apt`).
+
+- `postgresql_conf`: A hash (dictionary) of `postgresql.conf` options and
+  values. These options are not added to `postgresql.conf` directly - the role
+  adds a `conf.d` subdirectory in the configuration directory and an include
+  statement for that directory to `postgresql.conf`. Options set in
+  `postgresql_conf` are then set in `conf.d/25ansible_postgresql.conf`.
+
+  Due to YAML parsing, you must take care when defining values in
+  `postgresql_conf` to ensure they are properly written to the config file. For
+  example:
+
+  ```yaml
+  postgresql_conf:
+    max_connections: 250
+    archive_mode: "off"
+    work_mem: "'8MB'"
+  ```
+
+  Becomes the following in `25ansible_postgresql.conf`:
+
+  ```
+  max_connections = 250
+  archive_mode = off
+  work_mem: '8MB'
+  ```
+
+- `postgresql_pg_hba_conf`: A list of lines to add to `pg_hba.conf`
+
+- `postgresql_pg_hba_local_postgres_user`: If set to `false`, this will remove
+  the `postgres` user's entry from `pg_hba.conf` that is preconfigured on
+  Debian-based PostgreSQL installations. You probably do not want to do this
+  unless you know what you're doing.
+
+- `postgresql_pg_hba_local_socket`: If set to `false`, this will remove the
+  `local` entry from `pg_hba.conf` that is preconfigured by the PostgreSQL
+  package.
+
+- `postgresql_pg_hba_local_ipv4`: If set to `false`, this will remove the `host
+  ... 127.0.0.1/32` entry from `pg_hba.conf` that is preconfigured by the
+  PostgreSQL package.
+
+- `postgresql_pg_hba_local_ipv6`: If set to `false`, this will remove the `host
+  ... ::1/128` entry from `pg_hba.conf` that is preconfigured by the PostgreSQL
+  package.
+
+- `postgresql_pgdata_dir`: Only set this if you have changed the `$PGDATA`
+  directory from the package default. Note this does not configure PostgreSQL
+  to actually use a different directory, you will need to do that yourself, it
+  just allows the role to properly locate the directory.
+
+- `postgresql_conf_dir`: As with `postgresql_pgdata_dir` except for the
+  configuration directory.
+
+### Backups ###
+
+- `postgresql_backup_dir`: If set, enables [PITR][postgresql_pitr] backups. Set
+  this to a directory where your database will be backed up.
+
+- `postgresql_backup_local_dir`: Filesystem path on the PostgreSQL server where
+  backup scripts will be placed and working WALs will be written prior to a WAL
+  archive.
+
+- `postgresql_backup_[hour|minute]`: Controls what time the cron job will run
+  to perform a full backup. Defaults to 1:00 AM.
+
+- `postgresql_backup_[day|month|weekday]`: Additional cron controls for when
+  the full backup is performed (default: `*`).
+
+- `postgresql_backup_mail_recipient`: User or address that should receive mail
+  from the backup scripts.
+
+Dependencies
+------------
+
+None
+
+Example Playbook
+----------------
+
+Standard install: Default `postgresql.conf`, `pg_hba.conf` and default version
+for the OS:
+
+```yaml
+---
+
+- hosts: dbservers
+  remote_user: root
+  roles:
+    - postgresql
+```
+
+Use the pgdg packages on a Debian-based host:
+
+```yaml
+---
+
+- hosts: dbservers
+  remote_user: root
+  vars:
+    postgresql_flavor: pgdg
+  roles:
+    - postgresql
+```
+
+Use the PostgreSQL 9.3 packages and set some `postgresql.conf` options and
+`pg_hba.conf` entries:
+
+```yaml
+---
+
+- hosts: dbservers
+  remote_user: root
+  vars:
+    postgresql_version: 9.3
+    postgresql_conf:
+      listen_addresses: "''"    # disable network listening (listen on unix socket only)
+      max_connections: 50       # decrease connection limit
+    postgresql_pg_hba_conf:
+      - host all all 10.0.0.0/8 md5
+  roles:
+    - postgresql
+```
+
+Enable backups to /archive
+
+```yaml
+- hosts: all
+  remote_user: root
+  vars:
+    postgresql_backup_dir: /archive
+  roles:
+    - postgresql
+```
+
+License
+-------
+
+[Academic Free License ("AFL") v. 3.0][afl]
+
+[afl]: http://opensource.org/licenses/AFL-3.0
+
+Author Information
+------------------
+
+[Nate Coraor](https://github.com/natefoo)  

defaults/main.yml

@@ -0,0 +1,6 @@
+---
+
+postgresql_default_version: 9.4
+postgresql_backup_local_dir: ~postgres/backup
+postgresql_backup_active_dir: "{{ postgresql_backup_local_dir }}/active"
+postgresql_backup_mail_recipient: postgres

files/get_repo_rpm_release.py

@@ -0,0 +1,32 @@
+#!/usr/bin/env python
+"""
+Determine the latest version of the yum repository package.
+
+usage: get_repo_rpm_version.py url distribution
+
+e.g.:
+
+get_repo_rpm_version.py http://yum.postgresql.org/9.2/redhat/rhel-6-x86_64/ centos
+"""
+
+import re
+import sys
+import urllib2
+
+url, dist = sys.argv[1:]
+
+try:
+    repo = urllib2.urlopen(url)
+except urllib2.HTTPError, e:
+    print >>sys.stderr, "Failed to fetch directory list from %s" % url
+    raise
+
+pg_version = url.split('/')[3]
+re_pattern = 'href=[\'"]pgdg-%s%s-%s-([\d+]).noarch.rpm[\'"]' % (dist, pg_version.replace('.', ''), pg_version)
+match = re.findall(re_pattern, repo.read(), flags=re.I)
+
+assert match, "No matching %s pgdg repository packages found for version %s at %s" % (dist, pg_version, url)
+
+print max([ int(x) for x in match ])
+
+sys.exit(0)

handlers/main.yml

@@ -0,0 +1,4 @@
+---
+
+- name: Reload PostgreSQL
+  service: name={{ postgresql_service_name }} state=reloaded

tasks/backup.yml

@@ -0,0 +1,25 @@
+---
+
+- name: Create backup directories
+  file: owner=postgres group=postgres mode=0750 state=directory path={{ item }}
+  with_items:
+    - "{{ postgresql_backup_local_dir }}"
+    - "{{ postgresql_backup_local_dir }}/bin"
+    - "{{ postgresql_backup_active_dir }}"
+
+- name: Install backup scripts
+  template: src={{ item }}.j2 dest={{ postgresql_backup_local_dir }}/bin/{{ item }} owner=postgres group=postgres mode=0750
+  with_items:
+    - backup_working_wal.sh
+    - archive_wal.sh
+    - scheduled_backup.sh
+
+- name: Set WAL archive config options
+  template: src=20ansible_backup.conf.j2 dest={{ postgresql_conf_dir }}/conf.d/20ansible_backup.conf owner=postgres group=postgres backup=yes
+  notify: Reload PostgreSQL
+
+- name: Schedule backups
+  cron: name="PostgreSQL Backup" cron_file=ansible_postgresql_backup user=postgres hour={{ postgresql_backup_hour | default(1) }} minute={{ postgresql_backup_minute | default(0) }} day={{ postgresql_backup_day | default(omit) }} month={{ postgresql_backup_month | default(omit) }} weekday={{ postgresql_backup_weekday | default(omit) }} job={{ postgresql_backup_local_dir }}/bin/scheduled_backup.sh
+
+- name: Schedule PostgreSQL working WAL backup
+  cron: name="PostgreSQL WAL Backup" cron_file=ansible_postgresql_walbackup user=postgres job={{ postgresql_backup_local_dir }}/bin/backup_working_wal.sh

tasks/debian.yml

@@ -0,0 +1,22 @@
+---
+
+- name: Install pgdg package signing key (Debian/pgdg)
+  apt_key: keyserver=pgp.mit.edu id=ACCC4CF8
+  when: postgresql_flavor is defined and postgresql_flavor == "pgdg"
+  
+- name: Install pgdg repository (Debian/pgdg)
+  apt_repository: repo="deb http://apt.postgresql.org/pub/repos/apt/ {{ ansible_distribution_release }}-pgdg main" update_cache=yes
+  when: postgresql_flavor is defined and postgresql_flavor == "pgdg"
+
+- name: Install PostgreSQL (Debian)
+  apt: name=postgresql{{ '-' ~ postgresql_version if postgresql_version is defined else '' }}
+
+- name: Get installed version
+  command: dpkg-query -f ${Version;3} --show postgresql
+  when: postgresql_version is not defined
+  register: postgresql_version_query
+  changed_when: false
+
+- name: Set version fact
+  set_fact: postgresql_version={{ postgresql_version_query.stdout }}
+  when: postgresql_version is not defined

tasks/main.yml

@@ -0,0 +1,47 @@
+---
+
+# Convenient ordering - debian.yml sets postgresql_version if it is unset,
+# which is needed by the include_vars files.
+- include: debian.yml
+  when: ansible_os_family == "Debian"
+
+# For RedHat we use the value from defaults/main.yml
+- name: Set version fact
+  set_fact: postgresql_version={{ postgresql_default_version }}
+  when: ansible_os_family == "RedHat" and postgresql_version is not defined
+
+# Sets postgresql_pgdata_dir, postgresql_conf_dir
+- include_vars: "{{ ansible_os_family | lower }}.yml"
+
+- name: Set pgdata fact
+  set_fact: postgresql_pgdata={{ postgresql_pgdata_default }}
+  when: postgresql_pgdata is not defined
+
+- name: Set conf dir fact
+  set_fact: postgresql_conf_dir={{ postgresql_conf_dir_default }}
+  when: postgresql_conf_dir is not defined
+
+# Needs postgresql_pgdata_dir set
+- include: redhat.yml
+  when: ansible_os_family == "RedHat"
+
+- name: Create conf.d
+  file: path={{ postgresql_conf_dir }}/conf.d state=directory owner=postgres group=postgres
+
+- name: Set conf.d include in postgresql.conf
+  lineinfile: line="include_dir 'conf.d'" dest={{ postgresql_conf_dir }}/postgresql.conf backup=yes
+  notify: Reload PostgreSQL
+
+- name: Set config options
+  template: src=25ansible_postgresql.conf.j2 dest={{ postgresql_conf_dir }}/conf.d/25ansible_postgresql.conf owner=postgres group=postgres backup=yes
+  notify: Reload PostgreSQL
+
+- name: Install pg_hba.conf
+  template: src=pg_hba.conf.{{ ansible_os_family | lower }}.j2 dest={{ postgresql_conf_dir }}/pg_hba.conf owner=postgres group=postgres mode=0400 backup=yes
+  notify: Reload PostgreSQL
+
+- include: backup.yml
+  when: postgresql_backup_dir is defined
+
+- name: Ensure PostgreSQL is running
+  service: name={{ postgresql_service_name }} enabled=yes state=started

tasks/redhat.yml

@@ -0,0 +1,40 @@
+---
+
+# Using the rpm URL format of the yum module causes Ansible to download the rpm
+# every time to check whether it's installed, so, don't do that.
+- name: Check pgdg repository package (RedHat)
+  yum: name=pgdg-{{ postgresql_pgdg_dists[ansible_distribution] }}{{ postgresql_version | replace('.', '') }}
+  register: repo_pkg_installed
+  ignore_errors: yes
+
+# URLs for Fedora look like:
+#   http://yum.postgresql.org/<pg_version>/fedora/fedora-<os_major_version>-<arch>/pgdg-fedora<pg_version_without_period>-<pg_version>-<rpm_release_version>.noarch.rpm
+# URLs for RedHat and all derivatives look like:
+#   http://yum.postgresql.org/<pg_version>/redhat/rhel-<os_major_version>-<arch>/pgdg-<dist><pg_version_without_period>-<pg_version>-<rpm_release_version>.noarch.rpm
+
+# There's no direct way to determine the latest pacakge, so we have to use a
+# helper script to parse the directory list and figure it out.
+- name: Determine latest pgdg repository package (RedHat)
+  script: get_repo_rpm_release.py http://yum.postgresql.org/{{ postgresql_version }}/{{ postgresql_pgdg_families[ansible_distribution] | default("redhat") }}/{{ postgresql_pgdg_shortfamilies[ansible_distribution] | default("rhel") }}-{{ ansible_distribution_major_version }}-{{ ansible_architecture }}/ {{ postgresql_pgdg_dists[ansible_distribution] }}
+  register: pgdg_repo_pkg_release
+  when: repo_pkg_installed.failed is defined and repo_pkg_installed.failed
+
+- name: Install pgdg repository package (RedHat)
+  yum: name=http://yum.postgresql.org/{{ postgresql_version }}/{{ postgresql_pgdg_families[ansible_distribution] | default("redhat") }}/{{ postgresql_pgdg_shortfamilies[ansible_distribution] | default("rhel") }}-{{ ansible_distribution_major_version }}-{{ ansible_architecture }}/pgdg-{{ postgresql_pgdg_dists[ansible_distribution] }}{{ postgresql_version | replace('.', '') }}-{{ postgresql_version }}-{{ pgdg_repo_pkg_release.stdout.strip() }}.noarch.rpm
+  when: repo_pkg_installed.failed is defined and repo_pkg_installed.failed
+
+- name: Install PostgreSQL (RedHat)
+  yum: name=postgresql{{ postgresql_version | replace('.', '') }}-server
+
+- name: Check for pgdata directory
+  stat: path={{ postgresql_pgdata }}/base
+  register: pgdata_stat
+  failed_when: false
+
+- name: Initialize database (RedHat < 7)
+  command: /sbin/service postgresql-{{ postgresql_version }} initdb
+  when: ansible_distribution_major_version | int < 7 and (pgdata_stat.stat.isdir is not defined or not pgdata_stat.stat.isdir)
+
+- name: Initialize database (RedHat >= 7)
+  command: /usr/pgsql-{{ postgresql_version }}/bin/postgresql{{ postgresql_version | replace('.', '') }}-setup initdb
+  when: ansible_distribution_major_version | int >= 7 and (pgdata_stat.stat.isdir is not defined or not pgdata_stat.stat.isdir)

templates/20ansible_backup.conf.j2

@@ -0,0 +1,9 @@
+##
+## This file is maintained by Ansible - CHANGES WILL BE OVERWRITTEN
+##
+
+{% if postgresql_backup_dir is defined and postgresql_backup_local_dir is defined %}
+wal_level = archive
+archive_mode = on
+archive_command = '{{ postgresql_backup_local_dir | expanduser }}/bin/archive_wal.sh "%p" "%f" main'
+{% endif %}