Skip to content

Docs: refresh README usage guide #106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Docs: refresh README usage guide #106

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
275 changes: 134 additions & 141 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,187 +1,180 @@
[![Build Status](https://app.travis-ci.com/postgrespro/pg_wait_sampling.svg?branch=master)](https://app.travis-ci.com/postgrespro/pg_wait_sampling)
[![GitHub license](https://img.shields.io/badge/license-PostgreSQL-blue.svg)](https://raw.githubusercontent.com/postgrespro/pg_wait_sampling/master/LICENSE)

`pg_wait_sampling` – sampling based statistics of wait events
=============================================================
# pg_wait_sampling

Introduction
------------
Sampling-based statistics of wait events.

PostgreSQL provides information about current wait event of particular
process. However, in order to gather descriptive statistics of server
behavior user have to sample current wait event multiple times.
`pg_wait_sampling` is an extension for collecting sampling statistics of wait
events.
## Introduction

The module must be loaded by adding `pg_wait_sampling` to
`shared_preload_libraries` in postgresql.conf, because it requires additional
shared memory and launches background worker. This means that a server restart
is needed to add or remove the module.
PostgreSQL reports the current wait event for each backend. However, gathering descriptive statistics about server behavior requires repeatedly sampling wait events. `pg_wait_sampling` automates this process and provides sampling-based statistics for wait events.

When used with `pg_stat_statements` it is recommended to put `pg_stat_statements`
before `pg_wait_sampling` in `shared_preload_libraries` so queryIds of
utility statements are not rewritten by the former.
The extension must be loaded via `shared_preload_libraries`, because it allocates shared memory and runs a background worker. Any change to the preload list requires a server restart.

When `pg_wait_sampling` is enabled, it collects two kinds of statistics.
When `pg_stat_statements` is used together with `pg_wait_sampling`, list `pg_stat_statements` before `pg_wait_sampling` in `shared_preload_libraries`. That order keeps utility statement query IDs intact.

* History of waits events. It's implemented as in-memory ring buffer where
samples of each process wait events are written with given (configurable)
period. Therefore, for each running process user can see some number of
recent samples depending on history size (configurable). Assuming there is
a client who periodically read this history and dump it somewhere, user
can have continuous history.
* Waits profile. It's implemented as in-memory hash table where count
of samples are accumulated per each process and each wait event
(and each query with `pg_stat_statements`). This hash
table can be reset by user request. Assuming there is a client who
periodically dumps profile and resets it, user can have statistics of
intensivity of wait events among time.
Once enabled, `pg_wait_sampling` collects two types of statistics:

In combination with `pg_stat_statements` this extension can also provide
per query statistics.
- **History**: an in-memory ring buffer that stores recent wait-event samples for every backend. A client can periodically read and archive this history to build a continuous timeline.
- **Profile**: an in-memory hash table that aggregates wait-event sample counts per backend, wait event, and (optionally) query when `pg_stat_statements` is available. A client can periodically dump and reset the profile to monitor wait intensity over time.

`pg_wait_sampling` launches special background worker for gathering the
statistics above.
A dedicated background worker populates these statistics.

Availability
------------
## Availability

`pg_wait_sampling` is implemented as an extension and not available in default
PostgreSQL installation. It is available from
[github](https://github.com/postgrespro/pg_wait_sampling)
under the same license as
[PostgreSQL](http://www.postgresql.org/about/licence/)
and supports PostgreSQL 13+.

Installation
------------
`pg_wait_sampling` is distributed as a PostgreSQL extension under the PostgreSQL license. Source code is available on [GitHub](https://github.com/postgrespro/pg_wait_sampling), and PostgreSQL 13 or later is required.

Pre-built `pg_wait_sampling` packages are provided in official PostgreSQL
repository: https://download.postgresql.org/pub/repos/
## Installation

Manual build
------------
Pre-built packages are published in the official PostgreSQL repository: https://download.postgresql.org/pub/repos/

`pg_wait_sampling` is PostgreSQL extension which requires PostgreSQL 13 or
higher. Before build and install you should ensure following:

* PostgreSQL version is 13 or higher.
* You have development package of PostgreSQL installed or you built
PostgreSQL from source.
* Your PATH variable is configured so that `pg_config` command available, or
set PG_CONFIG variable.
### Manual build

Typical installation procedure may look like this:

$ git clone https://github.com/postgrespro/pg_wait_sampling.git
$ cd pg_wait_sampling
$ make USE_PGXS=1
$ sudo make USE_PGXS=1 install
`pg_wait_sampling` requires PostgreSQL 13 or newer. Before building from source, make sure:

Then add `shared_preload_libraries = pg_wait_sampling` to `postgresql.conf` and
restart the server.
- A supported PostgreSQL version is installed or built from source.
- The PostgreSQL development files are available.
- `pg_config` is on `PATH`, or `PG_CONFIG` points to it.

To test your installation:
Typical build sequence:

$ make USE_PGXS=1 installcheck
```sh
git clone https://github.com/postgrespro/pg_wait_sampling.git
cd pg_wait_sampling
make USE_PGXS=1
sudo make USE_PGXS=1 install
```

To create the extension in the target database:
Add `pg_wait_sampling` to `shared_preload_libraries` in `postgresql.conf` and restart the server.

CREATE EXTENSION pg_wait_sampling;
To run regression tests:

Compilation on Windows is not supported, since the extension uses symbols from PostgreSQL
that are not exported.
```sh
make USE_PGXS=1 installcheck
```

Usage
-----
Create the extension in the target database:

`pg_wait_sampling` interacts with user by set of views and functions.
```sql
CREATE EXTENSION pg_wait_sampling;
```

`pg_wait_sampling_current` view – information about current wait events for
all processed including background workers.
Compilation on Windows is not supported because the extension relies on PostgreSQL symbols that are undefined on that platform.

| Column name | Column type | Description |
| ----------- | ----------- | ----------------------- |
| pid | int4 | Id of process |
| event_type | text | Name of wait event type |
| event | text | Name of wait event |
| queryid | int8 | Id of query |
## Usage

`pg_wait_sampling_get_current(pid int4)` returns the same table for single given
process.
`pg_wait_sampling` exposes views, functions, and GUC parameters.

`pg_wait_sampling_history` view – history of wait events obtained by sampling into
in-memory ring buffer.
### Views

| Column name | Column type | Description |
| ----------- | ----------- | ----------------------- |
| pid | int4 | Id of process |
| ts | timestamptz | Sample timestamp |
| event_type | text | Name of wait event type |
| event | text | Name of wait event |
| queryid | int8 | Id of query |
#### `pg_wait_sampling_current`

`pg_wait_sampling_profile` view – profile of wait events obtained by sampling into
in-memory hash table.
Current wait events for all processes, including background workers.

| Column name | Column type | Description |
| ----------- | ----------- | ----------------------- |
| pid | int4 | Id of process |
| event_type | text | Name of wait event type |
| event | text | Name of wait event |
| queryid | int8 | Id of query |
| count | text | Count of samples |
| Column | Type | Description |
|--------|------|-------------|
| `pid` | `int4` | Process ID |
| `event_type` | `text` | Wait event type |
| `event` | `text` | Wait event name |
| `queryid` | `int8` | Query identifier |

`pg_wait_sampling_reset_profile()` function resets the profile.
#### `pg_wait_sampling_history`

The work of wait event statistics collector worker is controlled by following
GUCs.
History of wait events sampled into an in-memory ring buffer.

| Parameter name | Data type | Description | Default value |
|----------------------------------| --------- |---------------------------------------------|--------------:|
| pg_wait_sampling.history_size | int4 | Size of history in-memory ring buffer | 5000 |
| pg_wait_sampling.history_period | int4 | Period for history sampling in milliseconds | 10 |
| pg_wait_sampling.profile_period | int4 | Period for profile sampling in milliseconds | 10 |
| pg_wait_sampling.profile_pid | bool | Whether profile should be per pid | true |
| pg_wait_sampling.profile_queries | enum | Whether profile should be per query | top |
| pg_wait_sampling.sample_cpu | bool | Whether on CPU backends should be sampled | true |
| Column | Type | Description |
|--------|------|-------------|
| `pid` | `int4` | Process ID |
| `ts` | `timestamptz` | Sample timestamp |
| `event_type` | `text` | Wait event type |
| `event` | `text` | Wait event name |
| `queryid` | `int8` | Query identifier |

If `pg_wait_sampling.profile_pid` is set to false, sampling profile wouldn't be
collected in per-process manner. In this case the value of pid could would
be always zero and corresponding row contain samples among all the processes.
#### `pg_wait_sampling_profile`

If `pg_wait_sampling.profile_queries` is set to `none`, `queryid` field in
views will be zero. If it is set to `top`, queryIds only of top level statements
are recorded. If it is set to `all`, queryIds of nested statements are recorded.
Aggregated wait-event samples collected in an in-memory hash table.

If `pg_wait_sampling.sample_cpu` is set to true then processes that are not
waiting on anything are also sampled. The wait event columns for such processes
will be NULL.
| Column | Type | Description |
|--------|------|-------------|
| `pid` | `int4` | Process ID |
| `event_type` | `text` | Wait event type |
| `event` | `text` | Wait event name |
| `queryid` | `int8` | Query identifier |
| `count` | `int8` | Number of samples |

Values of these GUC variables can be changed only in config file or with ALTER SYSTEM.
Then you need to reload server's configuration (such as with pg_reload_conf function)
for changes to take effect.
### Functions

See
[PostgreSQL documentation](http://www.postgresql.org/docs/devel/static/monitoring-stats.html#WAIT-EVENT-TABLE)
for list of possible wait events.
- `pg_wait_sampling_get_current(pid int4)`: returns the same columns as `pg_wait_sampling_current` for a single backend.
- `pg_wait_sampling_reset_profile()`: clears the aggregated wait profile.

Contribution
------------
### Configuration parameters

Please, notice, that `pg_wait_sampling` is still under development and while
it's stable and tested, it may contains some bugs. Don't hesitate to raise
[issues at github](https://github.com/postgrespro/pg_wait_sampling/issues) with
your bug reports.
The background worker is controlled by these GUC parameters:

If you're lacking of some functionality in `pg_wait_sampling` and feeling power
to implement it then you're welcome to make pull requests.
| Parameter | Type | Description | Default |
|-----------|------|-------------|--------:|
| `pg_wait_sampling.history_size` | `int4` | Size of the in-memory history buffer | 5000 |
| `pg_wait_sampling.history_period` | `int4` | History sampling period in milliseconds | 10 |
| `pg_wait_sampling.profile_period` | `int4` | Profile sampling period in milliseconds | 10 |
| `pg_wait_sampling.profile_pid` | `bool` | Collect profile per PID | true |
| `pg_wait_sampling.profile_queries` | `enum` | Collect profile per query | top |
| `pg_wait_sampling.sample_cpu` | `bool` | Sample backends running on CPU | true |

Authors
-------
If `pg_wait_sampling.profile_pid` is `false`, the profile is aggregated across all processes and `pid` is always zero.

* Alexander Korotkov <[email protected]>, Postgres Professional,
Moscow, Russia
* Ildus Kurbangaliev <[email protected]>, Postgres Professional,
Moscow, Russia
If `pg_wait_sampling.profile_queries` is `none`, `queryid` is zero in all views. With `top`, only top-level statements get query IDs; with `all`, nested statements are included.

If `pg_wait_sampling.sample_cpu` is `true`, sessions that are not waiting on anything are also sampled and the wait-event columns are `NULL`.

Change these parameters in the configuration file or with `ALTER SYSTEM`, then reload the server configuration for changes to take effect.

Refer to the [PostgreSQL documentation](https://www.postgresql.org/docs/current/monitoring-stats.html#WAIT-EVENT-TABLE) for the list of wait events.

### Configuration

Add the extension to `shared_preload_libraries` and adjust sampling settings if needed:

```conf
# postgresql.conf
shared_preload_libraries = 'pg_stat_statements,pg_wait_sampling'
pg_wait_sampling.history_size = 10000 # samples kept in history
pg_wait_sampling.profile_period = 50 # ms between profile samples
```

Restart the server and create the extensions:

```sql
CREATE EXTENSION pg_stat_statements;
CREATE EXTENSION pg_wait_sampling;
```

> Note (PostgreSQL 14+): if you do not rely on `pg_stat_statements` for query IDs, set `compute_query_id = 'auto'` (or `'on'`) so `queryid` is populated.

### Example: top queries by wait samples

```sql
SELECT s.query,
SUM(p.count) AS samples
FROM pg_wait_sampling_profile AS p
JOIN pg_stat_statements AS s ON s.queryid = p.queryid
GROUP BY s.query
ORDER BY samples DESC
LIMIT 5;
```

This query highlights the statements that accumulated the largest number of wait-event samples.

### Load order

Load `pg_stat_statements` before `pg_wait_sampling` in `shared_preload_libraries` to keep query identifiers consistent.

## Contributing

`pg_wait_sampling` is actively developed and tested, but issues may still surface. Report problems or ideas on the [GitHub issue tracker](https://github.com/postgrespro/pg_wait_sampling/issues).

Contributions are welcome. If you plan to implement new functionality, open a discussion or submit a pull request.

## Authors

- Alexander Korotkov <[email protected]>, Postgres Professional, Moscow, Russia
- Ildus Kurbangaliev <[email protected]>, Postgres Professional, Moscow, Russia