From c1cf292169be1bafc8822e0e9e0823f6d058f4d5 Mon Sep 17 00:00:00 2001 From: fpagny Date: Tue, 8 Oct 2024 12:33:28 +0200 Subject: [PATCH 1/3] Create supported-postgresql-extensions.md Adding documentation for extensions support with Serverless SQL Database --- .../supported-postgresql-extensions.md | 93 +++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 serverless/sql-databases/reference-content/supported-postgresql-extensions.md diff --git a/serverless/sql-databases/reference-content/supported-postgresql-extensions.md b/serverless/sql-databases/reference-content/supported-postgresql-extensions.md new file mode 100644 index 0000000000..7c93aa0501 --- /dev/null +++ b/serverless/sql-databases/reference-content/supported-postgresql-extensions.md @@ -0,0 +1,93 @@ +--- +meta: + title: Supported PostgreSQL extensions + description: This page describes how to use PostgreSQL extensions with Serverless SQL Databases +content: + h1: Supported PostgreSQL extensions + paragraph: This page describes how to use PostgreSQL extensions with Serverless SQL Databases +tags: sql-databases serverless database extensions postgresql pgvector postgis timescaledb +dates: + validation: 2024-10-08 + posted: 2024-10-08 +categories: + - serverless +--- + +Serverless SQL Database supports most popular [PostgreSQL extensions](#install-an-extension). Due to autoscaling and built-in connection pooling, some [advanced features of extensions might be limited or require workarounds](#supported-extensions). + + + +- A Scaleway account logged into the [console](https://console.scaleway.com) +- [Owner](/identity-and-access-management/iam/concepts/#owner) status or [IAM permissions](/identity-and-access-management/iam/concepts/#permission) allowing you to perform actions in the intended Organization +- [Created a Serverless SQL Database](/serverless/sql-databases/how-to/create-a-database/) + +## Install an extension + +1. [Connect to your Serverless SQL Database](https://www.scaleway.com/en/docs/serverless/sql-databases/quickstart/#how-to-connect-to-a-database) with a PostgreSQL client. For instance with psql: + ```bash + psql "postgres://[user-or-application-id]:[api-secret-key]@[database-hostname]:5432/[database-name]?sslmode=require" + ``` + +2. Create extension using SQL: + ```sql + CREATE EXTENSION extension_name; + ``` + +3. Use extension with its supported features. Assuming you installed `pgvector` with `CREATE EXTENSION vector`, you can for instance create a vector and query it: + ```sql + CREATE TABLE songs (id bigserial PRIMARY KEY, embedding vector(3)); + INSERT INTO songs (embedding) VALUES ('[1,1,1]'), ('[2,2,2]'), ('[0,1,2]'); + SELECT * FROM songs ORDER BY embedding <-> '[3,3,3]' LIMIT 5; + ``` + This example will find the nearest vectors (using L2 distance) from '[3,3,3]': + ```bash + id | embedding + ----+----------- + 2 | [2,2,2] + 1 | [1,1,1] + 3 | [0,1,2] + ``` + +## Supported extensions + +The following PostgreSQL extensions are available with Serverless SQL Databases: + + +| Extension | Description and known limitations | +|------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------| +| address_standardizer | Parses addresses into their constituent elements, typically used in the geocoding normalization process. | +| address_standardizer_data_us | Provides a dataset example for the US Address Standardizer extension. | +| bloom | Implements a Bloom filter-based index, ideal for situations where space efficiency is crucial. | +| btree_gin | Adds support for indexing common data types using the GIN (Generalized Inverted Index) method. | +| btree_gist | Adds support for indexing common data types using the GiST (Generalized Search Tree) method. | +| citext | Provides a case-insensitive character string data type for easier text comparison. | +| cube | Defines a data type for representing multidimensional cubes, useful in mathematical and geometrical computations. | +| dict_int | Offers a text search dictionary template optimized for integer values. | +| dict_xsyn | Provides a text search dictionary template for advanced synonym processing, improving search accuracy. Features requiring direct access to the filesystem will not be supported (eg. custom synonym lists) | +| earthdistance | Calculates great-circle distances between points on Earth, useful in geographical applications. | +| fuzzystrmatch | Determines similarities and differences between strings, helpful for tasks like data deduplication. | +| hstore | Stores sets of key-value pairs in a single value, enabling a flexible schema for semi-structured data. | +| intagg | Aggregates and enumerates integer values, though it is considered obsolete since PG 8.4 and official support has been integrated in PostgreSQL directly | +| intarray | Provides functions, operators, and indexing support for one-dimensional arrays of integers, enhancing array processing. | +| isn | Supports data types for international product numbering standards, such as ISBN and EAN. | +| ltree | Defines a data type for representing tree-like structures in a hierarchical manner, facilitating complex data modeling. | +| pg_cron | Schedules jobs within PostgreSQL using a cron-like syntax, automating routine database tasks. `pg_cron` requires the database to be in active state permanently and thus that you setup a minimum vCPU above 1 to properly work. | +| pg_prewarm | Preloads relation data into memory to reduce initial disk I/O latency and improve performance. | +| pg_stat_statements | Tracks planning and execution statistics for all SQL statements, aiding in performance tuning. | +| pg_trgm | Measures text similarity and provides index searching based on trigrams, enhancing text search capabilities. | +| pgcrypto | Provides cryptographic functions for data encryption, decryption, and hashing, essential for secure data storage. | +| pgrouting | Extends PostgreSQL and PostGIS with geospatial routing capabilities, enabling the calculation of shortest paths and other routing operations. | +| pgrowlocks | Displays information about row-level locking, helping in the diagnosis of concurrency issues. | +| pgvector | Supports vector similarity search, often used for storing and searching embeddings in machine learning applications. [Query options](https://github.com/pgvector/pgvector?tab=readme-ov-file#query-options) using `SET` command require [to be used in a single transaction](https://www.scaleway.com/en/docs/serverless/sql-databases/reference-content/known-differences/#unsupported-configuration-commands) (ie. between `BEGIN; (...) COMMIT;`) | +| plpgsql | Enables the use of PL/pgSQL, a procedural language for PostgreSQL that allows complex control structures and query manipulation. | +| postgis | Adds support for geographic objects, allowing location queries to be run in SQL, and is the foundation of geographic information system (GIS) functionality in PostgreSQL. | +| postgis_raster | Implements raster data support in PostGIS, enabling analysis and storage of raster images alongside vector data. | +| postgis_sfcgal | Adds advanced 3D geometry functions to PostGIS, leveraging the SFCGAL library for complex spatial operations. | +| postgis_tiger_geocoder | Provides geocoding and reverse geocoding capabilities based on the US Census TIGER data within PostGIS. | +| postgis_topology | Supports topological data models in PostGIS, allowing for advanced spatial relationships and analyses. | +| tablefunc | Offers functions for manipulating entire tables, including crosstab operations for pivot tables. | +| timescaledb | Enables efficient handling and analysis of time-series data, extending PostgreSQL with specialized time-series functions. | +| tsm_system_rows | Implements a TABLESAMPLE method that limits the number of rows returned, useful for sampling large datasets. | +| tsm_system_time | Implements a TABLESAMPLE method that limits rows based on a time duration, useful for time-based sampling. | +| unaccent | Provides a text search dictionary that removes accents from characters, improving text search accuracy. | +| uuid-ossp | Generates universally unique identifiers (UUIDs), essential for ensuring data uniqueness across distributed systems. | From 69455985bc75ffad3ef2a46a41d786582a573357 Mon Sep 17 00:00:00 2001 From: SamyOubouaziz Date: Wed, 9 Oct 2024 11:40:25 +0200 Subject: [PATCH 2/3] docs(SRV): update --- ...md => supported-postgresql-extensions.mdx} | 30 ++++++++++--------- 1 file changed, 16 insertions(+), 14 deletions(-) rename serverless/sql-databases/reference-content/{supported-postgresql-extensions.md => supported-postgresql-extensions.mdx} (80%) diff --git a/serverless/sql-databases/reference-content/supported-postgresql-extensions.md b/serverless/sql-databases/reference-content/supported-postgresql-extensions.mdx similarity index 80% rename from serverless/sql-databases/reference-content/supported-postgresql-extensions.md rename to serverless/sql-databases/reference-content/supported-postgresql-extensions.mdx index 7c93aa0501..c5253d74a7 100644 --- a/serverless/sql-databases/reference-content/supported-postgresql-extensions.md +++ b/serverless/sql-databases/reference-content/supported-postgresql-extensions.mdx @@ -1,10 +1,10 @@ --- meta: title: Supported PostgreSQL extensions - description: This page describes how to use PostgreSQL extensions with Serverless SQL Databases + description: This page presents the PostgreSQL extensions supported by Serverless SQL Databases and the steps to use them content: h1: Supported PostgreSQL extensions - paragraph: This page describes how to use PostgreSQL extensions with Serverless SQL Databases + paragraph: This page presents the PostgreSQL extensions supported by Serverless SQL Databases and the steps to use them tags: sql-databases serverless database extensions postgresql pgvector postgis timescaledb dates: validation: 2024-10-08 @@ -13,7 +13,7 @@ categories: - serverless --- -Serverless SQL Database supports most popular [PostgreSQL extensions](#install-an-extension). Due to autoscaling and built-in connection pooling, some [advanced features of extensions might be limited or require workarounds](#supported-extensions). +Serverless SQL Databases support the most popular [PostgreSQL extensions](#install-an-extension). Due to autoscaling and built-in connection pooling, some [advanced features of extensions might be limited or require workarounds](#supported-extensions). @@ -23,24 +23,26 @@ Serverless SQL Database supports most popular [PostgreSQL extensions](#install-a ## Install an extension -1. [Connect to your Serverless SQL Database](https://www.scaleway.com/en/docs/serverless/sql-databases/quickstart/#how-to-connect-to-a-database) with a PostgreSQL client. For instance with psql: +1. [Connect to your Serverless SQL Database](/serverless/sql-databases/quickstart/#how-to-connect-to-a-database) with a PostgreSQL client (such as `psql`): ```bash psql "postgres://[user-or-application-id]:[api-secret-key]@[database-hostname]:5432/[database-name]?sslmode=require" ``` -2. Create extension using SQL: +2. Run the following command to create an extension using SQL: ```sql CREATE EXTENSION extension_name; ``` -3. Use extension with its supported features. Assuming you installed `pgvector` with `CREATE EXTENSION vector`, you can for instance create a vector and query it: +3. Use the supported features of the extension. + For instance, if you installed `pgvector` with `CREATE EXTENSION vector`, you can create a vector and query it: + ```sql CREATE TABLE songs (id bigserial PRIMARY KEY, embedding vector(3)); INSERT INTO songs (embedding) VALUES ('[1,1,1]'), ('[2,2,2]'), ('[0,1,2]'); SELECT * FROM songs ORDER BY embedding <-> '[3,3,3]' LIMIT 5; ``` This example will find the nearest vectors (using L2 distance) from '[3,3,3]': - ```bash + ```sql id | embedding ----+----------- 2 | [2,2,2] @@ -63,31 +65,31 @@ The following PostgreSQL extensions are available with Serverless SQL Databases: | citext | Provides a case-insensitive character string data type for easier text comparison. | | cube | Defines a data type for representing multidimensional cubes, useful in mathematical and geometrical computations. | | dict_int | Offers a text search dictionary template optimized for integer values. | -| dict_xsyn | Provides a text search dictionary template for advanced synonym processing, improving search accuracy. Features requiring direct access to the filesystem will not be supported (eg. custom synonym lists) | +| dict_xsyn | Provides a text search dictionary template for advanced synonym processing, improving search accuracy. Features requiring direct access to the filesystem will not be supported (eg. custom synonym lists). | | earthdistance | Calculates great-circle distances between points on Earth, useful in geographical applications. | | fuzzystrmatch | Determines similarities and differences between strings, helpful for tasks like data deduplication. | | hstore | Stores sets of key-value pairs in a single value, enabling a flexible schema for semi-structured data. | -| intagg | Aggregates and enumerates integer values, though it is considered obsolete since PG 8.4 and official support has been integrated in PostgreSQL directly | +| intagg | Aggregates and enumerates integer values, though it is considered obsolete since PG 8.4, and official support has been integrated in PostgreSQL directly | | intarray | Provides functions, operators, and indexing support for one-dimensional arrays of integers, enhancing array processing. | | isn | Supports data types for international product numbering standards, such as ISBN and EAN. | | ltree | Defines a data type for representing tree-like structures in a hierarchical manner, facilitating complex data modeling. | -| pg_cron | Schedules jobs within PostgreSQL using a cron-like syntax, automating routine database tasks. `pg_cron` requires the database to be in active state permanently and thus that you setup a minimum vCPU above 1 to properly work. | +| pg_cron | Schedules jobs within PostgreSQL using a cron-like syntax, automating routine database tasks. `pg_cron` requires the database to be in an active state permanently, and thus that you provision at least one vCPU for the extension to properly work. | | pg_prewarm | Preloads relation data into memory to reduce initial disk I/O latency and improve performance. | | pg_stat_statements | Tracks planning and execution statistics for all SQL statements, aiding in performance tuning. | | pg_trgm | Measures text similarity and provides index searching based on trigrams, enhancing text search capabilities. | | pgcrypto | Provides cryptographic functions for data encryption, decryption, and hashing, essential for secure data storage. | | pgrouting | Extends PostgreSQL and PostGIS with geospatial routing capabilities, enabling the calculation of shortest paths and other routing operations. | | pgrowlocks | Displays information about row-level locking, helping in the diagnosis of concurrency issues. | -| pgvector | Supports vector similarity search, often used for storing and searching embeddings in machine learning applications. [Query options](https://github.com/pgvector/pgvector?tab=readme-ov-file#query-options) using `SET` command require [to be used in a single transaction](https://www.scaleway.com/en/docs/serverless/sql-databases/reference-content/known-differences/#unsupported-configuration-commands) (ie. between `BEGIN; (...) COMMIT;`) | +| pgvector | Supports vector similarity search, often used for storing and searching embeddings in machine learning applications. [Query options](https://github.com/pgvector/pgvector?tab=readme-ov-file#query-options) using `SET` command require [to be used in a single transaction](/serverless/sql-databases/reference-content/known-differences/#unsupported-configuration-commands) (i.e. between `BEGIN; (...) COMMIT;`) | | plpgsql | Enables the use of PL/pgSQL, a procedural language for PostgreSQL that allows complex control structures and query manipulation. | -| postgis | Adds support for geographic objects, allowing location queries to be run in SQL, and is the foundation of geographic information system (GIS) functionality in PostgreSQL. | +| postgis | Adds support for geographic objects, allowing location queries to be run in SQL, and is the foundation of the geographic information system (GIS) functionality in PostgreSQL. | | postgis_raster | Implements raster data support in PostGIS, enabling analysis and storage of raster images alongside vector data. | | postgis_sfcgal | Adds advanced 3D geometry functions to PostGIS, leveraging the SFCGAL library for complex spatial operations. | | postgis_tiger_geocoder | Provides geocoding and reverse geocoding capabilities based on the US Census TIGER data within PostGIS. | | postgis_topology | Supports topological data models in PostGIS, allowing for advanced spatial relationships and analyses. | | tablefunc | Offers functions for manipulating entire tables, including crosstab operations for pivot tables. | | timescaledb | Enables efficient handling and analysis of time-series data, extending PostgreSQL with specialized time-series functions. | -| tsm_system_rows | Implements a TABLESAMPLE method that limits the number of rows returned, useful for sampling large datasets. | -| tsm_system_time | Implements a TABLESAMPLE method that limits rows based on a time duration, useful for time-based sampling. | +| tsm_system_rows | Implements a `TABLESAMPLE` method that limits the number of rows returned, useful for sampling large datasets. | +| tsm_system_time | Implements a `TABLESAMPLE` method that limits rows based on a time duration, useful for time-based sampling. | | unaccent | Provides a text search dictionary that removes accents from characters, improving text search accuracy. | | uuid-ossp | Generates universally unique identifiers (UUIDs), essential for ensuring data uniqueness across distributed systems. | From 54bba266f925619ea64d34977afacc17937c5614 Mon Sep 17 00:00:00 2001 From: SamyOubouaziz Date: Wed, 9 Oct 2024 11:41:55 +0200 Subject: [PATCH 3/3] docs(SRV): update --- menu/navigation.json | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/menu/navigation.json b/menu/navigation.json index db669cedbb..c873e9fd0a 100644 --- a/menu/navigation.json +++ b/menu/navigation.json @@ -4079,6 +4079,10 @@ "label": "Serverless SQL Databases overview", "slug": "serverless-sql-databases-overview" }, + { + "label": "Supported PostgreSQL extensions", + "slug": "supported-postgresql-extensions" + }, { "label": "Planned maintenance", "slug": "planned-maintenance"