Merge pull request #231821 from AssafFraenkel/assaff2-azure-docs-pr

[postgresql] Adding new pages for Flexible and correct typo for Single

Author: v-regandowner

articles/postgresql/TOC.yml

@@ -542,6 +542,8 @@
           displayName: logical decoding, logical replication
     - name: App Development
       items: 
+        - name: Best practices
+          href: flexible-server/application-best-practices.md
         - name: Connection resilience
           href: flexible-server/concepts-connectivity.md
     - name: Azure Advisor recommendations
@@ -566,6 +568,12 @@
       - name: Troubleshoot autovacuum 
         href: flexible-server/how-to-autovacuum-tuning.md
         displayName: Autovacuum troubleshooting, tuning
+      - name: Troubleshoot connections 
+        href: flexible-server/how-to-troubleshoot-common-connection-issues.md
+        displayName: Troubleshoot connections
+      - name: Optimize query statistics
+        href: flexible-server/how-to-optimize-query-stats-collection.md
+        displayName: Optimize query statistics
       - name: Best practices for bulk data upload 
         href: flexible-server/how-to-bulk-load-data.md
         displayName: Best practices for bulk data upload

articles/postgresql/flexible-server/application-best-practices.md

@@ -0,0 +1,112 @@
+---
+title: App development best practices - Azure Database for PostgreSQL flexible server
+description: Learn about best practices for building an app by using Azure Database for PostgreSQL flexible server.
+ms.service: postgresql
+ms.subservice: flexible-server
+ms.topic: conceptual
+author: assaff
+ms.author: assaff
+ms.reviewer: ""
+ms.date: 03/22/2023
+---
+
+# Best practices for building an application with Azure Database for PostgreSQL flexible server
+
+[!INCLUDE [applies-to-postgresql-flexible-server](../includes/applies-to-postgresql-flexible-server.md)]
+
+Here are some best practices to help you build a cloud-ready application by using Azure Database for PostgreSQL. These best practices can reduce development time for your app.
+
+## Configuration of application and database resources
+
+### Keep the application and database in the same region
+
+Make sure all your dependencies are in the same region when deploying your application in Azure. Spreading instances across regions or availability zones creates network latency, which might affect the overall performance of your application.
+
+### Keep your PostgreSQL server secure
+
+Configure your PostgreSQL server to be [secure](./concepts-security.md) and not accessible publicly. Use one of these options to secure your server:
+
+- [Firewall rules](./concepts-firewall-rules.md)
+- [Virtual networks](./concepts-networking.md)
+- [Private Networking](https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/private-networking-patterns-in-azure-database-for-postgres/ba-p/3007149)
+
+For security, you must always connect to your PostgreSQL server over SSL and configure your PostgreSQL server and your application to use TLS 1.2. See [How to connect SSL/TLS](./how-to-connect-tls-ssl.md).
+
+
+### Use environment variables for connection information
+
+Do not save your database credentials in your application code. Depending on the front end application, follow the guidance to set up environment variables. For App service use, see [how to configure app settings](../../app-service/configure-common.md#configure-app-settings) and for Azure Kubernetes service, see [how to use Kubernetes secrets](https://kubernetes.io/docs/concepts/configuration/secret/).
+
+## Performance and resiliency
+
+Here are a few tools and practices that you can use to help debug performance issues with your application.
+
+### Use Connection Pooling
+
+With connection pooling, a fixed set of connections is established at the startup time and maintained. This also helps reduce the memory fragmentation on the server that is caused by the dynamic new connections established on the database server. The connection pooling can be configured on the application side if the app framework or database driver supports it. If that is not supported, the other recommended option is to leverage a proxy connection pooler service like [PgBouncer](https://pgbouncer.github.io/) or [Pgpool](https://pgpool.net/mediawiki/index.php/Main_Page) running outside the application and connecting to the database server. Both PgBouncer and Pgpool are community based tools that work with Azure Database for PostgreSQL.
+
+### Retry logic to handle transient errors
+
+Your application might experience transient errors where connections to the database are dropped or lost intermittently. In such situations, the server is up and running after one to two retries in 5 to 10 seconds. A good practice is to wait for 5 seconds before your first retry. Then follow each retry by increasing the wait gradually, up to 60 seconds. Limit the maximum number of retries at which point your application considers the operation failed, so you can then further investigate. See [How to troubleshoot connection errors](./concepts-connectivity.md) to learn more.
+
+### Enable read replication to mitigate failovers
+
+You can use [Data-in Replication](./concepts-read-replicas.md) for failover scenarios. When you're using read replicas, no automated failover between source and replica servers occurs. You'll notice a lag between the source and the replica because the replication is asynchronous. Network lag can be influenced by many factors, like the size of the workload running on the source server and the latency between datacenters. In most cases, replica lag ranges from a few seconds to a couple of minutes.
+
+## Database deployment
+
+### Configure CI/CD deployment pipeline
+
+Occasionally, you need to deploy changes to your database. In such cases, you can use continuous integration (CI) through [GitHub Actions](https://github.com/Azure/postgresql/blob/master/README.md) for your PostgreSQL server to update the database by running a custom script against it.
+
+### Define manual database deployment process
+
+During manual database deployment, follow these steps to minimize downtime or reduce the risk of failed deployment:
+
+- Create a copy of a production database on a new database by using pg_dump.
+- Update the new database with your new schema changes or updates needed for your database.
+- Put the production database in a read-only state. You should not have write operations on the production database until deployment is completed.
+- Test your application with the newly updated database from step 1.
+- Deploy your application changes and make sure the application is now using the new database that has the latest updates.
+- Keep the old production database so that you can roll back the changes. You can then evaluate to either delete the old production database or export it on Azure Storage if needed.
+
+> [!NOTE]
+> If the application is like an e-commerce app and you can't put it in read-only state, deploy the changes directly on the production database after making a backup. Theses change should occur during off-peak hours with low traffic to the app to minimize the impact, because some users might experience failed requests. Make sure your application code also handles any failed requests.
+
+## Database schema and queries
+
+Here are few tips to keep in mind when you build your database schema and your queries.
+
+### Use BIGINT or UUID for Primary Keys
+
+When building custom application or some frameworks they maybe using `INT` instead of `BIGINT` for primary keys. When you use ```INT```, you run the risk of where the value in your database can exceed storage capacity of ```INT``` data type. Making this change to an existing production application can be time consuming with cost more development time. Another option is to use [UUID](https://www.postgresql.org/docs/current/datatype-uuid.html) for primary keys.This identifier uses an auto-generated 128-bit string, for example ```a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11```. Learn more about [PostgreSQL data types](https://www.postgresql.org/docs/8.1/datatype.html).
+
+### Use indexes
+
+There are many types of [indexes](https://www.postgresql.org/docs/9.1/indexes.html) in Postgres which can be used in different ways. Using an index helps the server find and retrieve specific rows much faster than it could do without an index. But indexes also add overhead to the database server, hence avoid having too many indexes.
+
+### Use autovacuum
+
+You can optimize your server with autovacuum on an Azure Database for PostgreSQL server. PostgreSQL allow greater database concurrency but with every update  results in insert and delete. For delete, the records are soft marked which will be purged later. To carry out these tasks, PostgreSQL runs a vacuum job. If you don't vacuum from time to time, the dead tuples that accumulate can result in:
+
+- Data bloat, such as larger databases and tables.
+- Larger suboptimal indexes.
+- Increased I/O.
+
+Learn more about [how to optimize with autovacuum](how-to-autovacuum-tuning.md).
+
+### Use pg_stats_statements
+
+Pg_stat_statements is a PostgreSQL extension that's enabled by default in Azure Database for PostgreSQL. The extension provides a means to track execution statistics for all SQL statements executed by a server. See [how to use pg_statement](how-to-optimize-query-stats-collection.md).
+
+### Use the Query Store
+
+The [Query Store](./concepts-query-store.md) feature in Azure Database for PostgreSQL provides a more effective method to track query statistics. We recommend this feature as an alternative to using pg_stats_statements.
+
+### Optimize bulk inserts and use transient data
+
+If you have workload operations that involve transient data or that insert large datasets in bulk, consider using unlogged tables. It provides atomicity and durability, by default. Atomicity, consistency, isolation, and durability make up the ACID properties. See [how to optimize bulk inserts](how-to-bulk-load-data.md).
+
+## Next Steps
+
+[Postgres Guide](http://postgresguide.com/)

articles/postgresql/flexible-server/how-to-optimize-query-stats-collection.md

@@ -0,0 +1,42 @@
+---
+title: Optimize query stats collection - Azure Database for PostgreSQL - flexible Server
+description: This article describes how you can optimize query stats collection on an Azure Database for PostgreSQL - flexible Server
+ms.service: postgresql
+ms.subservice: flexible-server
+ms.topic: how-to
+ms.author: assaff
+author: assaff
+ms.date: 03/23/2023
+---
+
+# Optimize query statistics collection on an Azure Database for PostgreSQL - flexible Server
+
+[!INCLUDE [applies-to-postgresql-flexible-server](../includes/applies-to-postgresql-flexible-server.md)]
+
+This article describes how to optimize query statistics collection on an Azure Database for PostgreSQL server.
+
+## Use pg_stat_statements
+
+**Pg_stat_statements** is a PostgreSQL extension that can be enabled in Azure Database for PostgreSQL. The extension provides a means to track execution statistics for all SQL statements executed by a server. This module hooks into every query execution and comes with a non-trivial performance cost. Enabling **pg_stat_statements** forces query text writes to files on disk.
+
+If you have unique queries with long query text or you don't actively monitor **pg_stat_statements**, disable **pg_stat_statements** for best performance. To do so, change the setting to `pg_stat_statements.track = NONE`.
+
+Some customer workloads have seen up to a 50 percent performance improvement when **pg_stat_statements** is disabled. The tradeoff you make when you disable pg_stat_statements is the inability to troubleshoot performance issues.
+
+To set `pg_stat_statements.track = NONE`:
+
+- In the Azure portal, go to the [PostgreSQL resource management page and select the server parameters blade](concepts-server-parameters.md).
+- Use the [Azure CLI](connect-azure-cli.md) az postgres server configuration set to `--name pg_stat_statements.track --resource-group myresourcegroup --server mydemoserver --value NONE`.
+
+## Use the Query Store
+
+The [Query Store](concepts-query-store.md) feature in Azure Database for PostgreSQL provides a more effective method to track query statistics. We recommend this feature as an alternative to using *pg_stat_statements*.
+
+## Next steps
+
+Consider setting `pg_stat_statements.track = NONE` in the [Azure portal](concepts-server-parameters.md) or by using the [Azure CLI](connect-azure-cli.md).
+
+For more information, see:
+
+- [Query Store usage scenarios](concepts-query-store-scenarios.md) 
+- [Query Store best practices](concepts-query-store-best-practices.md) 

articles/postgresql/flexible-server/how-to-troubleshoot-common-connection-issues.md

@@ -0,0 +1,61 @@
+---
+title: Troubleshoot connections - Azure Database for PostgreSQL - flexible Server
+description: Learn how to troubleshoot connection issues to Azure Database for PostgreSQL - flexible Server.
+ms.service: postgresql
+ms.subservice: flexible-server
+ms.topic: how-to
+ms.author: assaff
+author: assaff
+ms.date: 03/23/2023
+---
+
+# Troubleshoot connection issues to Azure Database for PostgreSQL - flexible Server
+
+[!INCLUDE [applies-to-postgresql-flexible-server](../includes/applies-to-postgresql-flexible-server.md)]
+
+Connection problems may be caused by various things, including:
+
+* Firewall settings
+* Connection time-out
+* Incorrect login information
+* Maximum limit reached on some Azure Database for PostgreSQL resources
+* Issues with the infrastructure of the service
+* Maintenance being performed in the service
+* The compute allocation of the server is changed by scaling the number of vCores or moving to a different service tier
+
+Generally, connection issues to Azure Database for PostgreSQL can be classified as follows:
+
+* Transient errors (short-lived or intermittent)
+* Persistent or non-transient errors (errors that regularly recur)
+
+## Troubleshoot transient errors
+
+Transient errors occur when maintenance is performed, the system encounters an error with the hardware or software, or you change the vCores or service tier of your server. The Azure Database for PostgreSQL service has built-in high availability and is designed to mitigate these types of problems automatically. However, your application loses its connection to the server for a short period of time of typically less than 60 seconds at most. Some events can occasionally take longer to mitigate, such as when a large transaction causes a long-running recovery.
+
+### Steps to resolve transient connectivity issues
+
+1. Check the [Microsoft Azure Service Dashboard](https://azure.microsoft.com/status) for any known outages that occurred during the time in which the errors were reported by the application.
+2. Applications that connect to a cloud service such as Azure Database for PostgreSQL should expect transient errors and implement retry logic to handle these errors instead of surfacing these as application errors to users. Review [Handling of transient connectivity errors for Azure Database for PostgreSQL](concepts-connectivity.md) for best practices and design guidelines for handling transient errors.
+3. As a server approaches its resource limits, errors can seem to be transient connectivity issue. See [Limitations in Azure Database for PostgreSQL](concepts-limits.md).
+4. If connectivity problems continue, or if the duration for which your application encounters the error exceeds 60 seconds or if you see multiple occurrences of the error in a given day, file an Azure support request by selecting **Get Support** on the [Azure Support](https://azure.microsoft.com/support/options) site.
+
+## Troubleshoot persistent errors
+
+If the application persistently fails to connect to Azure Database for PostgreSQL, it usually indicates an issue with one of the following:
+
+* Server firewall configuration: Make sure that the Azure Database for PostgreSQL server firewall is configured to allow connections from your client, including proxy servers and gateways.
+* Client firewall configuration: The firewall on your client must allow connections to your database server. IP addresses and ports of the server that you can't connect to must be allowed and the application names such as PostgreSQL in some firewalls.
+* User error: You might have mistyped connection parameters, such as the server name in the connection string or a missing *\@servername* suffix in the user name.
+* If you see the error _Server isn't configured to allow ipv6 connections_, note that the Basic tier doesn't support VNet service endpoints. You have to remove the Microsoft.Sql endpoint from the subnet that is attempting to connect to the Basic server.
+* If you see the connection error _sslmode value "***" invalid when SSL support is not compiled in_ error, it means your PostgreSQL client doesn't support SSL. Most probably, the client-side libpq hasn't been compiled with the "--with-openssl" flag. Try connecting with a PostgreSQL client that has SSL support.
+
+### Steps to resolve persistent connectivity issues
+
+1. Set up [firewall rules](concepts-firewall-rules.md) to allow the client IP address. For temporary testing purposes only, set up a firewall rule using 0.0.0.0 as the starting IP address and using 255.255.255.255 as the ending IP address. This will open the server to all IP addresses. If this resolves your connectivity issue, remove this rule and create a firewall rule for an appropriately limited IP address or address range.
+2. On all firewalls between the client and the internet, make sure that port 5432 is open for outbound connections.
+3. Verify your connection string and other connection settings.
+4. Check the service health in the dashboard. If you think there’s a regional outage, see [Overview of business continuity with Azure Database for PostgreSQL](concepts-business-continuity.md) for steps to recover to a new region.
+
+## Next steps
+
+* [Handling of transient connectivity errors for Azure Database for PostgreSQL](concepts-connectivity.md)

articles/postgresql/single-server/application-best-practices.md

@@ -1,6 +1,6 @@
 ---
-title: App development best practices - Azure Database for PostgreSQL
-description: Learn about best practices for building an app by using Azure Database for PostgreSQL.
+title: App development best practices - Azure Database for PostgreSQL single server
+description: Learn about best practices for building an app by using Azure Database for PostgreSQL single server.
 ms.service: postgresql
 ms.subservice: single-server
 ms.topic: conceptual
@@ -10,7 +10,7 @@ ms.reviewer: ""
 ms.date: 06/24/2022
 ---
 
-# Best practices for building an application with Azure Database for PostgreSQL
+# Best practices for building an application with Azure Database for PostgreSQL single server
 
 [!INCLUDE [applies-to-postgresql-single-server](../includes/applies-to-postgresql-single-server.md)]
 
@@ -70,7 +70,7 @@ During manual database deployment, follow these steps to minimize downtime or re
 - Deploy your application changes and make sure the application is now using the new database that has the latest updates.
 - Keep the old production database so that you can roll back the changes. You can then evaluate to either delete the old production database or export it on Azure Storage if needed.
 
->  [!NOTE]
+> [!NOTE]
 > If the application is like an e-commerce app and you can't put it in read-only state, deploy the changes directly on the production database after making a backup. Theses change should occur during off-peak hours with low traffic to the app to minimize the impact, because some users might experience failed requests. Make sure your application code also handles any failed requests.
 
 ## Database schema and queries