Skip to content

Commit b2436a0

Browse files
garycentricgarycentric
authored
Merge pull request #91864 from ambhatna/updateconnectiondoc
connection pooling
2 parents c8e419d + ebcc275 commit b2436a0

File tree

3 files changed

+305
-8
lines changed

3 files changed

+305
-8
lines changed

‎articles/mysql/TOC.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,8 @@
5757
items:
5858
- name: Azure CLI
5959
href: sample-scripts-azure-cli.md
60+
- name: Java connection pooling
61+
href: sample-scripts-java-connection-pooling.md
6062
- name: Concepts
6163
items:
6264
- name: DB & Servers

‎articles/mysql/concepts-connectivity.md

Lines changed: 74 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,17 @@
11
---
2-
title: Handling of transient connectivity errors for Azure Database for MySQL | Microsoft Docs
3-
description: Learn how to handle transient connectivity errors for Azure Database for MySQL.
4-
keywords: mysql connection,connection string,connectivity issues,transient error,connection error
2+
title: Handle transient errors and connect efficiently to Azure Database for MySQL | Microsoft Docs
3+
description: Learn how to handle transient connectivity errors and connect efficiently to Azure Database for MySQL.
4+
keywords: mysql connection,connection string,connectivity issues,transient error,connection error,connect efficiently
55
author: jan-eng
66
ms.author: janeng
77
ms.service: mysql
88
ms.topic: conceptual
99
ms.date: 11/09/2018
1010
---
1111

12-
# Handling of transient connectivity errors for Azure Database for MySQL
12+
# Handle transient errors and connect efficiently to Azure Database for MySQL
1313

14-
This article describes how to handle transient errors connecting to Azure Database for MySQL.
14+
This article describes how to handle transient errors and connect efficiently to Azure Database for MySQL.
1515

1616
## Transient errors
1717

@@ -31,13 +31,79 @@ The first and second case are fairly straight forward to handle. Try to open the
3131
* For each following retry, the increase the wait exponentially, up to 60 seconds.
3232
* Set a max number of retries at which point your application considers the operation failed.
3333

34-
When a connection with an active transaction fails, it is more difficult to handle the recovery correctly. There are two cases: If the transaction was read-only in nature, it is safe to reopen the connection and to retry the transaction. If however if the transaction was also writing to the database, you must determine if the transaction was rolled back, or if it succeeded before the transient error happened. In that case, you might just not have received the commit acknowledgement from the database server.
34+
When a connection with an active transaction fails, it is more difficult to handle the recovery correctly. There are two cases: If the transaction was read-only in nature, it is safe to reopen the connection and to retry the transaction. If however the transaction was also writing to the database, you must determine if the transaction was rolled back, or if it succeeded before the transient error happened. In that case, you might just not have received the commit acknowledgment from the database server.
3535

36-
One way of doing this, is to generate a unique ID on the client that is used for all the retries. You pass this unique ID as part of the transaction to the server and to store it in a column with a unique constraint. This way you can safely retry the transaction. It will succeed if the previous transaction was rolled back and the client generated unique ID does not yet exist in the system. It will fail indicating a duplicate key violation if the unique ID was previously stored because the previous transaction completed successfully.
36+
One way of doing this, is to generate a unique ID on the client that is used for all the retries. You pass this unique ID as part of the transaction to the server and to store it in a column with a unique constraint. This way you can safely retry the transaction. It will succeed if the previous transaction was rolled back and the client-generated unique ID does not yet exist in the system. It will fail indicating a duplicate key violation if the unique ID was previously stored because the previous transaction completed successfully.
3737

3838
When your program communicates with Azure Database for MySQL through third-party middleware, ask the vendor whether the middleware contains retry logic for transient errors.
3939

40-
Make sure to test you retry logic. For example, try to execute your code while scaling up or down the compute resources of you Azure Database for MySQL server. Your application should handle the brief downtime that is encountered during this operation without any problems.
40+
Make sure to test you retry logic. For example, try to execute your code while scaling up or down the compute resources of your Azure Database for MySQL server. Your application should handle the brief downtime that is encountered during this operation without any problems.
41+
42+
## Connect efficiently to Azure Database for MySQL
43+
44+
Database connections are a limited resource, so making effective use of connection pooling to access Azure Database for MySQL optimizes performance. The below section explains how to use connection pooling or persistent connections to more effectively access Azure Database for MySQL.
45+
46+
## Access databases by using connection pooling (recommended)
47+
48+
Managing database connections can have a significant impact on the performance of the application as a whole. To optimize the performance of your application, the goal should be to reduce the number of times connections are established and time for establishing connections in key code paths. We strongly recommend using database connection pooling or persistent connections to connect to Azure Database for MySQL. Database connection pooling handles the creation, management, and allocation of database connections. When a program requests a database connection, it prioritizes the allocation of existing idle database connections, rather than the creation of a new connection. After the program has finished using the database connection, the connection is recovered in preparation for further use, rather than simply being closed down.
49+
50+
For better illustration, this article provides [a piece of sample code](./sample-scripts-java-connection-pooling.md) that uses JAVA as an example. For more information, see [Apache common DBCP](http://commons.apache.org/proper/commons-dbcp/).
51+
52+
> [!NOTE]
53+
> The server configures a timeout mechanism to close a connection that has been in an idle state for some time to free up resources. Be sure to set up the verification system to ensure the effectiveness of persistent connections when you are using them. For more information, see [Configure verification systems on the client side to ensure the effectiveness of persistent connections](concepts-connectivity.md#configure-verification-mechanisms-in-clients-to-confirm-the-effectiveness-of-persistent-connections).
54+
55+
## Access databases by using persistent connections (recommended)
56+
57+
The concept of persistent connections is similar to that of connection pooling. Replacing short connections with persistent connections requires only minor changes to the code, but it has a major effect in terms of improving performance in many typical application scenarios.
58+
59+
## Access databases by using wait and retry mechanism with short connections
60+
61+
If you have resource limitations, we strongly recommend that you use database pooling or persistent connections to access databases. If your application use short connections and experience connection failures when you approach the upper limit on the number of concurrent connections,you can try wait and retry mechanism. You can set an appropriate wait time, with a shorter wait time after the first attempt. Thereafter, you can try waiting for events multiple times.
62+
63+
## Configure verification mechanisms in clients to confirm the effectiveness of persistent connections
64+
65+
The server configures a timeout mechanism to close a connection that’s been in an idle state for some time to free up resources. When the client accesses the database again, it’s equivalent to creating a new connection request between the client and the server. To ensure the effectiveness of connections during the process of using them, configure a verification mechanism on the client. As shown in the following example, you can use Tomcat JDBC connection pooling to configure this verification mechanism.
66+
67+
By setting the TestOnBorrow parameter, when there's a new request, the connection pool automatically verifies the effectiveness of any available idle connections. If such a connection is effective, its directly returned otherwise connection pool withdraws the connection. The connection pool then creates a new effective connection and returns it. This process ensures that database is accessed efficiently.
68+
69+
For information on the specific settings, see the [JDBC connection pool official introduction document](https://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html#Common_Attributes). You mainly need to set the following three parameters: TestOnBorrow (set to true), ValidationQuery (set to SELECT 1), and ValidationQueryTimeout (set to 1). The specific sample code is shown below:
70+
71+
```java
72+
public class SimpleTestOnBorrowExample {
73+
public static void main(String[] args) throws Exception {
74+
PoolProperties p = new PoolProperties();
75+
p.setUrl("jdbc:mysql://localhost:3306/mysql");
76+
p.setDriverClassName("com.mysql.jdbc.Driver");
77+
p.setUsername("root");
78+
p.setPassword("password");
79+
// The indication of whether objects will be validated by the idle object evictor (if any).
80+
// If an object fails to validate, it will be dropped from the pool.
81+
// NOTE - for a true value to have any effect, the validationQuery or validatorClassName parameter must be set to a non-null string.
82+
p.setTestOnBorrow(true);
83+
84+
// The SQL query that will be used to validate connections from this pool before returning them to the caller.
85+
// If specified, this query does not have to return any data, it just can't throw a SQLException.
86+
p.setValidationQuery("SELECT 1");
87+
88+
// The timeout in seconds before a connection validation queries fail.
89+
// This works by calling java.sql.Statement.setQueryTimeout(seconds) on the statement that executes the validationQuery.
90+
// The pool itself doesn't timeout the query, it is still up to the JDBC driver to enforce query timeouts.
91+
// A value less than or equal to zero will disable this feature.
92+
p.setValidationQueryTimeout(1);
93+
// set other usefull pool properties.
94+
DataSource datasource = new DataSource();
95+
datasource.setPoolProperties(p);
96+
97+
Connection con = null;
98+
try {
99+
con = datasource.getConnection();
100+
// execute your query here
101+
} finally {
102+
if (con!=null) try {con.close();}catch (Exception ignore) {}
103+
}
104+
}
105+
}
106+
```
41107

42108
## Next steps
43109

‎articles/mysql/sample-scripts-java-connection-pooling.md

Lines changed: 229 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,229 @@
1+
---
2+
title: Java samples to illustrate connection pooling
3+
description: This article lists java samples to illustrate connection pooling.
4+
author: ambhatna
5+
ms.author: ambhatna
6+
ms.service: mysql
7+
ms.topic: sample
8+
ms.date: 02/28/2018
9+
---
10+
# Java sample to illustrate connection pooling
11+
12+
The below sample code illustrates connection pooling in java.
13+
14+
```java
15+
import java.sql.Connection;
16+
import java.sql.DriverManager;
17+
import java.sql.ResultSet;
18+
import java.sql.SQLException;
19+
import java.sql.Statement;
20+
import java.util.HashSet;
21+
import java.util.Set;
22+
import java.util.Stack;
23+
24+
public class MySQLConnectionPool {
25+
private String databaseUrl;
26+
private String userName;
27+
private String password;
28+
private int maxPoolSize = 10;
29+
private int connNum = 0;
30+
31+
private static final String SQL_VERIFYCONN = "select 1";
32+
33+
Stack<Connection> freePool = new Stack<>();
34+
Set<Connection> occupiedPool = new HashSet<>();
35+
36+
/**
37+
* Constructor
38+
*
39+
* @param databaseUrl
40+
* The connection url
41+
* @param userName
42+
* user name
43+
* @param password
44+
* password
45+
* @param maxSize
46+
* max size of the connection pool
47+
*/
48+
public MySQLConnectionPool(String databaseUrl, String userName,
49+
String password, int maxSize) {
50+
this.databaseUrl = databaseUrl;
51+
this.userName = userName;
52+
this.password = password;
53+
this.maxPoolSize = maxSize;
54+
}
55+
56+
/**
57+
* Get an available connection
58+
*
59+
* @return An available connection
60+
* @throws SQLException
61+
* Fail to get an available connection
62+
*/
63+
public synchronized Connection getConnection() throws SQLException {
64+
Connection conn = null;
65+
66+
if (isFull()) {
67+
throw new SQLException("The connection pool is full.");
68+
}
69+
70+
conn = getConnectionFromPool();
71+
72+
// If there is no free connection, create a new one.
73+
if (conn == null) {
74+
conn = createNewConnectionForPool();
75+
}
76+
77+
// For Azure Database for MySQL, if there is no action on one connection for some
78+
// time, the connection is lost. By this, make sure the connection is
79+
// active. Otherwise reconnect it.
80+
makeAvailable(conn);
81+
return conn;
82+
}
83+
84+
/**
85+
* Return a connection to the pool
86+
*
87+
* @param conn
88+
* The connection
89+
* @throws SQLException
90+
* When the connection is returned already or it isn't gotten
91+
* from the pool.
92+
*/
93+
public synchronized void returnConnection(Connection conn)
94+
throws SQLException {
95+
if (conn == null) {
96+
throw new NullPointerException();
97+
}
98+
if (!occupiedPool.remove(conn)) {
99+
throw new SQLException(
100+
"The connection is returned already or it isn't for this pool");
101+
}
102+
freePool.push(conn);
103+
}
104+
105+
/**
106+
* Verify if the connection is full.
107+
*
108+
* @return if the connection is full
109+
*/
110+
private synchronized boolean isFull() {
111+
return ((freePool.size() == 0) && (connNum >= maxPoolSize));
112+
}
113+
114+
/**
115+
* Create a connection for the pool
116+
*
117+
* @return the new created connection
118+
* @throws SQLException
119+
* When fail to create a new connection.
120+
*/
121+
private Connection createNewConnectionForPool() throws SQLException {
122+
Connection conn = createNewConnection();
123+
connNum++;
124+
occupiedPool.add(conn);
125+
return conn;
126+
}
127+
128+
/**
129+
* Crate a new connection
130+
*
131+
* @return the new created connection
132+
* @throws SQLException
133+
* When fail to create a new connection.
134+
*/
135+
private Connection createNewConnection() throws SQLException {
136+
Connection conn = null;
137+
try {
138+
Class.forName("com.mysql.jdbc.Driver");
139+
conn = DriverManager.getConnection(databaseUrl, userName, password);
140+
} catch (ClassNotFoundException cnfe) {
141+
throw new SQLException(cnfe);
142+
}
143+
return conn;
144+
}
145+
146+
/**
147+
* Get a connection from the pool. If there is no free connection, return
148+
* null
149+
*
150+
* @return the connection.
151+
*/
152+
private Connection getConnectionFromPool() {
153+
Connection conn = null;
154+
if (freePool.size() > 0) {
155+
conn = freePool.pop();
156+
occupiedPool.add(conn);
157+
}
158+
return conn;
159+
}
160+
161+
/**
162+
* Make sure the connection is available now. Otherwise, reconnect it.
163+
*
164+
* @param conn
165+
* The connection for verification.
166+
* @return the available connection.
167+
* @throws SQLException
168+
* Fail to get an available connection
169+
*/
170+
private Connection makeAvailable(Connection conn) throws SQLException {
171+
if (isConnectionAvailable(conn)) {
172+
return conn;
173+
}
174+
175+
// If the connection is't available, reconnect it.
176+
occupiedPool.remove(conn);
177+
connNum--;
178+
conn.close();
179+
180+
conn = createNewConnection();
181+
occupiedPool.add(conn);
182+
connNum++;
183+
return conn;
184+
}
185+
186+
/**
187+
* By running a sql to verify if the connection is available
188+
*
189+
* @param conn
190+
* The connection for verification
191+
* @return if the connection is available for now.
192+
*/
193+
private boolean isConnectionAvailable(Connection conn) {
194+
try (Statement st = conn.createStatement()) {
195+
st.executeQuery(SQL_VERIFYCONN);
196+
return true;
197+
} catch (SQLException e) {
198+
return false;
199+
}
200+
}
201+
202+
// Just an Example
203+
public static void main(String[] args) throws SQLException {
204+
Connection conn = null;
205+
MySQLConnectionPool pool = new MySQLConnectionPool(
206+
"jdbc:mysql://mysqlaasdevintic-sha.cloudapp.net:3306/<Your DB name>",
207+
"<Your user>", "<Your Password>", 2);
208+
try {
209+
conn = pool.getConnection();
210+
try (Statement statement = conn.createStatement())
211+
{
212+
ResultSet res = statement.executeQuery("show tables");
213+
System.out.println("There are below tables:");
214+
while (res.next()) {
215+
String tblName = res.getString(1);
216+
System.out.println(tblName);
217+
}
218+
}
219+
}
220+
finally {
221+
if (conn != null) {
222+
pool.returnConnection(conn);
223+
}
224+
}
225+
}
226+
227+
}
228+
229+
```

0 commit comments

Comments
 (0)