I am investigating an issue where some commands periodically fail. In the course of this testing, I performed a stress test that seems to result in dtnma tools being unable to perform additional SQL queries.

To reproduce:

• Issue a PUT command to send an ARI and verify it works
• Execute the command several times in rapid succession
• Confirm that it's ability to send commands is locked up, even as other NM commands that do not invoke the DB work as expected

Logging confirms it is hanging in nm_rest.c between lines 475 and 485. civetweb creates a new task for each connection, so this doesn't affect other API behavior, but could prevent other activities (ie: logging received messages) from working.

The postgres library does not support multiple threada accessing the same connection simultaneously. It looks like we currently have a single connection used for all REST API operations, but civetweb will create a new task for each HTTP connection. That opens the probability of conflicting access for long-running queries or a busy system.

The solution is to either establish a new connection for each REST Connection (not recommended), add an additional mutex to protect access to the connection (quick solution), or (preferable) define a connection pool that can be reused.

Note: Connection pooling is that standard procedure in general, but is not a feature provided by the C library.