feat: options refactor and http timeout

Author: vlastahajek

CHANGELOG.md

@@ -2,6 +2,11 @@
 ## Version 3.4.0 (in progres)
 ### Features
  - [#89](https://github.com/tobiasschuerg/InfluxDB-Client-for-Arduino/pull/89) - ESP8266 only - Added Max Fragment Length Negotiation for TLS communicaton to reduce memory allocation. If server supports MFLN, it saves ~15kB. Standalone InfluxDB OSS server doesn't support MFLN, Cloud yes. To leverage MFLN for standalone OSS, a reverse proxy needs to be used. 
+ - [#91](https://github.com/tobiasschuerg/InfluxDB-Client-for-Arduino/pull/91) - Improved API for settings of write and HTTP options:
+  - Introduced `WriteOptions` to wrap the write related options (write precision, batch-size, etc). It offers fluent style API allowing to change only the required options. `InfluxDBClient` has overloaded `setWriteOptions(const WriteOptions& writeOptions)` method.
+  - Introduced `HTTPOptions` to wrap the HTTP related options (e.g. reusing connection). It offers fluent style API allowing to change only the required options. `InfluxDBClient` has `setHTTPOptions(const HTTPOptions& httpOptions)` method.
+  - Added possibility to set HTTP response read timeout (part of the `HTTPOptions`).
+  - Method `InfluxDBClient::void setWriteOptions(WritePrecision precision, uint16_t batchSize = 1, uint16_t bufferSize = 5, uint16_t flushInterval = 60, bool preserveConnection = true)` is deprecated and it will be removed in the next release.
 
 ### Documentation
  - [#87](https://github.com/tobiasschuerg/InfluxDB-Client-for-Arduino/pull/87) - Fixed include file name in the Readme

README.md

@@ -155,10 +155,10 @@ If points have no timestamp assigned, InfluxDB assigns timestamp at the time of
 
 InfuxDB allows sending timestamp in various precisions - nanoseconds, microseconds, milliseconds or seconds. The milliseconds precision is usually enough for using on Arduino. Maximum avavailable precision is microseconds. Setting to nanosecond will just add zeroes for microseconds fraction.
 
-The client has to be configured with time precision. The default settings is not using the timestamp. The `setWriteOptions` functions allow setting various parameters and one of them is __write precision__:
+The client has to be configured with a time precision. The default settings is not using the timestamp, which means server will assign timestamp. The `setWriteOptions` functions allow setting custom `WriteOptions` params and one of them is __write precision__:
 ``` cpp
 // Set write precision to milliseconds. Leave other parameters default.
-client.setWriteOptions(WritePrecision::MS);
+client.setWriteOptions(WriteOptions().writePrecision(WritePrecision::MS));
 ```
 When a write precision is configured, the client will automatically assign current time to the timestamp of each written point, which doesn't have a timestamp assigned. 
 
@@ -220,21 +220,22 @@ For example, if you would like to see updates (on the dashboard or in processing
 
 In case that data should be written in longer periods and gathered data consists of several points batch size should be set to an expected number of points.
 
-To set batch size we use [setWriteOptions](#write-options) function, where second parameter controls batch size:
+To set batch size we use `WriteOptions` object and  [setWriteOptions](#write-options) function:
 ```cpp
 // Enable messages batching
-client.setWriteOptions(WritePrecision::MS, 10);
+client.setWriteOptions(WriteOptions().batchSize(10));
 ```
 Writing point will add a point to the underlying buffer until the batch size is reached:
 ```cpp
 // Write first point to the buffer
+// Buffered write always returns `true`
 client.writePoint(point1);
 // Write second point to the buffer
 client.writePoint(point2);
 ..
-// Write nineth point to the buffer, returns 
+// Write nineth point to the buffer
 client.writePoint(point9);
-// Writing tenth point will cause flushing buffer
+// Writing tenth point will cause flushing buffer and returns actual write result.
 if(!client.writePoint(point10)) {
     Serial.print("InfluxDB write failed: ");
     Serial.println(client.getLastErrorMessage());
@@ -246,12 +247,12 @@ In case of a number of points is not always the same, set batch size to the maxi
 ## Buffer Handling and Retrying
 InfluxDB contains an underlying buffer for handling writing in batches and automatic retrying on server backpressure and connection failure.
 
-Its size is controled by the 3rd parameter of [setWriteOptions](#write-options) function:
+Its size is controled by the `bufferSize` param of [WriteOptions](#write-options) object:
 ```cpp
-// Enable messages batching
-client.setWriteOptions(WritePrecision::MS, 10, 30);
+// Increase buffer to allow caching of failed writes
+client.setWriteOptions(WriteOptions().bufferSize(50));
 ```
-The third parameter specifies the buffer size. The recommended size is at least 2 x batch size. 
+The recommended size is at least 2 x batch size. 
 
 State of the buffer can be determined via two functions:
  - `isBufferEmpty()` - Returns true if buffer is empty
@@ -278,15 +279,21 @@ Other functions for dealing with buffer:
 Check [SecureBatchWrite example](examples/SecureBatchWrite/SecureBatchWrite.ino) for example code of buffer handling functions.
 
 ## Write Options
-Writing points can be controlled via several parameters in `setWriteOptions` function:
+Writing points can be controlled via `WriteOptions`, which is set in `setWriteOptions` function:
 
 | Parameter | Default Value | Meaning |
 |-----------|---------------|---------| 
-| precision | `WritePrecision::NoTime` | Timestamp precision of written data |
+| writePrecision | `WritePrecision::NoTime` | Timestamp precision of written data |
 | batchSize | `1` | Number of points that will be written to the database at once |
 | bufferSize | `5` | Maximum number of points in buffer. Buffer contains new data that will be written to the database and also data that failed to be written due to network failure or server overloading |
 | flushInterval | `60` | Maximum time(in seconds) data will be held in buffer before are written to the db |
-| preserveConnection | `false` | true if underlying HTTP connection should be kept open |
+
+## HTTP Options
+`HTTPOptions` controls some aspects of HTTP communication and they are set via `setHTTPOptions` function:
+| Parameter | Default Value | Meaning |
+|-----------|---------------|---------| 
+| reuseConnection | `false` | Whether HTTP connection should be kept open after inital communicaton. Usable for frequent writes/queries. |
+| httpReadTimeout | `5000` | Timeout (ms) for reading server response |
 
 ## Secure Connection
 Connecting to a secured server requires configuring client to trust the server. This is achieved by providing client with a server certificate, certificate authority certificate or certificate SHA1 fingerprint. 

examples/SecureBatchWrite/SecureBatchWrite.ino

@@ -93,7 +93,7 @@ void setup() {
   }
 
   // Enable messages batching and retry buffer
-  client.setWriteOptions(WRITE_PRECISION, MAX_BATCH_SIZE, WRITE_BUFFER_SIZE);
+  client.setWriteOptions(WriteOptions().writePrecision(WRITE_PRECISION).batchSize(MAX_BATCH_SIZE).bufferSize(WRITE_BUFFER_SIZE));
 }
 
 void loop() {

src/InfluxDb.cpp

@@ -122,10 +122,10 @@ void Influxdb::begin() {
  */
 void Influxdb::prepare(InfluxData data) { 
   ++_preparedPoints;
-  if(_batchSize <= _preparedPoints) {
+  if(_writeOptions._batchSize <= _preparedPoints) {
     // for preparation, batchsize must be greater than number of prepared points, or it will send data right away
-    _batchSize = _preparedPoints+1;
-    reserveBuffer(2*_batchSize);
+    _writeOptions._batchSize = _preparedPoints+1;
+    reserveBuffer(2*_writeOptions._batchSize);
   }
   write(data);
 }

src/InfluxDbClient.cpp

@@ -68,93 +68,8 @@ static String precisionToString(WritePrecision precision, uint8_t version = 2) {
     }
 }
 
-Point::Point(String measurement):
-    _measurement(escapeKey(measurement, false)),
-    _tags(""),
-    _fields(""),
-    _timestamp("")
-{
-
-}
-
-void Point::addTag(String name, String value) {
-    if(_tags.length() > 0) {
-        _tags += ',';
-    }
-    _tags += escapeKey(name);
-    _tags += '=';
-    _tags += escapeKey(value);
-}
-
-void Point::addField(String name, const char *value) { 
-    putField(name, "\"" + escapeValue(value) + "\""); 
-}
-
-void Point::putField(String name, String value) {
-    if(_fields.length() > 0) {
-        _fields += ',';
-    }
-    _fields += escapeKey(name);
-    _fields += '=';
-    _fields += value;
-}
-
-String Point::toLineProtocol() const {
-    String line =  _measurement;
-    if(hasTags()) {
-        line += "," + _tags;
-    }
-    if(hasFields()) {
-        line += " " + _fields;
-    }
-    if(hasTime()) {
-        line += " " + _timestamp;
-    }
-    return line;
-}
-
-void  Point::setTime(WritePrecision precision) {
-    struct timeval tv;
-    gettimeofday(&tv, NULL);
-    
-    switch(precision) {
-        case WritePrecision::NS:
-            setTime(getTimeStamp(&tv,9));
-            break;
-        case WritePrecision::US:
-            setTime(getTimeStamp(&tv,6));
-            break;
-        case WritePrecision::MS: 
-            setTime(getTimeStamp(&tv,3));
-            break;
-        case WritePrecision::S:
-            setTime(getTimeStamp(&tv,0));
-            break;
-        case WritePrecision::NoTime:
-            _timestamp = "";
-            break;
-    }
-}
-
-void  Point::setTime(unsigned long long timestamp) {
-    _timestamp = timeStampToString(timestamp);
-}
-
-void  Point::setTime(String timestamp) {
-    _timestamp = timestamp;
-}
-
-void  Point::clearFields() {
-    _fields = "";
-    _timestamp = "";
-}
-
-void Point:: clearTags() {
-    _tags = "";
-}
-
 InfluxDBClient::InfluxDBClient() { 
-    _pointsBuffer = new String[_bufferSize];
+    _pointsBuffer = new String[_writeOptions._bufferSize];
 }
 
 InfluxDBClient::InfluxDBClient(const char *serverUrl, const char *db):InfluxDBClient() {
@@ -233,7 +148,7 @@ bool InfluxDBClient::init() {
     } else {
         _wifiClient = new WiFiClient;
     }
-    _httpClient.setReuse(false);
+    _httpClient.setReuse(_httpOptions._connectionReuse);
 
     _httpClient.setUserAgent(FPSTR(UserAgent));
     return true;
@@ -342,66 +257,72 @@ void InfluxDBClient::setUrls() {
         INFLUXDB_CLIENT_DEBUG("  writeUrl: %s\n", _writeUrl.c_str());
         INFLUXDB_CLIENT_DEBUG("  queryUrl: %s\n", _queryUrl.c_str());
     }
-    if(_writePrecision != WritePrecision::NoTime) {
+    if(_writeOptions._writePrecision != WritePrecision::NoTime) {
         _writeUrl += "&precision=";
-        _writeUrl += precisionToString(_writePrecision, _dbVersion);
+        _writeUrl += precisionToString(_writeOptions._writePrecision, _dbVersion);
         INFLUXDB_CLIENT_DEBUG("  writeUrl: %s\n", _writeUrl.c_str());
     }
 
 }
 
 void InfluxDBClient::setWriteOptions(WritePrecision precision, uint16_t batchSize, uint16_t bufferSize, uint16_t flushInterval, bool preserveConnection) {
-    if(_writePrecision != precision) {
-        _writePrecision = precision;
+    setWriteOptions(WriteOptions().writePrecision(precision).batchSize(batchSize).bufferSize(bufferSize).flushIntervalSec(flushInterval));
+    setHTTPOptions(_httpOptions.connectionReuse(preserveConnection));
+}
+
+void InfluxDBClient::setWriteOptions(const WriteOptions & writeOptions) {
+    if(_writeOptions._writePrecision != writeOptions._writePrecision) {
+        _writeOptions._writePrecision = writeOptions._writePrecision;
         setUrls();
     }
-    if(batchSize > 0) {
-        _batchSize = batchSize;
-    }
-    if(bufferSize < batchSize) {
-        bufferSize = 2*batchSize;
-        INFLUXDB_CLIENT_DEBUG("[D] Changing buffer size to %d\n", bufferSize);
+    if(writeOptions._batchSize > 0) {
+        _writeOptions._batchSize = writeOptions._batchSize;
     }
-    if(_bufferSize > 0 && bufferSize > 0 && _bufferSize != bufferSize) {
-        _bufferSize = bufferSize;
-        if(_bufferSize <  _batchSize) {
-            _bufferSize = 2*_batchSize;
-            INFLUXDB_CLIENT_DEBUG("[D] Changing buffer size to %d\n", _bufferSize);
+    if(_writeOptions._bufferSize > 0 && writeOptions._bufferSize > 0 && _writeOptions._bufferSize != writeOptions._bufferSize) {
+        _writeOptions._bufferSize = writeOptions._bufferSize;
+        if(_writeOptions._bufferSize <  _writeOptions._batchSize) {
+            _writeOptions._bufferSize = 2*_writeOptions._batchSize;
+            INFLUXDB_CLIENT_DEBUG("[D] Changing buffer size to %d\n", _writeOptions._bufferSize);
         }
         resetBuffer();
     }
-    _flushInterval = flushInterval;
-    _httpClient.setReuse(preserveConnection);
+    _writeOptions._flushInterval = writeOptions._flushInterval;
+}
+
+void InfluxDBClient::setHTTPOptions(const HTTPOptions & httpOptions) {
+    _httpOptions = httpOptions;
+    _httpClient.setReuse(_httpOptions._connectionReuse);
+    _httpClient.setTimeout(_httpOptions._httpReadTimeout);
 }
 
 void InfluxDBClient::resetBuffer() {
     if(_pointsBuffer) {
         delete [] _pointsBuffer;
     }
-    _pointsBuffer = new String[_bufferSize];
+    _pointsBuffer = new String[_writeOptions._bufferSize];
     _bufferPointer = 0;
     _batchPointer = 0;
     _bufferCeiling = 0;
 }
 
 void InfluxDBClient::reserveBuffer(int size) {
-    if(size > _bufferSize) {
+    if(size > _writeOptions._bufferSize) {
         String *newBuffer = new String[size];
-        INFLUXDB_CLIENT_DEBUG("Resising buffer from %d to %d\n", _bufferSize, size);
+        INFLUXDB_CLIENT_DEBUG("Resizing buffer from %d to %d\n", _writeOptions._bufferSize, size);
         for(int i=0;i<_bufferCeiling; i++) {
             newBuffer[i] = _pointsBuffer[i];
         }
 
         delete [] _pointsBuffer;
         _pointsBuffer = newBuffer;
-        _bufferSize = size;
+        _writeOptions._bufferSize = size;
     }
 }
 
 bool InfluxDBClient::writePoint(Point & point) {
     if (point.hasFields()) {
-        if(_writePrecision != WritePrecision::NoTime && !point.hasTime()) {
-            point.setTime(_writePrecision);
+        if(_writeOptions._writePrecision != WritePrecision::NoTime && !point.hasTime()) {
+            point.setTime(_writeOptions._writePrecision);
         }
         String line = point.toLineProtocol();
         return writeRecord(line);
@@ -412,15 +333,15 @@ bool InfluxDBClient::writePoint(Point & point) {
 bool InfluxDBClient::writeRecord(String &record) {
     _pointsBuffer[_bufferPointer] = record;
     _bufferPointer++;
-    if(_bufferPointer == _bufferSize) {
+    if(_bufferPointer == _writeOptions._bufferSize) {
         _bufferPointer = 0;
         INFLUXDB_CLIENT_DEBUG("[W] Reached buffer size, old points will be overwritten\n");
         if(isBufferFull()) {
             //if already isBufferFull
             _batchPointer = 0;
         }
     } 
-    if(_bufferCeiling < _bufferSize) {
+    if(_bufferCeiling < _writeOptions._bufferSize) {
         _bufferCeiling++;
     }
     if(isBufferFull() && _batchPointer < _bufferPointer) {
@@ -432,9 +353,9 @@ bool InfluxDBClient::writeRecord(String &record) {
 
 bool InfluxDBClient::checkBuffer() {
     // in case we (over)reach batchSize with non full buffer
-    bool bufferReachedBatchsize = !isBufferFull() && _bufferPointer - _batchPointer >= _batchSize;
+    bool bufferReachedBatchsize = !isBufferFull() && _bufferPointer - _batchPointer >= _writeOptions._batchSize;
     // or flush interval timed out
-    bool flushTimeout = _flushInterval > 0 && _lastFlushed > 0 && (millis()/1000 - _lastFlushed) > _flushInterval; 
+    bool flushTimeout = _writeOptions._flushInterval > 0 && _lastFlushed > 0 && (millis()/1000 - _lastFlushed) > _writeOptions._flushInterval; 
 
     if(bufferReachedBatchsize || flushTimeout || isBufferFull() ) {
         INFLUXDB_CLIENT_DEBUG("[D] Flushing buffer: is oversized %s, is timeout %s, is buffer full %s\n", bufferReachedBatchsize?"true":"false",flushTimeout?"true":"false", isBufferFull()?"true":"false");
@@ -464,9 +385,9 @@ bool InfluxDBClient::flushBuffer() {
             _lastFlushed = millis()/1000;
             _batchPointer += size;
             //did we got over top?
-            if(_batchPointer >= _bufferSize) {
+            if(_batchPointer >= _writeOptions._bufferSize) {
                 // restart _batchPointer in ring buffer from start
-                _batchPointer = _batchPointer - _bufferSize;
+                _batchPointer = _batchPointer - _writeOptions._bufferSize;
                 // we reached buffer size, that means buffer was full and now lower ceiling 
                 _bufferCeiling = _bufferPointer;
             }
@@ -493,7 +414,7 @@ char *InfluxDBClient::prepareBatch(int &size) {
     size = 0;
     int length = 0;
     char *buff = nullptr;
-    uint16_t top = _batchPointer+_batchSize;
+    uint16_t top = _batchPointer+_writeOptions._batchSize;
     INFLUXDB_CLIENT_DEBUG("[D] Prepare batch: bufferPointer: %d, batchPointer: %d, ceiling %d\n", _bufferPointer, _batchPointer, _bufferCeiling);
     if(top > _bufferCeiling ) {
         // are we returning to the begining?
@@ -510,14 +431,14 @@ char *InfluxDBClient::prepareBatch(int &size) {
     if(top > _batchPointer) { 
         size = top - _batchPointer;
     } else if(top < _batchPointer) {
-        size = _bufferSize - (_batchPointer - top);
+        size = _writeOptions._bufferSize - (_batchPointer - top);
     }
     INFLUXDB_CLIENT_DEBUG("[D] Prepare batch size %d\n", size);
     if(size) {
         int i = _batchPointer;
         for(int c=0; c < size; c++) {
             length += _pointsBuffer[i++].length();
-            if(i == _bufferSize) {
+            if(i == _writeOptions._bufferSize) {
                 i = 0;
             }
             yield();
@@ -530,7 +451,7 @@ char *InfluxDBClient::prepareBatch(int &size) {
             for(int c=0; c < size; c++) {
                 strcat(buff+strlen(buff), _pointsBuffer[i++].c_str());
                 strcat(buff+strlen(buff), "\n");
-                if(i == _bufferSize) {
+                if(i == _writeOptions._bufferSize) {
                     i = 0;
                 }
                 yield();