Performance improvements (#27)

* Developed batches to improve the performance of the metrics sending. Now its possible to control the batch size and frequency of the data importing. See read me for the details.
* Improved string messages saving.
* Updated libs.
* Supported the dashboard (compatible with Grafana 8.2.3) - tables have been updated as well. No its possible to sort data. 

Co-authored-by: Mikhail.Derevyanko <[email protected]>

Author: mderevyankoaqa

.idea/misc.xml

@@ -12,12 +12,19 @@ The plugin sends metrics to InfluxDB and provides the possibility to visualize t
 * Latency;
 * The response time (uses from the SampleResult.class, needs to make aggregate report).
 
+## Important notes 
+The plugin allows 5 errors happened one by one, then plugin will stop importing the results after that! See details in the logs.
+Counter will be refreshed at least you have 4 fails. This is protection to avoid OOM error.
+
+Pleas monitor the elapsed time of the data importing (see logs) to avoid issues with requests sending from JMeter.
+Pay attention on "Sending metrics performance tuning" chapter, see below.
 
 ## Compatibility
 The supported versions:
-* Java 11 - make sure that you have it.
+* Java 11 - make sure that you have it (its minimum version).
 * InfluxDB v2.0, see release notes: https://docs.influxdata.com/influxdb/v2.0/reference/release-notes/influxdb/  (1.8 is not supported)
-* JMeter 5.4.1 only (the previous supports 3.5)
+* JMeter 5.4.1 only.
+* The current board and plugin were tested on Grafana 8.2.3 and InfluxDB 2.0.9, JAVA 15.
 
 ## Deployment
 * Put '`jmeter-plugin-influxdb2-listener-<version>.jar`' file from [Releases](https://github.com/mderevyankoaqa/jmeter-influxdb2-listener-plugin/releases) to `~<JmeterPath<\lib\ext`; 
@@ -48,10 +55,13 @@ Let’s explain the plugin fields:
 * `testName` - the name of the test.
 * `nodeName` - the name of the server.
 * `runId` - the identification number of hte test run, can be dynamic.
-* `influxDBScheme` - InfluxDB server scheme (can be http or https) .
+* `influxDBScheme` - InfluxDB server scheme (can be http or https).
 * `influxDBHost` - the host name or ip of the InfluxDB server.
 * `influxDBPort` - the port of the InfluxDB server, the default is 8086.
 * `influxDBToken` - the influxdb bucket token, the default value should be updated, copy it from InfluxDB site.
+* `influxDBFlushInterval` - its interval to send data to InfluxDB, the default value is 4000 (4 seconds).
+* `influxDBMaxBatchSize` - the max size of the batch with metrics, the default 2000 (2000 items of JMeter results).
+
 
  ![](img/influx3.png)
 
@@ -76,6 +86,18 @@ You can modify the generated string in terms of your purposes.
 * `recordSubSamples` - allows to save the JMeter sub samples if set to 'true'.
 * `saveResponseBodyOfFailures` - allows to save the response body of the failures.
 
+## Sending metrics performance tuning
+The plugin imports batch with JMeter results each 4 seconds (by default settings). In the logs you will see records like this:
+`INFO o.m.j.i.v.InfluxDatabaseBackendListenerClient: Data has been imported successfully, batch size --> 68, elapsed time is --> 14 ms` (the elapsed time is the response time of the batch importing.)
+So you can control the flush interval depends on the load you have and adjust `influxDBFlushInterval` setting. Is not recommended having less 1 second.  
+Max batch protection -> send data when batch max size is occurred. For example, when batch size is 2000 items (it's the default setting of `influxDBMaxBatchSize`) plugin imports that batch, even when flush interval was not occurred.
+Using both options you can tune data importing and have optimal performance. 
+
+Make sure you have enough ram to aggregate huge batch and optimal flush period.
+
+Notes: when test has been interrupted from UI; the processes may not be finished properly, restart JMeter.
+ 
+
 ## Grafana dashboard capabilities
 See deployment instructions here https://grafana.com/grafana/dashboards/13644
 
@@ -87,24 +109,26 @@ Dashboard helps:
 * Overview and analise the response time, distribution as well.
 
  ![](img/grafana2.png)
+
+ ![](img/grafana3.png)
 
 * See aggregate report. 
   The table rendering may take an extra time. The table has hardware resources consuming queries from Influxdb side. If you have low hardware on the Influxdb server - recommended make the clone of the original dashboard and remove aggregate report.
   So the idea - it's to have one 'fast' dashboard for the online monitoring (has no aggregate report) to see the results while the test, the second (original with aggregate report) to see the final results.
-  
- ![](img/grafana3.png)
+
+ ![](img/grafana4.png)
 
 * Investigate errors. The table is interactive, it's possible to filter data in the columns and see details for the specific error.
 
- ![](img/grafana4.png)
+ ![](img/grafana5.png)
 
 * See network statistics, latency, processing time.
-
- ![](img/grafana5.png)
+  
+  ![](img/grafana6.png) 
 
 * Check individual request details.
 
- ![](img/grafana6.png)
+ ![](img/grafana7.png)
 
 
 

.idea/workspace.xml

@@ -14,7 +14,7 @@ repositories {
 }
 
 sourceCompatibility = 11
-version = '1.2'
+version = '1.3'
 def title = 'JMeterInfluxDBListener'
 def archiveName = 'jmeter-plugin-influxdb2-listener'