Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ingester-all/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<parent>
<groupId>io.greptime</groupId>
<artifactId>greptimedb-ingester</artifactId>
<version>0.14.5</version>
<version>0.14.6</version>
</parent>

<artifactId>ingester-all</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion ingester-bulk-protocol/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<parent>
<groupId>io.greptime</groupId>
<artifactId>greptimedb-ingester</artifactId>
<version>0.14.5</version>
<version>0.14.6</version>
</parent>

<artifactId>ingester-bulk-protocol</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion ingester-common/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<parent>
<groupId>io.greptime</groupId>
<artifactId>greptimedb-ingester</artifactId>
<version>0.14.5</version>
<version>0.14.6</version>
</parent>

<artifactId>ingester-common</artifactId>
Expand Down
4 changes: 3 additions & 1 deletion ingester-example/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,9 @@ This API is particularly well-suited for:

The Bulk Write API provides a high-performance, memory-efficient mechanism for ingesting large volumes of time-series data into GreptimeDB. It leverages Apache Arrow's Flight protocol and off-heap memory management to achieve optimal throughput when writing batches of data.

Unlike regular streaming, this API allows continuous writing to only one table per stream, but can handle very large data volumes (up to 200MB per write). It features sophisticated adaptive flow control mechanisms that automatically adjust to your data throughput requirements.
**Important Note**:
- This API is designed around streaming connections, which means each stream establishes a connection to only one database node. Unlike the regular write API, it lacks automatic load balancing for individual requests. However, if your use case involves multiple clients establishing multiple streams to the database, this limitation is not a concern.
- Unlike regular streaming, this API allows continuous writing to only one table per stream, but can handle very large data volumes (up to 200MB per write). It features sophisticated adaptive flow control mechanisms that automatically adjust to your data throughput requirements.

The Bulk Write API achieves superior performance compared to the Regular Write API through:
- Off-heap memory management using Arrow buffers
Expand Down
2 changes: 1 addition & 1 deletion ingester-example/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<parent>
<groupId>io.greptime</groupId>
<artifactId>greptimedb-ingester</artifactId>
<version>0.14.5</version>
<version>0.14.6</version>
</parent>

<artifactId>ingester-example</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ public static GreptimeDB connect() {
GreptimeOptions opts = GreptimeOptions.newBuilder(endpoints, database)
.writeMaxRetries(0)
.defaultStreamMaxWritePointsPerSecond(Integer.MAX_VALUE)
.maxInFlightWritePoints(Integer.MAX_VALUE)
.useZeroCopyWriteInBulkWrite(true)
.build();
LOG.info("Connect to db: {}, endpoint: {}", database, endpointsStr);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,9 @@ public class BatchingWriteBenchmark {
public static void main(String[] args) throws Exception {
boolean zstdCompression = SystemPropertyUtil.getBool("zstd_compression", true);
int batchSize = SystemPropertyUtil.getInt("batch_size_per_request", 64 * 1024);
int maxPointsPerSecond = SystemPropertyUtil.getInt("max_points_per_second", Integer.MAX_VALUE);

LOG.info("Using zstd compression: {}", zstdCompression);
LOG.info("Batch size: {}", batchSize);
LOG.info("Max points per second: {}", maxPointsPerSecond);

// Start a metrics exporter
MetricsExporter metricsExporter = new MetricsExporter(MetricsUtil.metricRegistry());
Expand Down
2 changes: 1 addition & 1 deletion ingester-grpc/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<parent>
<groupId>io.greptime</groupId>
<artifactId>greptimedb-ingester</artifactId>
<version>0.14.5</version>
<version>0.14.6</version>
</parent>

<artifactId>ingester-grpc</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion ingester-prometheus-metrics/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<parent>
<groupId>io.greptime</groupId>
<artifactId>greptimedb-ingester</artifactId>
<version>0.14.5</version>
<version>0.14.6</version>
</parent>
<artifactId>ingester-prometheus-metrics</artifactId>

Expand Down
2 changes: 1 addition & 1 deletion ingester-protocol/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<parent>
<groupId>io.greptime</groupId>
<artifactId>greptimedb-ingester</artifactId>
<version>0.14.5</version>
<version>0.14.6</version>
</parent>

<artifactId>ingester-protocol</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@
# See the License for the specific language governing permissions and
# limitations under the License.

client.version=0.14.5
client.version=0.14.6
2 changes: 1 addition & 1 deletion ingester-protocol/src/test/java/io/greptime/UtilTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,6 @@ public class UtilTest {
@Test
public void testClientVersion() {
String ver = Util.clientVersion();
Assert.assertEquals("0.14.5", ver);
Assert.assertEquals("0.14.6", ver);
}
}
2 changes: 1 addition & 1 deletion ingester-rpc/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<parent>
<groupId>io.greptime</groupId>
<artifactId>greptimedb-ingester</artifactId>
<version>0.14.5</version>
<version>0.14.6</version>
</parent>

<artifactId>ingester-rpc</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

<groupId>io.greptime</groupId>
<artifactId>greptimedb-ingester</artifactId>
<version>0.14.5</version>
<version>0.14.6</version>
<packaging>pom</packaging>

<name>${project.groupId}:${project.artifactId}</name>
Expand Down
Loading