Skip to content

Commit c0df97c

Browse files
author
mrrtree
authored
refactor: 1.0.0 API (#40)
* feat: remove http mangement * refactor: refactor sdk api * chore: remove dependency arvo * doc: update docs with new api * chore: rename metric to table in RouteClient * feat: supprt zstd batch decompress * doc: update version to 1.0.0.alpha * refactor: rename Value float64 to double * fix: fix WriteOkTest assert * chore: license header instread by simple one * doc: update changelog 1.0.0.alpha * oc: tranlate doc/* to en * fix: use simple license header * feat: support PointBuilder * feat: support database * feat: add database & remove tenant * feat: remove TablePointsBuilder * fix: fix problems in pr conversation * doc: rename metrics metric to table
1 parent ba05e59 commit c0df97c

File tree

237 files changed

+3732
-9368
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

237 files changed

+3732
-9368
lines changed

README.md

Lines changed: 63 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -102,27 +102,25 @@ CeresDB is a high-performance, distributed, schema-less, cloud native time-serie
102102
<dependency>
103103
<groupId>io.ceresdb</groupId>
104104
<artifactId>ceresdb-all</artifactId>
105-
<version>0.1.0</version>
105+
<version>1.0.0-alpha</version>
106106
</dependency>
107107
```
108108

109109
## Init CeresDB client
110110
```java
111-
// CeresDB options
112-
final CeresDBOptions opts = CeresDBOptions.newBuilder("127.0.0.1", 8831) //// CeresDB default grpc port 8831
113-
.tenant("public", "sub_test", "test_token") // tenant info
111+
final CeresDBOptions opts = CeresDBOptions.newBuilder("127.0.0.1", 8831, DIRECT) // CeresDB default grpc port 8831,use DIRECT RouteMode
112+
.database("public") // use database for client, can be overridden by the RequestContext in request
114113
// maximum retry times when write fails
115114
// (only some error codes will be retried, such as the routing table failure)
116115
.writeMaxRetries(1)
117116
// maximum retry times when read fails
118117
// (only some error codes will be retried, such as the routing table failure)
119-
.readMaxRetries(1)
120-
.build();
118+
.readMaxRetries(1).build();
121119

122120
final CeresDBClient client = new CeresDBClient();
123-
if (!client.init(this.opts)) {
121+
if (!client.init(opts)) {
124122
throw new IllegalStateException("Fail to start CeresDBClient");
125-
}
123+
}
126124
```
127125
For more configuration options, see [configuration](docs/configuration.md)
128126

@@ -132,105 +130,87 @@ CeresDB is a Schema-less time-series database, so creating table schema ahead of
132130
The following table creation statement(using the SQL API included in SDK )shows all field types supported by CeresDB:
133131

134132
```java
135-
SqlResult result = client.management().executeSql("CREATE TABLE MY_FIRST_TABL(" +
136-
"ts TIMESTAMP NOT NULL," +
137-
"c1 STRING TAG NOT NULL," +
138-
"c2 STRING TAG NOT NULL," +
139-
"c3 DOUBLE NULL," +
140-
"c4 STRING NULL," +
141-
"c5 INT64 NULL," +
142-
"c6 FLOAT NULL," +
143-
"c7 INT32 NULL," +
144-
"c8 INT16 NULL," +
145-
"c9 INT8 NULL," +
146-
"c10 BOOLEAN NULL,"
147-
"c11 UINT64 NULL,"
148-
"c12 UINT32 NULL,"
149-
"c13 UINT16 NULL,"
150-
"c14 UINT8 NULL,"
151-
"c15 TIMESTAMP NULL,"
152-
"c16 VARBINARY NULL,"
153-
"TIMESTAMP KEY(ts)) ENGINE=Analytic"
154-
);
133+
// Create table manually, creating table schema ahead of data ingestion is not required
134+
String createTableSql = "CREATE TABLE IF NOT EXISTS machine_table(" + "ts TIMESTAMP NOT NULL," + //
135+
"ts TIMESTAMP NOT NULL," +
136+
"city STRING TAG NOT NULL," +
137+
"ip STRING TAG NOT NULL," +
138+
"cpu DOUBLE NULL," +
139+
"mem DOUBLE NULL," +
140+
"TIMESTAMP KEY(ts)" + // timestamp column must be specified
141+
") ENGINE=Analytic";
142+
143+
Result<SqlQueryOk, Err> createResult = client.sqlQuery(new SqlQueryRequest(createTableSql)).get();
144+
if (!createResult.isOk()) {
145+
throw new IllegalStateException("Fail to create table");
146+
}
155147
```
156148

157-
## Write data example
149+
## How to build write data
158150
```java
159-
final long t0 = System.currentTimeMillis();
160-
final long t1 = t0 + 1000;
161-
final long t2 = t1 + 1000;
162-
final Rows data = Series.newBuilder("machine_metric")
163-
.tag("city", "Singapore")
164-
.tag("ip", "127.0.0.1")
165-
.toRowsBuilder()
166-
// codes below organizes 3 lines data (3 timestamps) for the `cpu` and `mem` column, this will just transport once through network. CeresDB encourage practices like this, because the SDK could use efficient compression algorithm to reduce network traffic and also be friendly to the sever side.
167-
.field(t0, "cpu", FieldValue.withDouble(0.23)) // first row, first column
168-
.field(t0, "mem", FieldValue.withDouble(0.55)) // first row, second column
169-
.field(t1, "cpu", FieldValue.withDouble(0.25)) // second row, first column
170-
.field(t1, "mem", FieldValue.withDouble(0.56)) // second row, second column
171-
.field(t2, "cpu", FieldValue.withDouble(0.21)) // third row, first column
172-
.field(t2, "mem", FieldValue.withDouble(0.52)) // third row, second column
151+
final Point point = Point.newPointBuilder("machine_table")
152+
.setTimestamp(t0)
153+
.addTag("city", "Singapore")
154+
.addTag("ip", "10.0.0.1")
155+
.addField("cpu", Value.withDouble(0.23))
156+
.addField("mem", Value.withDouble(0.55))
173157
.build();
158+
```
174159

175-
final CompletableFuture<Result<WriteOk, Err>> wf = client.write(data);
160+
## Write data example
161+
```java
162+
final CompletableFuture<Result<WriteOk, Err>> wf = client.write(new WriteRequest(pointList));
176163
// here the `future.get` is just for demonstration, a better async programming practice would be using the CompletableFuture API
177-
final Result<WriteOk, Err> wr = wf.get();
178-
179-
Assert.assertTrue(wr.isOk());
180-
Assert.assertEquals(3, wr.getOk().getSuccess());
181-
// `Result` class referenced the Rust language practice, provides rich functions (such as mapXXX, andThen) transforming the result value to improve programming efficiency. You can refer to the API docs for detail usage.
182-
Assert.assertEquals(3, wr.mapOr(0, WriteOk::getSuccess).intValue());
183-
Assert.assertEquals(0, wr.getOk().getFailed());
184-
Assert.assertEquals(0, wr.mapOr(-1, WriteOk::getFailed).intValue());
164+
final Result<WriteOk, Err> writeResult = wf.get();
165+
Assert.assertTrue(writeResult.isOk());
166+
// `Result` class referenced the Rust language practice, provides rich functions (such as mapXXX, andThen) transforming the result value to improve programming efficiency. You can refer to the API docs for detail usage.
167+
Assert.assertEquals(3, writeResult.getOk().getSuccess());
168+
Assert.assertEquals(3, writeResult.mapOr(0, WriteOk::getSuccess).intValue());
169+
Assert.assertEquals(0, writeResult.mapOr(-1, WriteOk::getFailed).intValue());
185170
```
186171
See [write](docs/write.md)
187172

188173
## Query data example
189174
```java
190-
final QueryRequest queryRequest = QueryRequest.newBuilder()
191-
.forMetrics("machine_metric") // table name is optional. If not provided, SQL parser will parse the `ql` to get the table name and do the routing automaticly
192-
.ql("select timestamp, cpu, mem from machine_metric") //
175+
final SqlQueryRequest queryRequest = SqlQueryRequest.newBuilder()
176+
.forTables("machine_table") // table name is optional. If not provided, SQL parser will parse the `sql` to get the table name and do the routing automaticly
177+
.sql("select * from machine_table where ts = %d", t0) //
193178
.build();
194-
final CompletableFuture<Result<QueryOk, Err>> qf = client.query(queryRequest);
179+
final CompletableFuture<Result<SqlQueryOk, Err>> qf = client.sqlQuery(queryRequest);
195180
// here the `future.get` is just for demonstration, a better async programming practice would be using the CompletableFuture API
196-
final Result<QueryOk, Err> qr = qf.get();
181+
final Result<SqlQueryOk, Err> queryResult = qf.get();
182+
183+
Assert.assertTrue(queryResult.isOk());
197184

198-
Assert.assertTrue(qr.isOk());
185+
final SqlQueryOk queryOk = queryResult.getOk();
186+
Assert.assertEquals(1, queryOk.getRowCount());
199187

200-
final QueryOk queryOk = qr.getOk();
188+
// get rows as list
189+
final List<Row> rows = queryOk.getRowList();
201190

202-
final List<Record> records = queryOk.mapToRecord().collect(Collectors.toList())
191+
// get rows as stream
192+
final Stream<Row> rowStream = queryOk.stream();
193+
rowStream.forEach(row -> System.out.println(row.toString()));
203194
```
204195
See [read](docs/read.md)
205196

206197
## stream write/read Example
207198
CeresDB support streaming writing and reading,suitable for large-scale data reading and writing。
208199
```java
209-
final Calendar time = Calendar.getInstance();
210-
final StreamWriteBuf<Rows, WriteOk> writeBuf = client.streamWrite("machine_metric");
211-
for (int i = 0; i < 1000; i++) {
212-
time.add(Calendar.MILLISECOND, 1);
213-
Collection<Rows> rows = new ArrayList<>();
214-
final long t0 = System.currentTimeMillis();
215-
final long t1 = t0 + 1000;
216-
final long t2 = t1 + 1000;
217-
final Rows data = Series.newBuilder("machine_metric").tag("city", "Singapore").tag("ip", "127.0.0.1")
218-
.toRowsBuilder()
219-
.field(t0, "cpu", FieldValue.withDouble(0.23))
220-
.field(t0, "mem", FieldValue.withDouble(0.55))
221-
.field(t1, "cpu", FieldValue.withDouble(0.25))
222-
.field(t1, "mem", FieldValue.withDouble(0.56))
223-
.field(t2, "cpu", FieldValue.withDouble(0.21))
224-
.field(t2, "mem", FieldValue.withDouble(0.52))
200+
final StreamWriteBuf<Point, WriteOk> writeBuf = client.streamWrite("machine_table");
201+
for (int i = 0; i < 1000; i++) {
202+
final Point point = Point.newPointBuilder("machine_table")
203+
.setTimestamp(timestamp)
204+
.addTag("city", "Beijing")
205+
.addTag("ip", "10.0.0.3")
206+
.addField("cpu", Value.withDouble(0.42))
207+
.addField("mem", Value.withDouble(0.67))
225208
.build();
226-
rows.add(data);
227-
writeBuf.writeAndFlush(data);
228-
}
229-
final CompletableFuture<WriteOk> writeOk = writeBuf.completed();
230-
Assert.assertEquals(1000, writeOk.join().getSuccess());
209+
writeBuf.writeAndFlush(Arrays.asList(point));
210+
timestamp = timestamp+1;
211+
}
231212

232-
final QueryRequest req = QueryRequest.newBuilder().ql("select * from %s", "machine_metric").build();
233-
final Iterator<Record> it = client.blockingStreamQuery(req, 3, TimeUnit.SECONDS);
213+
final CompletableFuture<WriteOk> writeOk = writeBuf.completed();
234214
```
235215
See [streaming](docs/streaming.md)
236216

0 commit comments

Comments
 (0)