Skip to content
This repository was archived by the owner on Sep 26, 2023. It is now read-only.

Commit 90e45b5

Browse files
rahulKQLigorbernstein2
authored andcommitted
Introducing interfaces for new Batching API (#692)
* Initial Commit for fresh approach * Batcher Implementation with blocking flush * Introducing new Batching API interfaces ## What this PR contain This contains interfaces required to perform batching with elements(`MutateRowsRequest.Entry` or `LogEntry`) object directly instead of RequestWrappers(`MutateRowsRequest` or `WriteLogEntriesRequest`). * reverting batching/BatchingSettings * Addressing the javadoc related comments * Removed `@Override` annotation from Javadoc code sample * removed access specifiers from sample code and some tiny language edits * Updated generic types name for Batcher related interfaces Have updated type names to increase user understanding by their name. Address some sample changes according to feedback. * Reverting type names and RequestBuilder#build removed return statement * Updated @param for types in batcher interfaces
1 parent 973b5a6 commit 90e45b5

File tree

3 files changed

+215
-0
lines changed

3 files changed

+215
-0
lines changed
Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
/*
2+
* Copyright 2019 Google LLC
3+
*
4+
* Redistribution and use in source and binary forms, with or without
5+
* modification, are permitted provided that the following conditions are
6+
* met:
7+
*
8+
* * Redistributions of source code must retain the above copyright
9+
* notice, this list of conditions and the following disclaimer.
10+
* * Redistributions in binary form must reproduce the above
11+
* copyright notice, this list of conditions and the following disclaimer
12+
* in the documentation and/or other materials provided with the
13+
* distribution.
14+
* * Neither the name of Google LLC nor the names of its
15+
* contributors may be used to endorse or promote products derived from
16+
* this software without specific prior written permission.
17+
*
18+
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
19+
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
20+
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
21+
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
22+
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
23+
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
24+
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
25+
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
26+
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
27+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28+
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29+
*/
30+
package com.google.api.gax.batching.v2;
31+
32+
import com.google.api.core.ApiFuture;
33+
34+
/**
35+
* Represents a batching context where individual elements will be accumulated and flushed in a
36+
* large batch request at some point in the future. The buffered elements can be flushed manually or
37+
* when triggered by an internal threshold. This is intended to be used for high throughput
38+
* scenarios at the cost of latency.
39+
*
40+
* @param <ElementT> The type of each individual element to be batched.
41+
* @param <ElementResultT> The type of the result for each individual element.
42+
*/
43+
public interface Batcher<ElementT, ElementResultT> extends AutoCloseable {
44+
45+
/**
46+
* Queues the passed in element to be sent at some point in the future.
47+
*
48+
* <p>The element will be sent as part of a larger batch request at some point in the future. The
49+
* returned {@link ApiFuture} will be resolved once the result for the element has been extracted
50+
* from the batch response.
51+
*
52+
* <p>Note: Cancelling returned result simply marks the future cancelled, It would not stop the
53+
* batch request.
54+
*/
55+
ApiFuture<ElementResultT> add(ElementT entry);
56+
57+
/**
58+
* Synchronously sends any pending elements as a batch and waits for all outstanding batches to be
59+
* complete.
60+
*/
61+
void flush() throws InterruptedException;
62+
63+
/**
64+
* Closes this Batcher by preventing new elements from being added and flushing the existing
65+
* elements.
66+
*/
67+
@Override
68+
void close() throws InterruptedException;
69+
}
Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
/*
2+
* Copyright 2019 Google LLC
3+
*
4+
* Redistribution and use in source and binary forms, with or without
5+
* modification, are permitted provided that the following conditions are
6+
* met:
7+
*
8+
* * Redistributions of source code must retain the above copyright
9+
* notice, this list of conditions and the following disclaimer.
10+
* * Redistributions in binary form must reproduce the above
11+
* copyright notice, this list of conditions and the following disclaimer
12+
* in the documentation and/or other materials provided with the
13+
* distribution.
14+
* * Neither the name of Google LLC nor the names of its
15+
* contributors may be used to endorse or promote products derived from
16+
* this software without specific prior written permission.
17+
*
18+
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
19+
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
20+
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
21+
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
22+
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
23+
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
24+
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
25+
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
26+
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
27+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28+
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29+
*/
30+
package com.google.api.gax.batching.v2;
31+
32+
import com.google.api.core.SettableApiFuture;
33+
import java.util.List;
34+
35+
/**
36+
* An adapter that packs and unpacks the elements in and out of batch requests and responses.
37+
*
38+
* <p>This interface should be implemented by either a service specific client or autogenerated by
39+
* gapic-generator.
40+
*
41+
* <p>Example implementation:
42+
*
43+
* <pre>{@code
44+
* class ListDescriptor implements BatchingDescriptor<String, String, List<String>, List<String>> {
45+
*
46+
* RequestBuilder<String, List<String>> newRequestBuilder(List<String> prototype) {
47+
* return new RequestBuilder<String, List<String>>() {
48+
*
49+
* void add(String element) {
50+
* list.add(element);
51+
* }
52+
*
53+
* List<String> build() {
54+
* return list.clone();
55+
* }
56+
* };
57+
* }
58+
*
59+
* void splitResponse(List<String> callableResponse, List<SettableApiFuture<String>> batch) {
60+
* for (int i = 0; i < batchResponse.size(); i++) {
61+
* batch.get(i).set(batchResponse.get(i);
62+
* }
63+
* }
64+
*
65+
* void splitException(Throwable throwable, List<SettableApiFuture<String>> batch) {
66+
* for (SettableApiFuture<String> result : batch) {
67+
* result.setException(throwable);
68+
* }
69+
* }
70+
*
71+
* long countBytes(String element) {
72+
* return element.length();
73+
* }
74+
* }
75+
* }</pre>
76+
*
77+
* @param <ElementT> The type of each individual element to be batched
78+
* @param <ElementResultT> The type of the result for each individual element
79+
* @param <RequestT> The type of the request that will contain the accumulated elements
80+
* @param <ResponseT> The type of the response that will be unpacked into individual element results
81+
*/
82+
public interface BatchingDescriptor<ElementT, ElementResultT, RequestT, ResponseT> {
83+
84+
/**
85+
* Creates a new wrapper for the underlying request builder. It's used to pack the current batch
86+
* request with elements.
87+
*/
88+
RequestBuilder<ElementT, RequestT> newRequestBuilder(RequestT prototype);
89+
90+
/** Unpacks the batch response into individual elements results. */
91+
void splitResponse(ResponseT batchResponse, List<SettableApiFuture<ElementResultT>> batch);
92+
93+
/** Unpacks the batch response error into individual element errors. */
94+
void splitException(Throwable throwable, List<SettableApiFuture<ElementResultT>> batch);
95+
96+
/** Returns the size of the passed element object in bytes. */
97+
long countBytes(ElementT element);
98+
}
Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
/*
2+
* Copyright 2019 Google LLC
3+
*
4+
* Redistribution and use in source and binary forms, with or without
5+
* modification, are permitted provided that the following conditions are
6+
* met:
7+
*
8+
* * Redistributions of source code must retain the above copyright
9+
* notice, this list of conditions and the following disclaimer.
10+
* * Redistributions in binary form must reproduce the above
11+
* copyright notice, this list of conditions and the following disclaimer
12+
* in the documentation and/or other materials provided with the
13+
* distribution.
14+
* * Neither the name of Google LLC nor the names of its
15+
* contributors may be used to endorse or promote products derived from
16+
* this software without specific prior written permission.
17+
*
18+
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
19+
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
20+
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
21+
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
22+
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
23+
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
24+
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
25+
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
26+
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
27+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28+
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29+
*/
30+
package com.google.api.gax.batching.v2;
31+
32+
/**
33+
* Adapter to pack individual elements into a larger batch request.
34+
*
35+
* <p>The implementation for this interface will be implemented by service specific client or auto
36+
* generated by the gapic-generator.
37+
*
38+
* @param <ElementT> The type of each individual element to be batched.
39+
* @param <RequestT> The type of the request that will contain the accumulated elements.
40+
*/
41+
public interface RequestBuilder<ElementT, RequestT> {
42+
43+
/** Adds element object into client specific batch request. */
44+
void add(ElementT element);
45+
46+
/** Returns the collected elements as a single client specific batch request. */
47+
RequestT build();
48+
}

0 commit comments

Comments
 (0)