Skip to content

Add stubs for channel pool health checking #2634

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 69 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
1cd5e4a
add stubs and make changes to BigtableChannelPool to allow for integr…
nicholsl Jul 16, 2025
08eb7af
add license headers
nicholsl Jul 16, 2025
cc76959
chore: fix viewConfig in AutomatedBackupPolicy (#2604)
mutianf Jun 16, 2025
b97ff6e
chore: integrate channel pool fixes (#2599)
nicholsl Jun 16, 2025
caa1f89
Update BigtableMaterializedViewIT.java (#2606)
ron-gal Jun 17, 2025
9210c56
test(bigtable): fix mv tests (#2608)
ron-gal Jun 18, 2025
8f15829
tests(bigtable): fix mv tests zone (#2609)
ron-gal Jun 20, 2025
8865a00
ci: allow nonexistent profile in google-cloud-bigtable (#2611)
suztomo Jun 23, 2025
22e6300
ci: add name elements for the POM.xml files (#2612)
suztomo Jun 23, 2025
8c1ac4c
fix: populate table id for materialized view (#2610)
mutianf Jun 24, 2025
1fd64f2
deps: update shared dependencies (#2605)
renovate-bot Jun 27, 2025
f522c8a
chore(main): release 2.61.0 (#2600)
release-please[bot] Jun 27, 2025
21b9278
chore(main): release 2.61.1-SNAPSHOT (#2615)
release-please[bot] Jun 30, 2025
2d08bd5
chore: Update generation configuration at Wed Jul 2 02:33:52 UTC 202…
cloud-java-bot Jul 2, 2025
5a0033c
feat(bigtable): Add schema bundle support (#2619)
ron-gal Jul 9, 2025
039c16a
feat: next release from main branch is 2.62.0 (#2621)
jinseopkim0 Jul 10, 2025
e76b8ca
deps: minor cleanup (#2623)
igorbernstein2 Jul 11, 2025
97a4930
chore: fix old build todo (#2625)
igorbernstein2 Jul 14, 2025
bc08faf
deps: update shared dependencies (#2616)
renovate-bot Jul 15, 2025
11938df
chore(main): release 2.62.0 (#2617)
release-please[bot] Jul 16, 2025
0372cd3
test: deflake prepare plan refresh timeout test (#2628)
jackdingilian Jul 16, 2025
f3c6734
lint
nicholsl Jul 16, 2025
48c7498
Merge branch 'main' into feature/channel-pool-health-checking-stub
nicholsl Jul 16, 2025
8d39504
documentation
nicholsl Jul 16, 2025
86958ad
lint
nicholsl Jul 16, 2025
2cc217c
lint
nicholsl Jul 17, 2025
4229883
lint
nicholsl Jul 17, 2025
c70f7ff
rearchitect so that channelpoolhealthchecker doesn't keep a separate …
nicholsl Jul 21, 2025
94895d5
have all operations related to proberesults be handled by the Channel…
nicholsl Jul 21, 2025
30af3b6
lint
nicholsl Jul 21, 2025
c6f8aef
Merge branch 'googleapis:main' into feature/channel-pool-health-check…
nicholsl Jul 21, 2025
243a5dd
Merge branch 'main' into feature/channel-pool-health-checking-stub
nicholsl Jul 22, 2025
1407bec
refactor channel health checking structure
nicholsl Jul 28, 2025
2b7f5fa
lint
nicholsl Jul 28, 2025
c477532
chore: Update generation configuration at Tue Jul 22 02:39:54 UTC 202…
cloud-java-bot Jul 22, 2025
2555141
chore: create LTS branch 2.60.x (#2635)
jinseopkim0 Jul 23, 2025
a713404
chore: Update generation configuration at Thu Jul 24 02:40:02 UTC 202…
cloud-java-bot Jul 24, 2025
b198022
chore: manual update of update_generation_config.sh (#2641)
diegomarquezp Jul 28, 2025
4614402
chore: Move channel priming out of InstantiatingGrpcTransportProvider
nicholsl Jul 29, 2025
15339b7
delete comment
nicholsl Jul 29, 2025
65bf6d9
fixes
nicholsl Jul 29, 2025
a6e7748
Update channel priming annotation
nicholsl Jul 29, 2025
c9629aa
Update google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/…
nicholsl Jul 29, 2025
8ada4d3
fixes
nicholsl Jul 29, 2025
80e57bf
add no op channel primer
nicholsl Jul 29, 2025
0d42d1a
add noop channel primer test
nicholsl Jul 29, 2025
20400bd
lint
nicholsl Jul 29, 2025
6344fdc
fixes
nicholsl Jul 29, 2025
84e555a
fixes
nicholsl Jul 29, 2025
7e12fea
feat/add-async-channel-ping
nicholsl Jul 29, 2025
9462a9c
lint
nicholsl Jul 29, 2025
68666d5
minor change to test
nicholsl Jul 29, 2025
463bb5b
clirr-ignored-differences.xml
nicholsl Jul 29, 2025
a33c53b
increase timeout for testing purposes
nicholsl Jul 29, 2025
db9278d
increase offset jitter to make more resilient to flakes
nicholsl Jul 30, 2025
1dc9526
more fault tolerance
nicholsl Jul 30, 2025
b082505
revert flaky test changes
nicholsl Jul 30, 2025
ec2ee14
deps: update shared dependencies (#2605)
renovate-bot Jun 27, 2025
fbd7091
chore(main): release 2.61.0 (#2600)
release-please[bot] Jun 27, 2025
0e25815
deps: update shared dependencies (#2616)
renovate-bot Jul 15, 2025
3c7d8dd
rebase onto chore/move-channel-priming-out
release-please[bot] Jul 16, 2025
e2eb9c1
rearchitect so that channelpoolhealthchecker doesn't keep a separate …
nicholsl Jul 21, 2025
eab4200
lint
nicholsl Jul 21, 2025
a40f0e5
refactor channel health checking structure
nicholsl Jul 28, 2025
ff20b87
lint
nicholsl Jul 28, 2025
9ba6b2c
add nonblocking calls for channel priming
nicholsl Jul 29, 2025
d413c0a
resolve extraneous merge stuff
nicholsl Jul 30, 2025
b9ff43b
add functionality and tests
nicholsl Aug 1, 2025
4897e68
licencse check
nicholsl Aug 1, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/release-please.yml
Original file line number Diff line number Diff line change
Expand Up @@ -79,5 +79,12 @@ branches:
- >-
google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/Version.java
branch: 2.61.x
- bumpMinorPreMajor: true
handleGHRelease: true
releaseType: java-backport
extraFiles:
- >-
google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/Version.java
branch: 2.60.x
extraFiles:
- google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/Version.java
10 changes: 6 additions & 4 deletions .github/scripts/update_generation_config.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
set -e
set -ex
# This script should be run at the root of the repository.
# This script is used to update googleapis_commitish, gapic_generator_version,
# and libraries_bom_version in generation configuration at the time of running
Expand All @@ -15,8 +15,10 @@ set -e
function get_latest_released_version() {
local group_id=$1
local artifact_id=$2
json_content=$(curl -s "https://search.maven.org/solrsearch/select?q=g:${group_id}+AND+a:${artifact_id}&core=gav&rows=500&wt=json")
latest=$(jq -r '.response.docs[] | select(.v | test("^[0-9]+(\\.[0-9]+)*$")) | .v' <<< "${json_content}" | sort -V | tail -n 1)
group_id_url_path="$(sed 's|\.|/|g' <<< "${group_id}")"
url="https://repo1.maven.org/maven2/${group_id_url_path}/${artifact_id}/maven-metadata.xml"
xml_content=$(curl -s --fail "${url}")
latest=$(xmllint --xpath 'metadata/versioning/latest/text()' - <<< "${xml_content}")
if [[ -z "${latest}" ]]; then
echo "The latest version of ${group_id}:${artifact_id} is empty."
echo "The returned json from maven.org is invalid: ${json_content}"
Expand Down Expand Up @@ -172,4 +174,4 @@ if [ -z "${pr_num}" ]; then
else
git push
gh pr edit "${pr_num}" --title "${title}" --body "${title}"
fi
fi
22 changes: 22 additions & 0 deletions .github/sync-repo-settings.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,28 @@ branchProtectionRules:
- conformance
- library_generation
- unmanaged_dependency_check
- pattern: 2.60.x
isAdminEnforced: true
requiredApprovingReviewCount: 1
requiresCodeOwnerReviews: true
requiresStrictStatusChecks: false
requiredStatusCheckContexts:
- dependencies (17)
- lint
- clirr
- units (11)
- units (17)
- units (21)
- units (24)
- 'Kokoro - Test: Integration'
- cla/google
- 'Kokoro - Test: Java GraalVM Native Image A'
- 'Kokoro - Test: Java GraalVM Native Image B'
- 'Kokoro - Test: Java GraalVM Native Image C'
- javadoc
- conformance
- library_generation
- unmanaged_dependency_check
permissionRules:
- team: yoshi-admins
permission: admin
Expand Down
7 changes: 4 additions & 3 deletions .github/workflows/update_generation_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ on:
schedule:
- cron: '0 2 * * *'
workflow_dispatch:

jobs:
update-generation-config:
runs-on: ubuntu-24.04
Expand All @@ -30,6 +29,9 @@ jobs:
with:
fetch-depth: 0
token: ${{ secrets.CLOUD_JAVA_BOT_TOKEN }}
- name: Install Dependencies
shell: bash
run: sudo apt-get update && sudo apt-get install -y libxml2-utils
- name: Update params in generation config to latest
shell: bash
run: |
Expand All @@ -40,5 +42,4 @@ jobs:
--base_branch "${base_branch}" \
--repo ${{ github.repository }}
env:
GH_TOKEN: ${{ secrets.CLOUD_JAVA_BOT_TOKEN }}

GH_TOKEN: ${{ secrets.CLOUD_JAVA_BOT_TOKEN }}
2 changes: 1 addition & 1 deletion generation_config.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
gapic_generator_version: 2.59.0
googleapis_commitish: 63161c080e7eb3dd1d4fe76db1fad108b6ac9a27
googleapis_commitish: 33d5137a234625ed94afd630031321c31bd16143
libraries_bom_version: 26.61.0
template_excludes:
- .gitignore
Expand Down
14 changes: 14 additions & 0 deletions google-cloud-bigtable/clirr-ignored-differences.xml
Original file line number Diff line number Diff line change
Expand Up @@ -382,4 +382,18 @@
<method>*</method>
<to>*</to>
</difference>
<difference>
<!-- InternalApi was updated -->
<differenceType>7004</differenceType>
<className>com/google/cloud/bigtable/gaxx/grpc/BigtableChannelPool</className>
<method>*create*</method>
<to>*</to>
</difference>
<difference>
<!-- InternalApi was updated -->
<differenceType>7004</differenceType>
<className>com/google/cloud/bigtable/gaxx/grpc/BigtableTransportChannelProvider</className>
<method>*create*</method>
<to>*</to>
</difference>
</differences>
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
*/
package com.google.cloud.bigtable.data.v2.stub;

import com.google.api.core.BetaApi;
import com.google.api.core.InternalApi;
import com.google.api.core.SettableApiFuture;
import com.google.api.gax.grpc.ChannelPrimer;
import com.google.auth.Credentials;
Expand All @@ -41,13 +41,13 @@

/**
* A channel warmer that ensures that a Bigtable channel is ready to be used before being added to
* the active {@link com.google.api.gax.grpc.ChannelPool}.
* the active {@link com.google.cloud.bigtable.gaxx.grpc.BigtableChannelPool}.
*
* <p>This implementation is subject to change in the future, but currently it will prime the
* channel by sending a ReadRow request for a hardcoded, non-existent row key.
*/
@BetaApi("Channel priming is not currently stable and might change in the future")
class BigtableChannelPrimer implements ChannelPrimer {
@InternalApi
public class BigtableChannelPrimer implements ChannelPrimer {
private static Logger LOG = Logger.getLogger(BigtableChannelPrimer.class.toString());

static final Metadata.Key<String> REQUEST_PARAMS =
Expand Down Expand Up @@ -96,10 +96,68 @@ public void primeChannel(ManagedChannel managedChannel) {
}

private void primeChannelUnsafe(ManagedChannel managedChannel) throws IOException {
sendPrimeRequests(managedChannel);
sendPrimeRequestsBlocking(managedChannel);
}

private void sendPrimeRequests(ManagedChannel managedChannel) {
private void sendPrimeRequestsBlocking(ManagedChannel managedChannel) {
try {
sendPrimeRequestsAsync(managedChannel).get(1, TimeUnit.MINUTES);
} catch (Throwable e) {
// TODO: Not sure if we should swallow the error here. We are pre-emptively swapping
// channels if the new
// channel is bad.
LOG.log(Level.WARNING, "Failed to prime channel", e);
}
}

public SettableApiFuture<PingAndWarmResponse> sendPrimeRequestsAsync(
ManagedChannel managedChannel) {
ClientCall<PingAndWarmRequest, PingAndWarmResponse> clientCall =
managedChannel.newCall(
BigtableGrpc.getPingAndWarmMethod(),
CallOptions.DEFAULT
.withCallCredentials(callCredentials)
.withDeadline(Deadline.after(1, TimeUnit.MINUTES)));

SettableApiFuture<PingAndWarmResponse> future = SettableApiFuture.create();
clientCall.start(
new ClientCall.Listener<PingAndWarmResponse>() {
private PingAndWarmResponse response;

@Override
public void onMessage(PingAndWarmResponse message) {
response = message;
}

@Override
public void onClose(Status status, Metadata trailers) {
if (status.isOk()) {
future.set(response);
} else {
// Propagate the gRPC error to the future.
future.setException(status.asException(trailers));
}
}
},
createMetadata(headers, request));

try {
// Send the request message.
clientCall.sendMessage(request);
// Signal that no more messages will be sent.
clientCall.halfClose();
// Request the response from the server.
clientCall.request(Integer.MAX_VALUE);
} catch (Throwable t) {
// If sending fails, cancel the call and notify the future.
clientCall.cancel("Failed to send priming request", t);
future.setException(t);
}

return future;
}

private void sendPrimeRequestsNonBlocking(ManagedChannel managedChannel) {
try {
ClientCall<PingAndWarmRequest, PingAndWarmResponse> clientCall =
managedChannel.newCall(
Expand Down Expand Up @@ -131,8 +189,6 @@ public void onClose(Status status, Metadata trailers) {
clientCall.sendMessage(request);
clientCall.halfClose();
clientCall.request(Integer.MAX_VALUE);

future.get(1, TimeUnit.MINUTES);
} catch (Throwable e) {
// TODO: Not sure if we should swallow the error here. We are pre-emptively swapping
// channels if the new
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
import com.google.api.gax.core.BackgroundResource;
import com.google.api.gax.core.CredentialsProvider;
import com.google.api.gax.core.FixedCredentialsProvider;
import com.google.api.gax.grpc.ChannelPrimer;
import com.google.api.gax.grpc.InstantiatingGrpcChannelProvider;
import com.google.api.gax.rpc.ClientContext;
import com.google.auth.Credentials;
Expand Down Expand Up @@ -121,20 +122,22 @@ public static BigtableClientContext create(EnhancedBigtableStubSettings settings
setupCookieHolder(transportProvider);
}

ChannelPrimer channelPrimer = NoOpChannelPrimer.create();

// Inject channel priming if enabled
if (builder.isRefreshingChannel()) {
transportProvider.setChannelPrimer(
channelPrimer =
BigtableChannelPrimer.create(
builder.getProjectId(),
builder.getInstanceId(),
builder.getAppProfileId(),
credentials,
builder.getHeaderProvider().getHeaders()));
builder.getHeaderProvider().getHeaders());
}

BigtableTransportChannelProvider btTransportProvider =
BigtableTransportChannelProvider.create(
(InstantiatingGrpcChannelProvider) transportProvider.build());
(InstantiatingGrpcChannelProvider) transportProvider.build(), channelPrimer);

builder.setTransportChannelProvider(btTransportProvider);
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
/*
* Copyright 2025 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.google.cloud.bigtable.data.v2.stub;

import com.google.api.core.InternalApi;
import com.google.api.gax.grpc.ChannelPrimer;
import io.grpc.ManagedChannel;

@InternalApi
public class NoOpChannelPrimer implements ChannelPrimer {
static NoOpChannelPrimer create() {
return new NoOpChannelPrimer();
}

private NoOpChannelPrimer() {}

@Override
public void primeChannel(ManagedChannel managedChannel) {
// No op
}
}
Loading
Loading