Skip to content

chore(x-goog-spanner-request-id): setup expectations for end-to-end request-id checks #3932

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 29, 2025
Merged

chore(x-goog-spanner-request-id): setup expectations for end-to-end request-id checks #3932

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
bf94ef7
chore(x-goog-spanner-request-id): setup expectations for end-to-end r…
odeke-em Jun 29, 2025
ffcf3f5
Suffix checks
odeke-em Jul 8, 2025
abb5f5d
Fix up slicing in comparison checks
odeke-em Jul 8, 2025
aa2cf65
Merge branch 'main' into x-goog-spanner-request-id-setup-e2e-expectat…
odeke-em Jul 16, 2025
e33cf68
Merge branch 'main' into x-goog-spanner-request-id-setup-e2e-expectat…
odeke-em Jul 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion google-cloud-spanner/src/main/java/com/google/cloud/spanner/DatabaseClientImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ class DatabaseClientImpl implements DatabaseClient {
@VisibleForTesting final MultiplexedSessionDatabaseClient multiplexedSessionDatabaseClient;
@VisibleForTesting final boolean useMultiplexedSessionPartitionedOps;
@VisibleForTesting final boolean useMultiplexedSessionForRW;
private final int dbId;
@VisibleForTesting final int dbId;
private final AtomicInteger nthRequest;
private final Map<String, Integer> clientIdToOrdinalMap;

Expand Down
3 changes: 0 additions & 3 deletions google-cloud-spanner/src/main/java/com/google/cloud/spanner/Options.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1098,9 +1098,6 @@ public boolean equals(Object o) {
return false;
}
RequestIdOption other = (RequestIdOption) o;
if (this.reqId == null || other.reqId == null) {
return this.reqId == null && other.reqId == null;
}
return Objects.equals(this.reqId, other.reqId);
}
}
Expand Down
4 changes: 2 additions & 2 deletions google-cloud-spanner/src/main/java/com/google/cloud/spanner/SessionClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ DatabaseId getDatabaseId() {
@Override
public XGoogSpannerRequestId nextRequestId(long channelId, int attempt) {
return XGoogSpannerRequestId.of(
this.nthId, this.nthRequest.incrementAndGet(), channelId, attempt);
this.nthId, channelId, this.nthRequest.incrementAndGet(), attempt);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that we can make this change without any tests failing (or needing to be changed) seems to indicate that we have a test gap. Will that be covered by the tests that are being added in this PR, but that are currently not verifying anything, as all verifications are in if (false) blocks? If not, can we add tests that cover this in a follow-up PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@olavloite I had made a much larger PR #3915 in which tests were all updated but deemed too large to get in at once hence I carved this one out. Indeed @tharoldD is sending a follow-up PR to remove all the false conditions and then more tests too.

}

/** Create a single session. */
Expand Down Expand Up @@ -423,7 +423,7 @@ private List<SessionImpl> internalBatchCreateSessions(
span.addAnnotation(String.format("Requesting %d sessions", sessionCount));
try (IScope s = spanner.getTracer().withSpan(span)) {
XGoogSpannerRequestId reqId =
XGoogSpannerRequestId.of(this.nthId, this.nthRequest.incrementAndGet(), channelHint, 1);
XGoogSpannerRequestId.of(this.nthId, channelHint, this.nthRequest.incrementAndGet(), 1);
List<com.google.spanner.v1.Session> sessions =
spanner
.getRpc()
Expand Down
1 change: 1 addition & 0 deletions google-cloud-spanner/src/main/java/com/google/cloud/spanner/spi/v1/GapicSpannerRpc.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2042,6 +2042,7 @@ <ReqT, RespT> GrpcCallContext newCallContext(
}
}
if (options != null) {
// TODO(@odeke-em): Infer the affinity if it doesn't match up with in the request-id.
context = withRequestId(context, options);
}
context = context.withExtraHeaders(metadataProvider.newExtraHeaders(resource, projectName));
Expand Down
83 changes: 83 additions & 0 deletions google-cloud-spanner/src/test/java/com/google/cloud/spanner/DatabaseClientImplTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2907,6 +2907,37 @@ public void testPartitionedDmlDoesNotTimeout() {
return null;
}));
assertEquals(ErrorCode.DEADLINE_EXCEEDED, e.getErrorCode());

DatabaseClientImpl dbImpl = ((DatabaseClientImpl) client);
int channelId = 0;
try (Session session = dbImpl.getSession()) {
channelId = ((PooledSessionFuture) session).getChannel();
}
int dbId = dbImpl.dbId;
long NON_DETERMINISTIC = XGoogSpannerRequestIdTest.NON_DETERMINISTIC;
XGoogSpannerRequestIdTest.MethodAndRequestId[] wantStreamingValues = {
XGoogSpannerRequestIdTest.ofMethodAndRequestId(
"google.spanner.v1.Spanner/ExecuteStreamingSql",
new XGoogSpannerRequestId(NON_DETERMINISTIC, channelId, 6, 1)),
};
if (false) { // TODO(@odeke-em): enable in next PRs.
xGoogReqIdInterceptor.checkExpectedStreamingXGoogRequestIds(wantStreamingValues);
}

XGoogSpannerRequestIdTest.MethodAndRequestId[] wantUnaryValues = {
XGoogSpannerRequestIdTest.ofMethodAndRequestId(
"google.spanner.v1.Spanner/BeginTransaction",
new XGoogSpannerRequestId(NON_DETERMINISTIC, channelId, 7, 1)),
XGoogSpannerRequestIdTest.ofMethodAndRequestId(
"google.spanner.v1.Spanner/CreateSession",
new XGoogSpannerRequestId(NON_DETERMINISTIC, 0, 1, 1)),
XGoogSpannerRequestIdTest.ofMethodAndRequestId(
"google.spanner.v1.Spanner/ExecuteSql",
new XGoogSpannerRequestId(NON_DETERMINISTIC, channelId, 8, 1)),
};
if (false) { // TODO(@odeke-em): enable in next PRs.
xGoogReqIdInterceptor.checkExpectedUnaryXGoogRequestIdsAsSuffixes(wantUnaryValues);
}
}
}

Expand Down Expand Up @@ -2989,6 +3020,38 @@ public void testPartitionedDmlWithHigherTimeout() {
.run(transaction -> transaction.executeUpdate(UPDATE_STATEMENT)));
assertThat(e.getErrorCode()).isEqualTo(ErrorCode.DEADLINE_EXCEEDED);
assertThat(updateCount).isEqualTo(UPDATE_COUNT);

DatabaseClientImpl dbImpl = ((DatabaseClientImpl) client);
int channelId = 0;
try (Session session = dbImpl.getSession()) {
channelId = ((PooledSessionFuture) session).getChannel();
}
int dbId = dbImpl.dbId;
long NON_DETERMINISTIC = XGoogSpannerRequestIdTest.NON_DETERMINISTIC;
XGoogSpannerRequestIdTest.MethodAndRequestId[] wantStreamingValues = {
XGoogSpannerRequestIdTest.ofMethodAndRequestId(
"google.spanner.v1.Spanner/ExecuteStreamingSql",
new XGoogSpannerRequestId(NON_DETERMINISTIC, channelId, 6, 1)),
};

if (false) { // TODO(@odeke-em): enable in next PRs.
xGoogReqIdInterceptor.checkExpectedStreamingXGoogRequestIds(wantStreamingValues);
}

XGoogSpannerRequestIdTest.MethodAndRequestId[] wantUnaryValues = {
XGoogSpannerRequestIdTest.ofMethodAndRequestId(
"google.spanner.v1.Spanner/BeginTransaction",
new XGoogSpannerRequestId(NON_DETERMINISTIC, channelId, 7, 1)),
XGoogSpannerRequestIdTest.ofMethodAndRequestId(
"google.spanner.v1.Spanner/CreateSession",
new XGoogSpannerRequestId(NON_DETERMINISTIC, 0, 1, 1)),
XGoogSpannerRequestIdTest.ofMethodAndRequestId(
"google.spanner.v1.Spanner/ExecuteSql",
new XGoogSpannerRequestId(NON_DETERMINISTIC, channelId, 8, 1)),
};
if (false) { // TODO(@odeke-em): enable in next PRs.
xGoogReqIdInterceptor.checkExpectedUnaryXGoogRequestIdsAsSuffixes(wantUnaryValues);
}
}
}

Expand Down Expand Up @@ -5314,6 +5377,26 @@ public void testSessionPoolExhaustedError_containsStackTraces() {
}
// Closing the transactions should return the sessions to the pool.
assertEquals(4, pool.getNumberOfSessionsInPool());

DatabaseClientImpl dbClient = (DatabaseClientImpl) client;
int channelId = 0;
try (Session session = dbClient.getSession()) {
channelId = ((PooledSessionFuture) session).getChannel();
}
int dbId = dbClient.dbId;
XGoogSpannerRequestIdTest.MethodAndRequestId[] wantStreamingValues = {};

xGoogReqIdInterceptor.checkExpectedStreamingXGoogRequestIds(wantStreamingValues);
long NON_DETERMINISTIC = XGoogSpannerRequestIdTest.NON_DETERMINISTIC;

XGoogSpannerRequestIdTest.MethodAndRequestId[] wantUnaryValues = {
XGoogSpannerRequestIdTest.ofMethodAndRequestId(
"google.spanner.v1.Spanner/CreateSession",
new XGoogSpannerRequestId(NON_DETERMINISTIC, 0, 1, 1)),
};
if (false) { // TODO(@odeke-em): enable in next PRs.
xGoogReqIdInterceptor.checkExpectedUnaryXGoogRequestIdsAsSuffixes(wantUnaryValues);
}
}
}

Expand Down
166 changes: 144 additions & 22 deletions google-cloud-spanner/src/test/java/com/google/cloud/spanner/XGoogSpannerRequestIdTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@
import io.grpc.ServerInterceptor;
import io.grpc.Status;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.Objects;
Expand All @@ -41,6 +43,7 @@

@RunWith(JUnit4.class)
public class XGoogSpannerRequestIdTest {
public static long NON_DETERMINISTIC = -1;

@Test
public void testEquals() {
Expand Down Expand Up @@ -157,40 +160,83 @@ private void assertMonotonicityOfIds(String prefix, List<XGoogSpannerRequestId>
+ String.join("\n\t", violations.toArray(new String[0])));
}

public static class methodAndRequestId {
String method;
String requestId;

public methodAndRequestId(String method, String requestId) {
this.method = method;
this.requestId = requestId;
}

public String toString() {
return "{" + this.method + ":" + this.requestId + "}";
}
}

public methodAndRequestId[] accumulatedUnaryValues() {
List<methodAndRequestId> accumulated = new ArrayList();
public MethodAndRequestId[] accumulatedUnaryValues() {
List<MethodAndRequestId> accumulated = new ArrayList();
this.unaryResults.forEach(
(String method, CopyOnWriteArrayList<XGoogSpannerRequestId> values) -> {
for (int i = 0; i < values.size(); i++) {
accumulated.add(new methodAndRequestId(method, values.get(i).toString()));
accumulated.add(new MethodAndRequestId(method, values.get(i)));
}
});
return accumulated.toArray(new methodAndRequestId[0]);
return accumulated.toArray(new MethodAndRequestId[0]);
}

public methodAndRequestId[] accumulatedStreamingValues() {
List<methodAndRequestId> accumulated = new ArrayList();
public MethodAndRequestId[] accumulatedStreamingValues() {
List<MethodAndRequestId> accumulated = new ArrayList();
this.streamingResults.forEach(
(String method, CopyOnWriteArrayList<XGoogSpannerRequestId> values) -> {
for (int i = 0; i < values.size(); i++) {
accumulated.add(new methodAndRequestId(method, values.get(i).toString()));
accumulated.add(new MethodAndRequestId(method, values.get(i)));
}
});
return accumulated.toArray(new methodAndRequestId[0]);
return accumulated.toArray(new MethodAndRequestId[0]);
}

public void checkExpectedUnaryXGoogRequestIds(MethodAndRequestId... wantUnaryValues) {
MethodAndRequestId[] gotUnaryValues = this.accumulatedUnaryValues();
sortValues(gotUnaryValues);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it correct that we are sorting these accumulated values here? Doesn't that mean that if we receive the request-ids out of the expected order on the (mock) server, this sort action hides any potential test failures due to the request-ids having been out of order?

for (int i = 0; i < gotUnaryValues.length && false; i++) {
System.out.println("\033[33misUnary: #" + i + ":: " + gotUnaryValues[i] + "\033[00m");
}
Comment on lines +188 to +190
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that this is not being printed due to the && false condition in the loop, but can we add a TODO that this should be removed in a follow-up PR.

assertEquals(wantUnaryValues, gotUnaryValues);
}

public void checkAtLeastHasExpectedUnaryXGoogRequestIds(MethodAndRequestId... wantUnaryValues) {
MethodAndRequestId[] gotUnaryValues = this.accumulatedUnaryValues();
sortValues(gotUnaryValues);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here regarding sorting the accumulated values as above.

for (int i = 0; i < gotUnaryValues.length && false; i++) {
System.out.println("\033[33misUnary: #" + i + ":: " + gotUnaryValues[i] + "\033[00m");
}
if (wantUnaryValues.length < gotUnaryValues.length) {
MethodAndRequestId[] gotSliced =
Arrays.copyOfRange(gotUnaryValues, 0, wantUnaryValues.length);
assertEquals(wantUnaryValues, gotSliced);
} else {
assertEquals(wantUnaryValues, gotUnaryValues);
}
}

public void checkExpectedUnaryXGoogRequestIdsAsSuffixes(MethodAndRequestId... wantUnaryValues) {
MethodAndRequestId[] gotUnaryValues = this.accumulatedUnaryValues();
sortValues(gotUnaryValues);
for (int i = 0; i < gotUnaryValues.length && false; i++) {
System.out.println("\033[33misUnary: #" + i + ":: " + gotUnaryValues[i] + "\033[00m");
}
if (wantUnaryValues.length < gotUnaryValues.length) {
MethodAndRequestId[] gotSliced =
Arrays.copyOfRange(
gotUnaryValues,
gotUnaryValues.length - wantUnaryValues.length,
gotUnaryValues.length);
assertEquals(wantUnaryValues, gotSliced);
} else {
assertEquals(wantUnaryValues, gotUnaryValues);
}
}

private void sortValues(MethodAndRequestId[] values) {
massageValues(values);
Arrays.sort(values, new MethodAndRequestIdComparator());
}

public void checkExpectedStreamingXGoogRequestIds(MethodAndRequestId... wantStreamingValues) {
MethodAndRequestId[] gotStreamingValues = this.accumulatedStreamingValues();
for (int i = 0; i < gotStreamingValues.length && false; i++) {
System.out.println(
"\033[32misStreaming: #" + i + ":: " + gotStreamingValues[i] + "\033[00m");
}
sortValues(gotStreamingValues);
assertEquals(wantStreamingValues, gotStreamingValues);
}

public void reset() {
Expand All @@ -199,4 +245,80 @@ public void reset() {
this.streamingResults.clear();
}
}

public static class MethodAndRequestId {
String method;
XGoogSpannerRequestId requestId;

public MethodAndRequestId(String method, XGoogSpannerRequestId requestId) {
this.method = method;
this.requestId = requestId;
}

public String toString() {
return "{" + this.method + ":" + this.requestId.debugToString() + "}";
}

@Override
public boolean equals(Object o) {
if (!(o instanceof MethodAndRequestId)) {
return false;
}
MethodAndRequestId other = (MethodAndRequestId) o;
return Objects.equals(this.method, other.method)
&& Objects.equals(this.requestId, other.requestId);
}
}

static class MethodAndRequestIdComparator implements Comparator<MethodAndRequestId> {
@Override
public int compare(MethodAndRequestId mr1, MethodAndRequestId mr2) {
int cmpMethod = mr1.method.compareTo(mr2.method);
if (cmpMethod != 0) {
return cmpMethod;
}

if (Objects.equals(mr1.requestId, mr2.requestId)) {
return 0;
}
if (mr1.requestId.isGreaterThan(mr2.requestId)) {
return +1;
}
return -1;
}
}

static void massageValues(MethodAndRequestId[] mreqs) {
for (int i = 0; i < mreqs.length; i++) {
MethodAndRequestId mreq = mreqs[i];
// BatchCreateSessions is so hard to control as the round-robin doling out
// hence we might need to be able to scrub the nth_request that won't match
// nth_req in consecutive order of nth_client.
if (mreq.method.compareTo("google.spanner.v1.Spanner/BatchCreateSessions") == 0) {
mreqs[i] =
new MethodAndRequestId(
mreq.method,
mreq.requestId
.withNthRequest(NON_DETERMINISTIC)
.withChannelId(NON_DETERMINISTIC)
.withNthClientId(NON_DETERMINISTIC));
} else if (mreq.method.compareTo("google.spanner.v1.Spanner/BeginTransaction") == 0
|| mreq.method.compareTo("google.spanner.v1.Spanner/ExecuteStreamingSql") == 0
|| mreq.method.compareTo("google.spanner.v1.Spanner/ExecuteSql") == 0
|| mreq.method.compareTo("google.spanner.v1.Spanner/CreateSession") == 0
|| mreq.method.compareTo("google.spanner.v1.Spanner/Commit") == 0) {
mreqs[i] =
new MethodAndRequestId(mreq.method, mreq.requestId.withNthClientId(NON_DETERMINISTIC));
}
}
}

public static MethodAndRequestId ofMethodAndRequestId(String method, String reqId) {
return new MethodAndRequestId(method, XGoogSpannerRequestId.of(reqId));
}

public static MethodAndRequestId ofMethodAndRequestId(
String method, XGoogSpannerRequestId reqId) {
return new MethodAndRequestId(method, reqId);
}
}
Loading