[Bug]: FirestoreV1 Beam connectors allow configuring inconsistent project/database IDs between RPC requests and routing headers

[pacoavila808]

What happened?

There are 3 places that can currently set Firestore project ID (and 2 for database ID), and they are not all used consistently.

GcpOptions.projectID is used for the routing header:

beam/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/firestore/FirestoreStatefulComponentFactory.java

Line 100 in c8d7ca0

+ gcpOptions.getProject()

FirestoreOptions.projectId used for RPC requests, but not routing header. 4df89c7

Database configured in WriteFn used for RPC requests if set (but not used in routing header).

beam/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/firestore/FirestoreV1WriteFn.java

Line 376 in c8d7ca0

BatchWriteRequest.newBuilder().setDatabase(databaseRootName.toString());

Fix needed here is to ensure that the precedence of WriteFn -> FirestoreOptions -> GcpOptions is respected for the routing header.

Can be reproduced by running the integration test with -PfirestoreDb set to anything other than "firestoredb", which is the value hard coded into the BaseFirestoreIT.

./gradlew integrationTest -p sdks/java/io/google-cloud-platform -PgcpProject=<project_id> -PfirestoreDb="(default)" --tests=org.apache.beam.sdk.io.gcp.firestore.it.FirestoreV1IT

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[pacoavila808]

.take-issue