Skip to content

Conversation

@mend-app-ghe
Copy link

@mend-app-ghe mend-app-ghe bot commented Sep 11, 2025

This PR contains the following updates:

Package Type Update Change
com.google.cloud.bigdataoss:gcs-connector compile patch hadoop3-2.2.28 -> 3.1.8

Warning

Some dependencies could not be looked up. Check the warning logs for more information.


Release Notes

GoogleCloudDataproc/hadoop-connectors (com.google.cloud.bigdataoss:gcs-connector)

v3.1.8: 3.1.8

Compare Source

  1. PR #​1511 - [Feature] Storage Client Caching for downscoped token
  2. Enable move by default.

v3.1.7: 3.1.7

Compare Source

  1. PR #​1484 - [Feature] Enabled write checksum by default
  2. PR #​1480 - [Bug] Fix infinite loop issue when EOF is reached during a skip operation in ReadChannel

v3.1.6: 3.1.6

Compare Source

  1. Add listStatusStartingFrom API.

v3.1.5: 3.1.5

Compare Source

  1. Fix cloud logging issue.

v3.1.4: 3.1.4

Compare Source

  1. Add checksum verification in upload.
  2. PR #​1463: [Bug] Fix for Hadoop directory copy failures in gRPC client.
  3. PR #​1420: Update readBytes via vectoredRead

v3.1.3: 3.1.3

Compare Source

  1. Skip Soft delete using move API during rename
  2. Use Apiary isHnBucket for HNS checks

v3.1.2: 3.1.2

Compare Source

  1. Upgrade to use cloud-storage libraries-bom
  2. Add AUTO_RANDOM as new fadvise mode.

v3.1.1: 3.1.1

  1. PR #​1323: Fix a bug where gRPC path was failing due to some of the dependencies not getting shaded.

v3.0.12: 3.0.12

Compare Source

  1. Enable move by default

v3.0.11: 3.0.11

Compare Source

  1. PR #​1484 - [Feature] Enabled write checksum by default
  2. PR #​1480 - [Bug] Fix infinite loop issue when EOF is reached during a skip operation in ReadChannel
  3. PR #​1463 - [Bug] Fix for Hadoop directory copy failures in gRPC client

v3.0.10: 3.0.10

Compare Source

  1. PR #​1396 - [Bug] Throw correct exception when DeleteFolderOperation is timed out
  2. PR #​1385 - [Bug] Correcting checksum calculation when received buffer is already moved
  3. PR #​1413 - [Bug] Fixing DeleteFolder stalling issue where high CPU usage can cause timeouts

v3.0.9: 3.0.9

Compare Source

  1. Add checksum verification in upload

v3.0.8: 3.0.8

Compare Source

  1. Skip Soft delete using move API during rename
  2. Use Apiary isHnBucket for HNS checks

v3.0.7: 3.0.7

Compare Source

  1. Optimize channel creation in vectoredRead
  2. Update GhfsThreadLocalStats with read vectored metrices

v3.0.6: 3.0.6

Compare Source

  1. Add AUTO_RANDOM as new fadvise mode.

v3.0.5: 3.0.5

Compare Source

  1. PR #​1274 - Upgrade google-cloud-storage to 2.44.1
  2. Downgrade grpc to 1.67.1

v3.0.4: 3.0.4

Compare Source

  1. PR #​1268 - [Bug] Fix a failure in gRPC path due to a dependency issue

v3.0.3: 3.0.3

Compare Source

  1. PR #​1244 - Add open() API which takes FileStatus as parameter
  2. Upgrade google-cloud-storage to 2.43.1
  3. Upgrade grpc to 1.68.0
  4. PR #​1253 - Add getFileStatusWithHint() API

v3.0.2: 3.0.2

Compare Source

  1. PR #​1230 - Avoid registering subscriber class multiple times
  2. PR #​1228 - Add backoff, retry and other metrics for GCS JSON API

v3.0.1: 3.0.1

Compare Source

  1. Add readVectored API implementation.
  2. Add gRPC configuration documentation.

v3.0.0: 3.0.0

  1. Remove Hadoop 2.x support.
  2. Update all dependencies to the latest versions.
  3. Add support for downscoped tokens in AccessTokenProvider.
  4. Implement FileSystem.openFile to take advantage of the FileStatus if
    passed.
  5. Remove an obsolete AuthorizationHandler and related properties:
    fs.gs.authorization.handler.impl
    fs.gs.authorization.handler.properties.<AUTHORIZATION_HANDLER_PROPERTY>
    
  6. Remove support for Apache HTTP transport and related property:
    fs.gs.http.transport.type
    
  7. Support GCS fine-grained action in AuthorizationHandlers.
  8. Decrease log level for hflush rate limit log message.
  9. Remove Cooperative Locking support for directory operations and related
    properties:
    fs.gs.cooperative.locking.enable
    fs.gs.cooperative.locking.expiration.timeout.ms
    fs.gs.cooperative.locking.max.concurrent.operations
    
  10. Migrate authentication to com.google.auth.Credentials and remove obsolete
    properties:
    fs.gs.auth.service.account.email
    fs.gs.auth.service.account.keyfile
    fs.gs.auth.service.account.private.key
    fs.gs.auth.service.account.private.key.id
    
  11. Refactor authentication configuration to use an explicit fs.gs.auth.type
    enum property, instead of relying on inference of the authentication type
    based on the set configuration properties, and remove obsolete properties:
    fs.gs.auth.null.enable
    fs.gs.auth.service.account.enable
    
  12. Add support for a new USER_CREDENTIALS authentication type that retrieves
    a refresh token using the authorisation code grant flow configured via the
    following properties:
    fs.gs.auth.client.id
    fs.gs.auth.client.secret
    fs.gs.auth.refresh.token
    
  13. Merge all output stream types functionality in the default output stream
    that behaves similarly to the FLUSHABLE_COMPOSITE stream, and remove
    obsolete fs.gs.outputstream.type property.
  14. Set default value for fs.gs.list.max.items.per.call property to 5000.
  15. Set socket read timeout (fs.gs.http.read-timeout) as early as possible on
    new sockets returned from the custom SSLSocketFactory. This guarantees the
    timeout is enforced during TLS handshakes when using Conscrypt as the
    security provider.
  16. The Google Cloud Storage Connector now can be used as a
    Hadoop Credential Provider.
  17. Added dependency on the Cloud Storage Client Library
    (google-cloud-storage).
  18. Rename fs.gs.rewrite.max.bytes.per.call property to
    fs.gs.rewrite.max.chunk.size.
  19. Remove support of the deprecated fs.gs.io.buffersize.write property.
  20. Add support for size suffixes (k, m, g, etc) in values of size-related
    properties:
    fs.gs.inputstream.inplace.seek.limit
    fs.gs.inputstream.min.range.request.size
    fs.gs.outputstream.buffer.size
    fs.gs.outputstream.pipe.buffer.size
    fs.gs.outputstream.upload.cache.size
    fs.gs.outputstream.upload.chunk.size
    fs.gs.rewrite.max.chunk.size
    
  21. Remove .ms suffix from names and add support for time suffixes (ms, s,
    m, etc) in values of time-related properties:
    fs.gs.http.connect-timeout
    fs.gs.http.read-timeout
    fs.gs.max.wait.for.empty.object.creation
    fs.gs.outputstream.sync.min.interval
    fs.gs.performance.cache.max.entry.age
    
  22. Change default values of properties:
    fs.gs.http.connect-timeout (default: 20s -> 5s)
    fs.gs.http.read-timeout (default: 20s -> 5s)
    fs.gs.outputstream.upload.chunk.size (default: 64m -> 24m)
    
  23. Upgrade Hadoop to 3.3.5.
  24. Upgrade java-storage to 2.25.0
  25. Add support for WORKLOAD_IDENTITY_FEDERATION_CREDENTIAL_CONFIG_FILE authentication type that retrieves a refresh token using workload identity federation configuraiton defined in: fs.gs.auth.workload.identity.federation.credential.config.file

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.


  • If you want to rebase/retry this PR, check this box

@github-actions github-actions bot added the BUILD label Sep 11, 2025
@mend-app-ghe mend-app-ghe bot force-pushed the whitesource-remediate/gcs-connector.version branch from de7ba3f to 06b0082 Compare September 26, 2025 17:06
@mend-app-ghe mend-app-ghe bot changed the title Update dependency com.google.cloud.bigdataoss:gcs-connector to v3.1.7 Update dependency com.google.cloud.bigdataoss:gcs-connector to v3.1.8 Sep 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants