Skip to content

Commit 7595793

Browse files
ddelgrosso1coryan
andauthored
feat(storage): Add bidiread and bidi appendable write to the async client. (#15078)
* feat(storage): Add BidiRead and BidiAppendableWrite to Async client This PR introduces all the Terraform files to create and maintains the resources for GCB builds. This includes a few buckets, the connection between GCB and GitHub, and the first two triggers. ci(gcb): prepare for upstream changes (#4) Upstream the `cloudbuild.yaml` file is becoming more configurable, but we need to set more substitution variables. ci(gcb): enable the checkers build (#6) Enable the `checkers` build. This is where we detect typos and formatting errors, which of course I had introduced while the build was disabled. Also made some changes to how the triggers are created. This will be handy when we add the next dozen builds or so. ci(gcb): only compile storage (#8) For ACv2 development we only need to compile the storage libraries and a few dependencies. This saves hours of CPU time per build, and simplifies the configuration for integration tests. refactor(gcb): move cloudbuild resource definitions (#7) The top-level `main.tf` was getting too bulky and I may want to create additional resources for integration tests. ci: disable public access to logs (#10) ci(gcb): enable asan build (#12) The AddressSanitizer build is the first Bazel-based build, and one of the best ways to find many types of "memory unsafe" errors. ci: disable tests against production (#15) The builds in the `pre-launch-acv2` branch are not working. I think they passed before because some script error masked the problem. In any case, the production environment is not ready to run all the integration tests. This change disables those tests, and restores the builds to a passing state. ci: create artifact registry (#14) We moved the builds to AR upstream, this creates the necessary AR repository in the build project for this repo. chore: merge from upstream (#13) Co-authored-by: Darren Bolduc <[email protected]> Co-authored-by: Anna Levenberg <[email protected]> Co-authored-by: Scott Hart <[email protected]> Co-authored-by: Bradley White <[email protected]> Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Anna Levenberg <[email protected]> chore: merge from public repository c.2024-06-13 (#23) Co-authored-by: Darren Bolduc <[email protected]> Co-authored-by: Scott Hart <[email protected]> Co-authored-by: Anna Levenberg <[email protected]> Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Neha Bajaj <[email protected]> Co-authored-by: jsrinnn <[email protected]> Co-authored-by: Marcel <[email protected]> impl(ACv2): interfaces for object descriptors (#21) This introduces the `ObjectDescriptorConnection` interface, and the member functions that would create them. This version has no implementation, it is just intended to unblock development. impl(ACv2): helper class for range state (#30) Part of the work for #20 impl(ACv2): helper function to open a descriptor (#29) impl(ACv2): manage open streams (#31) feat(ACv2): adapt `ReadRange` to use as `AsyncReader` (#33) We will want to use `ReadRange` instances as the underlying implementation of `storage_experimental::AsyncReader`. This PR introduces and adaptor between the two classes. impl(ACv2): object descriptor implementation (#37) Introduce the implementation for the `ObjectDescriptorConnection` interface, and some unit tests for this implementation. impl(ACv2): implement `AsyncConnection::Open` (#38) impl(ACv2): the surface `ObjectDescriptor` (#41) `ObjectDescriptor` implements the API we want external customers to use. It is implemented in terms of `ObjectDescriptorConnection`, which provides an API that can be mocked (no overloads, single parameter that can grow without breaking mocks). feat(ACv2): implement `AsyncClient::Open()` (#43) Finally, we can implement `AsyncClient::Open()`. This includes a unit test and a simple example using C++20 coroutines. impl(ACv2): handle redirect errors (#42) Connections closed with a `BidiReadObjectRedirectError` in the error details include information for the routing token. This can be used to speed up the reconnect. docs: howto guide for prelaunch repository (#39) impl(ACv2): `OpenObject()` performs the first read (#47) `OpenObject()` needs to perform the first `Read()` call or we may run into infinite retry / resume loops. This function used to create a streaming RPC, call `Start()`, and then call `Write()` to send the initial request. On errors it would call `Finish()`. This is not enough to detect if the service accepted the request. A successful `Write()` only indicates that the request was **sent**, we need to wait for the first `Read()` response to determine if the service accepted the request. refactor(ACv2): split functions to handle redirects (#51) ci: add `msan` build (#53) ci: add *san builds (#54) ci: add cxx20 and cxx14 builds (#55) impl(ACv2): handle redirects during startup (#52) fix(ACv2): avoid duplicate Finish() calls (#58) ci: add `noex` build (#60) This build compiles with exceptions disabled. Google happens to use C++ without exceptions. AFAICT, no other customer does, but them are the breaks. impl(ACv2): handle partial read range errors (#56) ci: add `libcxx` build (#61) libc++ is used inside Google, and by Apple. It is useful to test with it as it sometimes has surprinsingly different behavior. feat(ACv2): implement tracing decorator for ObjectDescriptorConnection (#46) chore: merge from public circa 2024-07-26 (#63) Co-authored-by: Denis DelGrosso <[email protected]> Co-authored-by: Scott Hart <[email protected]> Co-authored-by: Darren Bolduc <[email protected]> Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Siarhei Meilakh <[email protected]> Co-authored-by: jsrinnn <[email protected]> Co-authored-by: Yao Cui <[email protected]> Co-authored-by: Varun Naik <[email protected]> docs: fix prelaunch update document (#65) fix(ACv2): ifdef additional otel inclusion (#66) chore: merge from public circa 2024-08-27 (#68) * cleanup(storage): move test protos (#14508) * cleanup(generator): fewer query params already in body (#14560) * doc(bigquerycontrol): add job query sample (#14580) * cleanup(spanner): move test protos (#14586) * cleanup(bigtable): move test protos (#14589) * chore(deps): update googletest to v1.15.2 (#14590) * impl!: promote experimental LRO Start/Await methods to GA (#14588) * chore(deps): update protobuf to v27.3 (#14591) * chore(deps): update dependency protobuf to v27.3 (#14594) * cleanup(generator): parse api version from url pattern (#14595) * cleanup(generator): parse api version from url pattern * format * add tests * format * make regex string static * format * ci: reblance windows shards (#14593) * cleanup(generator): make static variable trivially destructible (#14599) * refactor(generator): prepare for dynamic query params (#14596) * chore: update vcpkg to 2024.07.12 (#14598) * docs(release): update changelog for the 2024-08 release (#14601) * chore: version bump to 2.28.0-rc (#14602) * chore(deps): update dependency googletest to v1.15.2 (#14600) * chore: update release notes (#14605) * chore: consolidate renovate-bot bzlmod PRs (#14606) * chore(deps): update dependency google_cloud_cpp to v2.27.0 (#14603) * cleanup: missing services script bzlmod (#14608) * docs: update mock LRO tips (#14609) * chore: update googleapis SHA circa 2024-08-01 (#14607) PiperOrigin-RevId: 658521163 * cleanup: regenerate libraries (#14610) * chore: skip absl types in check-api (#14613) * chore(deps): update abseil to v20240722 (#14533) * fix(otel): avoid infinite trace export loop (#14612) * fix(rest): prevent libcurl callback from reading bad address (#14615) * cleanup(generator): unused code (#14616) * chore: update googleapis SHA circa 2024-08-06 (#14619) * chore: update googleapis SHA circa 2024-08-06 PiperOrigin-RevId: 659991155 * impl(rest): set ReadFunctionAbort via RAII (#14617) * chore: update universe_domain demos WORKSPACE.bazel (#14621) * feat(grpc): add optional lb locality to otel metrics (#14624) * docs(release): update changelog for the 2024-08 release (#14628) * chore: version bump to 2.29.0-rc (#14629) * cleanup: save a manual vcpkg step (#14630) * ci: fix trigger to run on CI (#14632) * chore: update patch release process (#14627) * chore: update patch release process * how to make tag * fix: quickstarts build with bazel (#14633) * refactor(bigtable): no need for inline (#14631) * chore(compute): update_discovery_doc.sh now edits generator_config (#14623) * chore(compute): update discovery doc circa 20240805 (#14637) * chore(deps): update dependency google_cloud_cpp to v2.28.0 (#14636) * cleanup: missing links in conan docs (#14638) * chore(deps): update dependency build_bazel_rules_apple to v3.8.0 (#14635) * cleanup(compute): sort service_dirs (#14641) * fix: no need to link gmock_main in mocks (#14640) * cleanup(otel): move test, and guard it (#14642) * chore(deps): update dependency bazel to v7.3.0 (#14634) * doc(adr): googleapis SHA update policy (#14639) * doc(adr): googleapis SHA update policy * reworded per comments; changed status to accepted * ci: deflake policysimulator quickstart (#14644) * feat(compute): add missing services instant_snapshots and region_instant_snapshots (#14647) * doc: add googleapis update step for release (#14618) * chore(deps): update benchmark to v1.9.0 (#14648) * chore(deps): update dependency rules_python to v0.35.0 (#14649) * fix: do not persist the keys loaded from PKCS#12 on Windows (#14645) Do not persist the keys loaded from PKCS#12. Instead of getting the handle with `CryptAcquireCertificatePrivateKey`, we get it from a property of the certificate context. * fix(bigtable): sanitize RowRange proto input (#14651) * chore(deps): update gRPC to 1.65.5 (#14652) * chore(deps): update dependency bazel to v7.3.1 (#14653) * chore: update googleapis SHA circa 2024-08-22 (#14661) * chore: update googleapis SHA circa 2024-08-22 PiperOrigin-RevId: 666369744 * Update the protodeps/protolists * Regenerate libraries * fix(backupdr): include logging protos with cmake (#14662) * feat(gkeconnect): generate library (#14663) * feat(gkeconnect): generate library * Run generators and format their outputs * Add API baseline * Manually update READMEs, quickstart, and top-level stuff * fix * format * try add MODULE.bazel * regenerate deps * cleanup: remove manual step from scaffolding (#14665) * cleanup(generator): generalize return type strings (#14657) * cleanup(generator): generalize return type strings * add comment * fix typo * format * change * only generalize non-lro return type * remove unused * add test case * chore: update googleapis SHA circa 2024-08-25 (#14667) * chore: update googleapis SHA circa 2024-08-25 PiperOrigin-RevId: 666935281 * cleanup(quickstart): gkeconnect quickstart use correct inputs (#14666) * cleanup(quickstart): gkeconnect quickstart use correct inputs * fix * fix * format * cleanup(generator): generalize return type strings followup (#14669) * cleanup(generator): generalize return type strings followup * remove unused paramenters * remove useless variable * chore(compute): update discovery doc circa 20240813 (#14668) * chore: update typos to `1.24.1` (#14671) * chore(deps): update grpc (#14585) * chore(deps): update protobuf to v27.4 (#14672) * add bidi patch to module.bazel * checkers fix --------- Co-authored-by: Carlos O'Ryan <[email protected]> Co-authored-by: Darren Bolduc <[email protected]> Co-authored-by: Scott Hart <[email protected]> Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Yao Cui <[email protected]> Co-authored-by: Theodore Tsirpanis <[email protected]> impl(ACv2): Add options to ObjectDescriptorConnection (#69) impl(ACv2): Implement a tracing decorator for the Read operation in ReadRange (#70) chore: merge from public circa 2024-10-02 (#72) * feat: add utilities to detect gcp (#14655) * feat: add utilities to detect gcp spacing address feedback wip add windows header fix header do not initialize variable from getenv do not initialize variable from getenv use DWORD typings cast fix win32 test split logic across separate files use public / private methods fixes add additional windows tests * run checkers * simplify number of classes * win32 fixes * tidy fixes * remove stdlib.h * do not search root namespace * address feedback * use move on config * return statusor, remove unused headers, move declaration * fix status code * fix namespaces * use make_status helpers, log failures * fix win32 build * checkers * cleanup(generator): check extension existence in IsGRPCLongrunningOperation (#14675) * ci: reshard macOS builds (#14681) * feat(spanner): support instance edition (#14678) * chore: update googleapis SHA circa 2024-08-30 (#14680) * chore: update googleapis SHA circa 2024-08-30 PiperOrigin-RevId: 669375999 * docs(release): update changelog for the 2024-09 release (#14685) * chore: version bump to 2.30.0-rc (#14686) * chore(deps): update dependency google_cloud_cpp to v2.29.0 (#14687) * doc: update code of conduct POC (#14689) * chore: allow commit SHA override in renovate.sh (#14682) * chore(deps): update dependency bazelbuild/bazelisk to v1.21.0 (#14690) * fix: respect `GOOGLE_CLOUD_QUOTA_PROJECT` (#14684) * feat(otel): release GCM exporter (#14693) * ci: remove vc toolset 14.40 workaround (#14694) * chore(deps): update gcr.io/kaniko-project/executor docker tag to v1.23.2 (#14673) * ci: additional macos sharding (#14695) * feat(generativelanguage): generate library (#14698) * chore(deps): update protobuf to v28 (#14466) * chore(deps): update dependency com_google_protobuf to v28.1 (#14697) * chore(deps): update dependency protocolbuffers/protobuf to v28.1 (#14700) * feat(storage): Utilize gcp check to default to direct path (#14676) * feat: add utilities to detect gcp spacing address feedback wip add windows header fix header do not initialize variable from getenv do not initialize variable from getenv use DWORD typings cast fix win32 test split logic across separate files use public / private methods fixes add additional windows tests * run checkers * simplify number of classes * tidy fixes * address feedback * use move on config * return statusor, remove unused headers, move declaration * feat(storage): default to directpath if gcp can be detected * fix const * spacing * comment wording * remove detector options and pass to constructor * use const * clang * checkers * fix(deps): Remove dev_dependency = True for rules_proto and rules_python. (#14696) * feat(mixin): add mixin utils (#14691) * feat(mixin): add mixin utils * modify following comments * remove reference to YAML node string * docs(spanner): create a few code snippets as examples for using Spanner Graph using cpp (#14660) * doc(generativelanguage): add samples (#14701) * doc(aiplatform): add Vertex AI samples (#14703) * feat(bigquery): Added option to set JobCreationMode in QueryRequest (#14699) * chore(compute): update discovery doc circa 20240903 (#14708) * chore: update googleapis SHA circa 2024-09-16 (#14709) * chore: update googleapis SHA circa 2024-09-16 PiperOrigin-RevId: 675187467 * feat(mixin): add mixin support in http option utils (#14707) * feat(mixin): add mixin support in http option utils * add more test cases * chore(deps): update dependency protobuf to v28.1 (#14705) * chore(spanner): formatting and clang tidy fixes (#14714) * feat(storage): Add ability to restore soft deleted objects (#14710) * feat(storage): Add ability to restore soft deleted objects * bump testbench version * checkers * cleanup * client docs * additional checks in restore test * make IsIdempotent(RestoreObjectRequest) virtual not pure virtual * remove explicit, run checkers * ci: add universe-domain-demo build (#14715) * chore: remove redundant cmake add_subdirs for samples (#14717) * cleanup(generator): api version only from path (#14719) * chore(deps): update protobuf to v28.2 (#14718) * chore(deps): update dependency build_bazel_rules_apple to v3.9.0 (#14713) * ci: enable layering check in bazel (#14721) * feat(spanner): Add samples for backup schedule feature APIs (#14720) * cleanup(mixin): refactor mixin utils to use HttpRule (#14723) * cleanup(mixin): refactor mixin utils to use HttpRule * complete refactor * format * add comment * revise * revise * nit * chore: update googleapis SHA circa 2024-09-24 (#14726) PiperOrigin-RevId: 677952232 * chore: update Alpine Linux version (#14730) * feat(mixin): add mixin support in descriptor utils (#14727) * feat(mixin): add mixin support in descriptor utils * fix * fix based on changes on main * refactor * cleanup * Make test better * chore(deps): update dependency rules_python to v0.36.0 (#14725) * chore(deps): update dependency build_bazel_rules_apple to v3.9.2 (#14724) * impl: revert addition of generativelanguage (#14731) * fix(rest): promote buffer curl reads from to member variable (#14732) * feat(aiplatform): add `EvaluationServiceClient` (#14729) * chore: update googleapis SHA circa 2024-09-24 (#14736) * chore: update googleapis SHA circa 2024-09-24 PiperOrigin-RevId: 678307181 * feat: add support for API keys (#14737) * feat(dialogflow_es): add missing services (#14735) * chore(deps): update dependency ubuntu to v24 (#14734) * chore(deps): update dependency mozilla/sccache to v0.8.2 (#14741) * fix(storage): make notification, hmac, service account ops return unimplemented in gRPC (#14742) * fix(storage): make notification and hmac ops return unimplemented in gRPC * checkers * remove hmac tests from grpc/stub_test * make get service account return unimplemented in gRPC * remove grpc service account integration test * feat(storage): promote gRPC plugin to GA (#14712) * feat(storage): promote gRPC plugin to GA Move the gRPC plugin functions out of the `google::cloud::storage_experimental` namespace and remove the `experimental-` prefix from the CMake and Bazel targets. I left shims in place for backwards compatibility. The `AsyncClient` remains in the `storage_experimental` namespace, it is fully functional, but we may change some APIs. * continue from Carlos previous work * remove duplicate entry in readme * fix typo in cmake-split-install.sh * remove storage_control from transitional * remove more references to experimental-storage_grpc * fix naming in storage_grpc.cmake * more grpc build fixes * update store_grpc abi dump * change naming in quickstart build, remove more experimental references * checkers * fix quickstart * update customer facing references to direct path * feedback * checkers * remove inline from header and implement in .cc * add todo to storage/quickstart/build.bazel * checkers * namesapce * remove newline * fix namespace to exeperimental * update libraries.bzl * checkers --------- Co-authored-by: Carlos O'Ryan <[email protected]> * chore(deps): update dependency bazelbuild/bazelisk to v1.22.0 (#14743) * feat: API key support over REST transport (#14745) * docs: add samples for API key auth (#14740) * fix: ApiKeyOption without UserProjectOption (#14748) * impl(mixin): add mixin support in code generators (#14738) * code changes and generate code for datamigration * remove code generated for datamigration * recover quickstart * recover quickstart * recover quickstart * fix * fix * fix following comments * docs(pubsub): Add ingestion from GCS topic creation sample (#14749) * docs: explain api key restrictions (#14752) * tests(storage): add universe domain integration test (#14728) * tests(storage): add universe domain integration test add and modify builds adding perms checkers skip tests if missing UD vars cleanup checkers use tags to filter test not TEST_SKIP fix testoptions fix testoptions checkers skip test if environment variables missing remove test from cmake build remove module.bazel remove storage specific universe domain build scripts update copyright date create ud:bazel_test to copy environment variables to test env include common bazel args utilize runfiles to read ud_sa_key_file debug code more debug code symlink temp file fix vars remove runfiles remove testing code use run debug use test add sandbox_add_mount_pair to bazel test * temp printing to verify test * more debug * temporarily turn on all output * remove debug code * checkers * comment out secretenv setup --------- Co-authored-by: Yao Cui <[email protected]> Co-authored-by: Darren Bolduc <[email protected]> Co-authored-by: Scott Hart <[email protected]> Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Ryan Li <[email protected]> Co-authored-by: bharadwajvr <[email protected]> Co-authored-by: sachin purohit <[email protected]> Co-authored-by: aman-19 <[email protected]> Co-authored-by: Carlos O'Ryan <[email protected]> Co-authored-by: Mike Prieto <[email protected]> impl(ACv2): Add maximum range size option for BiDiReads (#77) chore: merge from public circa 2024-10-07 (#73) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc (#14751) * fix(builds): require grpc >= 1.65.4 for async_throughput_benchmark.cc * force unsigned int in SizeIs matcher * add todo * separate guards, fix logic * fix todo placement * ci(spanner): graph samples to use enterprise instances (#14757) * ci(gha): move aiplatform to its own shard (#14754) * cleanup(pubsub): samples (#14758) * refactor: avoid `ApiKeyOption` in sample (#14760) * impl: censor API key header in traces (#14755) * cleanup: revert `ApiKeyOption` changes (#14761) * feat(mixin): add manual changes for pubsub and generate mixin code (#14756) * Pubsub manual changes and the sample of generated library code * sort * remove unrelated * fix * add cmake dependency * change cmake dependency * fix linting * dependency * docs(release): update changelog for the 2024-10 release (#14764) * chore: version bump to v2.31.0-rc (#14767) * feat(mixin): generate mixins for libraries (#14766) * Add mixin for data_migration * Add mixin for the rest of libraries * cmake build * cmake build * cmake build * cmake install expectation * chore(storage): Remove references to notification / hmac / service ac… (#14768) * chore(storage): Remove references to notification / hmac / service account ops from internal * omit rpcs * fix typo * break into multiple lines * chore(deps): update dependency google_cloud_cpp to v2.30.0 (#14769) * chore(deps): update dependency bazel to v7.3.2 (#14753) --------- Co-authored-by: Scott Hart <[email protected]> Co-authored-by: Darren Bolduc <[email protected]> Co-authored-by: Yao Cui <[email protected]> Co-authored-by: Mend Renovate <[email protected]> fix: avoid lifetime extension to prevent holding lock (#79) * fix: avoid lifetime extension to prevent holding lock * checkers feat(storage): add checksumming of bidiread messages (#78) * feat(storage): add checksumming of bidiread messages * remove move feat(storage): Add ReadFromOffset and ReadLast convenience to object_… (#82) feat(storage): Add ReadFromOffset and ReadLast convenience to object_descriptor Chore: merge from public circa 2024-11-18 (#83) * docs(pubsub): Fix region tags for Pub/Sub ingestion from GCS samples (#14773) * chore(deps): update actions/checkout digest to eef6144 (#14772) * chore(deps): update opentelemetry to v1.17.0 (#14774) * chore(deps): update grpc to v1.67.0 (#14711) * cleanup: remove unused otel compile def (#14775) * impl: API key creds for gRPC (#14776) * cleanup(oauth2): change universe domain endpoint (#14777) * impl: add ApiKeyConfig, implement in gRPC (#14778) * feat: API key authentication (#14779) * refactor(oauth2): prepare for API key auth (#14780) * impl(compute): reduce specificity on integration test error message (#14782) * cleanup(oauth2): MinimalIamCredentialsRestStub use universe domain in endpoint (#14781) * cleanup(oauth2): MinimalIamCredentialsRestStub use universe domain in endpoint * test * cleanup * split unit tests * cleanup * fix win build * fix msan-pr * impl: API key auth over REST (#14785) * docs(storage): better suggestion for deprecated API (#14786) * cleanup(storage): add comment on why no API key support (#14788) * ci: pin python version for gsutil (#14792) * chore: update vcpkg to v2024.09.30 (#14790) * chore: update googleapis SHA circa 2024-10-17 (#14793) PiperOrigin-RevId: 686790780 * cleanup: cmake compute features (#14794) * chore(deps): update dependency bazelbuild/bazelisk to v1.22.1 (#14796) * chore(deps): update protobuf to v28.3 (#14798) * chore(deps): update dependency bazel to v7.4.0 (#14797) * chore(deps): update actions/checkout digest to 11bd719 (#14799) * chore: update googleapis SHA circa 2024-10-24 (#14801) * chore: update googleapis SHA circa 2024-10-24 PiperOrigin-RevId: 689456358 * Update the protodeps/protolists * Regenerate libraries * docs: add more cases for generating new libraries (#14806) * docs: add more cases for generating new libraries * fix * feat(oauth2): add support for external account workforce identity (#14800) * feat(oauth2): add support for external account workforce identity * move * avoid cmake dep * format * address the comments * ci: do not fail universe-domain-demo tests (#14811) * impl(mixin): add missing mixin headers to rest stub headers (#14808) * feat(parallelstore): generate library (#14805) * feat(parallelstore): generate library * Run generators and format their outputs * Add API baseline * Manually update READMEs, quickstart, and top-level stuff * use zone-id for quickstart input * refactor: prepare to parse ADC json from string (#14810) * chore(deps): update dependency build_bazel_rules_apple to v3.11.2 (#14802) * impl(compute): remove FutureReservationsClient as the service is not GA (#14812) * impl: parse impersonated ADC json (#14809) * cleanup(mixin): add one API test case for location mixin (#14813) * cleanup(mixin): add one API test case for location mixin * fix format * add test fix nit * chore: update googleapis SHA circa 2024-10-31 (#14817) PiperOrigin-RevId: 691873596 * chore(deps): update dependency rules_python to v0.37.2 (#14795) * cleanup(mixin): deduplicate mixin pb headers (#14819) * cleanup(mixin): add more test cases (#14818) * chore(compute): update discovery doc circa 20241015 (#14822) * cleanup: chrono literals (#14826) * chore(deps): update dependency rules_proto to v7 (#14827) * docs(release): update changelog for the 2024-11 release (#14830) * chore: version bump to 2.32.0-rc (#14834) * chore(deps): update dependency google_cloud_cpp to v2.31.0 (#14835) * chore(deps): update dependency rules_python to v0.38.0 (#14831) * chore(deps): update dependency build_bazel_rules_apple to v3.12.0 (#14837) * cleanup(mixin): add integration tests (#14829) * cleanup(mixin): add integration tests * fix * format * cleanup(quickstart): disable speech_quickstart_global (#14842) * cleanup(quickstart): disable speech_quickstart_global * format * ci: re-enable universe-domain-demo tests (#14843) * chore(deps): update dependency bazel to v7.4.1 (#14840) * chore(deps): update dependency rules_proto to v7.0.2 (#14839) Co-authored-by: Yao Cui <[email protected]> * feat(rest): support impersonated ADC (#14815) * chore(deps): update dependency build_bazel_rules_apple to v3.13.0 (#14844) * chore(deps): update dependency rules_python to v0.39.0 (#14845) * refactor: prepare for breaking change in Protobuf C++ API. (#14828) * cleanup(mixin): remove duplicated operations stub (#14838) * ci: enable global and add non-us region to speech quickstart (#14848) * docs(managedkafka): change old title to new title (#14846) * chore(compute): update discovery doc circa 20241112 (#14850) * impl(generator): handle deprecated services (#14849) * chore(deps): update dependency rules_python to v0.40.0 (#14847) * chore(deps): update dependency bazelbuild/bazelisk to v1.24.0 (#14851) * impl: warn but do not error on deprecated proto types (#14855) * impl(generator): remove deprecated declaration pragma (#14854) * ci: use installed cmake (#14857) * feat(rest): support generateIdToken in impersonation url (#14853) * ci: update cmake quickstart handling for storage grpc (#14856) * ci: prepare for new mdformat (#14859) * chore(deps): update dependency bazelbuild/bazelisk to v1.24.1 (#14858) --------- Co-authored-by: Mike Prieto <[email protected]> Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Darren Bolduc <[email protected]> Co-authored-by: Yao Cui <[email protected]> Co-authored-by: Scott Hart <[email protected]> Co-authored-by: evalon32 <[email protected]> Co-authored-by: Noah Dietz <[email protected]> Chore: merge from public circa 2024-12-05 (#86) chore(ACv2): Update bidi proto patch to rename the field from read_limit to read_length. (#90) impl(ACv2): Initial workflow for BidiWrite Appendable Object (#87) impl(ACv2): Add retry with routing_token for rpc Start (#91) build: build a prelaunch image (#95) * build: add build files to build a prelaunch image * naming * remove commented out code * copyright * checkers chore: merge from public circa 2025-02-07 (#98) * ci: disable deprecated warnings for windows GHA builds (#14875) * feat(spanner): add samples for MR CMEK (#14674) * docs(release): update changelog for the 2024-12 release (#14876) * docs(release): update changelog for the 2024-12 release * add lastest changes to release * add latest changelog * chore: update version to v2.33.0-rc (#14877) * chore(deps): update dependency rules_python to v1 (#14880) * chore(deps): update dependency google_cloud_cpp to v2.32.0 (#14879) * ci: fix spanner samples integration tests (#14883) * ci: fix spanner samples (#14885) * chore(deps): update dependency build_bazel_rules_apple to v3.16.0 (#14881) * chore(deps): update dependency curl to v8.8.0.bcr.2 (#14882) * ci: specify bazel version for quickstarts used in the quickstart-bazel build (#14892) * docs(storage): Update build instructions for gcs+grpc (#14833) * docs(storage): Update build instructions for gcs+grpc * and end mark * spacing + feedback on prometheus * checkers * cleanup(cmake): REGAPIC helper (#14894) * impl(bigquerycontrol): promote from experimental to transitional (#14887) * bazel: update gapic.bzl to work with REST transport (#14895) * fix(gkeconnect): service only supports REST endpoint (#14897) * impl(gkeconnect): only needs proto target (#14898) * ci: add 3PI(workforce) to SA impersonation integration tests for universe domain (#14878) * ci: add 3PI(workforce) to SA impersonation integration tests for universe domain * format * disable SC2046 * chore(deps): update dependency build_bazel_rules_apple to v3.16.1 (#14899) * impl(otel): include algorithm header (#14900) * chore(deps): update protobuf to v29.2 (#14903) * chore: update googleapis SHA circa 2024-12-13 (#14905) * chore: update googleapis SHA circa 2024-12-13 PiperOrigin-RevId: 706010293 * docs(release): update changelog for the second 2024-12 release (#14906) * chore: version bump to 2.34.0-rc (#14907) * chore(deps): update dependency zlib to v1.3.1.bcr.4 (#14909) * ci: fix renovate script (#14911) * docs: remove references to ADC environment variable (#14914) * ci: fix bazel/deps-cache.py (#14912) * chore(compute): regenerate protos in 2025 (#14916) * chore(deps): update dependency google_cloud_cpp to v2.33.0 (#14908) * chore(deps): update grpc to v1.69.0 (#14888) * chore(deps): update dependency mozilla/sccache to v0.9.1 (#14889) * impl(bigquery): Json parsing changes for custom BigQuery library (#14918) * chore(deps): update protobuf to v29.3 (#14919) * chore(deps): update dependency rules_proto to v7.1.0 (#14904) * chore(deps): update rules_cc to v0.0.17 (#14921) * impl(rest): support LRO operation types without name method (#14924) * chore(deps): update dependency com_github_zeux_pugixml to v1.15 (#14928) * chore: update googleapis SHA circa 2025-01-10 (#14926) PiperOrigin-RevId: 714068635 * chore(deps): update dependency platforms to v0.0.11 (#14927) * chore: update auth links (#14931) * chore: update auth links * manual changes * chore(compute): update discovery doc circa 20241231 (#14933) * feat(otel): copy service labels into GCM Metric (#14930) * chore(deps): update dependency google_benchmark to v1.9.0 (#14935) * chore(deps): update dependency pugixml to v1.15 (#14934) * feat(storage): add MoveObject functionality to JSON and gRPC (#14936) * feat(storage): add MoveObject functionality to JSON and gRPC * add moveobject integration test and update testbench version * checkers * add patchbucket call to integration test * use folder enabled bucket * create folder bucket in emulator * make non-pure virtual to fix abi issue * impl(bigquery): Fixed jobs and tables response for empty use case (#14938) * fix(otel): Exporter creating Monitored Resource with task_id for Cloud Run (#14923) When inside a Cloud Run environment, the `MonitoredResource` in a `CreateTimeSeriesRequest` to the Cloud Monitoring API does not include the necessary fields for the `generic_task` resource type, and is rejected. Should follow the well-tested Golang implementation where the `faas.instance` OTel Resource Attribute is mapped to `MonitoredResource` `task_id`. As the `service.namespace` OTel Resource Attribute is not set by the Resource Detector from within Cloud Run, it should be mapped as an empty string, rather than being left absent. https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/8da0f42dab085c916987891419461d583a2aa96e/internal/resourcemapping/resourcemapping.go#L153 * docs(release): update changelog for the 2025-01 release (#14939) * chore: version bump to 2.35.0-rc (#14943) * chore(deps): update dependency rules_python to v1.1.0 (#14946) * ci(spanner): use enterprise edition in instance autoscaler sample (#14949) * doc: update documentation to point to security best practice (#14942) * chore(otel): prepare for otel-cpp 1.19 (#14950) * chore(deps): update dependency google_cloud_cpp to v2.34.0 (#14945) * chore(deps): update dependency google_benchmark to v1.9.1 (#14937) * ci(optimization): pass quickstart if service is unavailable (#14955) * docs(pubsub): Add Pub/Sub ingestion from Kafka samples (#14954) * ci(gha): update sccache version and windows destination dir (#14956) * chore(deps): update abseil to v20240722.1 (#14952) * chore(deps): update opentelemetry to v1.19.0 (#14948) * fix: Make bool_flag public (#14961) * chore(deps): update dependency opentelemetry-cpp to v1.19.0 (#14960) * docs: add code formatting to `msbuild` (#14962) * chore: update googleapis SHA circa 2025-01-28 (#14964) * chore: update googleapis SHA circa 2025-01-28 PiperOrigin-RevId: 720741557 * ci: disable execution of resourcesettings quickstart (#14966) * docs(release): update changelog for the 2025-02 release (#14965) * chore: version bump to 2.36.0-rc (#14968) * cleanup: disable modernize-type-traits in .clang-tidy (#14973) * feat(parametermanager): generate library (#14971) * cleanup: changes following clang-tidy suggestions (#14976) * cleanup: changes following clang-tidy suggestions * fix * fix * chore(deps): update dependency google_cloud_cpp to v2.35.0 (#14970) * chore(deps): update dependency c-ares to v1.19.1 (#14975) * chore(deps): update dependency build_bazel_rules_apple to v3.17.1 (#14953) * chore(deps): update dependency bazel to v7.5.0 (#14959) * chore(deps): update dependency zlib to v1.3.1.bcr.5 (#14963) * cleanup: changes following clang-tidy suggestions (#14977) * chore(deps): update abseil to v20250127 (#14957) Co-authored-by: Yao Cui <[email protected]> * remove patches from builds, merge fixes --------- Co-authored-by: Scott Hart <[email protected]> Co-authored-by: panerorenn9541 <[email protected]> Co-authored-by: Yao Cui <[email protected]> Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Darren Bolduc <[email protected]> Co-authored-by: Carlos O'Ryan <[email protected]> Co-authored-by: jsrinnn <[email protected]> Co-authored-by: Douglas Heriot <[email protected]> Co-authored-by: Mike Prieto <[email protected]> Co-authored-by: Sven Grossmann <[email protected]> impl(ACv2): [ Appendable write ] Resume stream with write_handle (#94) Impl(ACv2): set timeout for appendable write (#100) fix: add x-goog-request-params header to fix routing (#103) * fix: add x-goog-request-params header to fix routing * checkers * Use ApplyRoutingHeaders method instead * remove unnecessary import --------- Co-authored-by: bajajnehaa <[email protected]> Impl(ACv2): add appendable takeover (#102) fix(ACv2): reset the shared_ptr of WriteObject to avoid infinite loop (#105) chore: merge from public circa 2025-02-27 (#104) * chore(deps): update googletest to v1.16.0 (#14983) * chore(deps): update dependency build_bazel_rules_apple to v3.18.0 (#14982) * chore(compute): update discovery doc circa 20250126 (#14984) * chore(bigquerycontrol): upgrade bigquerycontrol from transitive to GA (#14985) * chore: update googleapis SHA circa 2025-02-11 (#14987) * chore: update googleapis SHA circa 2025-02-11 PiperOrigin-RevId: 725444773 * doc: fix typo in doc link (#14990) * docs(storage): remove grpc docs from in-depth topics (#14989) * chore(deps): update dependency build_bazel_rules_apple to v3.19.0 (#14991) * refactor(generator): prepare for upcoming string_view return type change (#14997) * chore(deps): update dependency protoc-gen-validate to v1.2.1 (#14994) * chore(deps): update dependency mozilla/sccache to v0.10.0 (#14998) * chore(deps): update dependency curl to v8.8.0.bcr.3 (#14995) * chore(deps): update dependency c-ares to v1.19.1.bcr.1 (#14996) --------- Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Scott Hart <[email protected]> Co-authored-by: Yao Cui <[email protected]> fix: add utility include (#107) test: add additional unit tests for client context and writer connect… (#110) * test: add additional unit tests for client context and writer connection resumed * checkers * remove duplicate mock, clean up includes test(ACv2): add unit tests for appendable write (#109) chore: merge from public circa 2025-03-24 (#111) * chore: set gcs-sdk-team as CODEOWNERS (#15000) Replace outdated GCS codeowners name to gcs-sdk-team * chore: update googleapis SHA circa 2025-02-27 (#15003) * chore: update googleapis SHA circa 2025-02-27 PiperOrigin-RevId: 731731741 * chore(deps): update dependency rules_python to v1.2.0 (#15002) * ci: disable external account integration test (#15004) * refactor(storage): avoid initializing json object with empty initializer list (#15006) * docs(release): update changelog for the 2025-03 release (#15008) * docs(release): update changelog for the 2025-03 release * update changelog * update changelog * chore: version bump to 2.37.0-rc (#15012) * fix(spanner): update session bookkeeping for session NotFound (#15009) * chore(deps): update dependency google_cloud_cpp to v2.36.0 (#15010) * feat!: remove client library resourcesettings (#15014) * remove resourcesettings * checkers format changes * cleanup * exclude resourcesettings from quickstart cmake * add changelog * Chore: update googleapis SHA circa 2025-03-06 (#15016) * chore: update googleapis SHA circa 2025-03-06 PiperOrigin-RevId: 734192973 * Regenerate libraries * impl(spanner): lock mutex in total_sessions accessor (#15017) * checkers --------- Co-authored-by: Daniel B <[email protected]> Co-authored-by: Scott Hart <[email protected]> Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Yao Cui <[email protected]> impl(ACv2): Flush on close (#108) fix(ACv2): do not match ObjectChecksum in case of takeover on finalization (#113) merge fixes more merge fixes Remove unnecessary changes in merged code checkers remove infra directory cleanup gapic merge issues more cleanup of gapic merge issues more gapic cleanups fix merge issues with protos fix clang tidy errors remove unused env variable Fix cmake-oldest-deps-pr failure clean up read range test use cord workaround in read range test fix docs headers in async client.h add storage_experimental to skip for check-api CI failure fix remove constexpr from capture and declare as static for MSVC default lambda captures for windows capture by value try by copy skip resumeranges test on win32 Address review comments samples(storage): use istreambuf_iterator instead of istream_iterator in storage_object_samples (#15059) istream_iterator skips whitespace by default which results in data not being read as-is when it has whitespace symbols/bytes in it. Co-authored-by: Denis DelGrosso <[email protected]> * make params const ref, use std::move * convert parameter to enum class and do not default value * replace use of auto with specific types * add some unit tests for handle_redirect_error * checkers * remove unused statements * revert const ref change in open and utilize std::move * move TODO issues numbers from prelaunch to main repo --------- Co-authored-by: Carlos O'Ryan <[email protected]>
1 parent 440dcd6 commit 7595793

File tree

65 files changed

+6987
-23
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

65 files changed

+6987
-23
lines changed

ci/cloudbuild/builds/check-api.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,7 @@ function check_abi() {
124124
# characters in the string "internal", and it should again be followed
125125
# by some other number indicating the length of the symbol within the
126126
# "internal" namespace. See: https://en.wikipedia.org/wiki/Name_mangling
127-
-skip-internal-symbols "(8internal|_internal|4absl|4grpc|6google8protobuf|6google3rpc)\d"
127+
-skip-internal-symbols "(8internal|_internal|4absl|4grpc|6google8protobuf|6google3rpc|20storage_experimental)\d"
128128
# We ignore the raw gRPC Stub class. The generated gRPC headers that
129129
# contain these classes are installed alongside our headers. When a new
130130
# RPC is added to a service, these classes gain a pure virtual method. Our

google/cloud/storage/async/client.cc

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,26 @@ future<StatusOr<google::storage::v2::Object>> AsyncClient::InsertObject(
4949
internal::MergeOptions(std::move(opts), connection_->options())});
5050
}
5151

52+
future<StatusOr<ObjectDescriptor>> AsyncClient::Open(
53+
BucketName const& bucket_name, std::string object_name, Options opts) {
54+
auto spec = google::storage::v2::BidiReadObjectSpec{};
55+
spec.set_bucket(bucket_name.FullName());
56+
spec.set_object(std::move(object_name));
57+
return Open(std::move(spec), std::move(opts));
58+
}
59+
60+
future<StatusOr<ObjectDescriptor>> AsyncClient::Open(
61+
google::storage::v2::BidiReadObjectSpec spec, Options opts) {
62+
return connection_
63+
->Open({std::move(spec),
64+
internal::MergeOptions(std::move(opts), connection_->options())})
65+
.then([](auto f) -> StatusOr<ObjectDescriptor> {
66+
auto connection = f.get();
67+
if (!connection) return std::move(connection).status();
68+
return ObjectDescriptor(*std::move(connection));
69+
});
70+
}
71+
5272
future<StatusOr<std::pair<AsyncReader, AsyncToken>>> AsyncClient::ReadObject(
5373
BucketName const& bucket_name, std::string object_name, Options opts) {
5474
auto request = google::storage::v2::ReadObjectRequest{};
@@ -92,6 +112,65 @@ future<StatusOr<ReadPayload>> AsyncClient::ReadObjectRange(
92112
internal::MergeOptions(std::move(opts), connection_->options())});
93113
}
94114

115+
future<StatusOr<std::pair<AsyncWriter, AsyncToken>>>
116+
AsyncClient::StartAppendableObjectUpload(BucketName const& bucket_name,
117+
std::string object_name,
118+
Options opts) {
119+
auto request = google::storage::v2::BidiWriteObjectRequest{};
120+
auto& resource = *request.mutable_write_object_spec()->mutable_resource();
121+
122+
resource.set_bucket(BucketName(bucket_name).FullName());
123+
resource.set_name(std::move(object_name));
124+
request.mutable_write_object_spec()->set_appendable(true);
125+
126+
return StartAppendableObjectUpload(std::move(request), std::move(opts));
127+
}
128+
129+
future<StatusOr<std::pair<AsyncWriter, AsyncToken>>>
130+
AsyncClient::StartAppendableObjectUpload(
131+
google::storage::v2::BidiWriteObjectRequest request, Options opts) {
132+
return connection_
133+
->StartAppendableObjectUpload(
134+
{std::move(request),
135+
internal::MergeOptions(std::move(opts), connection_->options())})
136+
.then([](auto f) -> StatusOr<std::pair<AsyncWriter, AsyncToken>> {
137+
auto w = f.get();
138+
if (!w) return std::move(w).status();
139+
auto t = absl::holds_alternative<google::storage::v2::Object>(
140+
(*w)->PersistedState())
141+
? AsyncToken()
142+
: storage_internal::MakeAsyncToken(w->get());
143+
return std::make_pair(AsyncWriter(*std::move(w)), std::move(t));
144+
});
145+
}
146+
147+
future<StatusOr<std::pair<AsyncWriter, AsyncToken>>>
148+
AsyncClient::ResumeAppendableObjectUpload(BucketName const& bucket_name,
149+
std::string object_name,
150+
std::int64_t generation,
151+
Options opts) {
152+
auto request = google::storage::v2::BidiWriteObjectRequest{};
153+
auto& append_object_spec = *request.mutable_append_object_spec();
154+
155+
append_object_spec.set_bucket(BucketName(bucket_name).FullName());
156+
append_object_spec.set_object(std::move(object_name));
157+
append_object_spec.set_generation(generation);
158+
159+
return connection_
160+
->ResumeAppendableObjectUpload(
161+
{std::move(request),
162+
internal::MergeOptions(std::move(opts), connection_->options())})
163+
.then([](auto f) -> StatusOr<std::pair<AsyncWriter, AsyncToken>> {
164+
auto w = f.get();
165+
if (!w) return std::move(w).status();
166+
auto t = absl::holds_alternative<google::storage::v2::Object>(
167+
(*w)->PersistedState())
168+
? AsyncToken()
169+
: storage_internal::MakeAsyncToken(w->get());
170+
return std::make_pair(AsyncWriter(*std::move(w)), std::move(t));
171+
});
172+
}
173+
95174
future<StatusOr<std::pair<AsyncWriter, AsyncToken>>>
96175
AsyncClient::StartBufferedUpload(BucketName const& bucket_name,
97176
std::string object_name, Options opts) {

google/cloud/storage/async/client.h

Lines changed: 109 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717

1818
#include "google/cloud/storage/async/bucket_name.h"
1919
#include "google/cloud/storage/async/connection.h"
20+
#include "google/cloud/storage/async/object_descriptor.h"
2021
#include "google/cloud/storage/async/reader.h"
2122
#include "google/cloud/storage/async/rewriter.h"
2223
#include "google/cloud/storage/async/token.h"
@@ -231,6 +232,41 @@ class AsyncClient {
231232
google::storage::v2::WriteObjectRequest request, WritePayload contents,
232233
Options opts = {});
233234

235+
/**
236+
* Open an object descriptor to perform one or more ranged reads.
237+
*
238+
* @par Idempotency
239+
* This is a read-only operation and is always idempotent. The operation will
240+
* retry until the descriptor is successfully created. The descriptor itself
241+
* will resume any incomplete ranged reads if the connection(s) are
242+
* interrupted. Use `ResumePolicyOption` and `ResumePolicy` to control this.
243+
*
244+
* @param bucket_name the name of the bucket that contains the object.
245+
* @param object_name the name of the object to be read.
246+
* @param opts options controlling the behavior of this RPC, for example
247+
* the application may change the retry policy.
248+
*/
249+
future<StatusOr<ObjectDescriptor>> Open(BucketName const& bucket_name,
250+
std::string object_name,
251+
Options opts = {});
252+
253+
/**
254+
* Open an object descriptor to perform one or more ranged reads.
255+
*
256+
* @par Idempotency
257+
* This is a read-only operation and is always idempotent. The operation will
258+
* retry until the descriptor is successfully created. The descriptor itself
259+
* will resume any incomplete ranged reads if the connection(s) are
260+
* interrupted. Use `ResumePolicyOption` and `ResumePolicy` to control this.
261+
*
262+
* @param spec the BidiReadObjectSpec to use when retrieving the
263+
* ObjectDescriptor.
264+
* @param opts options controlling the behavior of this RPC, for example
265+
* the application may change the retry policy.
266+
*/
267+
future<StatusOr<ObjectDescriptor>> Open(
268+
google::storage::v2::BidiReadObjectSpec spec, Options opts = {});
269+
234270
/**
235271
* A streaming download for the contents of an object.
236272
*
@@ -243,7 +279,7 @@ class AsyncClient {
243279
* @par Idempotency
244280
* This is a read-only operation and is always idempotent. Once the download
245281
* starts, this operation will automatically resume the download if is
246-
* interrupted. Use `ResumePolicyOption` and `ResumePolicy` to control this
282+
* interrupted. Use `ResumePolicyOption` and `ResumePolicy` to control this.
247283
*
248284
* @param bucket_name the name of the bucket that contains the object.
249285
* @param object_name the name of the object to be read.
@@ -334,10 +370,82 @@ class AsyncClient {
334370
google::storage::v2::ReadObjectRequest request, std::int64_t offset,
335371
std::int64_t limit, Options opts = {});
336372

373+
/*
374+
[start-appendable-object-upload]
375+
Initiates a [resumable upload][resumable-link] for an appendable object.
376+
377+
Appendable objects allow you to create an object and upload data to it
378+
incrementally until it is finalized. This means you can start an upload
379+
and append data to the object later.
380+
381+
You can finalize an appendable object in the first call itself by providing
382+
all the data in the initial upload. You can also explicitly Flush to ensure
383+
the data is persisted.
384+
385+
The recovery can be done from most transient errors, including an unexpected
386+
closure of the streaming RPC used for the upload.
387+
388+
@par Example
389+
@snippet storage_async_samples.cc start-appendable-object-upload
390+
391+
@par Idempotency
392+
This function is always treated as idempotent, and the library will
393+
automatically retry the function on transient errors.
394+
395+
[resumable-link]: https://cloud.google.com/storage/docs/resumable-uploads
396+
[start-appendable-object-upload]
397+
*/
398+
399+
/**
400+
* Starts a new resumable upload session for appendable objects and
401+
* automatic recovery from transient failures.
402+
*
403+
* @snippet{doc} async/client.h start-appendable-object-upload
404+
*
405+
* @param bucket_name the name of the bucket that contains the object.
406+
* @param object_name the name of the object to be read.
407+
* @param opts options controlling the behavior of this RPC, for example
408+
* the application may change the retry policy.
409+
*/
410+
future<StatusOr<std::pair<AsyncWriter, AsyncToken>>>
411+
StartAppendableObjectUpload(BucketName const& bucket_name,
412+
std::string object_name, Options opts = {});
413+
414+
/**
415+
* Starts a new resumable upload session for appendable objects and
416+
* automatic recovery from transient failures.
417+
*
418+
* @snippet{doc} async/client.h start-appendable-object-upload
419+
*
420+
* @param request the request contents, it must include the bucket name and
421+
* object names. Many other fields are optional.
422+
* @param opts options controlling the behavior of this RPC, for example
423+
* the application may change the retry policy.
424+
*/
425+
future<StatusOr<std::pair<AsyncWriter, AsyncToken>>>
426+
StartAppendableObjectUpload(
427+
google::storage::v2::BidiWriteObjectRequest request, Options opts = {});
428+
429+
/**
430+
* Resume a resumable upload session for appendable objects and automatic
431+
* recovery from transient failures.
432+
*
433+
* @param bucket_name the name of the bucket that contains the object.
434+
* @param object_name the name of the object to be uploaded.
435+
* @param generation the object generation to be uploaded.
436+
* @param opts options controlling the behaviour of this RPC, for example the
437+
* application may change the retry policy.
438+
*/
439+
future<StatusOr<std::pair<AsyncWriter, AsyncToken>>>
440+
ResumeAppendableObjectUpload(BucketName const& bucket_name,
441+
std::string object_name, std::int64_t generation,
442+
Options opts = {});
443+
337444
/*
338445
[start-buffered-upload-common]
339446
This function always uses [resumable uploads][resumable-link]. The objects
340447
returned by this function buffer data until it is persisted on the service.
448+
341449
If the buffer becomes full, they stop accepting new data until the service
342450
has persisted enough data.
343451

0 commit comments

Comments
 (0)