Merging changes from mover/Stage2 to mover/c2c-stage#3391
Open
sgv-msft wants to merge 44 commits intomover/c2c-stagefrom
Open
Merging changes from mover/Stage2 to mover/c2c-stage#3391sgv-msft wants to merge 44 commits intomover/c2c-stagefrom
sgv-msft wants to merge 44 commits intomover/c2c-stagefrom
Conversation
* upload binaries to GH, removed upload to stg * removed publish to container var * remove version checker * changes * Fix the TokenStore getting stuck in a read lock * Bump golang.org/x/net from 0.36.0 to 0.38.0 Bumps [golang.org/x/net](https://github.com/golang/net) from 0.36.0 to 0.38.0. - [Commits](golang/net@v0.36.0...v0.38.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-version: 0.38.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * added change log * all deps in changelog * updated version * version checker logic * finsihed release and test * test for same or newer version * renamed version, added test * comments * remove glcm info * testing with tilde and previews * added transport to client * comments * removed version update step, txt file * remove typo * fix LMT timing issue * change pipeline * changed signature of version checker * version update * keep preview version * update gitignore, changeLog vars * fix CI failure with TestChange_DetectFileChangedDuringTransfer * removed version checker from --version * CI fix --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Adele Reed <adreed@microsoft.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Gauri Lamunion <51212198+gapra-msft@users.noreply.github.com>
* Move JobsClean and JobsRemove to library * Change log path location * fix single test * Fix shadow issue * Added another client test * Added tests * Added new e2e tests * remove trailing . * added copyright * updated date * addressed comments * undo delete of wrong file * Add jobs list to library (#3108) * Migrated jobs list to library * update jobs list test * get rid of last rpc ref * Marshal data into type users are used to seeing * Added tests * Move tests to job list * addressed feedback * ripped out RPC code (#3118) * ripped out RPC code * fixed test * import jobs admin * fix sync progress * Clean up of JobsAdmin (#3116) * jobs admin clean up * some more cleanup * Fix nil logger * remove jobs admin logging * Run tests * Fix error logs * remove oncer * Add jobs show to the library (#3120) * fixed codeql alert * remove redundant return * codespell * resolve Copilot comments * revert jppfn
* removed check * FileBlockBlob oauth * other types of blobs, tests * tests * add support for sync * removed old test * removed INFO log in copy, added sync tests * seperate sync and copy tests * fix CI, added version override * update to GA azblob1.6.2 * revert to beta * upgrade to 1.6.2 * replace azfile version * removed version txt file * weird reason, root.go changed
* Added properly error message for system to system container copy * Added properly error message for system to system container copy * Added properly error message for system to system container copy * Added properly error message for system to system container copy * Update common/util.go Co-authored-by: Gauri Lamunion <51212198+gapra-msft@users.noreply.github.com> * Added E2E test for system container copy scenario * added a check to see if the containername is encoded * Added E2E test * Added E2E test * Added E2E test * Added E2E test * Added E2E test * incorporated review comments * Introduce flags for NFS over REST support (#2968) * Added flags description (#2994) * Added support for copy/sync from Local linux <-> Azure Files NFS and Azure Files NFS <-> Azure Files NFS (#2987) * Fixed golangci-lint error * Add the --nfs flag check for download scenario (#3022) * resolved conflicts * Add NFS tests to E2E framework (#3036) * Default handling for hardlink for Azure Files NFS (#3033) * Add default permissions preservation logic if --preserve-permissions is set to false (#3047) * Default handling for symlink,hardlink and special files for Azure Files NFS - Phase-1 (#3046) * fixed golint error * fixed golint * Update hardlink flag name and check the sudo permission for the process (#3078) * Update ChangeLog for 10.30.0~preview.1 release * Update flag description and error message * Update ChangeLog for 10.30.0~preview.1 release * Update flag description and error message * Update ChangeLog.md Co-authored-by: Vikas Bhansali <64532198+vibhansa-msft@users.noreply.github.com> * Throw error and fail the job if destination share is not present for SMB and NFS both (S2S Copy) (#3099) * Remove --nfs flag and add NFS combinations in --from-to flag (#3102) * fixed golangci-lint errora * fixed golangci-lint errors * updates help message for NFS support (#3132) --------- Co-authored-by: wonwuakpa-msft <wonwuakpa@microsoft.com> Co-authored-by: Gauri Lamunion <51212198+gapra-msft@users.noreply.github.com> Co-authored-by: Vikas Bhansali <64532198+vibhansa-msft@users.noreply.github.com>
* Fix docker publish for preview release and mariner repo release * Fix github tag for preview release * Fix github tag for preview release * Fix github tag for preview release * removing platform flag as mariner image support arm64 as well * Added comment * Changing binary name as we have introduced a flder named azcopy in this version * Publish to linux repo stage should depend on VerifyArtifacts stage
* make dpkg step non interactive * force to use default * Added confnew * try removing problematic file * bypass using deb * use ado task * get rid of dep update * fix windows issue * fix mac m1 file name * fix assets for release * Changes azcopy binary name in UploadToStorage stage due to conflicting directory name * docker changes for azcopy binaries * fixed docker issues --------- Co-authored-by: dphulkar <dphulkar@microsoft.com>
* migrate code signing * updated version to preview for safety * changed version back * Default testing to false Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
GetJobSummary\ListJobSummary calls resets the list objects during the get call. This is problematic as its serve dual purpose and a caller can accidentally clear the objects prematurely. Added a parameter to conditionally reset the lists so the caller can decide. When build mode is Mover, regular GetJobSummary calls will skip resetting the lists so that Mover client gets the complete data.
…tor scale issue and other bug fixes - In sync orchestrator, we create traversers for each path prefix which were creating their own S3Client. This led to failures with scale runs. Using sync.once to make sure that S3Client is created once and reused thereafter. - Bug fix: S3 traverser not enumerating directory prefixes for sync orchestrator and double counting files - Adding setters for S3 in moverDependencies
… target If source object has readonly attribute, we set it to false in prologue so the object can be uploaded to target. Then in epilogue, we set the readonly property as needed. The bug is introduced due to pointer copy. When readonly is set to false in epilogue, it also modifies the original properties object so epilogue also sets it as false.
…NFS File share target
…averser, Adding a prefix to errors
…s for NFS file share target in error channel
…r precision time compare for nfs file share target
- Introduced sync scan counters for enumeration failure - Resolving the traverser root path when we follow symlinks. This handles the scenarios where local traverser is executed for a symlink to a folder - Incrementing scanning errors for file and folder during enumeration - Sending enumeration failure in the provided error channel for different error scenarios
…-azcopy into users/sanvern/c2cstage-mergesfrom-stage2
sgv-msft
requested review from
otendolkar,
samskan and
shnayak-msft
as code owners
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Merging latest changes from mover/stage2 into mover/c2c-stage
Type of Change
Reverse integration of mover/stage2 into mover/c2c-stage
How Has This Been Tested?
C2C Automation E2E tests - Yet to run,
Thank you for your contribution to AzCopy!