Add docs-assembler deploy [plan|apply] commands

[reakaleek]

Changes

This gives us the ability to analyze the plan and identify which files are being uploaded, deleted or skipped

Usage

docs-assembler repo clone-all --environment edge
docs-assembler repo build-all --environment edge
docs-assembler deploy plan --environment edge --s3-bucket-name elastic-docs-v3-website-edge --out out.plan 

# ... Do something else with the plan? E.g. optimize for cache invalidation.

docs-assembler deploy apply --environment edge --s3-bucket-name elastic-docs-v3-website-edge --plan-file out.plan

Example Output

info ::Program               :: Total files to sync: 14525
info ::Program               :: Total files to delete: 0
info ::Program               :: Total files to add: 0
info ::Program               :: Total files to update: 1725
info ::Program               :: Total files to skip: 12800
info ::Program               :: Plan written to out.plan

{
  "add": [
    { 
      "local_path": "/path-to-local/docs/add.md", 
      "destination_path": "/docs/add.md"
    }
  ],
  "update": [
    { 
      "local_path": "/path-to-local/docs/update.md",
      "destination_path": "/docs/update.md"
    }
  ],
  "skip": [
    { 
      "local_path": "/path-to-local/docs/skip.md",
      "destination_path": "/docs/skip.md"
    }
  ],
  "delete": [
    { 
      "destination_path": "/docs/delete.md"
    }
  ]
}

[Copilot]

Pull Request Overview

This PR adds deployment commands for the docs-assembler tool, enabling planning and applying file sync operations to an S3 bucket. Key changes include new test cases for sync planning and applying, the implementation of AWS S3 sync strategies (plan and apply), and CLI command integration to support deployment operations.

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/docs-assembler.Tests/docs-assembler.Tests.csproj	Added Moq package reference for unit tests
tests/docs-assembler.Tests/DocsSyncTests.cs	Introduced new tests for the plan and apply operations
src/tooling/docs-assembler/Program.cs	Registered the new deploy commands with the CLI app
src/tooling/docs-assembler/Deploying/DocsSync.cs	Added interfaces and record definitions for sync operations
src/tooling/docs-assembler/Deploying/AwsS3SyncPlanStrategy.cs	Implemented the sync planning logic with parallel processing and ETag calculation
src/tooling/docs-assembler/Deploying/AwsS3SyncApplyStrategy.cs	Implemented the sync applying logic with directory upload and batched file deletion
src/tooling/docs-assembler/Cli/DeployCommands.cs	Defined CLI commands for planning and applying sync operations
Directory.Packages.props	Updated package versions by adding the Moq package

Comments suppressed due to low confidence (1)

src/tooling/docs-assembler/Cli/DeployCommands.cs:23

• [nitpick] Consider using 'DeployCommands' as the logger category instead of 'Program' to ensure the log messages are clearly associated with this class.

var log = logger.CreateLogger<Program>();

[Mpdreamz]

Slight nit on library choice.

If we change our static files almost everything changes because of the hash inclusion on the references to these assets.

<link rel="stylesheet preload" as="style" type="text/css" href="/docs/_static/styles.css?v=673f1437ffe25ef1" crossorigin="">

Should we short circuit the plan in that case and always do a full sync? Although I guess this will still save us from uploading images all the time. NVMD 👍

Awesome addition! Can't wait to see how this will impact our upload times!

[reakaleek]

Should we short circuit the plan in that case and always do a full sync? Although I guess this will still save us from uploading images all the time. NVMD 👍

If there is a change in a static file the content-hash should change. This means the link tag content also changes and therefore it will basically do a full sync because all the ETags change.

Generally, I will only add the plan command to the assembler workflow for now and observe it for a while.

[Mpdreamz]

If there is a change in a static file the content-hash should change. This means the link tag content also changes and therefore it will basically do a full sync because all the ETags change.

I guess my question also was if in that case instead of writing all files out into the plan we write a special marker instead (fullsync: true) or something like that. But even if all content needs to be synced it makes sense to still do a partial update so that we don't have to upload all images.

[reakaleek]

If there is a change in a static file the content-hash should change. This means the link tag content also changes and therefore it will basically do a full sync because all the ETags change.

I guess my question also was if in that case instead of writing all files out into the plan we write a special marker instead (fullsync: true) or something like that. But even if all content needs to be synced it makes sense to still do a partial update so that we don't have to upload all images.

Ah, I see. Sorry for misunderstanding. Makes sense!