Skip to content

Conversation

@hgraeber
Copy link
Contributor

@hgraeber hgraeber commented Apr 14, 2025

Fix the copy command so that from-repository and from-repository-file works like they work with restic alone. That means

version: 2
profile:
  repository: r1
  ...
  copy:
    from-repository: r2
    ...

copies from repository r2 to r1 and not in the opposite direction, like it does now. That means the repository mentioned directly below profile (r1) is the destination similar to backup. The special case without the "from"-prefix mention in the docs works like before.

As a bonus the retention section can allow before-copy and after-copy analog to before-backup and after-backup. This will be reserved for a future PR.

@coderabbitai
Copy link

coderabbitai bot commented Apr 14, 2025

Walkthrough

The CopySection struct in the configuration is refactored to explicitly separate source and destination repository fields, introducing From* and To* prefixes. Associated methods and flag-generation logic are updated to support this distinction and maintain compatibility with different restic versions. Corresponding tests are updated and extended to validate the new structure and logic.

Changes

Cohort / File(s) Change Summary
CopySection Refactor and Logic Update
config/profile.go
Refactored CopySection to use explicit From* (source) and To* (destination) fields. Updated methods (resolve, setRootPath, getInitFlags, getCommandFlags) to handle the new structure and restic version differences. Added IsCopyTo() method.
Test Updates for CopySection
config/profile_test.go
Updated tests to use new From* and To* fields. Rewrote and extended tests to validate flag generation for both source and destination configurations and for different restic versions. Added TestGetCopyToStructFields.
Wrapper Test Field Name Updates
wrapper_test.go
Updated test struct field names in CopySection to match new To* destination naming in test setup. No changes to test logic or public interfaces.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~15–20 minutes

Suggested labels

enhancement

Note

⚡️ Unit Test Generation is now available in beta!

Learn more here, or try it out under "Finishing Touches" below.


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c05217b and dbd7da5.

📒 Files selected for processing (1)
  • config/profile.go (1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
config/profile.go (5)
util/maybe/bool.go (1)
  • Bool (11-13)
config/confidential.go (1)
  • ConfidentialValue (14-16)
shell/args.go (1)
  • Args (11-14)
constants/parameter.go (5)
  • ParameterRepository (23-23)
  • ParameterRepositoryFile (24-24)
  • ParameterPasswordFile (25-25)
  • ParameterPasswordCommand (26-26)
  • ParameterKeyHint (27-27)
shell/arg.go (2)
  • NewArg (40-51)
  • ArgConfigEscape (21-21)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Build and test (1.24, ubuntu-latest)
  • GitHub Check: Build and test (1.24, windows-latest)
  • GitHub Check: Analyze (go)
🔇 Additional comments (6)
config/profile.go (6)

347-363: Excellent refactoring of repository parameter separation.

The explicit separation of source (From*) and destination (To*) repository parameters provides clear semantic distinction and addresses the core issue mentioned in the PR objectives. The field descriptions are now accurate and consistent with their intended purpose.


367-367: Well-designed direction detection method.

The IsCopyTo() method provides a clear way to determine copy direction based on destination repository configuration, enabling proper branching logic in other methods.


369-377: Proper path normalization for both copy directions.

The conditional logic correctly normalizes the appropriate repository path based on copy direction, addressing the previous issue where FromRepository was not being normalized.


379-389: Consistent path handling across copy directions.

The conditional path normalization follows the same logical pattern as the resolve method, ensuring both copy directions properly handle file path resolution.


391-438: Complex but correct initialization logic for both copy directions.

The method properly handles repository initialization for both copy directions. When copying to a destination, it initializes the destination repository and uses the profile's source repository for chunker parameters. When copying from a source, it uses the source repository from the copy section. The logic correctly respects the InitializeCopyChunkerParams setting.


440-503: Comprehensive flag generation supporting multiple restic versions and copy directions.

The method correctly handles the complex requirements:

  • Proper branching for copy-to vs copy-from scenarios
  • Version-specific flag handling for restic < 0.14 (using repo2 style) vs >= 0.14 (using from- prefixes)
  • Appropriate confidential repository handling for both directions
  • Consistent flag building logic that aligns with the repository separation

The implementation addresses the core PR objective of ensuring consistent behavior with restic's native copy operations.

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@codecov
Copy link

codecov bot commented Jun 12, 2025

Codecov Report

❌ Patch coverage is 94.00000% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.39%. Comparing base (63dee21) to head (dbd7da5).
⚠️ Report is 8 commits behind head on master.

Files with missing lines Patch % Lines
config/profile.go 94.00% 4 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #486      +/-   ##
==========================================
+ Coverage   79.29%   79.39%   +0.10%     
==========================================
  Files         136      137       +1     
  Lines       13264    13390     +126     
==========================================
+ Hits        10517    10630     +113     
- Misses       2326     2340      +14     
+ Partials      421      420       -1     
Flag Coverage Δ
unittests 79.39% <94.00%> (+0.10%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@hgraeber hgraeber marked this pull request as ready for review June 12, 2025 21:26
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
config/profile.go (1)

354-355: Misleading description references “backup” instead of “copy”.

check-before / check-after sit in the copy section, yet the doc-strings still say “backup command”.
Rename to “copy command” to avoid confusion when reading the generated documentation or help.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 77b1f30 and 057f332.

📒 Files selected for processing (4)
  • config/profile.go (3 hunks)
  • config/profile_test.go (3 hunks)
  • wrapper.go (1 hunks)
  • wrapper_test.go (1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (2)
wrapper_test.go (1)
config/confidential.go (1)
  • NewConfidentialValue (81-83)
config/profile.go (4)
config/confidential.go (1)
  • ConfidentialValue (14-16)
shell/args.go (1)
  • Args (11-14)
constants/parameter.go (5)
  • ParameterRepository (23-23)
  • ParameterRepositoryFile (24-24)
  • ParameterPasswordFile (25-25)
  • ParameterPasswordCommand (26-26)
  • ParameterKeyHint (27-27)
shell/arg.go (2)
  • NewArg (40-51)
  • ArgConfigEscape (21-21)
🪛 GitHub Check: codecov/patch
wrapper.go

[warning] 152-155: wrapper.go#L152-L155
Added lines #L152 - L155 were not covered by tests


[warning] 160-163: wrapper.go#L160-L163
Added lines #L160 - L163 were not covered by tests


[warning] 169-170: wrapper.go#L169-L170
Added lines #L169 - L170 were not covered by tests


[warning] 174-177: wrapper.go#L174-L177
Added lines #L174 - L177 were not covered by tests


[warning] 182-185: wrapper.go#L182-L185
Added lines #L182 - L185 were not covered by tests

config/profile.go

[warning] 402-403: config/profile.go#L402-L403
Added lines #L402 - L403 were not covered by tests


[warning] 427-428: config/profile.go#L427-L428
Added lines #L427 - L428 were not covered by tests

🔇 Additional comments (4)
wrapper.go (1)

144-189: Consider extracting the pre-/post orchestration into a shared helper

The new Copy flow (check → retention → copy → retention → check) mirrors getBackupAction, but is hand-coded a second time.
Duplicated control-flow increases the risk of the two drifting and is already showing up as uncovered lines (Codecov hints).

A small, composable helper such as

func (r *resticWrapper) runWithGuards(
     beforeChecks, beforeRetention, main, afterRetention, afterChecks func() error) error { … }

would remove the duplication and make future maintenance & test coverage simpler.

[ suggest_optional_refactor ]

🧰 Tools
🪛 GitHub Check: codecov/patch

[warning] 152-155: wrapper.go#L152-L155
Added lines #L152 - L155 were not covered by tests


[warning] 160-163: wrapper.go#L160-L163
Added lines #L160 - L163 were not covered by tests


[warning] 169-170: wrapper.go#L169-L170
Added lines #L169 - L170 were not covered by tests


[warning] 174-177: wrapper.go#L174-L177
Added lines #L174 - L177 were not covered by tests


[warning] 182-185: wrapper.go#L182-L185
Added lines #L182 - L185 were not covered by tests

wrapper_test.go (1)

1632-1635: LGTM – test data now reflects the new To* fields

The switch to ToRepository / ToPasswordFile aligns the test with the refactored struct – no issues spotted.

config/profile_test.go (2)

1505-1607: Great coverage for source/destination flag mapping

These tests thoroughly exercise the new From*To* permutations across restic versions – nice work.

One small ask: add a require.NoError(t, copySection.Validate()) (or similar) before expectations to guard future validation logic.

[ suggest_nitpick ]


1610-1712: Symmetric “To” tests complete the matrix*

The complementary tests for destination-centric copies are clear and valuable.
No functional concerns.

@hgraeber hgraeber marked this pull request as draft June 17, 2025 18:23
resticprofile implements a "special case" for the copy command, which
does not use the "from-"-prefix and reverses the copy direction. With this
change the standard direction is used when the "from" prefix is explicitly
used.
@creativeprojects
Copy link
Owner

Hey! how is it going with this PR?

Tell me if you need any help 👍🏻

@hgraeber
Copy link
Contributor Author

It's the part of before-copy and after-copy that's missing (The same for check-before and check-after). I have made already two tries I never pushed, because I don't like the resulting code.

copy with from-repository is already working. Maybe I shall make it ready for release and leave the before-copy/after-copy part for another PR.

@creativeprojects
Copy link
Owner

If the first part of the PR is ready, you can definitely split it in two.
Smaller PR always makes the code quicker to review 👍🏻

@hgraeber hgraeber marked this pull request as ready for review July 29, 2025 17:43
@creativeprojects creativeprojects changed the title Fix copy Add copy from repository in configuration Jul 30, 2025
Copy link
Owner

@creativeprojects creativeprojects left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just have that one comment about the flag name, otherwise it looks good to me 👍🏻

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
config/profile.go (2)

391-438: Complex but logically sound implementation of init flag generation.

The method correctly handles both copy directions with appropriate chunker parameter and initialisation logic. However, the implementation is quite complex with profile copying and intricate conditional logic.

Consider if this complexity can be reduced through helper methods or clearer separation of concerns, though the current logic appears functionally correct.


440-503: Comprehensive handling of version-specific flag generation for both copy directions.

The method correctly implements flag generation for both copy scenarios while handling restic version differences appropriately. The repositoryArgs map approach provides clean parameter management.

The implementation is functionally sound but quite complex. Consider extracting helper methods for flag generation logic to improve maintainability and reduce code duplication between the copy-to and copy-from branches.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 893eb78 and 6d4c32b.

📒 Files selected for processing (1)
  • config/profile.go (1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
config/profile.go (5)
util/maybe/bool.go (1)
  • Bool (11-13)
config/confidential.go (1)
  • ConfidentialValue (14-16)
shell/args.go (1)
  • Args (11-14)
constants/parameter.go (5)
  • ParameterRepository (23-23)
  • ParameterRepositoryFile (24-24)
  • ParameterPasswordFile (25-25)
  • ParameterPasswordCommand (26-26)
  • ParameterKeyHint (27-27)
shell/arg.go (2)
  • NewArg (40-51)
  • ArgConfigEscape (21-21)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Build and test (1.24, windows-latest)
  • GitHub Check: Build and test (1.24, ubuntu-latest)
  • GitHub Check: Analyze (go)
🔇 Additional comments (4)
config/profile.go (4)

347-363: Excellent refactoring of CopySection fields for source/destination clarity.

The separation into explicit From* and To* fields provides clear semantics for copy operations. The field descriptions correctly identify source vs destination repositories, addressing the confusion highlighted in past reviews.


367-367: LGTM! Clean implementation of copy direction detection.

The IsCopyTo() method provides a clear way to determine copy operation direction based on destination repository configuration.


369-377: Excellent fix for repository path normalisation.

The conditional logic properly normalises both source and destination repository paths based on copy direction, ensuring consistent behaviour with other repository fields throughout the codebase.


379-389: Consistent implementation of conditional path resolution.

The method correctly applies root path resolution to the appropriate source or destination fields based on copy direction, maintaining consistency with the overall path handling approach.

This is consistent with the `init` section.
@creativeprojects
Copy link
Owner

creativeprojects commented Jul 30, 2025

Thanks for the change 👍🏻

Actually I found something during testing: when the initialise: true flag is set, we're trying to initialise the target repository if it's not existing yet. Which makes sense in a way that it's going to be a copy of the current one.

While leaving the initialise: true flag using a from-repository it doesn't try to initialise the source repository. And it kind of make sense as well since we should be copying from an existing repository.

2025/07/30 18:17:59 profile 'self': initializing repository (if not existing)
2025/07/30 18:17:59 starting command: /usr/local/bin/restic init --password-file=examples/key --repo=/Volumes/RAMDisk/self --verbose=1
2025/07/30 18:17:59 profile 'self': initializing secondary repository (if not existing)
2025/07/30 18:17:59 starting command: /usr/local/bin/restic init --copy-chunker-params --from-password-file=examples/key --from-repo=/Volumes/RAMDisk/copy-from --password-file=examples/key --repo=/Volumes/RAMDisk/self --verbose=1

So we have two choices:

  • we can disable trying to initialise the secondary (source) repository
  • we can be sending the source repository to the secondary init to fix the initialisation step: it makes things consistent (but not very useful)

Any preference?

@creativeprojects
Copy link
Owner

... which I realise is also opening yet another a can of worms 😆

In the order of initialisation, the source should be done first, then the target with the --copy-chunker-params flag so we'd need to change the order: right now the repository is done first, like any other step, then the one from the copy section 🤦🏻

@hgraeber
Copy link
Contributor Author

It is important to initialize the target repository to be consistent with the backup section. This is useful for making copies to external drives and initialize them on first use. Nor forgetting to use --copy-chunker-params is important here.

I wouldn't bother with initializing a source drive. This is really an edge case. Possibly forgetting --copy-chunker-params doesn't count here, because there is no source repo for this, which makes sense.

@creativeprojects
Copy link
Owner

Makes sense 😉
Right, I'll merge this first PR for now, thanks for the great work! 👍🏻

@creativeprojects creativeprojects merged commit f93ce20 into creativeprojects:master Jul 31, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants