feat: Implement atomic operations with rollback on error (#125)

* feat: Implement atomic operations with rollback on error

* fix: format

* fix: lint

* fix: format n lint

* chore: updated requirements

Author: ABHINAVGARG05

README.md

@@ -1,3 +1,30 @@
+### Atomic Transactions (Experimental)
+
+[Detailed design and usage docs](internal/atomic/README.md)
+
+Add and delete operations are executed via a lightweight transactional journal to prevent partial state (half-written manifests or orphaned chunks).
+
+Workflow:
+
+1. Start a transaction: new/replacement files written under `.txn/<id>/new/`.
+2. Deletions move originals to `.txn/<id>/trash/`.
+3. Commit promotes all staged files (atomic renames) and removes trash.
+4. Rollback removes staged files and restores trash originals.
+
+Manual recovery:
+
+```
+sietch recover --retention 24h
+```
+
+Scans `.txn/` for incomplete transactions and resumes or rolls them back. Completed journals older than the retention window are purged.
+
+Limitations / Next Steps:
+
+- Current scope covers `add` and `delete` commands.
+- Further commands (sync, sneakernet transfer) can adopt the same API later.
+- Fault injection hooks for deterministic testing are planned.
+
 # Sietch Vault
 
 [![CI](https://github.com/substantialcattle5/sietch/actions/workflows/ci.yml/badge.svg)](https://github.com/substantialcattle5/sietch/actions/workflows/ci.yml)
@@ -11,9 +38,9 @@ Sietch creates self-contained encrypted vaults that can sync over LAN, sneakerne
 
 Sietch Vault is designed for environments where:
 
-* Internet is scarce, censored, or unreliable
-* Data privacy is a necessity, not an optional feature
-* People work nomadically—researchers, journalists, activists
+- Internet is scarce, censored, or unreliable
+- Data privacy is a necessity, not an optional feature
+- People work nomadically—researchers, journalists, activists
 
 ## Quick Start
 
@@ -28,12 +55,14 @@ make build
 ### Basic Usage
 
 **Create a vault**
+
 ```bash
 sietch init --name dune --key-type aes        # AES-256-GCM encryption
 sietch init --name dune --key-type chacha20   # ChaCha20-Poly1305 encryption
 ```
 
 **Add files**
+
 ```bash
 # Single file
 sietch add ./secrets/thumper-plans.pdf documents/
@@ -46,13 +75,15 @@ sietch add ~/photos/img1.jpg ~/photos/img2.jpg vault/photos/
 ```
 
 **Sync over LAN**
+
 ```bash
 sietch sync /ip4/192.168.1.42/tcp/4001/p2p/QmPeerID
 # or auto-discover peers
 sietch sync
 ```
 
 **Retrieve files**
+
 ```bash
 sietch get thumper-plans.pdf ./retrieved/
 ```
@@ -70,30 +101,38 @@ sietch get thumper-plans.pdf ./retrieved/
 ## How It Works
 
 ### Chunking & Deduplication
-* Files are split into configurable chunks (default: 4MB)
-* Identical chunks across files are deduplicated to save space
-* Please Refer [this](internal/deduplication/README.md) documentation to understand how Deduplication works.
+
+- Files are split into configurable chunks (default: 4MB)
+- Identical chunks across files are deduplicated to save space
+- Please Refer [this](internal/deduplication/README.md) documentation to understand how Deduplication works.
 
 ### Encryption
+
 Each chunk is encrypted before storage using:
-* **Symmetric**: AES-256-GCM or ChaCha20-Poly1305 with passphrase
-* **Asymmetric**: GPG-compatible public/private keypairs
+
+- **Symmetric**: AES-256-GCM or ChaCha20-Poly1305 with passphrase
+- **Asymmetric**: GPG-compatible public/private keypairs
 
 ### Peer Discovery
+
 Peers discover each other via:
-* LAN gossip (UDP broadcast)
-* Manual IP whitelisting
-* QR-code sharing *(coming soon)*
+
+- LAN gossip (UDP broadcast)
+- Manual IP whitelisting
+- QR-code sharing _(coming soon)_
 
 ### Syncing
+
 Inspired by rsync, Sietch only transfers:
-* Missing chunks
-* Changed metadata
-* Over encrypted TCP connections with optional compression
+
+- Missing chunks
+- Changed metadata
+- Over encrypted TCP connections with optional compression
 
 ## Available Commands
 
 ### Core Operations
+
 ```bash
 sietch init [flags]                    # Initialize a new vault
 sietch add <source> <destination> [args...]  # Add files to vault (multiple file support)
@@ -103,13 +142,15 @@ sietch delete <filename>               # Delete files from vault
 ```
 
 ### Network Operations
+
 ```bash
 sietch discover [flags]                # Discover peers on local network
 sietch sync [peer-address]             # Sync with other vaults
 sietch sneak [flags]                   # Transfer via sneakernet (USB)
 ```
 
 ### Management
+
 ```bash
 sietch dedup stats                     # Show deduplication statistics
 sietch dedup gc                        # Run garbage collection
@@ -120,27 +161,31 @@ sietch scaffold [flags]                # Create vault from template
 ## Advanced Usage
 
 **View vault contents**
+
 ```bash
 sietch ls                              # List all files
 sietch ls docs/                        # List files in specific directory
 sietch ls --long                       # Show detailed information
 ```
 
 **Network synchronization**
+
 ```bash
 sietch discover                        # Find peers automatically
 sietch sync                            # Auto-discover and sync
 sietch sync /ip4/192.168.1.5/tcp/4001/p2p/QmPeerID  # Sync with specific peer
 ```
 
 **Sneakernet transfer**
+
 ```bash
 sietch sneak                           # Interactive mode
 sietch sneak --source /media/usb/vault # Transfer from USB vault
 sietch sneak --dry-run --source /backup/vault  # Preview transfer
 ```
 
 **Deduplication management**
+
 ```bash
 sietch dedup stats                     # Show statistics
 sietch dedup gc                        # Clean unreferenced chunks
@@ -168,26 +213,29 @@ sietch manifest
 ## Development
 
 ### Prerequisites
-* **Go 1.23+** – [Download](https://golang.org/dl/)
-* **Git** – Version control
+
+- **Go 1.23+** – [Download](https://golang.org/dl/)
+- **Git** – Version control
 
 ### Quick Development Setup
 
 1. **Clone and setup**
-    ```bash
-    git clone https://github.com/substantialcattle5/sietch.git
-    cd sietch
-    ./scripts/setup-hooks.sh
-    ```
+
+   ```bash
+   git clone https://github.com/substantialcattle5/sietch.git
+   cd sietch
+   ./scripts/setup-hooks.sh
+   ```
 
 2. **Verify installation**
-    ```bash
-    make check-versions
-    make build
-    make test
-    ```
+   ```bash
+   make check-versions
+   make build
+   make test
+   ```
 
 ### Available Commands
+
 ```bash
 make help            # List all commands
 make dev             # Format, test, build
@@ -203,13 +251,15 @@ For detailed development guidelines, see [CONTRIBUTING.md](CONTRIBUTING.md).
 We welcome contributions of all kinds! Whether you're fixing bugs, adding features, improving documentation, or enhancing UX.
 
 **Quick contribution steps:**
+
 1. Fork the repository
 2. Create a feature branch: `git checkout -b feature/stillsuit`
 3. Make your changes following our [style guidelines](CONTRIBUTING.md#styleguides)
 4. Commit using [conventional commits](CONTRIBUTING.md#commit-messages)
 5. Push and open a Pull Request
 
 See our [Contributing Guide](CONTRIBUTING.md) for detailed information about:
+
 - Development environment setup
 - Code style guidelines
 - Testing requirements
@@ -218,9 +268,10 @@ See our [Contributing Guide](CONTRIBUTING.md) for detailed information about:
 ## Inspiration & Credits
 
 Sietch draws inspiration from:
-* **Syncthing** - Decentralized file synchronization
-* **IPFS** - Content-addressed storage
-* **Obsidian Sync** - Seamless cross-device syncing
+
+- **Syncthing** - Decentralized file synchronization
+- **IPFS** - Content-addressed storage
+- **Obsidian Sync** - Seamless cross-device syncing
 
 Built with ❤️ in Go by the open source community.
 
@@ -308,4 +359,4 @@ Licensed under the **MIT License** – see the [LICENSE](LICENSE) file for detai
 
 ---
 
-> *"When you live in the desert, you develop a very strong survival instinct."* – Chani, *Dune*
+> _"When you live in the desert, you develop a very strong survival instinct."_ – Chani, _Dune_

cmd/add.go

@@ -6,21 +6,25 @@ package cmd
 import (
 	"context"
 	"fmt"
+	"io"
 	"os"
 	"path/filepath"
 	"strings"
 	"time"
 
 	"github.com/spf13/cobra"
 
+	"github.com/substantialcattle5/sietch/internal/atomic"
 	"github.com/substantialcattle5/sietch/internal/chunk"
 	"github.com/substantialcattle5/sietch/internal/config"
 	"github.com/substantialcattle5/sietch/internal/constants"
 	"github.com/substantialcattle5/sietch/internal/fs"
-	"github.com/substantialcattle5/sietch/internal/manifest"
+
+	// manifest raw storage removed in favor of transactional helper
 	"github.com/substantialcattle5/sietch/internal/progress"
 	"github.com/substantialcattle5/sietch/internal/ui"
 	"github.com/substantialcattle5/sietch/util"
+	"gopkg.in/yaml.v3"
 )
 
 // SpaceSavings represents space savings statistics for a file
@@ -140,6 +144,20 @@ Examples:
 			fmt.Printf("Starting batch processing of %d files...\n\n", len(filePairs))
 		}
 
+		// Begin transaction encompassing entire add set for atomicity
+		// vaultRoot already resolved earlier; reuse variable
+		txn, err := atomic.Begin(vaultRoot, map[string]any{"command": "add", "fileCount": len(filePairs)})
+		if err != nil {
+			return fmt.Errorf("begin transaction: %w", err)
+		}
+		committed := false
+		defer func() {
+			if !committed {
+				_ = txn.Rollback()
+				fmt.Println("txn rollback; add operation did not complete")
+			}
+		}()
+
 		for i, pair := range filePairs {
 			// Enhanced progress display for multiple files
 			if len(filePairs) > 1 {
@@ -220,7 +238,8 @@ Examples:
 
 			// Process the file and store chunks - using the appropriate chunking function
 			var chunkRefs []config.ChunkRef
-			chunkRefs, err = chunk.ChunkFile(ctx, actualSourcePath, chunkSize, vaultRoot, passphrase, progressMgr)
+			// Use transactional chunking to stage new chunks
+			chunkRefs, err = chunk.ChunkFileTransactional(ctx, actualSourcePath, chunkSize, vaultRoot, passphrase, progressMgr, txn)
 
 			if err != nil {
 				errorMsg := fmt.Sprintf("✗ %s: chunking failed - %v", filepath.Base(pair.Source), err)
@@ -254,8 +273,8 @@ Examples:
 			}
 
 			// Save the manifest
-			err = manifest.StoreFileManifest(vaultRoot, filepath.Base(pair.Source), fileManifest)
-			if err != nil {
+			// Store manifest via transaction (stage create)
+			if err := storeManifestTransactional(txn, vaultRoot, filepath.Base(pair.Source), fileManifest); err != nil {
 				if err.Error() == "skipped" {
 					errorMsg := fmt.Sprintf("✗ '%s': skipped", fileManifest.Destination+fileManifest.FilePath)
 					fmt.Println(errorMsg)
@@ -351,11 +370,15 @@ Examples:
 			}
 		}
 
-		// Return error only if all files failed
+		// Commit transaction if we had any successes
 		if successCount == 0 {
 			return fmt.Errorf("all files failed to process")
 		}
-
+		if err := txn.Commit(); err != nil {
+			return fmt.Errorf("commit transaction: %w", err)
+		}
+		committed = true
+		fmt.Println("txn successful; add committed")
 		return nil
 	},
 }
@@ -518,5 +541,48 @@ func init() {
 	addCmd.Flags().String("passphrase-file", "", "Read passphrase from file (file should have 0600 permissions)")
 }
 
+// storeManifestTransactional writes a manifest yaml via the transaction staging new file.
+func storeManifestTransactional(txn *atomic.Transaction, vaultRoot string, fileName string, m *config.FileManifest) error {
+	// Mirror logic from manifest.StoreFileManifest but stage instead of direct write.
+	manifestsDir := filepath.Join(vaultRoot, ".sietch", "manifests")
+	if err := os.MkdirAll(manifestsDir, 0o755); err != nil {
+		return fmt.Errorf("failed to create manifests directory: %v", err)
+	}
+	destination := strings.ReplaceAll(m.Destination, "/", ".")
+	uniqueFileIdentifier := destination + fileName + ".yaml"
+	relPath := filepath.ToSlash(filepath.Join(".sietch", "manifests", uniqueFileIdentifier))
+	// Prompt overwrite if exists in final location
+	finalPath := filepath.Join(manifestsDir, uniqueFileIdentifier)
+	if _, err := os.Stat(finalPath); err == nil {
+		message := fmt.Sprintf("'%s' exists. Overwrite? ", m.Destination+fileName)
+		response, err2 := util.ConfirmOverwrite(message, os.Stdin, os.Stdout)
+		if err2 != nil || !response {
+			return fmt.Errorf("skipped")
+		}
+		// Stage replace instead of create
+		w, err2 := txn.StageReplace(relPath)
+		if err2 != nil {
+			return err2
+		}
+		defer w.Close()
+		return writeManifestYAML(w, m)
+	}
+	w, err := txn.StageCreate(relPath)
+	if err != nil {
+		return err
+	}
+	defer w.Close()
+	return writeManifestYAML(w, m)
+}
+
+func writeManifestYAML(w io.Writer, m *config.FileManifest) error {
+	enc := yaml.NewEncoder(w)
+	enc.SetIndent(2)
+	if err := enc.Encode(m); err != nil {
+		return fmt.Errorf("encode manifest: %w", err)
+	}
+	return nil
+}
+
 //TODO: Need to check how symlinks will be handled
 //TODO: Interactive mode with real time progress indicators