Merge pull request dimitri-yatsenko#62 from dimitri-yatsenko/claude/improve-transactions-section-GJsFb

Clarify when explicit transactions are needed

Author: dimitri-yatsenko

book/40-operations/040-transactions.ipynb

@@ -453,110 +453,13 @@
    "cell_type": "markdown",
    "id": "cell-best-practices",
    "metadata": {},
-   "source": [
-    "## Best Practices\n",
-    "\n",
-    "### Keep Transactions Short\n",
-    "\n",
-    "Long-running transactions can:\n",
-    "- Lock database resources, blocking other users\n",
-    "- Increase the risk of deadlocks\n",
-    "- Consume server memory for maintaining transaction state\n",
-    "\n",
-    "```python\n",
-    "# Good: Short, focused transaction\n",
-    "with conn.transaction:\n",
-    "    Table.insert(prepared_data)\n",
-    "\n",
-    "# Bad: Long transaction with computation inside\n",
-    "with conn.transaction:\n",
-    "    data = fetch_from_external_api()  # Could take minutes!\n",
-    "    processed = heavy_computation(data)  # Even longer!\n",
-    "    Table.insert(processed)\n",
-    "```\n",
-    "\n",
-    "**Better approach:** Do computation outside the transaction, then insert:\n",
-    "\n",
-    "```python\n",
-    "# Prepare data outside transaction\n",
-    "data = fetch_from_external_api()\n",
-    "processed = heavy_computation(data)\n",
-    "\n",
-    "# Quick transaction for database operations only\n",
-    "with conn.transaction:\n",
-    "    Table.insert(processed)\n",
-    "```\n",
-    "\n",
-    "### Avoid User Interaction Inside Transactions\n",
-    "\n",
-    "Never wait for user input while holding a transaction open:\n",
-    "\n",
-    "```python\n",
-    "# Bad: User could take coffee break!\n",
-    "with conn.transaction:\n",
-    "    Table.insert(data)\n",
-    "    confirm = input(\"Confirm? (y/n)\")  # DON'T DO THIS\n",
-    "    if confirm != 'y':\n",
-    "        raise Exception(\"Cancelled\")\n",
-    "```\n",
-    "\n",
-    "### Handle Deadlocks Gracefully\n",
-    "\n",
-    "When multiple transactions compete for the same resources, deadlocks can occur.\n",
-    "MySQL automatically detects and resolves deadlocks by rolling back one transaction.\n",
-    "\n",
-    "```python\n",
-    "import time\n",
-    "\n",
-    "def insert_with_retry(data, max_retries=3):\n",
-    "    \"\"\"Insert data with deadlock retry logic.\"\"\"\n",
-    "    for attempt in range(max_retries):\n",
-    "        try:\n",
-    "            with dj.conn().transaction:\n",
-    "                Table.insert(data)\n",
-    "            return  # Success\n",
-    "        except Exception as e:\n",
-    "            if \"Deadlock\" in str(e) and attempt < max_retries - 1:\n",
-    "                time.sleep(0.1 * (2 ** attempt))  # Exponential backoff\n",
-    "                continue\n",
-    "            raise\n",
-    "```\n",
-    "\n",
-    "### Use Transactions for Related Inserts\n",
-    "\n",
-    "When inserting data that spans multiple tables and must be consistent:\n",
-    "\n",
-    "```python\n",
-    "# Insert related data atomically\n",
-    "with dj.conn().transaction:\n",
-    "    Subject.insert1(subject_data)\n",
-    "    Session.insert(session_data)  # References Subject\n",
-    "    Recording.insert(recording_data)  # References Session\n",
-    "```"
-   ]
+   "source": "## Best Practices\n\n### When to Use Explicit Transactions\n\nA single `insert()` or `update1()` call is already atomic—it either fully succeeds or fully fails.\nExplicit transactions using `conn.transaction` are only needed when **multiple operations** must succeed or fail together:\n\n```python\n# No need for explicit transaction - single insert is already atomic\nTable.insert(data)\n\n# Explicit transaction needed - multiple operations must be atomic\nwith dj.conn().transaction:\n    Table1.insert(data1)\n    Table2.insert(data2)\n    Table3.update1(data3)\n```\n\n### Transactions and Long Computations\n\nWhen computations take a long time, there's a tension between keeping transactions short (to avoid blocking) and maintaining data integrity (ensuring inputs haven't changed).\n\nDataJoint's `make()` method handles this through a three-part pattern that refetches and verifies input data inside the transaction.\nSee {doc}`055-make` for details on implementing safe computations.\n\n### Avoid User Interaction Inside Transactions\n\nNever wait for user input while holding a transaction open:\n\n```python\n# Bad: User could take coffee break!\nwith conn.transaction:\n    Table.insert(data)\n    confirm = input(\"Confirm? (y/n)\")  # DON'T DO THIS\n    if confirm != 'y':\n        raise Exception(\"Cancelled\")\n```\n\n### Handle Deadlocks Gracefully\n\nWhen multiple transactions compete for the same resources, deadlocks can occur.\nMySQL automatically detects and resolves deadlocks by rolling back one transaction.\n\n```python\nimport time\n\ndef safe_operation(max_retries=3):\n    \"\"\"Execute operations with deadlock retry logic.\"\"\"\n    for attempt in range(max_retries):\n        try:\n            with dj.conn().transaction:\n                Table1.insert(data1)\n                Table2.insert(data2)\n            return  # Success\n        except Exception as e:\n            if \"Deadlock\" in str(e) and attempt < max_retries - 1:\n                time.sleep(0.1 * (2 ** attempt))  # Exponential backoff\n                continue\n            raise\n```"
   },
   {
    "cell_type": "markdown",
    "id": "cell-summary",
    "metadata": {},
-   "source": [
-    "## Summary\n",
-    "\n",
-    "Transactions are fundamental to maintaining data integrity in DataJoint:\n",
-    "\n",
-    "| Feature | Description |\n",
-    "|---------|-------------|\n",
-    "| `conn.transaction` | Context manager for explicit transactions |\n",
-    "| Automatic rollback | On exception, all changes are undone |\n",
-    "| `make()` wrapping | Computed table computations are automatically transactional |\n",
-    "| Master-part atomicity | Insert master and parts together safely |\n",
-    "\n",
-    "**Key takeaways:**\n",
-    "1. Use transactions when multiple operations must succeed or fail together\n",
-    "2. Keep transactions short to avoid blocking other users\n",
-    "3. DataJoint's `populate()` automatically handles transactions for you\n",
-    "4. Master-part relationships provide built-in transactional semantics"
-   ]
+   "source": "## Summary\n\nTransactions are fundamental to maintaining data integrity in DataJoint:\n\n| Feature | Description |\n|---------|-------------|\n| Single operations | Already atomic—no explicit transaction needed |\n| `conn.transaction` | Context manager for grouping multiple operations |\n| Automatic rollback | On exception, all changes are undone |\n| `make()` wrapping | Computed table computations are automatically transactional |\n| Master-part atomicity | Insert master and parts together safely |\n\n**Key takeaways:**\n1. Use explicit transactions only when multiple operations must succeed or fail together\n2. Single inserts and updates are already atomic\n3. DataJoint's `populate()` automatically handles transactions for you\n4. Master-part relationships provide built-in transactional semantics\n5. For long computations, see {doc}`055-make` for safe patterns"
   }
  ],
  "metadata": {