docs(rust): add cloudfetch pipeline redesign spec and sprint plan

[eric-wang-1990]

Summary

• Add CloudFetch pipeline redesign spec with channel-based architecture design
• Add sprint plan with task breakdown and DoD checklist
• Update spec with channel design details (bounded result channel, unbounded download channel)

This is a design-only PR with no code changes.

Stacked PRs

• [stack/task-0-cloudfetch-design](this PR) — design docs
  • stack/task-1-config-types-and-pipeline-types — impl: config + pipeline types
    • stack/task-2-scheduler-workers-consumer-rewrite — impl: scheduler, workers, consumer

🤖 Generated with Claude Code

[eric-wang-1990]

Why 401/403/404 counts against max_retries? It should only counts to max_refresh_retries? What is CSharp ADBC behavior?

[eric-wang-1990]

✅ Fixed — updated the retry table so 401/403/404 counts only against max_refresh_retries, not max_retries.

C# behavior for reference: In the C# driver, 401/403 inadvertently increments the max_retries loop counter because the continue statement advances the for (int retry = 0; retry < _maxRetries; retry++) variable — but the explicit guard is RefreshAttempts >= _maxUrlRefreshAttempts. 404 is not specially handled in C# and falls through to EnsureSuccessStatusCode(), which throws and gets caught with a sleep delay.

For the Rust redesign we cleanly separate the two counters: 401/403/404 exhausting max_refresh_retries is a distinct failure mode from transient network errors exhausting max_retries. Added a note in the spec documenting the C# divergence.

---

This comment was generated with GitHub MCP.

[eric-wang-1990]

✅ Also updated the design spec (cloudfetch-pipeline-redesign.md) retry table — 401/403/404 now shows No for max_retries column, matching the sprint plan fix.

---

This comment was generated with GitHub MCP.

[jadewang-db]

the new bazel command does not need a planning parameter anymore in fact it read the design doc for the list of PRs and commits

[vikrantpuppala]

my biggest concern is wrt to the change in defaults for parallel downloads, max_chunks_in_memory, etc. these were configured inline with oss jdbc which has generally been the benchmark in terms of performance. to make sure these new defaults do not cause regressions in latencies, we should also perform benchmarking - go/oss-odbc-perf

[eric-wang-1990]

@vikrantpuppala Good call — updated both spec files to keep the original defaults aligned with OSS JDBC:

• max_retries: stays 5 (was going to change to 3)
• retry_delay: stays 1500ms (was going to change to 500ms)
• max_chunks_in_memory: stays 16 (was going to change to 10)

C# equivalents are noted for reference only. This avoids any perf regression risk. We can tune later with benchmarking data from the dashboard you linked.

---

This comment was generated with GitHub MCP.

[eric-wang-1990]

@jadewang-db Noted — thanks for the heads up on the bazel command change.

---

This comment was generated with GitHub MCP.