Improve getting started / testing guides for Humans and Agents

Author: alamb

AGENTS.md

@@ -2,33 +2,18 @@
 
 ## Developer Documentation
 
+- [Quick Start Setup](docs/source/contributor-guide/development_environment.md#quick-start)
+- [Testing Quick Start](docs/source/contributor-guide/testing.md#testing-quick-start)
+- [Before Submitting a PR](docs/source/contributor-guide/index.md#before-submitting-a-pr)
 - [Contributor Guide](docs/source/contributor-guide/index.md)
 - [Architecture Guide](docs/source/contributor-guide/architecture.md)
 
 ## Before Committing
 
-Before committing any changes, you **must** run the following checks and fix any issues:
-
-```bash
-cargo fmt --all
-cargo clippy --all-targets --all-features -- -D warnings
-```
-
-- `cargo fmt` ensures consistent code formatting across the project.
-- `cargo clippy` catches common mistakes and enforces idiomatic Rust patterns. All warnings must be resolved (treated as errors via `-D warnings`).
-
-Do not commit code that fails either of these checks.
+See [Before Submitting a PR](docs/source/contributor-guide/index.md#before-submitting-a-pr)
+for the required formatting and lint checks.
 
 ## Testing
 
-Run relevant tests before submitting changes:
-
-```bash
-cargo test --all-features
-```
-
-For SQL logic tests:
-
-```bash
-cargo test -p datafusion-sqllogictest
-```
+See the [Testing quick start](docs/source/contributor-guide/testing.md#testing-quick-start)
+for the recommended pre-PR test commands.

docs/source/contributor-guide/development_environment.md

@@ -21,7 +21,38 @@
 
 This section describes how you can get started at developing DataFusion.
 
-## Windows setup
+## Quick Start
+
+For the fastest path to a working local environment, follow these steps
+from the repository root:
+
+```shell
+# 1. Install Rust (https://rust-lang.org/tools/install/) and verify version with
+rustup show
+
+# 2. Install protoc 3.15+ (see details below)
+protoc --version
+
+# 3. Download test data used by examples and many tests
+git submodule update --init --recursive
+
+# 4. Build the workspace
+cargo build
+
+# 5. Verify that Rust integration tests can be run
+cargo test -p datafusion --test parquet_integration
+
+# 6. Verify that sqllogictests can run
+cargo test --profile=ci --test sqllogictests
+```
+
+Notes:
+
+- The pinned Rust version is defined in `rust-toolchain.toml`.
+- `protoc` is required to compile DataFusion from source.
+- Some tests and examples rely on git submodule data being present locally.
+
+## Windows Setup
 
 ```shell
 wget https://az792536.vo.msecnd.net/vms/VMBuild_20190311/VirtualBox/MSEdge/MSEdge.Win10.VirtualBox.zip
@@ -34,19 +65,19 @@ cargo build
 
 DataFusion has support for [dev containers](https://containers.dev/) which may be used for
 developing DataFusion in an isolated environment either locally or remote if desired. Using dev containers for developing
-DataFusion is not a requirement by any means but is available for those where doing local development could be tricky
+DataFusion is not a requirement but is available where doing local development could be tricky
 such as with Windows and WSL2, those with older hardware, etc.
 
 For specific details on IDE support for dev containers see the documentation for [Visual Studio Code](https://code.visualstudio.com/docs/devcontainers/containers),
 [IntelliJ IDEA](https://www.jetbrains.com/help/idea/connect-to-devcontainer.html),
 [Rust Rover](https://www.jetbrains.com/help/rust/connect-to-devcontainer.html), and
 [GitHub Codespaces](https://docs.github.com/en/codespaces/setting-up-your-project-for-codespaces/adding-a-dev-container-configuration/introduction-to-dev-containers).
 
-## Protoc Installation
+## `protoc` Installation
 
 Compiling DataFusion from sources requires an installed version of the protobuf compiler, `protoc`.
 
-On most platforms this can be installed from your system's package manager
+On most platforms this can be installed from your system's package manager. For example
 
 ```
 # Ubuntu
@@ -71,7 +102,7 @@ libprotoc 3.15.0
 
 Alternatively a binary release can be downloaded from the [Release Page](https://github.com/protocolbuffers/protobuf/releases) or [built from source](https://github.com/protocolbuffers/protobuf/blob/main/src/README.md).
 
-## Bootstrap environment
+## Bootstrap Environment
 
 DataFusion is written in Rust and it uses a standard rust toolkit:
 

docs/source/contributor-guide/index.md

@@ -32,8 +32,10 @@ community as well as get more familiar with Rust and the relevant codebases.
 
 ## Development Environment
 
-Setup your development environment [here](development_environment.md), and learn
-how to test the code [here](testing.md).
+Start with the [Development Environment Quick Start](development_environment.md#quick-start).
+
+For more detail, see the full [development environment guide](development_environment.md)
+and the [testing guide](testing.md).
 
 ## Finding and Creating Issues to Work On
 
@@ -99,6 +101,19 @@ If you are concerned that a larger design will be lost in a string of small PRs,
 
 Note all commits in a PR are squashed when merged to the `main` branch so there is one commit per PR after merge.
 
+## Before Submitting a PR
+
+Before submitting a PR, run the standard formatting and lint checks and fix any
+issues they report:
+
+```bash
+./ci/scripts/rust_fmt.sh
+./ci/scripts/rust_clippy.sh
+```
+
+These scripts are the same checks run in CI for Rust formatting and clippy.
+You should also run any relevant commands from the [testing quick start](testing.md#testing-quick-start).
+
 ## Conventional Commits & Labeling PRs
 
 We generate change logs for each release using an automated process that will categorize PRs based on the title

docs/source/contributor-guide/testing.md

@@ -23,6 +23,39 @@ Tests are critical to ensure that DataFusion is working properly and
 is not accidentally broken during refactorings. All new features
 should have test coverage and the entire test suite is run as part of CI.
 
+## Testing Quick Start
+
+While developing a feature or bug fix, best practice is to run the smallest set
+of tests that gives confidence for your change, then expand as needed.
+
+Initially, run the tests in the crates you changed. For example if you made changes
+to files in `datafusion-optimizer/src`, run the corresponding crate tests:
+
+```shell
+cargo test -p datafusion-optimizer
+```
+
+Then, run the `sqllogictest` suite, which is the main regression suite for SQL
+behavior and covers most DataFusion features.
+
+```shell
+# run sqllogictests
+cargo test --profile=ci --test sqllogictests
+```
+
+Finally, before submitting a PR, run the tests for the core `datafusion` and
+`datafusion-cli` crates
+
+```shell
+cargo test -p datafusion
+cargo test -p datafusion-cli
+```
+
+Some integration tests require optional external services such as Docker-backed
+containers and may skip when unavailable.
+
+## Testing Overview
+
 DataFusion has several levels of tests in its [Test Pyramid] and tries to follow
 the Rust standard [Testing Organization] described in [The Book].