feat(cli): Multi-Config Resolution by Env Var or pyproject.toml

Author: cofin

AGENTS.md

@@ -415,6 +415,46 @@ Dialect.classes["custom"] = CustomDialect
 - Use `wrap_exceptions` context manager in adapter layer
 - Two-tier pattern: graceful skip (DEBUG) for expected conditions, hard errors for malformed input
 
+### Click Environment Variable Pattern
+
+When adding CLI options that should support environment variables:
+
+**Use Click's native `envvar` parameter instead of custom parsing:**
+
+```python
+# Good - Click handles env var automatically
+@click.option(
+    "--config",
+    help="Dotted path to SQLSpec config(s) (env: SQLSPEC_CONFIG)",
+    required=False,
+    type=str,
+    envvar="SQLSPEC_CONFIG",  # Click handles precedence: CLI flag > env var
+)
+def command(config: str | None):
+    pass
+
+# Bad - Custom env var parsing
+import os
+
+@click.option("--config", required=False, type=str)
+def command(config: str | None):
+    if config is None:
+        config = os.getenv("SQLSPEC_CONFIG")  # Don't do this!
+```
+
+**Benefits:**
+- Click automatically handles precedence (CLI flag always overrides env var)
+- Help text automatically shows env var name
+- Support for multiple fallback env vars via `envvar=["VAR1", "VAR2"]`
+- Less code, fewer bugs
+
+**For project file discovery (pyproject.toml, etc.):**
+- Use custom logic as fallback after Click env var handling
+- Walk filesystem from cwd to find config files
+- Return `None` if not found to trigger helpful error message
+
+**Reference implementation:** `sqlspec/cli.py` (lines 26-65), `sqlspec/utils/config_discovery.py`
+
 ### CLI Sync/Async Dispatch Pattern
 
 When implementing CLI commands that support both sync and async database adapters:

docs/usage/cli.rst

@@ -204,17 +204,24 @@ that generate static completion files (system-wide bash, zsh completion director
 Available Commands
 ==================
 
-The SQLSpec CLI provides commands for managing database migrations. All commands
-require a ``--config`` option pointing to your SQLSpec configuration.
+The SQLSpec CLI provides commands for managing database migrations. Configure the CLI
+using ``--config`` flag, ``SQLSPEC_CONFIG`` environment variable, or ``[tool.sqlspec]``
+section in pyproject.toml.
 
 Configuration Loading
 ---------------------
 
-The ``--config`` option accepts a dotted path to either:
+SQLSpec CLI supports three ways to specify your database configuration, in order of precedence:
 
-1. **A single config object**: ``myapp.config.db_config``
-2. **A config list**: ``myapp.config.configs``
-3. **A callable function**: ``myapp.config.get_configs()``
+1. **CLI Flag** (``--config``) - Explicit override for one-off commands
+2. **Environment Variable** (``SQLSPEC_CONFIG``) - Convenient for development workflows
+3. **pyproject.toml** (``[tool.sqlspec]``) - Project-wide default configuration
+
+The ``--config`` option and ``SQLSPEC_CONFIG`` environment variable accept a dotted path to either:
+
+- **A single config object**: ``myapp.config.db_config``
+- **A config list**: ``myapp.config.configs``
+- **A callable function**: ``myapp.config.get_configs()``
 
 Example configuration file (``myapp/config.py``):
 
@@ -225,11 +232,66 @@ Example configuration file (``myapp/config.py``):
    :end-before: # end-example
    :caption: `configuration loading`
 
+Config Discovery Methods
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+**Method 1: CLI Flag (Highest Priority)**
+
+.. code-block:: bash
+
+   sqlspec --config myapp.config:get_configs upgrade head
+
+Use for one-off commands or to override other config sources.
+
+**Method 2: Environment Variable**
+
+.. code-block:: bash
+
+   export SQLSPEC_CONFIG=myapp.config:get_configs
+   sqlspec upgrade head  # Uses environment variable
+
+Convenient for development. Add to your shell profile:
+
+.. code-block:: bash
+
+   # ~/.bashrc or ~/.zshrc
+   export SQLSPEC_CONFIG=myapp.config:get_configs
+
+Multiple configs (comma-separated):
+
+.. code-block:: bash
+
+   export SQLSPEC_CONFIG="app.db:primary_config,app.db:analytics_config"
+
+**Method 3: pyproject.toml (Project Default)**
+
+.. code-block:: toml
+
+   [tool.sqlspec]
+   config = "myapp.config:get_configs"
+
+Best for team projects - config is version controlled.
+
+Multiple configs (array):
+
+.. code-block:: toml
+
+   [tool.sqlspec]
+   config = [
+       "myapp.config:primary_config",
+       "myapp.config:analytics_config"
+   ]
+
+**Precedence:** CLI flag > Environment variable > pyproject.toml
+
+If no config is found from any source, SQLSpec will show a helpful error message with examples.
+
 Global Options
 --------------
 
 ``--config PATH``
-   **Required**. Dotted path to SQLSpec config(s) or callable function.
+   Dotted path to SQLSpec config(s) or callable function. Optional when using
+   environment variable or pyproject.toml config discovery.
 
    Example: ``--config myapp.config.get_configs``
 

pyproject.toml

@@ -25,7 +25,7 @@ asyncmy = ["asyncmy"]
 asyncpg = ["asyncpg"]
 attrs = ["attrs", "cattrs"]
 bigquery = ["google-cloud-bigquery", "google-cloud-storage"]
-cli = ["rich-click"]
+cli = ["rich-click", "tomli>=2.0.0; python_version < '3.11'"]
 cloud-sql = ["cloud-sql-python-connector"]
 duckdb = ["duckdb"]
 fastapi = ["fastapi"]

sqlspec/cli.py

@@ -26,24 +26,44 @@ def get_sqlspec_group() -> "Group":
     @click.group(name="sqlspec")
     @click.option(
         "--config",
-        help="Dotted path to SQLSpec config(s) or callable function (e.g. 'myapp.config.get_configs')",
-        required=True,
+        help="Dotted path to SQLSpec config(s) or callable function (env: SQLSPEC_CONFIG)",
+        required=False,
+        default=None,
         type=str,
+        envvar="SQLSPEC_CONFIG",
     )
     @click.option(
         "--validate-config", is_flag=True, default=False, help="Validate configuration before executing migrations"
     )
     @click.pass_context
-    def sqlspec_group(ctx: "click.Context", config: str, validate_config: bool) -> None:
+    def sqlspec_group(ctx: "click.Context", config: str | None, validate_config: bool) -> None:
         """SQLSpec CLI commands."""
         from rich import get_console
 
         from sqlspec.exceptions import ConfigResolverError
+        from sqlspec.utils.config_discovery import discover_config_from_pyproject
         from sqlspec.utils.config_resolver import resolve_config_sync
 
         console = get_console()
         ctx.ensure_object(dict)
 
+        # Click already handled: CLI flag > SQLSPEC_CONFIG env var
+        # Now check pyproject.toml as final fallback
+        if config is None:
+            config = discover_config_from_pyproject()
+            if config:
+                console.print("[dim]Using config from pyproject.toml[/]")
+
+        # No config from any source - show helpful error
+        if config is None:
+            console.print("[red]Error: No SQLSpec config found.[/]")
+            console.print("\nSpecify config using one of:")
+            console.print("  1. CLI flag:        sqlspec --config myapp.config:get_configs <command>")
+            console.print("  2. Environment var: export SQLSPEC_CONFIG=myapp.config:get_configs")
+            console.print("  3. pyproject.toml:  [tool.sqlspec]")
+            console.print('                      config = "myapp.config:get_configs"')
+            ctx.exit(1)
+
         # Add current working directory to sys.path to allow loading local config modules
         cwd = str(Path.cwd())
         cwd_added = False
@@ -52,11 +72,34 @@ def sqlspec_group(ctx: "click.Context", config: str, validate_config: bool) -> N
             cwd_added = True
 
         try:
-            config_result = resolve_config_sync(config)
-            if isinstance(config_result, Sequence) and not isinstance(config_result, str):
-                ctx.obj["configs"] = list(config_result)
-            else:
-                ctx.obj["configs"] = [config_result]
+            # Split comma-separated config paths and resolve each
+            all_configs: list[AsyncDatabaseConfig[Any, Any, Any] | SyncDatabaseConfig[Any, Any, Any]] = []
+            for config_path in config.split(","):
+                config_path = config_path.strip()
+                if not config_path:
+                    continue
+                config_result = resolve_config_sync(config_path)
+                if isinstance(config_result, Sequence) and not isinstance(config_result, str):
+                    all_configs.extend(config_result)
+                else:
+                    all_configs.append(
+                        cast("AsyncDatabaseConfig[Any, Any, Any] | SyncDatabaseConfig[Any, Any, Any]", config_result)
+                    )
+
+            # Deduplicate by bind_key (later configs override earlier ones)
+            configs_by_key: dict[
+                str | None, AsyncDatabaseConfig[Any, Any, Any] | SyncDatabaseConfig[Any, Any, Any]
+            ] = {}
+            for cfg in all_configs:
+                configs_by_key[cfg.bind_key] = cfg
+
+            ctx.obj["configs"] = list(configs_by_key.values())
+
+            # Check for empty configs after resolution
+            if not ctx.obj["configs"]:
+                console.print("[red]Error: No valid configs found after resolution.[/]")
+                console.print("\nEnsure your config path returns valid config instance(s).")
+                ctx.exit(1)
 
             ctx.obj["validate_config"] = validate_config
 

sqlspec/utils/__init__.py

@@ -7,6 +7,7 @@
 """
 
 from sqlspec.utils import (
+    config_discovery,
     deprecation,
     fixtures,
     logging,
@@ -19,6 +20,7 @@
 )
 
 __all__ = (
+    "config_discovery",
     "deprecation",
     "fixtures",
     "logging",

sqlspec/utils/config_discovery.py

@@ -0,0 +1,108 @@
+"""Config discovery for SQLSpec CLI.
+
+Provides pyproject.toml config discovery for CLI convenience.
+Environment variable support is handled natively by Click via envvar parameter.
+"""
+
+import sys
+from pathlib import Path
+
+if sys.version_info >= (3, 11):
+    import tomllib
+else:
+    import tomli as tomllib
+
+
+__all__ = ("discover_config_from_pyproject", "find_pyproject_toml", "parse_pyproject_config")
+
+
+def discover_config_from_pyproject() -> str | None:
+    """Find and parse pyproject.toml for SQLSpec config.
+
+    Walks filesystem upward from current directory to find pyproject.toml.
+    Parses [tool.sqlspec] section for 'config' key.
+
+    Returns:
+        Config path(s) as string (comma-separated if list), or None if not found.
+    """
+    pyproject_path = find_pyproject_toml()
+    if pyproject_path is None:
+        return None
+
+    return parse_pyproject_config(pyproject_path)
+
+
+def find_pyproject_toml() -> "Path | None":
+    """Walk filesystem upward to find pyproject.toml.
+
+    Starts from current working directory and walks up to filesystem root.
+    Stops at .git directory boundary (repository root) if found.
+
+    Returns:
+        Path to pyproject.toml, or None if not found.
+    """
+    current = Path.cwd()
+
+    while True:
+        pyproject = current / "pyproject.toml"
+        if pyproject.exists():
+            return pyproject
+
+        # Stop at .git boundary (repository root)
+        if (current / ".git").exists():
+            return None
+
+        # Stop at filesystem root
+        if current == current.parent:
+            return None
+
+        current = current.parent
+
+
+def parse_pyproject_config(pyproject_path: "Path") -> str | None:
+    """Parse pyproject.toml for [tool.sqlspec] config.
+
+    Args:
+        pyproject_path: Path to pyproject.toml file.
+
+    Returns:
+        Config path(s) as string (converts list to comma-separated), or None if not found.
+
+    Raises:
+        ValueError: If [tool.sqlspec].config has invalid type (not str or list[str]).
+    """
+    try:
+        with pyproject_path.open("rb") as f:
+            data = tomllib.load(f)
+    except Exception as e:
+        msg = f"Failed to parse {pyproject_path}: {e}"
+        raise ValueError(msg) from e
+
+    # Navigate to [tool.sqlspec] section
+    tool_section = data.get("tool", {})
+    if not isinstance(tool_section, dict):
+        return None
+
+    sqlspec_section = tool_section.get("sqlspec", {})
+    if not isinstance(sqlspec_section, dict):
+        return None
+
+    # Extract config value
+    config = sqlspec_section.get("config")
+    if config is None:
+        return None
+
+    # Handle string config
+    if isinstance(config, str):
+        return config
+
+    # Handle list config (convert to comma-separated)
+    if isinstance(config, list):
+        if not all(isinstance(item, str) for item in config):
+            msg = f"Invalid [tool.sqlspec].config in {pyproject_path}: list items must be strings"
+            raise ValueError(msg)
+        return ",".join(config)
+
+    # Invalid type
+    msg = f"Invalid [tool.sqlspec].config in {pyproject_path}: must be string or list of strings, got {type(config).__name__}"
+    raise ValueError(msg)