ready to ship 0.2.0

Author: lzjever

CHANGELOG.md

@@ -5,6 +5,35 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.2.0] - 2025-01-02
+
+### Added
+- **Explicit Required/Optional Configuration**: All fields must explicitly specify `required=True` or `optional=True` in metadata. No inference allowed.
+- **Automatic Model Defaults**: Model defaults are automatically applied as base layer. No need to explicitly include `sources.Defaults` in sources list.
+- **Model-Driven Source Filtering**: All sources (Env, CLI, DotEnv, Etcd) now filter variables/arguments based on model fields. Only model-defined fields are loaded.
+- **Required Field Validation**: New `Config.validate()` method to validate required fields independently. `Config.load()` now has optional `validate` parameter.
+- **Comprehensive Error Messages**: When required fields are missing, error messages include:
+  - List of missing fields with descriptions
+  - Source mapping rules and examples for each active source
+  - Actionable guidance on how to provide missing parameters
+- **Field Metadata Support**: Support for `description` and `help` in field metadata for better documentation and CLI help text.
+- **New Modules**:
+  - `varlord.metadata`: Field information extraction and utilities
+  - `varlord.validation`: Model definition validation and configuration validation
+  - `varlord.source_help`: Source mapping examples and error message formatting
+
+### Changed
+- **BREAKING**: `Env` source no longer accepts `prefix` parameter. All environment variables are filtered by model fields.
+- **BREAKING**: All fields must have explicit `required` or `optional` metadata. `ModelDefinitionError` is raised if missing.
+- **BREAKING**: `Config.from_model()` no longer accepts `env_prefix` parameter.
+- **BREAKING**: `Defaults` source is now internal. Model defaults are automatically applied, no need to include in sources list.
+- **BREAKING**: Empty strings and empty collections are now considered valid values for required fields (only presence is checked, not emptiness).
+
+### Fixed
+- Improved error messages for missing required fields
+- Better CLI help text with field descriptions
+- Consistent model filtering across all sources
+
 ## [0.1.0] - 2025-12-31
 
 ### Added

LICENSE

@@ -11,5 +11,6 @@ Complete API documentation for all Varlord classes and functions.
    sources
    policy
    validators
+   model_validation
    logging
 

docs/source/api_reference/index.rst

@@ -0,0 +1,63 @@
+Model Validation
+================
+
+This module provides validation functions for model definitions and configuration structure.
+
+.. automodule:: varlord.model_validation
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Functions
+---------
+
+**validate_model_definition**
+   Validates that all fields in a dataclass model have explicit required/optional metadata.
+
+   Example:
+
+   .. code-block:: python
+
+      from varlord.model_validation import validate_model_definition
+      from dataclasses import dataclass, field
+
+      @dataclass
+      class Config:
+          api_key: str = field(metadata={"required": True})  # OK
+          host: str = field()  # Missing metadata - will raise ModelDefinitionError
+
+      validate_model_definition(Config)  # Raises ModelDefinitionError
+
+**validate_config**
+   Validates that all required fields exist in a configuration dictionary.
+
+   Example:
+
+   .. code-block:: python
+
+      from varlord.model_validation import validate_config, RequiredFieldError
+      from dataclasses import dataclass, field
+
+      @dataclass
+      class Config:
+          api_key: str = field(metadata={"required": True})
+
+      config_dict = {}  # Missing api_key
+      try:
+          validate_config(Config, config_dict, [])
+      except RequiredFieldError as e:
+          print(e)  # Shows missing fields and source help
+
+Exceptions
+----------
+
+**VarlordError**
+   Base exception for all varlord errors.
+
+**ModelDefinitionError**
+   Raised when a field is missing required/optional metadata in the model definition.
+
+**RequiredFieldError**
+   Raised when required fields are missing from the configuration dictionary.
+   Includes comprehensive error messages with source mapping help.
+

docs/source/api_reference/model_validation.rst

@@ -55,4 +55,4 @@ The validators are organized into the following categories:
 
 **Custom Validators**
    - :func:`validate_custom` - Validate using custom function
-   - :func:`validate_config` - Validate configuration object
+   - :func:`apply_validators` - Apply validators to configuration object

docs/source/api_reference/validators.rst

@@ -18,14 +18,14 @@ Basic Usage
 
 .. code-block:: python
 
-   from dataclasses import dataclass
+   from dataclasses import dataclass, field
    from varlord import Config, sources
 
    @dataclass(frozen=True)
    class AppConfig:
-       host: str = "127.0.0.1"
-       port: int = 8000
-       debug: bool = False
+       host: str = field(default="127.0.0.1", metadata={"optional": True})
+       port: int = field(default=8000, metadata={"optional": True})
+       debug: bool = field(default=False, metadata={"optional": True})
 
 **Step 2: Create and load configuration**
 
@@ -34,8 +34,7 @@ Basic Usage
    cfg = Config(
        model=AppConfig,
        sources=[
-           sources.Defaults(model=AppConfig),
-           sources.Env(prefix="APP_"),
+           sources.Env(),  # Model defaults applied automatically
            sources.CLI(),  # Model auto-injected
        ],
    )
@@ -49,8 +48,7 @@ Basic Usage
 
    cfg = Config.from_model(
        AppConfig,
-       env_prefix="APP_",
-       cli=True,
+       cli=True,  # env_prefix removed - all env vars filtered by model
    )
 
    app = cfg.load()
@@ -65,8 +63,7 @@ Priority is determined by sources order (later sources override earlier ones):
    cfg = Config(
        model=AppConfig,
        sources=[
-           sources.Defaults(model=AppConfig),  # Lowest priority
-           sources.Env(prefix="APP_"),
+           sources.Env(),  # Model defaults applied first, then env
            sources.CLI(),  # Highest priority (last)
        ],
    )
@@ -134,7 +131,6 @@ Use ConfigStore for dynamic configuration updates:
    cfg = Config(
        model=AppConfig,
        sources=[
-           sources.Defaults(model=AppConfig),
            sources.Etcd(..., watch=True),  # Enable watch here
        ],
    )

docs/source/quickstart.rst

@@ -23,19 +23,19 @@ Sometimes you need different priority orders for different keys. Use
    :linenos:
 
    import os
-   from dataclasses import dataclass
+   from dataclasses import dataclass, field
    from varlord import Config, sources, PriorityPolicy
 
    @dataclass(frozen=True)
    class AppConfig:
-       host: str = "0.0.0.0"
-       port: int = 8000
-       api_key: str = "default-key"
+       host: str = field(default="0.0.0.0", metadata={"optional": True})
+       port: int = field(default=8000, metadata={"optional": True})
+       api_key: str = field(default="default-key", metadata={"optional": True})
 
-   # Set environment variables
-   os.environ["APP_HOST"] = "env-host"
-   os.environ["APP_PORT"] = "9000"
-   os.environ["APP_API_KEY"] = "env-key"
+   # Set environment variables (no prefix needed)
+   os.environ["HOST"] = "env-host"
+   os.environ["PORT"] = "9000"
+   os.environ["API_KEY"] = "env-key"
 
    # Define priority policy
    policy = PriorityPolicy(
@@ -49,8 +49,7 @@ Sometimes you need different priority orders for different keys. Use
    cfg = Config(
        model=AppConfig,
        sources=[
-           sources.Defaults(model=AppConfig),
-           sources.Env(prefix="APP_"),
+           sources.Env(),  # Defaults applied automatically
        ],
        policy=policy,
    )
@@ -112,8 +111,8 @@ You can create custom sources by extending the ``Source`` base class:
 
    @dataclass(frozen=True)
    class AppConfig:
-       host: str = "0.0.0.0"
-       port: int = 8000
+       host: str = field(default="0.0.0.0", metadata={"optional": True})
+       port: int = field(default=8000, metadata={"optional": True})
 
    # Create JSON file
    import tempfile
@@ -125,8 +124,7 @@ You can create custom sources by extending the ``Source`` base class:
    cfg = Config(
        model=AppConfig,
        sources=[
-           sources.Defaults(model=AppConfig),
-           JSONFileSource(json_path),
+           JSONFileSource(json_path),  # Defaults applied automatically
        ],
    )
 
@@ -222,18 +220,18 @@ Here are some best practices for complex scenarios:
 
    @dataclass(frozen=True)
    class DatabaseConfig:
-       host: str = "localhost"
-       port: int = 5432
+       host: str = field(default="localhost", metadata={"optional": True})
+       port: int = field(default=5432, metadata={"optional": True})
 
    @dataclass(frozen=True)
    class CacheConfig:
-       host: str = "localhost"
-       port: int = 6379
+       host: str = field(default="localhost", metadata={"optional": True})
+       port: int = field(default=6379, metadata={"optional": True})
 
    @dataclass(frozen=True)
    class AppConfig:
-       db: DatabaseConfig = field(default_factory=lambda: DatabaseConfig())
-       cache: CacheConfig = field(default_factory=lambda: CacheConfig())
+       db: DatabaseConfig = field(default_factory=lambda: DatabaseConfig(), metadata={"optional": True})
+       cache: CacheConfig = field(default_factory=lambda: CacheConfig(), metadata={"optional": True})
 
 **2. Use Environment-Specific Defaults**
 
@@ -244,19 +242,20 @@ Here are some best practices for complex scenarios:
 
    @dataclass(frozen=True)
    class AppConfig:
-       debug: bool = os.getenv("ENV") != "production"
-       log_level: str = "DEBUG" if os.getenv("ENV") != "production" else "INFO"
+       debug: bool = field(default=os.getenv("ENV") != "production", metadata={"optional": True})
+       log_level: str = field(default="DEBUG" if os.getenv("ENV") != "production" else "INFO", metadata={"optional": True})
 
 **3. Validate Critical Fields**
 
 .. code-block:: python
    :linenos:
 
+   from dataclasses import field
    from varlord.validators import validate_not_empty, validate_port
 
    @dataclass(frozen=True)
    class AppConfig:
-       api_key: str = ""
+       api_key: str = field(default="", metadata={"optional": True})
 
        def __post_init__(self):
            validate_not_empty(self.api_key)  # Fail fast if missing
@@ -277,28 +276,28 @@ Here's a complete example combining multiple advanced features:
 
    @dataclass(frozen=True)
    class DBConfig:
-       host: str = "localhost"
-       port: int = 5432
+       host: str = field(default="localhost", metadata={"optional": True})
+       port: int = field(default=5432, metadata={"optional": True})
 
        def __post_init__(self):
            validate_not_empty(self.host)
            validate_port(self.port)
 
    @dataclass(frozen=True)
    class AppConfig:
-       host: str = "0.0.0.0"
-       port: int = 8000
-       db: DBConfig = field(default_factory=lambda: DBConfig())
+       host: str = field(default="0.0.0.0", metadata={"optional": True})
+       port: int = field(default=8000, metadata={"optional": True})
+       db: DBConfig = field(default_factory=lambda: DBConfig(), metadata={"optional": True})
 
        def __post_init__(self):
            validate_port(self.port)
 
    def main():
-       # Set environment variables
-       os.environ["APP_HOST"] = "0.0.0.0"
-       os.environ["APP_PORT"] = "9000"
-       os.environ["APP_DB__HOST"] = "db.example.com"
-       os.environ["APP_DB__PORT"] = "3306"
+       # Set environment variables (no prefix needed)
+       os.environ["HOST"] = "0.0.0.0"
+       os.environ["PORT"] = "9000"
+       os.environ["DB__HOST"] = "db.example.com"
+       os.environ["DB__PORT"] = "3306"
 
        # Use PriorityPolicy for fine-grained control
        policy = PriorityPolicy(
@@ -311,8 +310,7 @@ Here's a complete example combining multiple advanced features:
        cfg = Config(
            model=AppConfig,
            sources=[
-               sources.Defaults(model=AppConfig),
-               sources.Env(prefix="APP_", separator="__"),
+               sources.Env(),  # Defaults applied automatically
            ],
            policy=policy,
        )