lazy settings evaluation with environment variables & with_full_overwrite (#26)

Changes:
- allow lazy evaluating env variables
- Allow unsetting settings via with_settings by using False or "".
- docs
- ruff formatting
- bump version

Author: devkral

.pre-commit-config.yaml

@@ -13,7 +13,7 @@ repos:
       - id: end-of-file-fixer
       - id: trailing-whitespace
   - repo: https://github.com/charliermarsh/ruff-pre-commit
-    rev: v0.7.3
+    rev: v0.11.5
     hooks:
       - id: ruff
         args: ["--fix"]

docs/release-notes.md

@@ -5,6 +5,19 @@ hide:
 
 # Release notes
 
+## Version 0.4.0
+
+### Added
+
+- Add `with_full_overwrite` helper method which sets multiple contexts.
+- Add `evaluate_settings_with` parameter to `with_settings`.
+
+### Changed
+
+- When string or class is provided by a callable for settings it is parsed and cached.
+  This allows lazy parsing of environment variables, so they can be changed programmatically.
+- Allow unsetting settings via with_settings by using False or "".
+
 ## Version 0.3.0
 
 ### Breaking

docs/settings.md

@@ -18,26 +18,13 @@ settings object.
 ### Example: Child Package
 
 ```python
-import os
-from monkay import Monkay
-
-monkay = Monkay(
-    globals(),
-    settings_path=os.environ.get("MONKAY_CHILD_SETTINGS", "foo.test:example") or ""
-)
+{!> ../docs_src/settings/forwarding_child.py !}
 ```
 
 ### Example: Main Package
 
 ```python
-import os
-import child
-
-monkay = Monkay(
-    globals(),
-    settings_path=os.environ.get("MONKAY_MAIN_SETTINGS", "foo.test:example") or ""
-)
-child.monkay.settings = lambda: monkay.settings
+{!> ../docs_src/settings/forwarding_main.py !}
 ```
 
 With this setup, the child package will use the settings from the main package, ensuring that all configurations
@@ -53,29 +40,11 @@ evaluation of settings until later in the application lifecycle.
 ### Example:
 
 ```python
-import os
-from monkay import Monkay
-
-monkay = Monkay(
-    globals(),
-    # Required for initializing settings feature
-    settings_path=""
-)
-
-# Lazy setup based on environment variables
-if not os.environ.get("DEBUG"):
-    monkay.settings = os.environ.get("MONKAY_MAIN_SETTINGS", "foo.test:example") or ""
-elif os.environ.get("PERFORMANCE"):
-    monkay.settings = DebugSettings
-else:
-    monkay.settings = DebugSettings()
-
-# Now the settings are applied
-monkay.evaluate_settings()
+{!> ../docs_src/settings/lazy_loader.py !}
 ```
 
 This approach allows for flexible configuration of the application, based on the environment, while deferring the
-actual evaluation of settings.
+actual evaluation of settings and os.environ.
 
 ---
 
@@ -88,19 +57,7 @@ and handle errors more gracefully.
 ### Example:
 
 ```python
-import os
-from monkay import Monkay
-
-monkay = Monkay(
-    globals(),
-    # Required for initializing settings feature
-    settings_path=""
-)
-
-def find_settings():
-    for path in ["a.settings", "b.settings.develop"]:
-        if monkay.evaluate_settings(ignore_import_errors=True):
-            break
+{!> ../docs_src/settings/multi_stage.py !}
 ```
 
 In this example, **Monkay** tries to evaluate settings from both `a.settings` and `b.settings.develop`, ignoring
@@ -147,16 +104,11 @@ It is reset when new settings are assigned and is initially set to `False` for i
 **Monkay** supports various ways of assigning settings:
 
 - **String or Class**: Initialization happens the first time the settings are accessed and are cached afterward.
-- **Function**: The function gets evaluated each time the settings are accessed. If caching is required,
-- it is up to the function to handle it (e.g., forwarding settings can rely on caching in the main settings).
+- **Function**: The function gets evaluated each time the settings are accessed when returning an instance (useful for forwarding).
+  Otherwise for types like str and class, the result is cached and used instead until the settings cache is cleared.
 
 You can also use the `settings_path` parameter to assign a settings location directly, using either a string or
-class reference.
-
-### Caching Behavior:
-- **String or Class**: These types are cached after the first evaluation.
-- **Function**: Functions are re-evaluated on every access. If needed, the caching mechanism can be handled within
-the function (e.g., caching results in the main settings).
+class reference. The caching behavior is the same.
 
 ---
 
@@ -166,20 +118,7 @@ Sometimes, you may need to forward old settings to the **Monkay** settings. Whil
 creating a forwarder is easy. Here's an example:
 
 ```python
-from typing import Any, cast, TYPE_CHECKING
-
-if TYPE_CHECKING:
-    from .global_settings import EdgySettings
-
-class SettingsForward:
-    def __getattribute__(self, name: str) -> Any:
-        import edgy
-        return getattr(edgy.monkay.settings, name)
-
-# Pretend the forward is the real object
-settings = cast("EdgySettings", SettingsForward())
-
-__all__ = ["settings"]
+{!> ../docs_src/settings/forwarder.py !}
 ```
 
 ### Note:

docs/testing.md

@@ -15,7 +15,7 @@ For testing purposes, **Monkay** provides three context manager methods that all
 
 ### Available Context Managers:
 
-1. **`with_settings(settings)`**: Temporarily overwrites the settings for the scope of the context.
+1. **`with_settings(settings, *, evaluate_settings_with=None)`**: Temporarily overwrites the settings for the scope of the context. Optionally evaluate settings with provided parameters as dict. Enable evaluation by providing a dict.
 2. **`with_extensions(extensions_dict, *, apply_extensions=False)`**: Temporarily overwrites the extensions for the scope. The `apply_extensions` flag controls whether extensions are applied during the overwrite.
 3. **`with_instance(instance, *, apply_extensions=False, use_extensions_overwrite=True)`**: Temporarily overwrites the instance for the scope. You can also apply extensions and control the overwrite behavior using the respective flags.
 
@@ -26,10 +26,10 @@ These context managers provide a flexible way to test different configurations o
 ```python
 from monkay import Monkay
 
-monkay = Monkay(globals())
+monkay = Monkay(globals(), settings_path="", with_extensions=True, with_instance=True)
 
-# Overwrite settings temporarily
-with monkay.with_settings({"setting_name": "new_value"}):
+# Overwrite settings temporarily and evaluate it
+with monkay.with_settings({"setting_name": "new_value"}, evaluate_settings_with={}):
     assert monkay.settings.setting_name == "new_value"
 
 # Overwrite extensions temporarily
@@ -43,6 +43,38 @@ with monkay.with_instance(new_instance, apply_extensions=False):
 
 These context managers are especially useful for writing isolated and repeatable tests, where you may want to modify the environment without affecting the global state.
 
+### Full overwrite
+
+If multiple attributes should be simulated, e.g. for a virtual monkay environment, you can use
+`with_full_overwrite`. The advantage: you have already the right order set.
+Note: when not enabling a feature in monkay it will cause an error to provide a parameter for it.
+
+#### Parameters
+
+Parameters which need a feature enabled when creating the monkay instance:
+
+**extensions** - Set an extra extensions set like with `with_extensions`. You might want to set it to {} for a clean extensions set.
+**settings** - Overwrite the settings if specified. It types matches the `with_settings` contextmanager.
+**instance** - Overwrite the instance if specified. It types matches the `with_instance` contextmanager.
+
+Extra parameters:
+
+- **apply_extensions** - Apply extensions. Only used when `extensions` parameter is used.
+- **evaluate_settings_with** - Pass options to `with_settings` parameter `evaluate_settings_with`. Only used when `settings` parameter is used.
+  To enable evaluation pass `{}`
+
+#### Example:
+
+```python
+from monkay import Monkay
+
+monkay = Monkay(globals(), settings_path="", with_extensions=True, with_instance=True)
+
+# Overwrite everything temporarily and use a clean extensions set
+with monkay.with_full_overwrite(extensions={}, settings=..., instance=..., evaluate_settings_with={}):
+    assert monkay.instance is new_instance
+```
+
 ---
 
 ## Check Imports and Exports

docs/tutorial.md

@@ -26,49 +26,18 @@ pip install monkay
 Below is an example of how to set up **Monkay** in your project. You can use **Monkay** to manage dynamic imports, lazy loading, settings, extensions, and more.
 
 ```python title="foo/__init__.py"
-from monkay import Monkay
-
-monkay = Monkay(
-    # Required for auto-hooking
-    globals(),
-    with_extensions=True,
-    with_instance=True,
-    settings_path="settings_path:Settings",
-    preloads=["tests.targets.module_full_preloaded1:load"],
-    # Warning: settings names have a catch
-    settings_preloads_name="preloads",
-    settings_extensions_name="extensions",
-    uncached_imports=["settings"],
-    lazy_imports={
-        "bar": "tests.targets.fn_module:bar",
-        "settings": lambda: monkay.settings,
-    },
-    deprecated_lazy_imports={
-        "deprecated": {
-            "path": "tests.targets.fn_module:deprecated",
-            "reason": "old.",
-            "new_attribute": "super_new",
-        }
-    },
-)
+{!>../docs_src/tutorial/full_example_init.py}
 ```
 
 This configuration sets up **Monkay** with several features:
 - **Lazy imports** for `bar` and `settings`.
+- **Lazy evaluated settings_path** for being able to update the environment variable in code.
 - **Deprecated lazy imports** for `deprecated`.
 - **Preloads** and **extensions** for dynamic configuration.
 - **Uncached imports** to prevent caching specific imports like settings.
 
 ```python title="foo/main.py"
-from foo import monkay
-
-def get_application():
-    # sys.path updates
-    important_preloads = [...]
-    monkay.evaluate_preloads(important_preloads, ignore_import_errors=False)
-    extra_preloads = [...]
-    monkay.evaluate_preloads(extra_preloads)
-    monkay.evaluate_settings()
+{!>../docs_src/tutorial/full_example_main.py}
 ```
 
 In `main.py`, the application is initialized by evaluating preloads and settings, ensuring that all required dependencies are loaded before use.
@@ -79,7 +48,7 @@ In `main.py`, the application is initialized by evaluating preloads and settings
 
 After providing **Monkay**, if you need more control over the `__all__` variable, you can disable the automatic update of `__all__` by setting `skip_all_update=True`. You can later update it manually using `Monkay.update_all_var`.
 
-**Warning**: Using `settings_preloads_name` or `settings_extensions_name` can sometimes cause circular dependency issues. To avoid such issues, ensure that you call `evaluate_settings()` later in the setup process. For more information, refer to [Settings Preloads and Extensions](#settings-preloads-andextensions).
+**Warning**: Using `settings_preloads_name` or `settings_extensions_name` can sometimes cause circular dependency issues. To avoid such issues, ensure that you call `evaluate_settings()` later in the setup process. For more information, refer to [Settings Preloads and Extensions](#settings-extensions-and-preloads).
 
 ---
 
@@ -136,15 +105,16 @@ monkay = Monkay(
     globals(),
     with_extensions=True,
     with_instance=True,
-    settings_path=os.environ.get("MONKAY_SETTINGS", "example.default.path.settings:Settings"),
+    settings_path=lambda: os.environ.get("MONKAY_SETTINGS", "example.default.path.settings:Settings"),
     settings_preloads_name="preloads",
     settings_extensions_name="extensions",
     uncached_imports=["settings"],
     lazy_imports={"settings": lambda: monkay.settings}
 )
 ```
 
-Here, the `settings_path` is determined by an environment variable, and **Monkay** will use `settings` as the main settings object.
+Here, the `settings_path` is determined by an environment variable when accessed, and **Monkay** will use `settings` as the main settings object.
+The object will be cached.
 
 ```python title="settings.py"
 from pydantic_settings import BaseSettings
@@ -238,7 +208,7 @@ monkay = Monkay(
     globals(),
     with_extensions=True,
     with_instance=True,
-    settings_path=os.environ.get("MONKAY_SETTINGS", "example.default.path.settings:settings"),
+    settings_path=lambda: os.environ.get("MONKAY_SETTINGS", "example.default.path.settings:settings"),
     settings_preloads_name="preloads",
     settings_extensions_name="extensions",
 )
@@ -338,3 +308,10 @@ monkay = Monkay(
 ```
 
 This example updates `__all__` dynamically in the debug environment and ensures that lazy imports are added to `__all__`.
+
+
+### Sub monkay environment
+
+Sometimes you want to provide temporily a different environment for a code path. You can do this with:
+
+[`with_full_overwrite`](testing.md#full-overwrite)

docs_src/settings/forwarder.py

@@ -0,0 +1,17 @@
+from typing import TYPE_CHECKING, Any, cast
+
+if TYPE_CHECKING:
+    from .global_settings import EdgySettings
+
+
+class SettingsForward:
+    def __getattribute__(self, name: str) -> Any:
+        import edgy
+
+        return getattr(edgy.monkay.settings, name)
+
+
+# Pretend the forward is the real object
+settings = cast("EdgySettings", SettingsForward())
+
+__all__ = ["settings"]

docs_src/settings/forwarding_child.py

@@ -0,0 +1,8 @@
+import os
+
+from monkay import Monkay
+
+monkay = Monkay(
+    globals(),
+    settings_path=lambda: os.environ.get("MONKAY_CHILD_SETTINGS", "foo.test:example") or "",
+)

docs_src/settings/forwarding_main.py

@@ -0,0 +1,12 @@
+import os
+
+import child
+
+from monkay import Monkay
+
+monkay = Monkay(
+    globals(),
+    settings_path=lambda: os.environ.get("MONKAY_MAIN_SETTINGS", "foo.test:example") or "",
+)
+# because monkay.settings is an instance it uses not a cache for the settings
+child.monkay.settings = lambda: monkay.settings

docs_src/settings/lazy_loader.py

@@ -0,0 +1,41 @@
+import os
+from dataclasses import dataclass
+
+from monkay import Monkay
+
+
+@dataclass
+class Settings:
+    env: str
+
+
+@dataclass
+class ProductionSettings(Settings):
+    env: str = "production"
+
+
+@dataclass
+class DebugSettings(Settings):
+    env: str = "debug"
+
+
+def lazy_loader():
+    # Lazy setup based on environment variables
+    if not os.environ.get("DEBUG"):
+        return os.environ.get("MONKAY_MAIN_SETTINGS", "foo.test:example") or ""
+    elif os.environ.get("PERFORMANCE"):
+        # must be class to be cached
+        return ProductionSettings
+    else:
+        # not a class, will evaluated always on access
+        return DebugSettings()
+
+
+monkay = Monkay(
+    globals(),
+    # Required for initializing settings feature
+    settings_path=lazy_loader,
+)
+
+# Now the settings are applied
+monkay.evaluate_settings()