Merge pull request #302 from viccie30/yamlfix-toml-configuration

Add support for yamlfix.toml configuration files

Author: lyz-code

docs/index.md

@@ -92,15 +92,15 @@ variable, use `fix_code`:
 
 `yamlfix` uses the `maison` library to find and parse configuration from standard locations, and can additionally be configured through environment variables.
 
-Any configuration found in the [YamlfixConfig class](./reference/#yamlfix.model.YamlfixConfig) can be set through your projects `pyproject.toml`, a custom `toml`-file or through the environment by providing an environment variable like `{yamlfix_env_prefix}_{yamlfix_config_key}`.
+Any configuration found in the [YamlfixConfig class](./reference/#yamlfix.model.YamlfixConfig) can be set through your projects' `pyproject.toml`, a `.yamlfix.toml` file, a custom `toml`-file or through the environment by providing an environment variable like `{yamlfix_env_prefix}_{yamlfix_config_key}`.
 
 Configuration options that are provided through environment variables have higher priority than options provided through configuration files and will override those keys.
 
 All provided [configuration options](#configuration-options), be it through `pyproject.toml`, config-files or env-vars, will be parsed by `pydantic`, so the target value type (str, bool, int, etc.) will be enforced, even if the provided value has the wrong type (for example all env vars in linux systems are strings, but pydantic will parse them to bools/numbers where necessary).
 
-## Auto-Configure through `pyproject.toml`
+## Auto-Configure through `pyproject.toml` or `.yamlfix.toml`
 
-The `maison` library will automatically pick up your `yamlfix` configuration through your projects `pyproject.toml`. It will look in the section named `tool.yamlfix` and apply the provided [configuration options](#configuration-options). For example:
+The `maison` library will automatically pick up your `yamlfix` configuration through your projects' `pyproject.toml`. It will look in the section named `tool.yamlfix` and apply the provided [configuration options](#configuration-options). For example:
 
 ```toml
 # pyproject.toml
@@ -111,15 +111,57 @@ line_length = 80
 none_representation = "null"
 ```
 
-## Provide config-files
+When running `yamlfix` as a standalone cli application it might be desireable to provide a config file containing just the configuration related to `yamlfix`. The `maison` library will therefore also read `.yamlfix.toml` and `yamlfix.toml` if they exist. No section headers are necessary for these configuration files, as the expected behaviour is, that those files contain only configuration related to `yamlfix`. For example:
 
-When running `yamlfix` as a standalone cli application it might be desireable to provide a config file containing just the configuration related to `yamlfix`. A cli-argument `-c` (`--config-file`) can be provided multiple times to read configuration values from `toml` formatted files. The rightmost value-files override the value-files preceding them, only trumped by environment variables. No section headers are necessary for these configuration files, as the expected behaviour is, that those files contain only configuration related to `yamlfix`. For example:
+```toml
+# .yamlfix.toml
+allow_duplicate_keys = true
+line_length = 80
+none_representation = "null"
+```
+
+The configurations from `pyproject.toml`, `yamlfix.toml`, and `.yamlfix.toml` are merged from left to right. This means that settings in `yamlfix.toml` override those in `pyproject.toml` and settings in `.yamlfix.toml` override those in `pyproject.toml` and `yamlfix.toml`. Environment variables override all settings set in configuration files. For example:
+
+```toml
+# pyproject.toml
+allow_duplicate_keys = false
+line_length = 100
+preserve_quotes = false
+```
+
+```toml
+# yamlfix.toml
+allow_duplicate_keys = true
+preserve_quotes = true
+```
+
+```toml
+# .yamlfix.toml
+preserve_quotes = false
+```
+
+These provided configuration files would result in a merged runtime-configuration of:
+```toml
+# merged configuration
+allow_duplicate_keys = true
+line_length = 100
+preserve_quotes = false
+```
+
+## Custom config-files
+
+ A cli-argument `-c` (`--config-file`) can be provided multiple times to read configuration values from `toml` formatted files. If `-c` is provided, the default configuration files are ignored. The rightmost value-files override the value-files preceding them, only trumped by environment variables. No section headers are necessary for these configuration files, as the expected behaviour is, that those files contain only configuration related to `yamlfix`. For example:
 
 ```bash
 # run yamlfix with two config files
 yamlfix -c base.toml --config-file environment.toml file.yaml
 ```
 
+```toml
+# .yamlfix.toml
+preserve_quotes = true
+```
+
 ```toml
 # base.toml
 allow_duplicate_keys = false
@@ -206,13 +248,13 @@ Environment variable override:
 export YAMLFIX_WHITELINES="0"
 ```
 
-This option allows to keep a speficied number of whitelines between lines. 
+This option allows to keep a speficied number of whitelines between lines.
 
-It's useful if, for one, you like to separate GitHub Actions job steps or Docker-Compose 
+It's useful if, for one, you like to separate GitHub Actions job steps or Docker-Compose
 service definitions with a blank line.
 
-Bear in mind that, like **Comments Whitelines**, it won't insert whitelines if there's no 
-whitelines in the first place. It will only fix 1 or more whitelines to the desired 
+Bear in mind that, like **Comments Whitelines**, it won't insert whitelines if there's no
+whitelines in the first place. It will only fix 1 or more whitelines to the desired
 amount (or remove them completely by default).
 
 ### Comments Whitelines
@@ -268,7 +310,7 @@ Environment variable override:
 export YAMLFIX_CONFIG_PATH="/etc/yamlfix/"
 ```
 
-Configure the base config-path that `maison` will look for a `pyproject.toml` configuration file. This path is traversed upwards until such a file is found.
+Configure the base config-path that `maison` will look for a `pyproject.toml`, `.yamlfix.toml`, or `yamlfix.toml` configuration file. This path is traversed upwards until such a file is found.
 
 ### Explicit Document Start
 

src/yamlfix/config.py

@@ -22,6 +22,13 @@ def configure_yamlfix(
         if config_path_env:
             config_path = Path(config_path_env)
 
+    if not config_files:
+        config_files = [
+            "pyproject.toml",
+            "yamlfix.toml",
+            ".yamlfix.toml",
+        ]
+
     config: UserConfig = UserConfig(
         schema=YamlfixConfig,
         merge_configs=True,

tests/conftest.py

@@ -1 +1,11 @@
 """Store the classes and fixtures used throughout the tests."""
+
+import os
+
+import pytest
+
+
+@pytest.fixture(autouse=True)
+def _unset_yamlfix_config_path() -> None:
+    """Unset YAMLFIX_CONFIG_PATH environment variable."""
+    os.environ.pop("YAMLFIX_CONFIG_PATH", None)

tests/e2e/test_cli.py

@@ -215,7 +215,81 @@ def test_check_one_file_no_changes(runner: CliRunner, tmp_path: Path) -> None:
     assert test_file.read_text() == test_file_source
 
 
-def test_config_parsing(runner: CliRunner, tmp_path: Path) -> None:
+def test_default_config_parsing(runner: CliRunner, tmp_path: Path) -> None:
+    """Default configuration files are parsed, merged, and applied correctly."""
+    os.environ["YAMLFIX_CONFIG_PATH"] = str(tmp_path)
+    pyproject_config = dedent(
+        """\
+        [tool.yamlfix]
+        line_length = 90
+        quote_basic_values = "true"
+        quote_representation = "'"
+        none_represtation = ""
+        """
+    )
+    pyproject_config_file = tmp_path / "pyproject.toml"
+    pyproject_config_file.write_text(pyproject_config)
+
+    nodot_toml_config = dedent(
+        """\
+        none_representation = "null"
+        quote_representation = '"'
+        """
+    )
+    nodot_toml_config_file = tmp_path / "yamlfix.toml"
+    nodot_toml_config_file.write_text(nodot_toml_config)
+
+    dot_toml_config = dedent(
+        """\
+        none_representation = "~"
+        """
+    )
+    dot_toml_config_file = tmp_path / ".yamlfix.toml"
+    dot_toml_config_file.write_text(dot_toml_config)
+
+    test_source = dedent(
+        f"""\
+        ---
+        really_long_string: >
+          {("abcdefghij " * 10).strip()}
+        single_quoted_string: 'value1'
+        double_quoted_string: "value2"
+        unquoted_string: value3
+        none_value:
+        none_value2: ~
+        none_value3: null
+        none_value4: NULL
+        """
+    )
+    test_source_file = tmp_path / "source.yaml"
+    test_source_file.write_text(test_source)
+
+    result = runner.invoke(
+        cli,
+        [
+            str(test_source_file),
+        ],
+    )
+
+    assert result.exit_code == 0
+    assert test_source_file.read_text() == dedent(
+        f"""\
+        ---
+        really_long_string: >
+          {("abcdefghij " * 9).strip()}
+          abcdefghij
+        single_quoted_string: "value1"
+        double_quoted_string: "value2"
+        unquoted_string: "value3"
+        none_value: ~
+        none_value2: ~
+        none_value3: ~
+        none_value4: ~
+        """
+    )
+
+
+def test_custom_config_parsing(runner: CliRunner, tmp_path: Path) -> None:
     """Provided config options are parsed, merged, and applied correctly."""
     os.environ["YAMLFIX_CONFIG_PATH"] = str(tmp_path)
     pyproject_config = dedent(
@@ -227,13 +301,14 @@ def test_config_parsing(runner: CliRunner, tmp_path: Path) -> None:
     )
     pyproject_config_file = tmp_path / "pyproject.toml"
     pyproject_config_file.write_text(pyproject_config)
+
     toml_config = dedent(
         """\
         none_representation = "null"
         quote_representation = '"'
         """
     )
-    toml_config_file = tmp_path / "yamlfix.toml"
+    toml_config_file = tmp_path / "custom.toml"
     toml_config_file.write_text(toml_config)
 
     # the ini config is currenlty parsed incorrectly and it is not possible to provide
@@ -247,8 +322,9 @@ def test_config_parsing(runner: CliRunner, tmp_path: Path) -> None:
         none_representation = "~"
         """
     )
-    ini_config_file = tmp_path / "yamlfix.ini"
+    ini_config_file = tmp_path / "custom.ini"
     ini_config_file.write_text(ini_config)
+
     test_source = dedent(
         f"""\
         ---
@@ -266,6 +342,14 @@ def test_config_parsing(runner: CliRunner, tmp_path: Path) -> None:
     test_source_file = tmp_path / "source.yaml"
     test_source_file.write_text(test_source)
 
+    ignored_config = dedent(
+        """\
+        quote_keys_and_basic_values = true
+        """
+    )
+    ignored_config_file = tmp_path / ".yamlfix.toml"
+    ignored_config_file.write_text(ignored_config)
+
     # we have to provide the pyproject.toml as a relative path to YAMLFIX_CONFIG_PATH
     # until this is fixed: https://github.com/dbatten5/maison/issues/141
     pyproject_config_file_name = "pyproject.toml"