Add YAML file helpers and mixed-format configuration loading (#4)

b7binw13 · web-flow · commit e20e5efca218 · 2026-03-19T10:11:50.000+01:00
* feat: add config handler utilities for TOML-based configuration loading

This commit introduces a new ConfigHandler API and a load_config helper for loading and merging multiple TOML configuration files.

Included in this update:

- add ConfigHandler with nested attribute access and dot-path lookups

- add support for merging multiple TOML config files into one configuration object

- export the new config helpers from the public package interface

- make TOML writing handle missing tomli_w imports more safely

- add tests for merged config loading and access patterns

- update the README and bump the package version to 1.3.1

* docs: expand load_config documentation and refine public API

This commit improves the documentation for load_config and cleans up how the configuration helper is exposed in the public interface.

Included in this update:

- document load_config in the README, quickstart guide, and tools index

- add a dedicated load_config tool documentation page with examples and behavior notes

- remove ConfigHandler from the top-level public exports to keep the API more focused

- add docstrings to the config handler implementation for clarity and maintainability

- update the related tests to reflect the refined public usage

* test: reorganize static tests and expand config-based IO coverage

This commit restructures the static test setup and extends the test runner to cover split TOML configuration loading and merged config access.

Included in this update:

- move the static test module from tmp_tests to static_tests

- add a dedicated static test package structure

- expand the static IO test runner to write and merge multiple TOML files

- add coverage for nested config access and merged configuration values

- keep the JSON read and write flow as part of the static integration-style test run

* feat: add YAML IO support and extend mixed-format config loading

This commit introduces YAML read and write helpers, expands load_config to support TOML, JSON, and YAML inputs, and updates the docs and tests accordingly.

Included in this update:

- add read_yaml and write_yaml file helpers

- add YAML-specific IO exceptions and the PyYAML dependency

- extend load_config to merge TOML, JSON, YAML, and YML files in order

- validate that loaded config roots are mappings before merging

- update the README, quickstart guide, and tool docs for YAML and mixed-format config loading

- replace the old static test runner with focused tool-level tests for config loading and YAML IO
diff --git a/README.md b/README.md
@@ -5,7 +5,7 @@
 ## What You Get
 
 - simple text and binary file helpers
-- JSON and TOML read/write helpers
+- JSON, TOML, and YAML read/write helpers
 - masking for tokens, secrets, IDs, and similar values
 - global configuration for shared error behavior
 - ready-to-use console and file logging
@@ -37,6 +37,8 @@ loaded = read_json("tmp/config.json")
 logger.info("Loaded config: %s", loaded)
 ```
 
+For split configuration setups, `load_config()` merges multiple TOML, JSON, or YAML files into one config object with dot access.
+
 ## Public API Overview
 
 ### Core
@@ -53,6 +55,9 @@ logger.info("Loaded config: %s", loaded)
 - `write_json`
 - `read_toml`
 - `write_toml`
+- `read_yaml`
+- `write_yaml`
+- `load_config`
 
 ### Logging helpers
 
diff --git a/docs/quickstart.md b/docs/quickstart.md
@@ -26,6 +26,15 @@ loaded = read_json("tmp/config.json")
 logger.info("Loaded config: %s", loaded)
 ```
 
+If your configuration is split across multiple TOML, JSON, or YAML files, `load_config()` merges it into one object:
+
+```python
+from clevertools import load_config
+
+config = load_config("config/settings.toml", "config/content.yaml")
+print(config.pipelines.ai.enabled)
+```
+
 Next steps:
 
 - Read [Tools](./tools/README.md) for every public helper.
diff --git a/docs/tools/README.md b/docs/tools/README.md
@@ -16,6 +16,9 @@ Each public helper has its own page with signature, behavior, and examples.
 - [write_json](./write_json.md)
 - [read_toml](./read_toml.md)
 - [write_toml](./write_toml.md)
+- [read_yaml](./read_yaml.md)
+- [write_yaml](./write_yaml.md)
+- [load_config](./load_config.md)
 
 ## Logging helpers
 
diff --git a/docs/tools/load_config.md b/docs/tools/load_config.md
@@ -0,0 +1,51 @@
+# `load_config`
+
+`load_config()` reads multiple configuration files, merges overlapping sections, and returns one combined configuration object.
+
+## Signature
+
+```python
+load_config(*file_paths: str | Path, on_error: ErrorMode | None = None)
+```
+
+## What it does
+
+- Reads each file in the order you pass it in.
+- Supports `.toml`, `.json`, `.yaml`, and `.yml`.
+- Merges nested tables recursively.
+- Combines shared sections such as `pipelines.ai` across multiple files.
+- Exposes the merged result through attribute access and dot-path lookups.
+- Applies the shared error policy when a file cannot be read or parsed.
+
+## Returns
+
+Returns one merged configuration object.
+
+You can access values like this:
+
+- `config.pipelines.ai.enabled`
+- `config.get("pipelines.ai.ai_model")`
+
+## Example
+
+```python
+from clevertools import load_config
+
+config = load_config(
+    "config/settings.toml",
+    "config/content.json",
+    "config/content.yaml",
+)
+
+print(config.pipelines.ai.enabled)
+print(config.pipelines.ai.ai_model)
+print(config.get("pipelines.publishing.default_post_status"))
+```
+
+## Notes
+
+- Nested mappings are merged recursively across supported file types.
+- If the same non-mapping key exists in multiple files, the later file wins.
+- Missing files and parse errors follow the shared error policy from the corresponding reader.
+- Each loaded document must have a mapping object at its root so it can be merged safely.
+- Use `as_dict()` when you need the merged result as a plain dictionary.
diff --git a/docs/tools/read_yaml.md b/docs/tools/read_yaml.md
@@ -0,0 +1,26 @@
+# `read_yaml`
+
+`read_yaml()` reads a YAML file and deserializes it with `yaml.safe_load()`.
+
+## Signature
+
+```python
+read_yaml(
+    file_path: str | Path,
+    on_error: Literal["raise", "log", "silent"] | None = None,
+) -> Any | None
+```
+
+## Example
+
+```python
+from clevertools import read_yaml
+
+config = read_yaml("config.yaml")
+print(config["service"])
+```
+
+## Notes
+
+- Invalid YAML returns `None` unless the active error mode raises.
+- Missing files and directory paths are handled through the shared error policy.
diff --git a/docs/tools/write_yaml.md b/docs/tools/write_yaml.md
@@ -0,0 +1,39 @@
+# `write_yaml`
+
+`write_yaml()` serializes a Python value to YAML and writes it to disk.
+
+## Signature
+
+```python
+write_yaml(
+    file_path: str | Path,
+    data: Any,
+    create_if_missing: bool = True,
+    allow_unicode: bool = True,
+    sort_keys: bool = False,
+    on_error: Literal["raise", "log", "silent"] | None = None,
+) -> None
+```
+
+## Example
+
+```python
+from clevertools import write_yaml
+
+write_yaml(
+    "config.yaml",
+    {
+        "service": "clevertools",
+        "enabled": True,
+        "labels": ["yaml", "config"],
+    },
+    allow_unicode=True,
+    sort_keys=False,
+)
+```
+
+## Notes
+
+- `data` must not be `None`.
+- With `create_if_missing=True`, parent folders are created automatically.
+- Serialization errors are handled through the shared error policy.
diff --git a/pyproject.toml b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "clevertools"
-version = "1.2.1"
+version = "1.3.1"
 description = "Clevertools is a utility library providing practical tools for common workflows."
 readme = "README.md"
 requires-python = ">=3.11"
diff --git a/requirements.txt b/requirements.txt
@@ -5,4 +5,5 @@ mypy
 faker
 
 # === For tools to work === #
-tomli_w
+tomli_w
+pyyaml
diff --git a/src/clevertools/__init__.py b/src/clevertools/__init__.py
@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 from .configuration import configure
+from .system.config_handler import load_config
 
 from .errors.exceptions import *  # noqa: F403
 
@@ -13,6 +14,7 @@
 from .file.default_io import read, write
 from .file.json_io import read_json, write_json
 from .file.toml_io import read_toml, write_toml
+from .file.yaml_io import read_yaml, write_yaml
 
 from .system.mask_handler import mask
 
@@ -30,6 +32,8 @@
     "write_json",
     "read_toml",
     "write_toml",
+    "read_yaml",
+    "write_yaml",
     "configure_logger",
     "get_logger",
     "CleverToolsFormatter",
@@ -38,6 +42,7 @@
     "reset_handlers",
     "resolve_logger_options",
     "configure",
+    "load_config",
     
     # === Exceptions === #
     "AppError",
diff --git a/src/clevertools/errors/exceptions.py b/src/clevertools/errors/exceptions.py
@@ -1110,9 +1110,22 @@ class InternalProtocolError(ExecutionError):
 
 class ExternalProtocolMismatchError(ProtocolError):
     """Raised when external protocol versions or formats mismatch."""
-    
+
+class YamlIOError(Exception):
+    """Base exception for YAML IO operations."""
+
+
+class YamlReadError(YamlIOError):
+    """Raised when reading YAML fails."""
+
+
+class YamlWriteError(YamlIOError):
+    """Raised when writing YAML fails."""
 
 __all__ = (
+    "YamlIOError",
+    "YamlReadError",
+    "YamlWriteError",
     "AppError",
     "RecoverableError",
     "FatalError",
diff --git a/src/clevertools/file/toml_io.py b/src/clevertools/file/toml_io.py
@@ -2,7 +2,6 @@
 
 from typing import Any, Mapping, Optional
 from pathlib import Path
-import tomli_w
 import tomllib
 
 from ..errors.policy import handle_error
@@ -72,6 +71,8 @@ def write_toml(
         return handle_error(TypeError(f"TOML data must be a mapping, got {type(data).__name__}."), on_error=on_error, fallback=None)
 
     try:
+        import tomli_w
+
         if create_if_missing:
             path.parent.mkdir(parents=True, exist_ok=True)
         else:
@@ -83,5 +84,5 @@ def write_toml(
         toml_text = tomli_w.dumps(dict(data))
         path.write_text(toml_text, encoding="utf-8")
 
-    except (OSError, TypeError, ValueError) as exc:
-        handle_error(exc, on_error=on_error, fallback=None)
+    except (ImportError, OSError, TypeError, ValueError) as exc:
+        handle_error(exc, on_error=on_error, fallback=None)
diff --git a/src/clevertools/file/yaml_io.py b/src/clevertools/file/yaml_io.py
@@ -0,0 +1,90 @@
+from __future__ import annotations
+
+from typing import Any, Optional
+from pathlib import Path
+import yaml
+
+from ..errors.exceptions import YamlReadError, YamlWriteError
+from ..errors.policy import handle_error
+from ..configuration import ErrorMode
+
+
+def read_yaml(file_path: Path | str, on_error: Optional[ErrorMode] = None) -> Any | None:
+    """
+    Read and deserialize a YAML file.
+
+    Args:
+        file_path: Path to the YAML file that should be loaded.
+        on_error: Error handling mode. If omitted, the global default from
+            `configure()` is used.
+
+    Returns:
+        The parsed YAML value. Returns `None` when the file cannot be read or
+        parsed and the selected error mode does not raise.
+    """
+
+    path = Path(file_path)
+
+    if not path.exists():
+        return handle_error(FileNotFoundError(f"YAML file not found: {path}"), on_error=on_error, fallback=None)
+
+    if not path.is_file():
+        return handle_error(IsADirectoryError(f"YAML is not a file: {path}"), on_error=on_error, fallback=None)
+
+    try:
+        with path.open("r", encoding="utf-8") as file:
+            return yaml.safe_load(file)
+    except yaml.YAMLError as e:
+        return handle_error(YamlReadError(f"Invalid YAML format: {e}"), on_error=on_error, fallback=None)
+    except Exception as e:
+        return handle_error(YamlReadError(f"Failed to read YAML: {e}"), on_error=on_error, fallback=None)
+
+def write_yaml(
+    file_path: Path | str,
+    data: Any,
+    create_if_missing: Optional[bool] = True,
+    allow_unicode: bool = True,
+    sort_keys: bool = False,
+    on_error: ErrorMode | None = None,
+) -> None:
+    """
+    Serialize data as YAML and write it to a file.
+
+    Args:
+        file_path: Target file path.
+        data: YAML-serializable value to write.
+        create_if_missing: When `True`, missing parent directories are created
+            automatically. When `False`, the target file must already exist.
+        allow_unicode: Forwarded to `yaml.safe_dump()` to control whether
+            Unicode characters are emitted directly.
+        sort_keys: Forwarded to `yaml.safe_dump()` to control whether mapping
+            keys are sorted in the output.
+        on_error: Error handling mode. If omitted, the global default from
+            `configure()` is used.
+
+    Returns:
+        `None`. If serialization or writing fails, the outcome depends on the
+        selected error mode.
+    """
+
+    path = Path(file_path)
+
+    if data is None:
+        return handle_error(ValueError("YAML data must not be None."), on_error=on_error, fallback=None)
+
+    try:
+        if create_if_missing:
+            path.parent.mkdir(parents=True, exist_ok=True)
+        else:
+            if not path.exists():
+                return handle_error(FileNotFoundError(f"YAML file not found: {path}"), on_error=on_error, fallback=None)
+            
+            if not path.is_file():
+                return handle_error(IsADirectoryError(f"YAML is not a file: {path}"), on_error=on_error, fallback=None)
+
+        with path.open("w", encoding="utf-8") as file:
+            yaml.safe_dump(data, file, allow_unicode=allow_unicode, sort_keys=sort_keys)
+    except yaml.YAMLError as e:
+        return handle_error(YamlWriteError(f"Invalid YAML data: {e}"), on_error=on_error, fallback=None)
+    except Exception as e:
+        return handle_error(YamlWriteError(f"Failed to write YAML: {e}"), on_error=on_error, fallback=None)
diff --git a/src/clevertools/system/config_handler.py b/src/clevertools/system/config_handler.py
diff --git a/tests/tmp_tests/__init__.py b/tests/tmp_tests/__init__.py
diff --git a/tests/tmp_tests/static_test.py b/tests/tmp_tests/static_test.py
diff --git a/tests/tools/config_handler_test.py b/tests/tools/config_handler_test.py
diff --git a/tests/tools/yaml_io_test.py b/tests/tools/yaml_io_test.py

-Original file line number
+Diff line change
 faker
 # === For tools to work === #
 -tomli_w
 +tomli_w
 +pyyaml