Skip to content

Commit b3a5a12

Browse files
committed
feat: add YAML IO support and extend mixed-format config loading
This commit introduces YAML read and write helpers, expands load_config to support TOML, JSON, and YAML inputs, and updates the docs and tests accordingly. Included in this update: - add read_yaml and write_yaml file helpers - add YAML-specific IO exceptions and the PyYAML dependency - extend load_config to merge TOML, JSON, YAML, and YML files in order - validate that loaded config roots are mappings before merging - update the README, quickstart guide, and tool docs for YAML and mixed-format config loading - replace the old static test runner with focused tool-level tests for config loading and YAML IO
1 parent 5822553 commit b3a5a12

15 files changed

Lines changed: 376 additions & 261 deletions

File tree

README.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
## What You Get
66

77
- simple text and binary file helpers
8-
- JSON and TOML read/write helpers
8+
- JSON, TOML, and YAML read/write helpers
99
- masking for tokens, secrets, IDs, and similar values
1010
- global configuration for shared error behavior
1111
- ready-to-use console and file logging
@@ -37,7 +37,7 @@ loaded = read_json("tmp/config.json")
3737
logger.info("Loaded config: %s", loaded)
3838
```
3939

40-
For split TOML setups, `load_config()` merges multiple files into one config object with dot access.
40+
For split configuration setups, `load_config()` merges multiple TOML, JSON, or YAML files into one config object with dot access.
4141

4242
## Public API Overview
4343

@@ -55,6 +55,8 @@ For split TOML setups, `load_config()` merges multiple files into one config obj
5555
- `write_json`
5656
- `read_toml`
5757
- `write_toml`
58+
- `read_yaml`
59+
- `write_yaml`
5860
- `load_config`
5961

6062
### Logging helpers

docs/quickstart.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,12 +26,12 @@ loaded = read_json("tmp/config.json")
2626
logger.info("Loaded config: %s", loaded)
2727
```
2828

29-
If your TOML configuration is split across multiple files, `load_config()` merges it into one object:
29+
If your configuration is split across multiple TOML, JSON, or YAML files, `load_config()` merges it into one object:
3030

3131
```python
3232
from clevertools import load_config
3333

34-
config = load_config("config/settings.toml", "config/content.toml")
34+
config = load_config("config/settings.toml", "config/content.yaml")
3535
print(config.pipelines.ai.enabled)
3636
```
3737

docs/tools/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ Each public helper has its own page with signature, behavior, and examples.
1616
- [write_json](./write_json.md)
1717
- [read_toml](./read_toml.md)
1818
- [write_toml](./write_toml.md)
19+
- [read_yaml](./read_yaml.md)
20+
- [write_yaml](./write_yaml.md)
1921
- [load_config](./load_config.md)
2022

2123
## Logging helpers

docs/tools/load_config.md

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,21 @@
11
# `load_config`
22

3-
`load_config()` reads multiple TOML files, merges overlapping sections, and returns one combined configuration object.
3+
`load_config()` reads multiple configuration files, merges overlapping sections, and returns one combined configuration object.
44

55
## Signature
66

77
```python
8-
load_config(*file_paths: str | Path)
8+
load_config(*file_paths: str | Path, on_error: ErrorMode | None = None)
99
```
1010

1111
## What it does
1212

13-
- Reads each TOML file in the order you pass it in.
13+
- Reads each file in the order you pass it in.
14+
- Supports `.toml`, `.json`, `.yaml`, and `.yml`.
1415
- Merges nested tables recursively.
1516
- Combines shared sections such as `pipelines.ai` across multiple files.
1617
- Exposes the merged result through attribute access and dot-path lookups.
18+
- Applies the shared error policy when a file cannot be read or parsed.
1719

1820
## Returns
1921

@@ -29,7 +31,11 @@ You can access values like this:
2931
```python
3032
from clevertools import load_config
3133

32-
config = load_config("config/settings.toml", "config/content.toml")
34+
config = load_config(
35+
"config/settings.toml",
36+
"config/content.json",
37+
"config/content.yaml",
38+
)
3339

3440
print(config.pipelines.ai.enabled)
3541
print(config.pipelines.ai.ai_model)
@@ -38,7 +44,8 @@ print(config.get("pipelines.publishing.default_post_status"))
3844

3945
## Notes
4046

41-
- Nested TOML tables are merged recursively.
42-
- If the same non-table key exists in multiple files, the later file wins.
43-
- Missing files and TOML parse errors follow the shared error policy from `read_toml()`.
47+
- Nested mappings are merged recursively across supported file types.
48+
- If the same non-mapping key exists in multiple files, the later file wins.
49+
- Missing files and parse errors follow the shared error policy from the corresponding reader.
50+
- Each loaded document must have a mapping object at its root so it can be merged safely.
4451
- Use `as_dict()` when you need the merged result as a plain dictionary.

docs/tools/read_yaml.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# `read_yaml`
2+
3+
`read_yaml()` reads a YAML file and deserializes it with `yaml.safe_load()`.
4+
5+
## Signature
6+
7+
```python
8+
read_yaml(
9+
file_path: str | Path,
10+
on_error: Literal["raise", "log", "silent"] | None = None,
11+
) -> Any | None
12+
```
13+
14+
## Example
15+
16+
```python
17+
from clevertools import read_yaml
18+
19+
config = read_yaml("config.yaml")
20+
print(config["service"])
21+
```
22+
23+
## Notes
24+
25+
- Invalid YAML returns `None` unless the active error mode raises.
26+
- Missing files and directory paths are handled through the shared error policy.

docs/tools/write_yaml.md

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# `write_yaml`
2+
3+
`write_yaml()` serializes a Python value to YAML and writes it to disk.
4+
5+
## Signature
6+
7+
```python
8+
write_yaml(
9+
file_path: str | Path,
10+
data: Any,
11+
create_if_missing: bool = True,
12+
allow_unicode: bool = True,
13+
sort_keys: bool = False,
14+
on_error: Literal["raise", "log", "silent"] | None = None,
15+
) -> None
16+
```
17+
18+
## Example
19+
20+
```python
21+
from clevertools import write_yaml
22+
23+
write_yaml(
24+
"config.yaml",
25+
{
26+
"service": "clevertools",
27+
"enabled": True,
28+
"labels": ["yaml", "config"],
29+
},
30+
allow_unicode=True,
31+
sort_keys=False,
32+
)
33+
```
34+
35+
## Notes
36+
37+
- `data` must not be `None`.
38+
- With `create_if_missing=True`, parent folders are created automatically.
39+
- Serialization errors are handled through the shared error policy.

requirements.txt

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,5 @@ mypy
55
faker
66

77
# === For tools to work === #
8-
tomli_w
8+
tomli_w
9+
pyyaml

src/clevertools/__init__.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
from .file.default_io import read, write
1515
from .file.json_io import read_json, write_json
1616
from .file.toml_io import read_toml, write_toml
17+
from .file.yaml_io import read_yaml, write_yaml
1718

1819
from .system.mask_handler import mask
1920

@@ -31,6 +32,8 @@
3132
"write_json",
3233
"read_toml",
3334
"write_toml",
35+
"read_yaml",
36+
"write_yaml",
3437
"configure_logger",
3538
"get_logger",
3639
"CleverToolsFormatter",

src/clevertools/errors/exceptions.py

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1110,9 +1110,22 @@ class InternalProtocolError(ExecutionError):
11101110

11111111
class ExternalProtocolMismatchError(ProtocolError):
11121112
"""Raised when external protocol versions or formats mismatch."""
1113-
1113+
1114+
class YamlIOError(Exception):
1115+
"""Base exception for YAML IO operations."""
1116+
1117+
1118+
class YamlReadError(YamlIOError):
1119+
"""Raised when reading YAML fails."""
1120+
1121+
1122+
class YamlWriteError(YamlIOError):
1123+
"""Raised when writing YAML fails."""
11141124

11151125
__all__ = (
1126+
"YamlIOError",
1127+
"YamlReadError",
1128+
"YamlWriteError",
11161129
"AppError",
11171130
"RecoverableError",
11181131
"FatalError",

src/clevertools/file/yaml_io.py

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
from __future__ import annotations
2+
3+
from typing import Any, Optional
4+
from pathlib import Path
5+
import yaml
6+
7+
from ..errors.exceptions import YamlReadError, YamlWriteError
8+
from ..errors.policy import handle_error
9+
from ..configuration import ErrorMode
10+
11+
12+
def read_yaml(file_path: Path | str, on_error: Optional[ErrorMode] = None) -> Any | None:
13+
"""
14+
Read and deserialize a YAML file.
15+
16+
Args:
17+
file_path: Path to the YAML file that should be loaded.
18+
on_error: Error handling mode. If omitted, the global default from
19+
`configure()` is used.
20+
21+
Returns:
22+
The parsed YAML value. Returns `None` when the file cannot be read or
23+
parsed and the selected error mode does not raise.
24+
"""
25+
26+
path = Path(file_path)
27+
28+
if not path.exists():
29+
return handle_error(FileNotFoundError(f"YAML file not found: {path}"), on_error=on_error, fallback=None)
30+
31+
if not path.is_file():
32+
return handle_error(IsADirectoryError(f"YAML is not a file: {path}"), on_error=on_error, fallback=None)
33+
34+
try:
35+
with path.open("r", encoding="utf-8") as file:
36+
return yaml.safe_load(file)
37+
except yaml.YAMLError as e:
38+
return handle_error(YamlReadError(f"Invalid YAML format: {e}"), on_error=on_error, fallback=None)
39+
except Exception as e:
40+
return handle_error(YamlReadError(f"Failed to read YAML: {e}"), on_error=on_error, fallback=None)
41+
42+
def write_yaml(
43+
file_path: Path | str,
44+
data: Any,
45+
create_if_missing: Optional[bool] = True,
46+
allow_unicode: bool = True,
47+
sort_keys: bool = False,
48+
on_error: ErrorMode | None = None,
49+
) -> None:
50+
"""
51+
Serialize data as YAML and write it to a file.
52+
53+
Args:
54+
file_path: Target file path.
55+
data: YAML-serializable value to write.
56+
create_if_missing: When `True`, missing parent directories are created
57+
automatically. When `False`, the target file must already exist.
58+
allow_unicode: Forwarded to `yaml.safe_dump()` to control whether
59+
Unicode characters are emitted directly.
60+
sort_keys: Forwarded to `yaml.safe_dump()` to control whether mapping
61+
keys are sorted in the output.
62+
on_error: Error handling mode. If omitted, the global default from
63+
`configure()` is used.
64+
65+
Returns:
66+
`None`. If serialization or writing fails, the outcome depends on the
67+
selected error mode.
68+
"""
69+
70+
path = Path(file_path)
71+
72+
if data is None:
73+
return handle_error(ValueError("YAML data must not be None."), on_error=on_error, fallback=None)
74+
75+
try:
76+
if create_if_missing:
77+
path.parent.mkdir(parents=True, exist_ok=True)
78+
else:
79+
if not path.exists():
80+
return handle_error(FileNotFoundError(f"YAML file not found: {path}"), on_error=on_error, fallback=None)
81+
82+
if not path.is_file():
83+
return handle_error(IsADirectoryError(f"YAML is not a file: {path}"), on_error=on_error, fallback=None)
84+
85+
with path.open("w", encoding="utf-8") as file:
86+
yaml.safe_dump(data, file, allow_unicode=allow_unicode, sort_keys=sort_keys)
87+
except yaml.YAMLError as e:
88+
return handle_error(YamlWriteError(f"Invalid YAML data: {e}"), on_error=on_error, fallback=None)
89+
except Exception as e:
90+
return handle_error(YamlWriteError(f"Failed to write YAML: {e}"), on_error=on_error, fallback=None)

0 commit comments

Comments
 (0)