Add bootstrap logging, path preparation helpers, and broader tooling/documentation improvements (#5)

* feat: add bootstrap logging workflow and refresh package docs

Introduce runtime bootstrap logging with start_file_logger() and stop_file_logger() so early startup records can be buffered, stdout/stderr can be captured, and buffered messages can be flushed once configure_logger() installs the final handlers.

Expose the new bootstrap logging helpers through the public API, update console handler creation to support explicit streams, flush buffered records during logger configuration, and harden config loading by importing YAML support lazily so environments without PyYAML fail through the shared error policy instead of at module import time.

Refine typing and packaging details by adding overloads for handle_error(), marking the PyYAML import for type checking, expanding pyproject metadata and optional dev dependencies, splitting runtime and development requirements, adding pytest discovery settings, and broadening .gitignore coverage for Python, tooling, editor, and OS artifacts.

Rework the test suite around pytest fixtures and temporary cache directories, remove the old bootstrap/path helpers, simplify test modules into direct function-based tests, and extend logger coverage to verify buffered bootstrap records plus stdout/stderr capture behavior alongside the existing file, config, masking, and serialization checks.

Rewrite the README and large parts of the documentation to better onboard new users, clarify the library's purpose, document the error-handling and logging concepts in more depth, expand the tool pages with practical examples and usage guidance, and add dedicated reference pages for start_file_logger() and stop_file_logger().

This commit is still a broad work-in-progress snapshot. The new logging flow and the larger documentation/testing refresh are in place, but the state should continue to be validated and may still receive follow-up adjustments, cleanup, and refinement.

* feat: add path preparation helpers for files and directories

Introduce ensure_file() and ensure_dir() as new public helpers for preparing filesystem targets before later reads, writes, logging, cache generation, or export steps. The new path utility module normalizes single-path and multi-path inputs, creates missing parent directories as needed, and routes validation or filesystem failures through the shared clevertools error policy.

Expose the new helpers through the package root API so they can be imported directly from clevertools, and document them in the main README plus the tools index to make the expanded filesystem helper surface easier to discover.

Add dedicated documentation pages for ensure_file() and ensure_dir() with signatures, behavior notes, and practical examples covering single paths, multiple paths, nested directory preparation, replacement content, and binary file initialization.

Extend the test suite with focused coverage for the new path utilities, including creation of missing files and directories, list input support, content replacement behavior, binary payload handling, preservation of existing files by default, and validation failures for wrong target types or invalid path inputs.

This snapshot appears to be a smaller incremental feature update rather than a large refactor, but it is still a work in progress and may receive follow-up cleanup, polish, or additional validation in later commits.

* fix: harden bootstrap logging and path helper error handling

Prevent duplicated bootstrap console output when stdout or stderr capture is enabled and the final logger configuration also writes to the console.

Update the runtime bootstrap flow to keep explicit stream redirect objects, flush buffered partial output before restoring the original streams, and ensure early startup messages are emitted exactly once through the configured logger handlers.

Align ensure_file() and ensure_dir() with the package-wide error policy by restoring None-based on_error defaults so configured global error handling is respected automatically.

Clean up path helper validation messages so conflicting file and directory targets report the concrete path value instead of a literal placeholder.

Expand logger regression coverage with a dedicated test that verifies captured bootstrap stdout and stderr do not appear twice on the console while still being persisted to the log file.

* fix: tighten logger formatter behavior and coverage

Make the datetime logging preset honor custom date_format values instead of always rendering the built-in default date layout.

Route console color detection through the actual handler stream so ANSI coloring follows the real output target rather than relying on sys.stdout.

Pass the console handler stream into CleverToolsFormatter and keep the formatter logic aligned with the configured logger output path.

Expand logger regression coverage with focused tests for custom datetime formatting and stream-aware color handling, alongside the earlier bootstrap logging protections.

This keeps the logger API behavior consistent with its configuration surface while preserving clean test and mypy results in the project venv.

Author: b7binw13

.gitignore

@@ -2,6 +2,50 @@
 __pycache__/
 *.py[cod]
 *$py.class
+*.pyd
+*.pyo
+*.py,cover
+
+# Python environments and local version files
+.env
+.env.*
+.venv/
+.python-version
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Python packaging, builds, and installers
+build/
+dist/
+site/
+.eggs/
+*.egg-info/
+*.egg
+pip-wheel-metadata/
+share/python-wheels/
+wheelhouse/
+sdist/
+develop-eggs/
+downloads/
+parts/
+var/
+wheels/
+MANIFEST
+
+# Python tooling and dependency managers
+__pypackages__/
+poetry.toml
+.pdm.toml
+.pdm-build/
+.pdm-python
+dmypy.json
+.pyre/
+.pytype/
+.pybuilder/
+cython_debug/
 
 # Test, lint, and type-checker caches
 .pytest_cache/
@@ -14,37 +58,60 @@ __pycache__/
 .coverage.*
 htmlcov/
 coverage.xml
-
-# Build and packaging artifacts
-build/
-dist/
-.eggs/
-*.egg-info/
-*.egg
-pip-wheel-metadata/
-
-# Virtual environments
-.env
-.venv/
-env/
-venv/
-ENV/
-
-# Local tool configuration
-poetry.toml
-.pdm.toml
+nosetests.xml
+*.cover
+*.py.cover
+.cache/
 
 # Documentation builds
 docs/_build/
+_build/
 
-# Editors
+# Jupyter and notebooks
+.ipynb_checkpoints/
+
+# Editors and IDEs
 .vscode/
 .vscode/*
 !.vscode/settings.json
 !.vscode/tasks.json
 !.vscode/launch.json
 !.vscode/extensions.json
 !.vscode/*.code-snippets
+.vscode-test/
 .idea/
+.fleet/
 .history/
-.ionide/
+.ionide/
+*.code-workspace
+*.iml
+*.suo
+*.user
+*.userossc
+*.rsuser
+*.sln.docstates
+*.swp
+*.swo
+*~
+
+# OS and file manager artifacts
+.DS_Store
+.DS_Store?
+Thumbs.db
+Desktop.ini
+$RECYCLE.BIN/
+.Spotlight-V100/
+.Trashes/
+._*
+
+# Logs and runtime files
+*.log
+*.out
+*.err
+*.pid
+*.pid.lock
+
+# Local archives and temporary files
+*.tmp
+*.temp
+*.bak

README.md

@@ -1,95 +1,190 @@
 # clevertools
 
-`clevertools` is a compact Python utility library for everyday workflows like file I/O, masking sensitive values, runtime configuration, and logger setup.
+`clevertools` is a practical Python helper library for everyday application code. It focuses on the boring but important parts that show up in many projects: reading and writing files, handling JSON/TOML/YAML, masking sensitive values, loading split configuration, and setting up logging without repetitive boilerplate.
 
-## What You Get
+The package is intentionally small, but it still gives you enough structure that a new teammate can open the project and understand how configuration, file access, and logging are supposed to work.
 
-- simple text and binary file helpers
-- JSON, TOML, and YAML read/write helpers
-- masking for tokens, secrets, IDs, and similar values
-- global configuration for shared error behavior
-- ready-to-use console and file logging
-- lower-level logging utilities when you need more control
+## What clevertools is for
 
-## Quick Example
+Use `clevertools` when you want:
+
+- a consistent way to read and write text, bytes, JSON, TOML, and YAML
+- one shared error-handling model across helpers
+- simple masking for API keys, tokens, emails, IDs, and similar values
+- merged configuration from multiple files with dot access
+- fast logger setup for console and file logging
+- lower-level logging primitives when you need custom control
+
+## What is included
+
+### Core helpers
+
+- `configure()` stores package-wide defaults such as error handling and logger overrides
+- `log` is the shared default logger
+- `mask()` partially hides sensitive strings before you log or display them
+
+### File helpers
+
+- `read()` and `write()` handle plain text and raw bytes
+- `ensure_file()` and `ensure_dir()` make sure file and directory paths exist
+- `read_json()` and `write_json()` handle JSON files
+- `read_toml()` and `write_toml()` handle TOML files
+- `read_yaml()` and `write_yaml()` handle YAML files
+- `load_config()` merges multiple config files into one object
+
+### Logging helpers
+
+- `configure_logger()` creates or refreshes a logger with console and file handlers
+- `start_file_logger()` captures early startup output before final logger setup is ready
+- `stop_file_logger()` restores captured `stdout` and `stderr`
+- `get_logger()` returns a named logger without modifying it
+- `CleverToolsFormatter` adds structured time fields and optional colors
+- `build_console_handler()` and `build_file_handler()` expose the low-level handler builders
+- `reset_handlers()` safely clears existing handlers
+- `resolve_logger_options()` shows the final logger settings after overrides are applied
+
+## Quick example
 
 ```python
-from clevertools import configure, configure_logger, mask, read_json, write_json
+from clevertools import (
+    configure,
+    configure_logger,
+    load_config,
+    mask,
+    read_json,
+    write_json,
+)
 
 configure(
     error_mode="raise",
     logger_overrides={
         "level": "INFO",
+        "format_preset": "datetime",
         "console_enabled": True,
     },
 )
 
-logger = configure_logger(name="demo", use_colors=False)
+logger = configure_logger(name="billing", use_colors=False)
 
 payload = {
     "service": "billing",
     "token": mask("sk-demo-123456789", 4, 3),
+    "retries": 3,
 }
 
-write_json("tmp/config.json", payload)
+write_json("tmp/config.json", payload, indent=2, ensure_ascii=False)
 loaded = read_json("tmp/config.json")
 
 logger.info("Loaded config: %s", loaded)
+
+config = load_config("config/base.toml", "config/local.yaml")
+logger.info("Feature enabled: %s", config.get("features.billing.enabled"))
 ```
 
-For split configuration setups, `load_config()` merges multiple TOML, JSON, or YAML files into one config object with dot access.
+## Typical workflows
 
-## Public API Overview
+### 1. Read and write structured files
 
-### Core
+```python
+from clevertools import read_toml, write_yaml
 
-- `configure`
-- `log`
-- `mask`
+settings = read_toml("settings.toml", on_error="raise")
+write_yaml("build/settings.yaml", settings, sort_keys=False)
+```
 
-### File helpers
+### 2. Hide secrets before logging
 
-- `read`
-- `write`
-- `read_json`
-- `write_json`
-- `read_toml`
-- `write_toml`
-- `read_yaml`
-- `write_yaml`
-- `load_config`
+```python
+from clevertools import mask, log
 
-### Logging helpers
+token = "sk-prod-abcdef1234567890"
+log.info("Using token %s", mask(token, 5, 4))
+```
+
+### 3. Merge config from multiple sources
+
+```python
+from clevertools import load_config
+
+config = load_config(
+    "config/defaults.toml",
+    "config/team.json",
+    "config/local.yaml",
+)
+
+print(config.database.host)
+print(config.get("features.search.enabled", False))
+```
+
+### 4. Set up application logging
+
+```python
+from clevertools import configure_logger
+
+logger = configure_logger(
+    name="worker",
+    level="INFO",
+    format_preset="datetime",
+    console_enabled=True,
+    file_logging_enabled=True,
+    file_log_path="logs/worker.log",
+    file_write_mode="buffered",
+    use_colors=False,
+)
+
+logger.info("worker started")
+```
+
+## Error handling model
+
+Most helpers support `on_error`. If you do not pass it, the package-wide default from `configure()` is used.
+
+- `"raise"` raises the original exception
+- `"log"` logs the error and returns the helper's fallback value
+- `"silent"` suppresses the error and returns the helper's fallback value
+
+Example:
+
+```python
+from clevertools import configure, read_json
+
+configure(error_mode="silent")
+
+payload = read_json("missing.json")
+print(payload)  # None
+```
+
+## Documentation map
+
+If this is your first time in the repository, read the docs in this order:
 
-- `configure_logger`
-- `get_logger`
-- `CleverToolsFormatter`
-- `build_console_handler`
-- `build_file_handler`
-- `reset_handlers`
-- `resolve_logger_options`
+1. [docs/README.md](./docs/README.md)
+2. [docs/installation.md](./docs/installation.md)
+3. [docs/quickstart.md](./docs/quickstart.md)
+4. [docs/tools/README.md](./docs/tools/README.md)
+5. [docs/concepts/README.md](./docs/concepts/README.md)
 
-## Why It Exists
+Important reference pages:
 
-`clevertools` stays intentionally small. It focuses on practical helpers that are easy to drop into scripts and small projects without turning simple tasks into a framework.
+- [docs/getting-started.md](./docs/getting-started.md) for a guided first run
+- [docs/tools/README.md](./docs/tools/README.md) for the full API map
+- [docs/concepts/error-handling.md](./docs/concepts/error-handling.md) for shared fallback behavior
+- [docs/concepts/logging.md](./docs/concepts/logging.md) for logger architecture and write modes
 
-## Repository Guide
+## Repository layout
 
-- [docs/README.md](./docs/README.md) is the documentation entry point
-- [docs/installation.md](./docs/installation.md) shows local setup
-- [docs/quickstart.md](./docs/quickstart.md) shows an end-to-end example
-- [docs/tools/README.md](./docs/tools/README.md) lists every tool page
-- [docs/concepts/README.md](./docs/concepts/README.md) explains shared behavior
-- [src/clevertools](./src/clevertools) contains the implementation
-- [tests](./tests) contains the test suite
+- `src/clevertools/` contains the library code
+- `tests/` contains the automated test suite
+- `docs/` contains the user-facing documentation
+- `README.md` is the first overview for new users
 
 ## Installation
 
 ```bash
 pip install -e .
 ```
 
-For a full local setup, see [docs/installation.md](./docs/installation.md).
+For a full local setup including virtual environments and dependencies, see [docs/installation.md](./docs/installation.md).
 
 ## Requirements