diff --git a/CHANGELOG.md b/CHANGELOG.md index 37ef579..f106147 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,36 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [Unreleased] + +### Added +- **framework:** `AddUnaryInterceptor`, `AddStreamInterceptor`, and `AddHTTPMiddleware` on `App` — bundles can register gRPC interceptors and HTTP middleware during `Initialize`. +- **framework:** `WithLogging`, `WithObservability`, and `WithHealthRegistry` options to inject custom infrastructure managers, plus an `Observability()` accessor. +- **prometheus:** Automatic HTTP and gRPC request metrics — adding the bundle now records request metrics with no per-handler code. +- **jwt:** `StreamServerInterceptor` so streaming RPCs are authenticated (previously only unary RPCs were), and a `TrustedProxyHeader` config flag gating `X-Forwarded-Proto` trust. +- **httpclient:** Optional `AllowedHosts` allowlist (defense-in-depth SSRF guard) covering both requests and redirect targets. +- **health:** Background check runner honoring per-check `Interval`/`InitialDelay`; probe endpoints serve cached results so a slow dependency cannot stall Kubernetes probes. + +### Changed (breaking) +- **API naming:** `Get`-prefixed getters renamed to drop the prefix — `Tracer`, `Meter`, `RegisteredChecks`, `CheckConfig`, `HealthySummary`, `MetricsHandler`, `SecureMetricsHandler`, `ConfigInfo`, `CircuitBreakerState`, `CircuitBreakerCounts`, and `jwt.ServiceID`/`jwt.ServiceName`. (`CredentialProvider.GetAPIKey`/`GetJWTToken` are unchanged — they are context-taking fetch operations, not accessors.) +- **health:** `HealthStatus` renamed to `Report`; constructors `NewHealthyStatus`/`NewUnhealthyStatus`/`NewUnknownStatus` renamed to `NewHealthyReport`/`NewUnhealthyReport`/`NewUnknownReport`. +- **errors package:** moved from `github.com/datariot/forge/errors` to `github.com/datariot/forge/forgeerrors` (package `forgeerrors`) so it no longer shadows the standard library `errors`. +- **postgresql:** switched the driver from `lib/pq` (maintenance mode) to `pgx/v5/stdlib`. Connection-string compatible; no config changes required. +- Health check HTTP responses no longer include raw per-check error strings (which could leak internal hostnames/ports); full detail remains in logs. +- gRPC reflection now requires an explicit `EnableReflection` flag instead of being on by default in development. + +### Security +- Bumped `google.golang.org/grpc` (GO-2026-4762, authorization bypass) and `go.opentelemetry.io/otel/sdk` (GO-2026-4394, code execution); `govulncheck` reports no vulnerabilities affecting the code. +- **httpclient:** auth headers are stripped on cross-host redirects. +- Opt-in HTTP Basic Auth for the `/metrics` endpoint; pprof is now blocked outside development. + +### Migration from 0.1.0 + +- Replace imports of `github.com/datariot/forge/errors` with `github.com/datariot/forge/forgeerrors` and update references from `errors.X` to `forgeerrors.X`. +- Drop the `Get` prefix from the renamed getters listed above. +- Replace `health.HealthStatus` with `health.Report` and the three `New*Status` constructors with `New*Report`. +- No action needed for the pgx driver swap unless you referenced the `"postgres"` driver name directly. + ## [0.1.0] - 2026-02-25 ### Added @@ -28,4 +58,5 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Overall coverage raised from 29.6% to 70.0% - Added unit tests across all packages: errors (100%), health (96.5%), prometheus (76.4%), httpclient (75.9%), config (73.8%), testutil (71.1%), framework (70.0%), configloader (66.7%), jwt (62.0%), redis (36.0%), postgresql (30.6%) +[Unreleased]: https://github.com/datariot/forge/compare/v0.1.0...HEAD [0.1.0]: https://github.com/datariot/forge/releases/tag/v0.1.0 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..ce4cdf6 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,65 @@ +# Contributing to Forge + +Thanks for your interest in improving Forge. This guide covers how to get set up, what's expected of a change, and how to get it merged. + +## Prerequisites + +- **Go 1.25+** +- **[Task](https://taskfile.dev)** (optional but recommended — `brew install go-task`) +- **Docker + Docker Compose** (only for integration tests) +- **[golangci-lint](https://golangci-lint.run) v2.12.x** — CI pins this version; match it locally to avoid surprises. + +## Getting started + +```bash +git clone https://github.com/datariot/forge +cd forge +task test # or: go test ./... +``` + +## Development workflow + +```bash +task test # unit tests with the race detector +task test:coverage # coverage report (opens coverage.html) +task test:integration # integration tests — starts PostgreSQL + Redis via Docker, runs, tears down +task lint # gofmt check + go vet + golangci-lint +task fmt # format the tree +task build:all # build the framework and every example +task --list # all available tasks +``` + +The raw `go` commands (`go test ./...`, `go build ./...`, `go vet ./...`) work without Task if you prefer. + +## What a change needs + +Before opening a pull request, make sure: + +1. **It builds and passes** — `go build ./...`, `go test ./... -race`, and every module under `examples/` builds. CI runs all three. +2. **It's formatted** — `gofmt -l .` reports nothing. CI fails on unformatted code. +3. **It lints clean** — `golangci-lint run` reports 0 issues. +4. **It's tested** — new behavior comes with tests. The project targets **70%+ coverage** overall (CI enforces the threshold); the health and framework core aim higher. +5. **Exported symbols are documented** — godoc comments on exported types, functions, and methods, starting with the symbol name. pkg.go.dev is the API reference, so this matters. + +### Adding a bundle + +Bundles live under `bundles//` and follow a consistent shape: + +- A `Config` struct with a `Validate() error` method and a `DefaultConfig()` constructor. +- `NewBundle(config Config) *Bundle`. +- `Name() string`, `Initialize(app *framework.App) error`, and `Stop(ctx context.Context) error`. +- Health checks via the `HealthContributor` interface where the bundle manages a dependency. +- **Evaluate third-party dependencies carefully** — Forge is "batteries included" but deliberately lightweight. A new direct dependency should earn its place; prefer the standard library where it's close enough. + +Model new bundles on `bundles/postgresql` or `bundles/redis`. + +## Commit and PR conventions + +- Use [Conventional Commits](https://www.conventionalcommits.org) (`feat:`, `fix:`, `docs:`, `refactor:`, `chore:`; use `!` for breaking changes, e.g. `refactor!:`). +- Keep pull requests focused; a reviewable diff beats a sweeping one. +- Note any breaking API change in the PR description and in [CHANGELOG.md](CHANGELOG.md) under "Unreleased". +- CI (unit tests with race detection, formatting, linting, example builds, coverage) must be green before merge. + +## Reporting issues + +Open a GitHub issue with the Forge version (or commit), Go version, a minimal reproduction if you can, and what you expected versus what happened. diff --git a/README.md b/README.md index 717744b..8ef20af 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,50 @@ # Forge -A batteries-included Go framework for building production-ready microservices. +[![CI](https://github.com/datariot/forge/actions/workflows/test.yml/badge.svg)](https://github.com/datariot/forge/actions/workflows/test.yml) +[![Go Reference](https://pkg.go.dev/badge/github.com/datariot/forge.svg)](https://pkg.go.dev/github.com/datariot/forge) +[![Go Report Card](https://goreportcard.com/badge/github.com/datariot/forge)](https://goreportcard.com/report/github.com/datariot/forge) +[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) +![Go 1.25+](https://img.shields.io/badge/Go-1.25%2B-00ADD8?logo=go) -## Overview +A batteries-included Go framework for building production-ready microservices — inspired by DropWizard, designed for Go's strengths. -Forge is inspired by DropWizard but designed specifically for Go's strengths - interfaces, goroutines, and clean composition. It provides opinionated defaults while maintaining flexibility through a pluggable architecture. +Forge wires up the boilerplate every service re-implements — lifecycle orchestration, health checks, structured logging, tracing, metrics, graceful shutdown — behind a small, composable API, so you write business logic instead of plumbing. -## Features +```go +app, err := framework.New( + framework.WithConfig(&cfg), + framework.WithComponent(myService), + framework.WithBundle(postgresql.NewBundle(dbConfig)), +) +if err != nil { + log.Fatal(err) +} +log.Fatal(app.Run(context.Background())) +``` + +## Why Forge + +**Use Forge when** you're running more than one Go service and are tired of copy-pasting the same `main.go` — the health endpoints, the OTel setup, the signal handling, the "did I remember to drain connections on shutdown" checklist. Forge gives you opinionated, production-grade defaults for all of it and a bundle system to add PostgreSQL, Redis, JWT auth, a resilient HTTP client, and Prometheus metrics with one line each. + +**What you get out of the box:** + +- **Lifecycle orchestration** — ordered startup, reverse-order graceful shutdown, timeout handling, and startup/shutdown hooks. +- **Health checks** — Kubernetes-style `/health`, `/health/ready`, `/health/live` plus a gRPC health service, run on a background schedule and cached so a slow dependency can't stall your probes. +- **Observability** — OpenTelemetry tracing (W3C propagation, configurable sampling), structured JSON logging via zerolog, and automatic HTTP/gRPC request metrics when the Prometheus bundle is added. +- **Bundles** — PostgreSQL (pooled, via pgx), Redis (cache, pub/sub, distributed locks, rate limiting), JWT service auth (unary + stream interceptors), a circuit-breaking HTTP client, Prometheus, and multi-source config loading with hot reload. +- **Config with validation** — a common `BaseConfig` with environment-aware defaults; layer YAML files and environment overrides via the configloader bundle. + +**Forge is probably not for you if** you want a full web framework with routing, middleware, and an ORM (reach for Echo/Gin/Fiber + your data layer), or if your service is a single small binary where the framework's structure is more than you need. Forge is opinionated about *operability*, not about how you write handlers. + +**Status:** pre-1.0. The API is settling and may still change between minor versions; pin your dependency and read the [CHANGELOG](CHANGELOG.md) before upgrading. + +## Install + +```bash +go get github.com/datariot/forge +``` -- **Clean Architecture**: Component-based design with clear separation of concerns -- **Observability Built-in**: OpenTelemetry tracing, structured logging, Prometheus metrics -- **Health Checks**: Comprehensive liveness and readiness checks -- **Graceful Lifecycle**: Sophisticated startup and shutdown orchestration -- **Configuration Management**: Validated config with YAML + env overrides via the configloader bundle -- **Database Integration**: PostgreSQL connection pooling with health checks -- **Redis Integration**: Caching, pub/sub, distributed locks, and rate limiting -- **Resilient HTTP Client**: Circuit breaker, retries, and backoff built in -- **Security First**: No hardcoded credentials, explicit validation requirements +Requires Go 1.25+. ## Quick Start @@ -27,137 +55,89 @@ import ( "context" "log" - "github.com/datariot/forge/bundles/postgresql" "github.com/datariot/forge/config" "github.com/datariot/forge/framework" + "github.com/datariot/forge/health" ) +type Greeter struct{} + +func (Greeter) Start(ctx context.Context) error { log.Println("greeter up"); return nil } +func (Greeter) Stop(ctx context.Context) error { log.Println("greeter down"); return nil } + +func (Greeter) HealthChecks() []health.Check { + return []health.Check{health.NewAlwaysHealthyCheck("greeter")} +} + func main() { cfg := config.DefaultBaseConfig() - cfg.ServiceName = "my-service" - - pgConfig := postgresql.DefaultConfig() - pgConfig.DatabaseURL = "postgres://user:pass@localhost:5432/mydb" + cfg.ServiceName = "greeter" + svc := Greeter{} app, err := framework.New( framework.WithConfig(&cfg), framework.WithVersion("1.0.0"), - framework.WithComponent(NewMyComponent(&cfg)), - framework.WithBundle(postgresql.NewBundle(pgConfig)), + framework.WithComponent(svc), + framework.WithHealthContributor(svc), ) if err != nil { log.Fatal(err) } + // Run blocks until SIGINT/SIGTERM, then shuts down gracefully. if err := app.Run(context.Background()); err != nil { log.Fatal(err) } } ``` -## Installation +Run it, then hit the health endpoints on the HTTP server (`:8081` by default): ```bash -go get github.com/datariot/forge +curl localhost:8081/health # overall status +curl localhost:8081/health/ready # readiness probe +curl localhost:8081/health/live # liveness probe ``` -## Development - -### Prerequisites +The gRPC server (`:8080` by default) starts only when you register a gRPC service with `framework.WithGRPCRegistrar(...)`, so an HTTP-only service like the one above runs without it. -- Go 1.25+ -- Docker & Docker Compose (for integration tests) -- [Task](https://taskfile.dev) (optional, for task runner) +See [`examples/`](examples/) for complete services using each bundle, and the [Getting Started guide](docs/getting-started.md) for a step-by-step walkthrough. -### Common Commands +## Bundles -```bash -# Run tests -task test -# OR -go test ./... +Add a bundle with `framework.WithBundle(...)`; it self-initializes during startup, contributes its own health checks, and cleans up on shutdown. -# Run tests with coverage -task test:coverage +| Bundle | What it provides | +|--------|------------------| +| `bundles/postgresql` | Pooled `database/sql` access via pgx, with a readiness check | +| `bundles/redis` | Cache, pub/sub, distributed locks, sliding-window rate limiting | +| `bundles/jwt` | Service-to-service auth with gRPC unary + stream interceptors and HTTP middleware | +| `bundles/httpclient` | HTTP client with circuit breaker, retries, backoff, and an optional host allowlist | +| `bundles/prometheus` | Metrics registry and **automatic** HTTP/gRPC request metrics | +| `bundles/configloader` | YAML + environment config loading with validation and hot reload | -# Run integration tests (requires Docker) -task test:integration +Full reference: [pkg.go.dev/github.com/datariot/forge](https://pkg.go.dev/github.com/datariot/forge). -# Build framework and examples -task build:all - -# Format and lint code -task lint - -# See all available tasks -task --list -``` - -### Testing +## Documentation -**Unit Tests** (no external dependencies): -```bash -task test -``` +- [Getting Started](docs/getting-started.md) — build your first service step by step +- [Bundles Guide](docs/bundles.md) — configuring and using each integration +- [API Reference](https://pkg.go.dev/github.com/datariot/forge) — full godoc on pkg.go.dev +- [Examples](examples/) — runnable services +- [Contributing](CONTRIBUTING.md) — dev setup and PR workflow -**Integration Tests** (requires Docker): -```bash -task docker:up # Start PostgreSQL + Redis -task test:integration # Run integration tests -task docker:down # Stop services -``` +## Development -**Coverage Report**: ```bash -task test:coverage # Generates coverage.html +task test # unit tests (race-enabled) +task test:coverage # coverage report +task lint # gofmt check + go vet + golangci-lint +task build:all # framework + examples +task --list # everything else ``` -Current test coverage: **70.0%** (Target: 70%+) - -See [TESTING.md](TESTING.md) for comprehensive testing strategy. - -## Architecture - -Forge follows Clean Architecture principles: - -- **Framework**: Core application lifecycle and interfaces -- **Bundles**: Pre-built integrations (PostgreSQL, Redis, JWT, HTTP client, Prometheus, configloader) -- **Components**: Your business logic implementing framework interfaces -- **Config**: Common service configuration with validation - -## Documentation - -- [Getting Started](docs/getting-started.md) -- [API Reference](docs/api-reference.md) -- [Bundles Guide](docs/bundles.md) -- [Examples](examples/) -- [Development Guide](CLAUDE.md) - -## CI/CD - -GitHub Actions automatically runs on all PRs: -- Unit tests with race detection -- Code formatting checks -- Linting (go vet + golangci-lint) -- Build verification (framework + examples) -- Coverage reporting (70% threshold) - -See [.github/workflows/test.yml](.github/workflows/test.yml) for details. +`go test ./...` and `go build ./...` work without Task if you prefer. Integration tests (PostgreSQL + Redis) run under `task test:integration` and require Docker. See [CONTRIBUTING.md](CONTRIBUTING.md). ## License -MIT License - see LICENSE file for details. - -## Contributing - -1. Fork the repository -2. Create a feature branch -3. Add tests for your changes -4. Ensure `task ci` passes -5. Submit a pull request - -All contributions must: -- Include tests (maintain 70%+ coverage) -- Follow Go best practices -- Include documentation -- Pass CI/CD checks +MIT — see [LICENSE](LICENSE). diff --git a/docs/api-reference.md b/docs/api-reference.md index ad2a5d9..f7ff3c9 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -6,429 +6,61 @@ permalink: /api-reference/ # API Reference -Complete API documentation for the Forge framework. +The complete, always-current API reference is generated from the source and hosted on pkg.go.dev: -## Core Framework +## → [pkg.go.dev/github.com/datariot/forge](https://pkg.go.dev/github.com/datariot/forge) -### Application (`framework.App`) +Rather than duplicate godoc here (where it drifts out of date), this page orients you to the packages. Follow the links for full type and method documentation. -The main application struct that orchestrates the entire microservice lifecycle. +## Packages -```go -app, err := framework.New(options ...AppOption) -``` - -**Configuration Options:** - -- `WithConfig(*config.BaseConfig)` - Set base configuration -- `WithVersion(string)` - Set service version -- `WithComponent(Component)` - Add business logic component -- `WithBundle(Bundle)` - Add integration bundle -- `WithGRPCRegistrar(Registrar)` - Add gRPC service -- `WithHealthContributor(HealthContributor)` - Add health checks -- `WithUnaryInterceptor(grpc.UnaryServerInterceptor)` - Add gRPC interceptor -- `WithStartupHook(StartupHook)` - Add startup hook -- `WithShutdownHook(ShutdownHook)` - Add shutdown hook - -**Methods:** +| Package | Purpose | Reference | +|---------|---------|-----------| +| `framework` | Application lifecycle, the `App` builder, `Component`/`Bundle`/`HealthContributor` interfaces, functional options | [godoc](https://pkg.go.dev/github.com/datariot/forge/framework) | +| `config` | `BaseConfig`, environment-aware defaults, validation | [godoc](https://pkg.go.dev/github.com/datariot/forge/config) | +| `health` | `Report`, `Check`, `Registry`, liveness/readiness model | [godoc](https://pkg.go.dev/github.com/datariot/forge/health) | +| `forgeerrors` | Structured `DomainError` with codes and classification | [godoc](https://pkg.go.dev/github.com/datariot/forge/forgeerrors) | +| `testutil` | Test helpers for framework consumers | [godoc](https://pkg.go.dev/github.com/datariot/forge/testutil) | +| `bundles/postgresql` | Pooled PostgreSQL via pgx | [godoc](https://pkg.go.dev/github.com/datariot/forge/bundles/postgresql) | +| `bundles/redis` | Cache, pub/sub, locks, rate limiting | [godoc](https://pkg.go.dev/github.com/datariot/forge/bundles/redis) | +| `bundles/jwt` | Service-to-service JWT auth | [godoc](https://pkg.go.dev/github.com/datariot/forge/bundles/jwt) | +| `bundles/httpclient` | Resilient HTTP client | [godoc](https://pkg.go.dev/github.com/datariot/forge/bundles/httpclient) | +| `bundles/prometheus` | Metrics + automatic request instrumentation | [godoc](https://pkg.go.dev/github.com/datariot/forge/bundles/prometheus) | +| `bundles/configloader` | Multi-source config with hot reload | [godoc](https://pkg.go.dev/github.com/datariot/forge/bundles/configloader) | -- `Run(context.Context) error` - Start application and block until shutdown -- `Start(context.Context) error` - Start application components -- `Stop(context.Context) error` - Gracefully stop application -- `Config() *config.BaseConfig` - Get application configuration -- `HealthRegistry() *health.Registry` - Get health check registry -- `IsRunning() bool` - Check if application is running +## Core building blocks -### Interfaces - -#### Component - -Your business logic implements this interface: +Your service implements these framework interfaces: ```go +// Business logic — started in registration order, stopped in reverse. type Component interface { Start(ctx context.Context) error Stop(ctx context.Context) error } -``` - -#### Bundle -Pre-built integrations implement this interface: - -```go -type Bundle interface { - Name() string - Initialize(app *App) error -} -``` - -#### HealthContributor - -Components providing health checks implement this interface: - -```go +// Optional: contribute health checks. type HealthContributor interface { HealthChecks() []health.Check } -``` - -#### Registrar -gRPC services implement this interface: - -```go +// Optional: register gRPC services. type Registrar interface { RegisterGRPC(server *grpc.Server) error } ``` -## Configuration (`config`) - -### BaseConfig - -Standard configuration for all Forge services: - -```go -type BaseConfig struct { - // Service identification - ServiceName string `yaml:"service_name" env:"SERVICE_NAME"` - AppEnv string `yaml:"app_env" env:"APP_ENV"` - - // Server configuration - GRPCAddr string `yaml:"grpc_addr" env:"GRPC_ADDR"` - HTTPAddr string `yaml:"http_addr" env:"HTTP_ADDR"` - - // Logging - LogLevel string `yaml:"log_level" env:"LOG_LEVEL"` - - // Timeouts - ShutdownTimeout time.Duration `yaml:"shutdown_timeout" env:"SHUTDOWN_TIMEOUT"` - ReadinessInitialDelay time.Duration `yaml:"readiness_initial_delay" env:"READINESS_INITIAL_DELAY"` - - // Infrastructure URLs - DatabaseURL string `yaml:"database_url" env:"DATABASE_URL"` - RedisURL string `yaml:"redis_url" env:"REDIS_URL"` - - // Features - EnablePprof bool `yaml:"enable_pprof" env:"ENABLE_PPROF"` - EnableReflection bool `yaml:"enable_reflection" env:"ENABLE_REFLECTION"` - EnableMetrics bool `yaml:"enable_metrics" env:"ENABLE_METRICS"` -} -``` - -**Methods:** - -- `Validate() error` - Validate configuration -- `IsDevelopment() bool` - Check if development environment -- `IsProduction() bool` - Check if production environment -- `ShouldEnableReflection() bool` - Check if gRPC reflection should be enabled - -## Health Checks (`health`) - -### Check Interface - -Health checks implement this interface: - -```go -type Check interface { - Name() string - Liveness(ctx context.Context) error - Readiness(ctx context.Context) error -} -``` - -### Registry - -Health check registry manages all health checks: - -```go -registry := health.NewRegistry(logger) -registry.Register(check, config) -``` - -**Methods:** - -- `Register(Check, CheckConfig) error` - Register health check -- `CheckLiveness(context.Context) Report` - Check liveness -- `CheckReadiness(context.Context) Report` - Check readiness -- `CheckHealth(context.Context) Report` - Check overall health - -### Built-in Checks - -- `NewAlwaysHealthyCheck(name)` - Always reports healthy -- `NewAlwaysUnhealthyCheck(name, err)` - Always reports unhealthy -- `NewBasicCheck(config, liveness, readiness)` - Custom check functions - -## Error Handling (`forgeerrors`) - -### DomainError - -Structured errors with context: - -```go -type DomainError struct { - Code string `json:"code"` - Message string `json:"message"` - Cause error `json:"cause,omitempty"` -} -``` - -**Methods:** - -- `WithMessage(format, args...) DomainError` - Add context message -- `WithCause(error) DomainError` - Add underlying cause -- `Unwrap() error` - For error unwrapping - -### Predefined Errors - -```go -var ( - ErrInvalidConfiguration = DomainError{Code: "INVALID_CONFIGURATION", ...} - ErrRepositoryUnavailable = DomainError{Code: "REPOSITORY_UNAVAILABLE", ...} - ErrAuthenticationFailed = DomainError{Code: "AUTHENTICATION_FAILED", ...} - ErrServiceUnavailable = DomainError{Code: "SERVICE_UNAVAILABLE", ...} - // ... more predefined errors -) -``` - -### Error Classification - -- `IsTransientError(error) bool` - Check if error is retryable -- `IsAuthenticationError(error) bool` - Check if authentication related -- `IsValidationError(error) bool` - Check if validation related -- `IsConfigurationError(error) bool` - Check if configuration related - -## Bundle APIs - -### PostgreSQL Bundle - -```go -import "github.com/datariot/forge/bundles/postgresql" - -bundle := postgresql.NewBundle(postgresql.Config{ - DatabaseURL: "postgres://...", - MaxOpenConns: 25, - MaxIdleConns: 10, - ConnMaxLifetime: 30 * time.Minute, - HealthCheckTimeout: 5 * time.Second, -}) - -db := bundle.DB() // Get *sql.DB instance -``` - -### Redis Bundle - -```go -import "github.com/datariot/forge/bundles/redis" - -bundle := redis.NewBundle(redis.Config{ - RedisURL: "redis://localhost:6379/0", - PoolSize: 10, -}) - -// High-level interfaces -cache := bundle.Cache() -pubsub := bundle.PubSub() -lock := bundle.NewDistributedLock("resource", 30*time.Second) -limiter := bundle.NewRateLimiter("api") - -// Cache operations -cache.Set(ctx, "key", value, 1*time.Hour) -cache.Get(ctx, "key", &dest) -cache.Delete(ctx, "key1", "key2") - -// Pub/sub operations -subscription := pubsub.Subscribe(ctx, "channel") -defer subscription.Close() -pubsub.Publish(ctx, "channel", message) - -// Distributed locking -acquired, err := lock.TryLock(ctx) -if acquired { - defer lock.Unlock(ctx) - // Critical section -} - -// Rate limiting -allowed, err := limiter.Allow(ctx, userID, 100, 1*time.Minute) -``` - -### JWT Authentication Bundle +And compose the application with functional options: ```go -import "github.com/datariot/forge/bundles/jwt" - -bundle := jwt.NewBundle(jwt.Config{ - SecretKey: []byte("secret"), - Issuer: "auth-service", - Audience: "services", - TokenDuration: 1 * time.Hour, -}) - -// Server-side authentication app, err := framework.New( - framework.WithUnaryInterceptor(bundle.UnaryServerInterceptor()), -) - -// Client-side token injection -conn, err := grpc.Dial("target:8080", - grpc.WithUnaryInterceptor(bundle.UnaryClientInterceptor()), -) - -// HTTP middleware -mux.Handle("/api/", bundle.HTTPMiddleware(protectedHandler)) - -// Permission-based protection -mux.Handle("/admin/", bundle.RequirePermissions("admin")(adminHandler)) - -// Access claims in handlers -claims := jwt.ClaimsFromContext(ctx) -serviceID := jwt.ServiceID(ctx) -hasPermission := jwt.HasPermission(ctx, "read:users") -``` - -### HTTP Client Bundle - -```go -import "github.com/datariot/forge/bundles/httpclient" - -bundle := httpclient.NewBundle(httpclient.Config{ - BaseURL: "https://api.example.com", - Timeout: 30 * time.Second, - RetryConfig: httpclient.RetryConfig{ - MaxRetries: 3, - InitialInterval: 100 * time.Millisecond, - }, -}) - -client := bundle.Client() - -// Type-safe requests -var user User -err := client.Get(ctx, "/users/123", &user) - -var response CreateResponse -err := client.Post(ctx, "/users", createRequest, &response) - -// Raw requests -resp, err := client.RawRequest(ctx, "GET", "/custom", headers, body) - -// Circuit breaker monitoring -state := client.CircuitBreakerState() -counts := client.CircuitBreakerCounts() -``` - -### Prometheus Bundle - -```go -import "github.com/datariot/forge/bundles/prometheus" - -bundle := prometheus.NewBundle(prometheus.Config{ - Namespace: "myservice", - EnableDefaultMetrics: true, - EnableHTTPMetrics: true, -}) - -// Custom metrics -counter, err := bundle.CreateCustomCounter( - "operations_total", - "Total operations", - []string{"type", "status"}, -) - -histogram, err := bundle.CreateCustomHistogram( - "duration_seconds", - "Operation duration", - []string{"operation"}, + framework.WithConfig(&cfg), + framework.WithVersion("1.0.0"), + framework.WithComponent(svc), + framework.WithBundle(postgresql.NewBundle(dbConfig)), + framework.WithHealthContributor(svc), + framework.WithGRPCRegistrar(svc), // starts the gRPC server ) - -// Record metrics -bundle.RecordHTTPRequest("GET", "/api/users", 200, duration) -bundle.RecordGRPCRequest("UserService.GetUser", "OK", duration) -bundle.RecordHealthCheck("database", "readiness", true, duration) - -// Get metrics handler -handler := bundle.MetricsHandler() ``` -### Configuration Loading Bundle - -```go -import "github.com/datariot/forge/bundles/configloader" - -bundle := configloader.NewBundle(configloader.Config{ - ConfigPaths: []string{"./config.yaml"}, - EnvPrefix: "MYSERVICE", - WatchFiles: true, -}) - -// Load configuration -var cfg MyServiceConfig -result, err := bundle.Loader().Load(&cfg) - -// Hot reload -bundle.Loader().OnConfigChange(func(newConfig interface{}) { - // Handle configuration changes -}) - -// Generic loading -cfg, result, err := configloader.LoadConfig[MyServiceConfig]("./config.yaml") -``` - -## HTTP Endpoints - -All Forge services automatically expose: - -| Endpoint | Purpose | Method | -|----------|---------|--------| -| `/health` | Overall health status | GET | -| `/health/ready` | Readiness probe | GET | -| `/health/live` | Liveness probe | GET | -| `/metrics` | Prometheus metrics | GET | -| `/debug/pprof/*` | Debug endpoints (dev only) | GET | - -## Environment Variables - -Standard environment variables for all services: - -| Variable | Purpose | Default | -|----------|---------|---------| -| `SERVICE_NAME` | Service identifier | forge-service | -| `APP_ENV` | Environment (development/staging/production) | development | -| `GRPC_ADDR` | gRPC server address | :8080 | -| `HTTP_ADDR` | HTTP server address | :8081 | -| `LOG_LEVEL` | Logging level | info | -| `SHUTDOWN_TIMEOUT` | Graceful shutdown timeout | 30s | -| `DATABASE_URL` | PostgreSQL connection string | - | -| `REDIS_URL` | Redis connection string | - | -| `JWT_SECRET` | JWT signing secret | - | -| `ENABLE_PPROF` | Enable debug endpoints | false | -| `ENABLE_REFLECTION` | Enable gRPC reflection | false (true in dev) | -| `ENABLE_METRICS` | Enable metrics collection | true | - -## Error Codes - -Standard error codes across all Forge services: - -| Code | HTTP Status | Description | -|------|-------------|-------------| -| `INVALID_CONFIGURATION` | 500 | Configuration validation failed | -| `REPOSITORY_UNAVAILABLE` | 503 | Database/cache unavailable | -| `AUTHENTICATION_FAILED` | 401 | Authentication required | -| `INVALID_CREDENTIAL` | 401 | Invalid or expired credential | -| `INSUFFICIENT_PERMISSIONS` | 403 | Missing required permissions | -| `SERVICE_UNAVAILABLE` | 503 | External service unavailable | -| `RATE_LIMIT_EXCEEDED` | 429 | Rate limit exceeded | - -## Examples Repository - -Complete examples are available in the [examples directory](https://github.com/datariot/forge/tree/main/examples): - -- `simple-service/` - Basic framework usage -- `postgresql-service/` - Database integration -- `redis-service/` - Caching and messaging -- `jwt-service/` - Authentication patterns -- `httpclient-service/` - Service communication -- `prometheus-service/` - Metrics and observability -- `config-service/` - Configuration management \ No newline at end of file +See [Getting Started](getting-started.html) for a full walkthrough and [Bundles](bundles.html) for per-integration configuration. diff --git a/docs/getting-started.md b/docs/getting-started.md index a8c326e..731ee1e 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -89,12 +89,12 @@ go run main.go ``` Your service will start with: -- **gRPC server** on `:8080` - **HTTP health endpoints** on `:8081` - `http://localhost:8081/health` - Overall health - `http://localhost:8081/health/ready` - Readiness probe - `http://localhost:8081/health/live` - Liveness probe - - `http://localhost:8081/metrics` - Prometheus metrics + - `http://localhost:8081/metrics` - Prometheus metrics (when the prometheus bundle is added) +- **gRPC server** on `:8080` — started only when you register a gRPC service with `framework.WithGRPCRegistrar(...)`. The example above is HTTP-only, so no gRPC server runs. ## Adding Database Connectivity @@ -178,7 +178,9 @@ func main() { app, err := framework.New( framework.WithConfig(&cfg), framework.WithBundle(jwtBundle), // Add authentication + // Register BOTH interceptors — the unary one does not cover streaming RPCs. framework.WithUnaryInterceptor(jwtBundle.UnaryServerInterceptor()), + framework.WithStreamInterceptor(jwtBundle.StreamServerInterceptor()), framework.WithComponent(service), ) } @@ -214,8 +216,8 @@ go run main.go ## Next Steps - **[Explore Bundles](bundles.html)** - Learn about available integrations -- **[View Examples](examples.html)** - See complete working examples -- **[API Reference](api-reference.html)** - Detailed API documentation +- **[View Examples](https://github.com/datariot/forge/tree/main/examples)** - Complete working services +- **[API Reference](https://pkg.go.dev/github.com/datariot/forge)** - Full godoc on pkg.go.dev ## Key Concepts @@ -238,6 +240,7 @@ Bundles are pre-built integrations that add functionality to your application: type Bundle interface { Name() string Initialize(app *App) error + Stop(ctx context.Context) error } ``` diff --git a/docs/index.md b/docs/index.md index b9a0fcf..9d6cd1b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -123,7 +123,7 @@ Forge follows Clean Architecture principles with these key concepts: - Automatic HTTP/gRPC metrics collection - Health check monitoring and alerting - Distributed tracing with OpenTelemetry -- Pre-built Grafana dashboards +- A starter Grafana dashboard ## Getting Started diff --git a/framework/http.go b/framework/http.go index 05c767d..8cbe0a8 100644 --- a/framework/http.go +++ b/framework/http.go @@ -1,14 +1,14 @@ -// Package framework HTTP server enhancements provide production-ready HTTP endpoints. -// -// The enhanced HTTP server includes: +// The HTTP server built by HTTPServerBuilder is the production-ready HTTP +// endpoint layer every Forge service gets automatically. It provides: // - Health endpoints (/health, /health/ready, /health/live) -// - Metrics endpoint (/metrics) with Prometheus integration +// - A metrics endpoint (/metrics) with Prometheus integration // - Debug endpoints (/debug/pprof/*) when enabled // - Request logging middleware // - CORS support with configuration -// - Graceful shutdown handling +// - Graceful shutdown handling via the http.Server built by Build // -// All endpoints are configurable and can be enabled/disabled based on environment. +// All endpoints are configurable via HTTPServerConfig and can be enabled or +// disabled based on environment. package framework diff --git a/framework/logging.go b/framework/logging.go index 9d0ca28..c2b1789 100644 --- a/framework/logging.go +++ b/framework/logging.go @@ -10,20 +10,31 @@ import ( "github.com/datariot/forge/config" ) -// LoggingManager manages logging configuration and provides logger instances. +// LoggingManager owns the application's zerolog configuration and hands out +// logger instances tagged with service and component context. Call +// Initialize before using Logger, WithService, or WithContext so the +// underlying logger has its output format (pretty console in development, +// JSON in production) and log level configured. type LoggingManager struct { config *config.BaseConfig logger zerolog.Logger } -// NewLoggingManager creates a new logging manager with the given configuration. +// NewLoggingManager creates a LoggingManager bound to the given +// configuration. The returned manager's logger is not yet configured; call +// Initialize before using it. func NewLoggingManager(config *config.BaseConfig) *LoggingManager { return &LoggingManager{ config: config, } } -// Initialize sets up the logging configuration. +// Initialize configures the manager's base logger from its BaseConfig: it +// sets the global zerolog level (falling back to Info if LogLevel doesn't +// parse), switches between a human-readable console writer in development +// and JSON output otherwise, and tags every log line with the service name +// and environment. The App calls this automatically during New, before +// observability and bundles are initialized. func (lm *LoggingManager) Initialize() error { // Set global log level based on configuration level, err := zerolog.ParseLevel(lm.config.LogLevel) @@ -53,12 +64,15 @@ func (lm *LoggingManager) Initialize() error { return nil } -// Logger returns the base logger instance. +// Logger returns a pointer to the manager's base zerolog.Logger, already +// tagged with service and environment fields by Initialize. func (lm *LoggingManager) Logger() *zerolog.Logger { return &lm.logger } -// WithService creates a logger with service and component context. +// WithService returns a copy of the base logger with additional "service" +// and "component" fields attached, for use by a specific subsystem (e.g. the +// gRPC or HTTP server) so its log lines can be filtered independently. func (lm *LoggingManager) WithService(service, component string) zerolog.Logger { return lm.logger.With(). Str("service", service). @@ -66,7 +80,10 @@ func (lm *LoggingManager) WithService(service, component string) zerolog.Logger Logger() } -// WithContext creates a logger with additional context fields. +// WithContext returns a copy of the base logger with an arbitrary set of +// key-value fields attached, one call to Interface per map entry, for +// callers that need structured fields beyond the fixed service/component +// pair provided by WithService. func (lm *LoggingManager) WithContext(fields map[string]any) zerolog.Logger { ctx := lm.logger.With() for key, value := range fields { @@ -75,7 +92,10 @@ func (lm *LoggingManager) WithContext(fields map[string]any) zerolog.Logger { return ctx.Logger() } -// NewHealthLogger creates a logger adapter for the health system. +// NewHealthLogger wraps a LoggingManager in an adapter satisfying the +// health package's logger interface, so the health registry's background +// checks log through the same structured, service-tagged zerolog output as +// the rest of the application instead of a separate logging path. func NewHealthLogger(logging *LoggingManager) *healthLoggerAdapter { return &healthLoggerAdapter{ logger: logging.WithService(logging.config.ServiceName, "health"), diff --git a/framework/observability.go b/framework/observability.go index 0ea4d1a..3fcb88c 100644 --- a/framework/observability.go +++ b/framework/observability.go @@ -21,7 +21,9 @@ import ( "github.com/datariot/forge/config" ) -// ObservabilityConfig contains configuration for observability (tracing, metrics). +// ObservabilityConfig contains the settings used to initialize OpenTelemetry +// tracing and metrics: the service's identity (name, version, environment), +// the OTLP collector endpoint to export to, and the trace sampling rate. type ObservabilityConfig struct { ServiceName string ServiceVersion string @@ -30,7 +32,9 @@ type ObservabilityConfig struct { SampleRate float64 } -// NewObservabilityConfig creates a new observability config from base config. +// NewObservabilityConfig derives an ObservabilityConfig from the +// application's BaseConfig and the given service version, copying over the +// service name, environment, OTEL endpoint, and sample rate. func NewObservabilityConfig(baseConfig *config.BaseConfig, version string) *ObservabilityConfig { return &ObservabilityConfig{ ServiceName: baseConfig.ServiceName, @@ -41,7 +45,11 @@ func NewObservabilityConfig(baseConfig *config.BaseConfig, version string) *Obse } } -// ObservabilityManager manages OpenTelemetry tracing and metrics. +// ObservabilityManager owns the OpenTelemetry tracing and metrics providers +// for the life of the application. Call Initialize once during startup to +// wire up exporters from the manager's config, use Tracer and Meter to +// obtain instrumentation handles, and call Shutdown during graceful +// shutdown to flush pending telemetry and release exporter resources. type ObservabilityManager struct { config *ObservabilityConfig traceProvider *trace.TracerProvider @@ -49,14 +57,21 @@ type ObservabilityManager struct { initialized bool } -// NewObservabilityManager creates a new observability manager. +// NewObservabilityManager creates an ObservabilityManager for the given +// configuration. The returned manager is not yet initialized; call +// Initialize before relying on Tracer or Meter to return non-no-op +// instrumentation. func NewObservabilityManager(config *ObservabilityConfig) *ObservabilityManager { return &ObservabilityManager{ config: config, } } -// Initialize sets up OpenTelemetry tracing and metrics. +// Initialize sets up OpenTelemetry tracing and metrics using the manager's +// configuration. It is idempotent: once initialization has succeeded, +// subsequent calls return nil without doing any work. The App calls this +// automatically during Start, before bundles are initialized, so components +// and bundles can safely assume tracing/metrics are ready by the time they run. func (om *ObservabilityManager) Initialize(ctx context.Context) error { if om.initialized { return nil @@ -206,7 +221,10 @@ func (om *ObservabilityManager) initializeMetrics(ctx context.Context) error { return nil } -// Shutdown gracefully shuts down the observability components. +// Shutdown flushes and shuts down the trace and meter providers created by +// Initialize, returning a joined error if either fails to shut down +// cleanly. It is a no-op if Initialize was never called or did not +// complete successfully. func (om *ObservabilityManager) Shutdown(ctx context.Context) error { if !om.initialized { return nil @@ -236,7 +254,12 @@ func (om *ObservabilityManager) Shutdown(ctx context.Context) error { return nil } -// Tracer returns a tracer for the given name. +// Tracer returns an OpenTelemetry Tracer for the given instrumentation name +// (conventionally the calling package or component name). Use it to start +// spans around units of work you want traced, e.g. +// om.Tracer("myservice").Start(ctx, "operation-name"). If Initialize has not +// configured a trace provider (no OTEL endpoint set), this falls back to the +// global otel.Tracer, which produces no-op spans rather than erroring. func (om *ObservabilityManager) Tracer(name string) oteltrace.Tracer { if om.traceProvider == nil { return otel.Tracer(name) @@ -244,7 +267,12 @@ func (om *ObservabilityManager) Tracer(name string) oteltrace.Tracer { return om.traceProvider.Tracer(name) } -// Meter returns a meter for the given name. +// Meter returns an OpenTelemetry Meter for the given instrumentation name +// (conventionally the calling package or component name). Use it to create +// instruments — counters, histograms, gauges, etc. — for recording metrics. +// If Initialize has not configured a meter provider (no OTEL endpoint set), +// this falls back to the global otel.Meter, which produces no-op +// instruments rather than erroring. func (om *ObservabilityManager) Meter(name string) otelmetric.Meter { if om.meterProvider == nil { return otel.Meter(name) diff --git a/health/registry.go b/health/registry.go index 4053426..c00e3c4 100644 --- a/health/registry.go +++ b/health/registry.go @@ -82,9 +82,16 @@ type Logger interface { // NoopLogger is a logger that does nothing. type NoopLogger struct{} +// Debug discards a debug-level message. func (NoopLogger) Debug(msg string, fields ...any) {} -func (NoopLogger) Info(msg string, fields ...any) {} -func (NoopLogger) Warn(msg string, fields ...any) {} + +// Info discards an info-level message. +func (NoopLogger) Info(msg string, fields ...any) {} + +// Warn discards a warning-level message. +func (NoopLogger) Warn(msg string, fields ...any) {} + +// Error discards an error-level message. func (NoopLogger) Error(msg string, fields ...any) {} // NewRegistry creates a new health check registry.