diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6a434c79a..34077fef8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -8,184 +8,138 @@ # Contributing to openframe-oss-lib -Thank you for contributing to **openframe-oss-lib**! This guide covers everything you need to know: code style, branching strategy, commit conventions, pull request process, and security requirements. +Thank you for contributing to `openframe-oss-lib`! This document outlines how to get involved, our code conventions, branching strategy, commit standards, and the pull request process. --- ## Community First -All contribution discussions happen on the [OpenMSP Slack Community](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA). We do **not** use GitHub Issues or GitHub Discussions. +> **All project discussion, support, and contributions are coordinated through the OpenMSP Slack community.** +> We do not use GitHub Issues or GitHub Discussions. +> +> πŸ’¬ **Join**: [https://www.openmsp.ai/](https://www.openmsp.ai/) +> +> πŸ”— **Direct invite**: [https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) -Before starting a large contribution: - -1. Join the [OpenMSP Slack](https://www.openmsp.ai/) -2. Describe what you're planning in the `#openframe-dev` channel -3. Get feedback from maintainers before investing significant effort +Before opening a pull request for a significant change, please discuss it in Slack to align on the approach with the maintainers. --- -## Development Environment Setup - -### Prerequisites - -| Tool | Minimum Version | -|------|----------------| -| Java (JDK) | 21 | -| Apache Maven | 3.9+ | -| Git | 2.x | -| Docker | 24.x | - -### Initial Setup - -```bash -# Clone the repository -git clone https://github.com/flamingo-stack/openframe-oss-lib.git -cd openframe-oss-lib - -# Configure GitHub Packages credentials -# Add to ~/.m2/settings.xml (see Prerequisites guide) - -# Build all modules (skip tests for speed) -mvn install -DskipTests -``` - -### Recommended IDE +## Getting Started -**IntelliJ IDEA** (Community or Ultimate) is the recommended IDE. After importing the project: +Before contributing, ensure your development environment is set up correctly. See the full guides: -1. Set **Project SDK** to Java 21 -2. Enable **Annotation Processors** for Lombok (`Settings β†’ Compiler β†’ Annotation Processors`) -3. Enable Maven delegate: `Settings β†’ Build Tools β†’ Maven β†’ Runner β†’ Delegate IDE build/run actions to Maven` -4. Increase heap: `Help β†’ Change Memory Settings β†’ Xmx: 4096 MB` +- [Prerequisites](./docs/getting-started/prerequisites.md) β€” Required tools and versions +- [Local Development](./docs/development/setup/local-development.md) β€” Clone, build, and run locally +- [Environment Setup](./docs/development/setup/environment.md) β€” IDE and toolchain configuration --- ## Code Style and Conventions -### Java Style +### Java Conventions | Convention | Rule | |-----------|------| -| Indentation | 4 spaces (no tabs) | -| Line length | Max 120 characters | -| Imports | No wildcard imports; organize by static β†’ java β†’ jakarta β†’ spring β†’ other | -| Braces | Opening brace on same line | -| Naming | `camelCase` methods/fields, `PascalCase` classes, `UPPER_SNAKE` constants | +| **Indentation** | 4 spaces (no tabs) | +| **Line length** | 120 characters maximum | +| **Imports** | No wildcard imports | +| **Naming** | Standard Java naming (camelCase for methods/fields, PascalCase for classes) | +| **Access modifiers** | Use the most restrictive access level possible | +| **Final fields** | Use `final` for fields that are not reassigned | +| **Null handling** | Use `Optional` for nullable return values | ### Lombok Usage -Use Lombok to reduce boilerplate: +Prefer the following Lombok annotations: ```java -// Prefer these annotations -@Data // getters, setters, equals, hashCode, toString -@Value // immutable value objects -@Builder // fluent builders -@RequiredArgsConstructor // constructor injection -@Slf4j // logging -``` - -### Spring Boot Conventions - -Use constructor injection (via `@RequiredArgsConstructor`) over field injection: +// Immutable DTOs +@Value +public class AgentRegistrationResponse { + String machineId; + String clientSecret; +} -```java -// Correct: Constructor injection +// Constructor injection (preferred pattern) @Service @RequiredArgsConstructor -public class MyService { - private final MyRepository repository; - private final AnotherService anotherService; +public class DeviceService { + private final MachineRepository machineRepository; + private final TagService tagService; } -``` -Prefer `@ConfigurationProperties` over `@Value` for configuration classes. - -### Multi-Tenancy Requirements - -Every service and repository method that accesses tenant-scoped data **must** include `tenantId` in queries: - -```java -// Correct: Always scope to tenant -Optional findByIdAndTenantId(String id, String tenantId); - -// Wrong: Missing tenant scope β€” never do this for tenant-scoped data -Optional findById(String id); +// Complex object construction +@Builder +public class NotificationMessage { + private String title; + private String description; + private NotificationSeverity severity; +} ``` -### Exception Handling +### Spring Conventions -Use the standard exception hierarchy from `openframe-exception`: - -```java -throw new NotFoundException("Organization not found: " + id); -throw new BadRequestException("Invalid email format"); -throw new ForbiddenException("Access denied for tenant: " + tenantId); -throw new ConflictException("Email already exists"); -throw new ValidationException("Required field missing: name"); -``` - -Never throw raw `RuntimeException` or `Exception` directly. +- **Constructor injection** is preferred over field injection +- Use `@RequiredArgsConstructor` with `final` fields for Spring components +- Avoid `@Autowired` on fields +- Keep controllers thin β€” delegate all business logic to service classes +- Controllers must return DTOs, not domain documents --- ## Branch Naming -| Type | Pattern | Example | -|------|---------|---------| -| Feature | `feature/` | `feature/add-nats-retry-logic` | -| Bug fix | `fix/` | `fix/tenant-context-not-cleared` | -| Refactor | `refactor/` | `refactor/notification-repository` | -| Documentation | `docs/` | `docs/update-kafka-readme` | -| Dependency updates | `deps/` | `deps/upgrade-spring-boot-3.4` | +| Type | Format | Example | +|------|--------|---------| +| Feature | `feature/short-description` | `feature/add-notification-webhooks` | +| Bug fix | `fix/short-description` | `fix/agent-registration-timeout` | +| Refactor | `refactor/short-description` | `refactor/simplify-tenant-resolution` | +| Documentation | `docs/short-description` | `docs/update-gateway-readme` | +| Chore | `chore/short-description` | `chore/upgrade-spring-boot-3.4` | + +**Rules:** +- Use lowercase and hyphens β€” no spaces or underscores +- Keep names short but descriptive (3–6 words) +- Branch from `main` (or the current release branch) unless instructed otherwise --- ## Commit Message Format -Follow the [Conventional Commits](https://www.conventionalcommits.org/) specification: +This project uses the **Conventional Commits** specification: ```text -(): +(): [optional body] [optional footer] ``` -### Types +### Commit Types -| Type | When to Use | +| Type | When to use | |------|------------| -| `feat` | New feature or capability | -| `fix` | Bug fix | -| `refactor` | Code change that neither fixes a bug nor adds a feature | +| `feat` | A new feature or capability | +| `fix` | A bug fix | +| `refactor` | Code restructuring without behavior change | | `test` | Adding or updating tests | -| `docs` | Documentation only changes | -| `chore` | Build system, dependency updates, CI changes | +| `docs` | Documentation-only changes | +| `chore` | Build system, dependency updates, CI config | | `perf` | Performance improvements | +| `style` | Formatting changes (no behavior change) | ### Scope -Use the module name (without the `openframe-` prefix) as scope: - -```text -feat(security-core): add PKCE utility for code challenge generation -fix(data-mongo-sync): resolve tenant context leak in batch operations -refactor(gateway-service-core): extract rate limit logic into service -test(data-nats): add integration test for notification broadcast -chore(deps): upgrade spring-boot to 3.3.2 -``` - -### Example Commit +Use the module name (without `openframe-` prefix) as the scope: ```text -feat(authorization-service-core): add Microsoft SSO provider strategy - -Implements MicrosoftClientRegistrationStrategy to support Microsoft -Entra ID (Azure AD) authentication flows alongside existing Google SSO. - -Closes: #discussion in #openframe-dev slack +feat(api-service-core): add bulk device status update endpoint +fix(gateway-service-core): resolve rate limit header not set on 429 response +refactor(data-mongo-sync): simplify cursor pagination in notification repository +test(authorization-service-core): add SSO invitation acceptance integration test +chore: upgrade spring-boot to 3.3.1 ``` --- @@ -194,167 +148,162 @@ Closes: #discussion in #openframe-dev slack ### Before Opening a PR -- [ ] Build passes: `mvn install -DskipTests` -- [ ] Tests pass: `mvn test -pl ` -- [ ] No secrets committed β€” review all changed files -- [ ] Follows code style: Lombok, constructor injection, tenant scoping -- [ ] Edge cases covered: unit tests added for new logic +1. **Discuss in Slack** β€” Align with maintainers for significant changes +2. **Write tests** β€” New features require unit tests; bug fixes require regression tests +3. **Check tests pass** β€” Run `mvn test` locally before pushing +4. **Follow code style** β€” Format your code before committing -### PR Title Format +### PR Title + +Use the same Conventional Commits format: ```text -feat(security-core): add PKCE utility for authorization flows -fix(data-mongo-sync): resolve tenant context leak +feat(api-service-core): add GraphQL subscription for device status changes ``` ### PR Description Template ```markdown -## What does this PR do? +## Summary +Brief description of what this PR does. -Brief description of the change. +## Changes +- List key changes made +- Include any migration steps if needed -## Why? +## Testing +- Describe how you tested the changes +- List new tests added -Motivation for the change. +## Related +- Link to Slack thread or task (if applicable) +``` -## How was it tested? +### Review Process -- [ ] Unit tests added/updated -- [ ] Integration tests added/updated -- [ ] Manual testing performed +1. At least **one maintainer approval** is required to merge +2. All CI checks (build + tests) must pass +3. Resolve all review comments before requesting re-review +4. Squash commits if the history is noisy (maintainer may do this at merge) -## Checklist +--- -- [ ] No secrets in code or tests -- [ ] All new endpoints have authorization rules -- [ ] New DB queries are tenant-scoped -- [ ] Input validation on all new DTOs -- [ ] No breaking changes (or breaking changes are documented) -``` +## Review Checklist ---- +Use this checklist when reviewing or self-reviewing a PR: -## Testing Guidelines +### Correctness +- [ ] The change does what the PR description says +- [ ] Edge cases are handled (null inputs, empty collections, concurrent access) +- [ ] Error handling is appropriate (no swallowed exceptions) -### Test Structure +### Security +- [ ] New MongoDB queries always filter by `tenantId` +- [ ] New API endpoints are covered by Gateway authorization rules +- [ ] Sensitive data uses `EncryptionService` for at-rest protection +- [ ] No secrets or credentials are hardcoded or logged -```text -openframe-/ -└── src/ - └── test/java/com/openframe/... - β”œβ”€β”€ FooTest.java # Unit test (runs in mvn test) - └── FooIT.java # Integration test (runs in mvn verify) -``` +### Testing +- [ ] Unit tests cover the happy path and key error scenarios +- [ ] Integration tests are added for new repository methods +- [ ] Existing tests still pass -### Running Tests +### Code Quality +- [ ] No unnecessary complexity added +- [ ] Follows the processor/extension pattern for lifecycle hooks +- [ ] Lombok annotations are used appropriately +- [ ] Constructor injection is used (not field injection) +- [ ] DTOs are used in controllers (not domain documents) -```bash -# Unit tests for a specific module -mvn test -pl openframe-core +### Documentation +- [ ] Public interfaces and complex methods have Javadoc +- [ ] Configuration properties are documented -# Integration tests (requires Docker) -mvn verify -pl openframe-data-mongo-sync +--- -# Skip all tests during build -mvn install -DskipTests -``` +## Adding a New Module -### Unit Test Template +If your contribution requires a new module: -```java -@ExtendWith(MockitoExtension.class) -class MyServiceTest { +1. Follow the existing module structure conventions +2. Add the module to the root `pom.xml` `` section +3. Add it to `` with `${revision}` version +4. Use `com.openframe.oss` as the `groupId` and follow the `openframe-*` artifact naming convention +5. Ensure the module has a meaningful `` in its `pom.xml` +6. Add appropriate unit and integration tests before opening the PR - @Mock - private MyRepository repository; +--- + +## Testing Guidelines - @InjectMocks - private MyService service; +### Running Tests - @Test - void shouldReturnExpectedResult() { - // Given - when(repository.findById("id-1")).thenReturn(Optional.of(new MyEntity("id-1"))); +```bash +# Run all unit tests (no Docker required) +mvn test - // When - var result = service.getById("id-1"); +# Run integration tests (requires Docker) +mvn verify - // Then - assertThat(result).isPresent(); - assertThat(result.get().getId()).isEqualTo("id-1"); - } -} +# Run tests for a specific module +mvn test -pl openframe-api-service-core ``` -### Test Conventions +### Test Naming Conventions -- Use **Given / When / Then** structure in test methods -- Test method names should describe behavior: `shouldReturnNotFoundWhenEntityDoesNotExist` -- Always clean up test data in `@AfterEach` for integration tests -- Integration tests use Testcontainers β€” Docker must be running +| Test Type | File Suffix | Example | +|-----------|------------|---------| +| Unit Test | `*Test.java` | `CommandDispatchServiceTest` | +| Integration Test | `*IT.java` | `NotificationServiceIT` | +| UI Test | `*UITest.java` | `DeviceRemoteTest` | ---- +### Coverage Guidelines -## Security Requirements +| Test Type | Recommended Coverage | +|-----------|---------------------| +| Unit tests | β‰₯ 80% for service and utility classes | +| Integration tests | All repository methods with real infrastructure | +| E2E tests | All critical user flows (auth, device, ticket, org) | -Before opening a PR, verify: +Focus coverage on business logic in `*Service` classes, repository query methods, mapper/converter classes, and security-sensitive code. -- [ ] No secrets, tokens, or keys in code or test fixtures -- [ ] All new endpoints have explicit authorization rules -- [ ] New database queries include `tenantId` scope -- [ ] Input validation annotations on all request DTOs -- [ ] Sensitive fields are encrypted at rest using `EncryptionService` -- [ ] No raw SQL/NoSQL string interpolation +--- -**For security vulnerabilities**, do **not** open a public GitHub issue. Contact the team via the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) in a direct message to the maintainers, or visit [flamingo.run](https://flamingo.run) for the security contact. +## Security Guidelines for Contributors ---- +When writing or reviewing security-relevant code: -## Adding a New Module +- [ ] All new DTOs have `@Valid` annotations on request-body parameters +- [ ] New MongoDB queries always filter by `tenantId` +- [ ] New API endpoints are covered by Gateway path authorization rules +- [ ] Sensitive fields use `EncryptionService` for at-rest protection +- [ ] No secrets or keys are logged +- [ ] New OAuth flows use PKCE +- [ ] New external integrations use stored, encrypted credentials -When adding a new module to the library: - -1. Create the module directory following the naming convention: `openframe-/` -2. Add a `pom.xml` that inherits from the parent POM -3. Add the module to the parent `pom.xml` `` section -4. Add the module to `` in the parent with `${revision}` -5. Write unit tests before submitting -6. Update the README and architecture documentation - -```xml - -openframe-my-new-module - - - - com.openframe.oss - openframe-my-new-module - ${revision} - -``` +For full security architecture details, see [Security Guide](./docs/development/security/README.md). --- -## Versioning +## Release Process -All modules are versioned together using `${revision}` in the parent POM. Version bumps are managed by the maintainers. Contributors do **not** need to update version numbers in PRs. +The project uses **flat Maven versioning** via the `${revision}` property in the parent POM. The current version is `6.0.10`. -Versioning follows [Semantic Versioning](https://semver.org/): +Versions are published to GitHub Packages at: -- **Major:** Breaking API changes -- **Minor:** New backward-compatible features -- **Patch:** Backward-compatible bug fixes +```text +https://maven.pkg.github.com/flamingo-stack/openframe-oss-lib +``` ---- +Release coordination happens through the OpenMSP Slack `#engineering` channel. -## Getting Help +--- -Stuck? Reach out on the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) in `#openframe-dev`. +## Questions? -- 🌐 **OpenFrame Platform:** [https://openframe.ai](https://openframe.ai) -- πŸ’¬ **OpenMSP Community:** [https://www.openmsp.ai/](https://www.openmsp.ai/) -- 🦩 **Flamingo:** [https://flamingo.run](https://flamingo.run) +> Join the **OpenMSP Slack** community for questions, discussions, and support. +> +> πŸ’¬ [https://www.openmsp.ai/](https://www.openmsp.ai/) --- diff --git a/README.md b/README.md index 3ccf01ff2..e6cbf3010 100644 --- a/README.md +++ b/README.md @@ -12,28 +12,30 @@ # openframe-oss-lib -**openframe-oss-lib** is the foundational backend library powering the [OpenFrame](https://openframe.ai) platform β€” the AI-driven, open-source MSP infrastructure stack built by [Flamingo](https://flamingo.run). +**`openframe-oss-lib`** is the modular backend foundation of the [OpenFrame platform](https://openframe.ai) β€” the unified AI-driven MSP platform by [Flamingo](https://flamingo.run). It is a production-ready, multi-tenant, event-driven, analytics-enabled backend library that powers IT support operations at scale. -This repository provides the **shared Spring Boot libraries** that every OpenFrame service is built upon: multi-tenant authentication, API layers (REST + GraphQL), reactive gateway routing, event streaming, real-time messaging, polyglot persistence, and distributed management workflows. +Built on **Spring Boot 3** and **Java 21**, this repository ships as a set of 30+ reusable library modules that compose together to form a fully functional MSP backend platform supporting both OSS single-tenant deployments and SaaS multi-tenant environments. -[![OpenFrame: 5-Minute MSP Platform Walkthrough - Cut Vendor Costs & Automate Ops](https://img.youtube.com/vi/er-z6IUnAps/maxresdefault.jpg)](https://www.youtube.com/watch?v=er-z6IUnAps) +--- + +[![OpenFrame: 5-Minute MSP Platform Walkthrough - Cut Vendor Costs & Automate Ops](https://img.youtube.com/vi/er-z6IUnAps/maxresdefault.jpg)](https://www.youtube.com/watch?v=er-z6IUnAps) --- ## Features -- **Multi-Tenant by Default** β€” Every module implements strict tenant isolation: per-tenant RSA key pairs, tenant-scoped cache keys, tenant-scoped scheduler locks, and tenant ID embedded in every JWT -- **Multi-Tenant OAuth2 Authorization Server** β€” Per-tenant RSA key pairs, JWT issuance, PKCE support, dynamic client registration, SSO integration (Google, Microsoft), and invitation-based onboarding -- **Relay-Compliant GraphQL + REST APIs** β€” Netflix DGS GraphQL with DataLoaders for N+1 prevention alongside versioned REST controllers; cursor-based pagination throughout -- **Reactive Edge Gateway** β€” Spring Cloud Gateway + WebFlux + Netty; multi-issuer JWT validation, API key rate limiting, WebSocket proxying, and tool upstream resolution -- **Event Ingestion & Streaming** β€” Kafka / Debezium CDC processing pipeline with tool-specific deserialization, event enrichment, unified event type normalization, and Kafka Streams enrichment -- **Real-Time Messaging** β€” NATS JetStream publish/subscribe for device and tool notifications; persist-first notification strategy with read-state tracking -- **Polyglot Persistence** β€” MongoDB (sync + reactive), Redis (tenant-aware caching), Apache Pinot (analytics), Apache Cassandra (log storage) -- **Distributed Schedulers** β€” ShedLock + Redis for cluster-safe distributed job scheduling; offline device detection, API key stats sync, MDM fleet setup -- **External REST API** β€” API key–authenticated integration surface for third-party tools with OpenAPI documentation -- **Tool SDKs** β€” First-class Java clients for Tactical RMM, Fleet MDM, and MeshCentral -- **Modular Maven Structure** β€” 30+ independent modules; include only what your service needs -- **Spring Boot 3.3 / Java 21** β€” Modern LTS Java with Spring Security OAuth2 and virtual thread readiness +- **Multi-Tenant Architecture** β€” Every entity is tenant-scoped at every layer (JWT claims, MongoDB queries, gateway routing) +- **REST + GraphQL API** β€” Relay-compliant GraphQL with cursor-based pagination and N+1-safe DataLoaders, plus a full REST surface +- **OAuth2 / OIDC Authorization Server** β€” Multi-tenant JWT issuers, per-tenant RSA signing keys, PKCE enforcement, and dynamic Google/Microsoft SSO support +- **Reactive API Gateway** β€” Spring Cloud Gateway with JWT validation, API key auth, WebSocket proxying, rate limiting, and upstream tool routing +- **Hybrid Event-Driven Messaging** β€” Kafka for durable ordered streaming; NATS JetStream for real-time low-latency agent communication +- **High-Performance Analytics** β€” Apache Pinot integration for sub-second time-series log and device metric queries with faceted filtering +- **MongoDB Operational Persistence** β€” Tenant-aware document model with cursor pagination, aggregations, and Debezium CDC +- **Agent Ingress & Command Execution** β€” Endpoint agent registration, OAuth token issuance, heartbeat tracking, and NATS-based command dispatch +- **Tool Integration SDKs** β€” Strongly-typed SDK adapters for Tactical RMM and Fleet MDM +- **Operational Management** β€” Distributed schedulers (ShedLock/Redis), Mongock data migrations, and system bootstrapping +- **Extensible by Design** β€” Processor interface pattern for lifecycle hooks (agent registration, invitations, SSO, tool saves) +- **Defense-in-Depth Security** β€” Multi-layered security: rate limiting, multi-issuer JWT, RBAC, field-level AES encryption, BCrypt password hashing --- @@ -41,40 +43,98 @@ This repository provides the **shared Spring Boot libraries** that every OpenFra ```mermaid flowchart TD - Client["Client / Browser / Agent"] --> Gateway["Gateway Service Core"] - Gateway --> ExternalAPI["External API Service Core"] - Gateway --> ApiCore["API Service Core (GraphQL + REST)"] - - ApiCore --> Authz["Authorization Service Core (OAuth2 / JWT)"] - ApiCore --> Stream["Stream Service Core (Kafka / Debezium)"] - ApiCore --> Management["Management Service Core (Schedulers)"] - - Authz --> Mongo["MongoDB"] - ApiCore --> Mongo - ApiCore --> Redis["Redis"] - ApiCore --> Pinot["Apache Pinot"] - Stream --> Kafka["Apache Kafka"] - Stream --> Cassandra["Apache Cassandra"] - Management --> NATS["NATS JetStream"] + subgraph Edge["Edge"] + Browser["Browser / SPA"] + Agent["Endpoint Agent"] + External["External API Consumer"] + end + + subgraph GatewayLayer["Gateway Service Core"] + Gateway["Reactive API Gateway"] + end + + subgraph SecurityLayer["Authorization & Security"] + AuthServer["Authorization Service Core"] + SecurityCore["Security OAuth & JWT"] + end + + subgraph ApiLayer["API Layer"] + Rest["REST Controllers"] + GraphQL["GraphQL Layer"] + DataLoaders["GraphQL DataLoaders"] + end + + subgraph DomainLayer["Business & Mapping"] + Services["Business Services"] + Mapping["Mapping & Domain Services"] + end + + subgraph PersistenceLayer["Persistence"] + MongoSync["Mongo Sync Repositories"] + MongoDocs["Mongo Documents"] + end + + subgraph MessagingLayer["Messaging"] + Kafka["Apache Kafka"] + Nats["NATS JetStream"] + end + + subgraph StreamLayer["Stream Processing & Analytics"] + StreamCore["Kafka Streams / Debezium CDC"] + Pinot["Apache Pinot"] + end + + subgraph AgentIngress["Agent Ingress"] + AgentAPI["Agent Registration & Auth"] + AgentListeners["NATS Listeners"] + end + + Browser --> Gateway + Agent --> Gateway + External --> Gateway + + Gateway --> AuthServer + Gateway --> Rest + Gateway --> GraphQL + AuthServer --> SecurityCore + + Rest --> Services + GraphQL --> DataLoaders + DataLoaders --> Services + + Services --> Mapping + Mapping --> MongoSync + MongoSync --> MongoDocs + + Services --> Kafka + Services --> Nats + + Kafka --> StreamCore + StreamCore --> Pinot + + Nats --> AgentListeners + AgentListeners --> Services ``` --- ## Technology Stack -| Layer | Technology | -|-------|-----------| -| Language | Java 21 | -| Framework | Spring Boot 3.3 | -| Build Tool | Apache Maven 3.9+ | -| Auth | Spring Authorization Server, Spring Security OAuth2 | -| API | Netflix DGS (GraphQL), Spring MVC (REST) | -| Gateway | Spring Cloud Gateway + WebFlux + Netty | -| Persistence | MongoDB (sync + reactive), Redis, Cassandra, Apache Pinot | -| Messaging | Apache Kafka / Debezium CDC, NATS JetStream | -| Testing | JUnit 5, Testcontainers, RestAssured | -| Distributed Locking | ShedLock + Redis | -| Multi-tenancy | ThreadLocal tenant context, per-tenant RSA keys | +| Technology | Version | Role | +|------------|---------|------| +| **Java** | 21 | Primary language (Virtual Threads, Records, Sealed Classes) | +| **Spring Boot** | 3.3.0 | Application framework | +| **Spring Cloud Gateway** | 2023.0.3 | Reactive API gateway | +| **Spring Authorization Server** | 1.3.1 | OAuth2/OIDC authorization server | +| **Spring Data MongoDB** | 4.2.0 | MongoDB persistence | +| **Netflix DGS** | 9.0.3 | GraphQL framework | +| **Apache Kafka** | via Spring Cloud | Durable event streaming | +| **NATS** | 0.6.2+3.5 | Real-time agent messaging | +| **Apache Pinot** | 1.2.0 | Analytics query engine | +| **Testcontainers** | 1.21.4 | Integration test infrastructure | +| **Lombok** | 1.18.30 | Code generation | +| **JJWT** | 0.11.5 | JWT utilities | +| **gRPC** | 1.58.0 | Internal service communication | --- @@ -82,21 +142,14 @@ flowchart TD ### Prerequisites -- Java 21 JDK ([Eclipse Temurin 21](https://adoptium.net/) recommended) -- Apache Maven 3.9+ -- Docker 24.x (for integration tests) -- GitHub Personal Access Token (PAT) with `read:packages` scope - -### 1. Clone the Repository - -```bash -git clone https://github.com/flamingo-stack/openframe-oss-lib.git -cd openframe-oss-lib -``` +- **Java 21** (OpenJDK or Eclipse Temurin) +- **Apache Maven 3.9+** +- **Docker 24+** (for integration tests) +- **GitHub Personal Access Token** with `read:packages` scope -### 2. Configure GitHub Packages +### 1. Configure GitHub Packages -Add your GitHub credentials to `~/.m2/settings.xml`: +Add your credentials to `~/.m2/settings.xml`: ```xml @@ -110,42 +163,50 @@ Add your GitHub credentials to `~/.m2/settings.xml`: ``` -### 3. Build the Library +### 2. Clone and Build ```bash +git clone https://github.com/flamingo-stack/openframe-oss-lib.git +cd openframe-oss-lib + +# Build all 30+ modules (skip tests for speed) mvn install -DskipTests ``` -### 4. Add a Module as a Dependency +### 3. Use in Your Service + +Add individual modules or the BOM to your Spring Boot service: ```xml + com.openframe.oss openframe-oss-lib - 5.79.3 + 6.0.10 pom import - - - com.openframe.oss - openframe-api-service-core - - + + + com.openframe.oss + openframe-api-service-core + ``` -### 5. Run Tests +### 4. Run Unit Tests ```bash -# Unit tests for a specific module -mvn test -pl openframe-core +mvn test +``` -# Integration tests (requires Docker) +### 5. Run Integration Tests (requires Docker) + +```bash mvn verify -pl openframe-data-mongo-sync ``` @@ -155,46 +216,43 @@ mvn verify -pl openframe-data-mongo-sync | Module | Description | |--------|-------------| -| `openframe-core` | Core utilities, pagination, validation | -| `openframe-exception` | Standard exception hierarchy | -| `openframe-core-crypto` | Encryption utilities | -| `openframe-security-core` | JWT, PKCE, cookie service | -| `openframe-security-oauth` | OAuth2 BFF layer | -| `openframe-authorization-service-core` | Multi-tenant OAuth2 authorization server | -| `openframe-api-lib` | API contracts, filter DTOs, cursor pagination | -| `openframe-api-service-core` | REST + GraphQL API service layer | -| `openframe-gateway-service-core` | Reactive gateway, routing, security | +| `openframe-core` | Core utilities (pagination, constants, slug) | +| `openframe-exception` | Shared exception hierarchy and error codes | +| `openframe-security-core` | JWT encoder/decoder, PKCE utilities | +| `openframe-security-oauth` | OAuth BFF controller and token cookies | +| `openframe-api-lib` | Shared DTO contracts and domain services | +| `openframe-api-service-core` | REST + GraphQL API service core | +| `openframe-authorization-service-core` | OAuth2/OIDC Authorization Server | +| `openframe-gateway-service-core` | Reactive API Gateway with rate limiting | +| `openframe-client-core` | Agent registration and ingress | +| `openframe-management-service-core` | Management services and schedulers | | `openframe-data-mongo-common` | MongoDB domain documents | -| `openframe-data-mongo-sync` | Synchronous MongoDB repositories | -| `openframe-data-mongo-reactive` | Reactive MongoDB repositories | -| `openframe-data-redis` | Redis tenant-aware cache | -| `openframe-data-kafka` | Kafka multi-tenant configuration | -| `openframe-data-nats` | NATS real-time messaging | -| `openframe-data-cassandra` | Cassandra log storage | -| `openframe-data-pinot` | Apache Pinot analytics queries | -| `openframe-management-service-core` | Distributed schedulers, startup initializers | -| `openframe-stream-service-core` | Kafka streams, Debezium event enrichment | -| `openframe-external-api-service-core` | External REST API for integrations | -| `sdk/fleetmdm` | Fleet MDM Java SDK | -| `sdk/tacticalrmm` | Tactical RMM Java SDK | +| `openframe-data-mongo-sync` | MongoDB sync repositories | +| `openframe-data-redis` | Redis cache and rate-limit support | +| `openframe-data-kafka` | Kafka producers and configuration | +| `openframe-data-nats` | NATS messaging publishers | +| `openframe-data-pinot` | Apache Pinot query repositories | +| `openframe-stream-service-core` | Kafka Streams and Debezium CDC processing | +| `openframe-external-api-service-core` | External REST API for third-party consumers | +| `sdk/tacticalrmm` | Tactical RMM SDK | +| `sdk/fleetmdm` | Fleet MDM SDK | --- -## Documentation - -πŸ“š See the [Documentation](./docs/README.md) for comprehensive guides covering architecture, setup, development workflows, and reference documentation. +## Community & Support ---- +> **All discussions happen on OpenMSP Slack** β€” we do not use GitHub Issues or GitHub Discussions. -## Community +- πŸ’¬ **Join Slack**: [https://www.openmsp.ai/](https://www.openmsp.ai/) +- πŸ”— **Direct Invite**: [https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) +- 🌐 **Flamingo Platform**: [https://flamingo.run](https://flamingo.run) +- πŸ”§ **OpenFrame**: [https://openframe.ai](https://openframe.ai) -All discussions, questions, and support happen on the **[OpenMSP Slack Community](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA)**. We do not use GitHub Issues or GitHub Discussions. +--- -- 🌐 **OpenFrame Platform:** [https://openframe.ai](https://openframe.ai) -- πŸ’¬ **OpenMSP Slack:** [https://www.openmsp.ai/](https://www.openmsp.ai/) -- 🦩 **Flamingo:** [https://flamingo.run](https://flamingo.run) +## Documentation -[![OpenFrame v0.5.2: Autonomous AI Agent Architecture for MSPs](https://img.youtube.com/vi/PexpoNdZtUk/maxresdefault.jpg)](https://www.youtube.com/watch?v=PexpoNdZtUk) +πŸ“š See the [Documentation](./docs/README.md) for comprehensive guides including architecture reference, getting started tutorials, and development workflows. --- diff --git a/docs/README.md b/docs/README.md index f5d99817c..247f3c309 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,108 +1,150 @@ # openframe-oss-lib Documentation -Welcome to the documentation for **openframe-oss-lib** β€” the foundational backend library powering the [OpenFrame](https://openframe.ai) platform. - -This section contains comprehensive guides covering getting started, development workflows, security practices, testing strategies, and full reference architecture documentation for every module. +Welcome to the documentation hub for **`openframe-oss-lib`** β€” the modular backend foundation of the [OpenFrame platform](https://openframe.ai) by [Flamingo](https://flamingo.run). --- ## πŸ“š Table of Contents -### Getting Started +- [Getting Started](#getting-started) +- [Development](#development) +- [Reference Architecture](#reference-architecture) +- [Architecture Diagrams](#architecture-diagrams) +- [Quick Links](#quick-links) + +--- + +## Getting Started -- [Introduction](./getting-started/introduction.md) β€” What is openframe-oss-lib, key features, and high-level architecture -- [Prerequisites](./getting-started/prerequisites.md) β€” Required software, Java setup, GitHub Packages access, IDE configuration -- [Quick Start](./getting-started/quick-start.md) β€” Clone, build, and add modules as dependencies in 5 minutes -- [First Steps](./getting-started/first-steps.md) β€” Module structure, domain model exploration, security modules, and gateway overview +New to `openframe-oss-lib`? Start here. + +| Document | Description | +|----------|-------------| +| [Introduction](./getting-started/introduction.md) | What is openframe-oss-lib, key features, and architecture overview | +| [Prerequisites](./getting-started/prerequisites.md) | Required tools, infrastructure dependencies, and environment setup | +| [Quick Start](./getting-started/quick-start.md) | Clone, configure, and build in under 5 minutes | +| [First Steps](./getting-started/first-steps.md) | Explore key modules, set up local infrastructure, and write your first extension | --- -### Development +## Development + +Guides for building, testing, and contributing to the library. -- [Development Overview](./development/README.md) β€” Index of all development documentation and quick navigation -- [Environment Setup](./development/setup/environment.md) β€” IDE configuration (IntelliJ / VS Code), Maven setup, Docker, environment variables -- [Local Development](./development/setup/local-development.md) β€” Clone, build, iterate, debug, and manage dependencies locally -- [Architecture Overview](./development/architecture/README.md) β€” System design, component relationships, data flows, and key design decisions -- [Security Best Practices](./development/security/README.md) β€” Auth patterns, multi-tenant key isolation, secrets management, input validation -- [Testing Overview](./development/testing/README.md) β€” Test structure, running tests, integration test infrastructure, writing new tests -- [Contributing Guidelines](./development/contributing/guidelines.md) β€” Code style, branching, commit conventions, and pull request process +| Document | Description | +|----------|-------------| +| [Development Overview](./development/README.md) | Technology stack, repository structure, and module dependency layers | +| [Environment Setup](./development/setup/environment.md) | IDE configuration, toolchain installation, and recommended plugins | +| [Local Development](./development/setup/local-development.md) | Build workflow, local infrastructure, hot reload tips, and debugging | +| [Testing Guide](./development/testing/README.md) | Unit tests, integration tests, Testcontainers patterns, and E2E framework | +| [Security Guide](./development/security/README.md) | JWT architecture, OAuth2/PKCE flows, RBAC, secrets management | +| [Architecture Overview](./development/architecture/README.md) | High-level component interactions and key design decisions | +| [Contributing Guidelines](./development/contributing/guidelines.md) | Code style, branch naming, commit conventions, and PR process | --- -### Reference Architecture - -- [Repository Overview](./reference/architecture/README.md) β€” Full module index and end-to-end system view - -#### API Foundation -- [API Contracts and Pagination](./reference/architecture/api-contracts-and-pagination/api-contracts-and-pagination.md) β€” Relay-style pagination, cursor codec, mutation inputs -- [API Domain Filters & DTOs](./reference/architecture/api-domain-filters-dtos/api-domain-filters-dtos.md) β€” Strongly-typed filter DTOs for devices, events, logs, and organizations -- [API Lib Core Services](./reference/architecture/api-lib-core-services/api-lib-core-services.md) β€” Reusable domain services: tool connections, ticket queries, device status -- [API Organization Mapping](./reference/architecture/api-organization-mapping/api-organization-mapping.md) β€” Organization-level API mapping layer - -#### API Service Core -- [API Service Core Overview](./reference/architecture/api-service-core-user-sso-services-and-processors/api-service-core-user-sso-services-and-processors.md) β€” User SSO services and processors -- [API Service Core Services](./reference/architecture/api-service-core-user-sso-services-and-processors/services.md) β€” Core service implementations -- [API Service Core Processors](./reference/architecture/api-service-core-user-sso-services-and-processors/processors.md) β€” Processor implementations -- [REST Controllers](./reference/architecture/api-service-core-rest-controllers/api-service-core-rest-controllers.md) β€” Internal REST endpoints: organizations, devices, users, invitations, API keys -- [GraphQL Data Fetchers](./reference/architecture/api-service-core-graphql-datafetchers/api-service-core-graphql-datafetchers.md) β€” Relay-compliant GraphQL execution layer -- [GraphQL DTOs](./reference/architecture/api-service-core-graphql-dtos/api-service-core-graphql-dtos.md) β€” GraphQL input/output types -- [DataLoaders](./reference/architecture/api-service-core-dataloaders/api-service-core-dataloaders.md) β€” Batched DataLoader implementations for N+1 prevention -- [Relay Type Resolution](./reference/architecture/api-service-core-relay-type-resolution/api-service-core-relay-type-resolution.md) β€” Polymorphic Relay node type resolution -- [Config and Security](./reference/architecture/api-service-core-config-and-security/api-service-core-config-and-security.md) β€” JWT resource server, multi-issuer support, OAuth client initialization - -#### Authorization Service Core -- [Server and Tenant](./reference/architecture/authorization-service-core-server-and-tenant/authorization-service-core-server-and-tenant.md) β€” Multi-tenant OAuth2 authorization server, tenant discovery and registration -- [Auth Controllers and DTOs](./reference/architecture/authorization-service-core-auth-controllers-and-dtos/authorization-service-core-auth-controllers-and-dtos.md) β€” Authentication controllers and data transfer objects -- [Keys and Authorization Persistence](./reference/architecture/authorization-service-core-keys-and-authorization-persistence/authorization-service-core-keys-and-authorization-persistence.md) β€” Per-tenant RSA key pairs, JWT issuance, persistence -- [SSO Flow and Utils](./reference/architecture/authorization-service-core-sso-flow-and-utils/authorization-service-core-sso-flow-and-utils.md) β€” Google and Microsoft SSO flows, PKCE support - -#### Gateway Service Core -- [Gateway Security and Routing](./reference/architecture/gateway-service-core-security-and-routing/gateway-service-core-security-and-routing.md) β€” Reactive edge gateway, JWT validation, API key rate limiting, WebSocket proxying - -#### Stream Service Core -- [Kafka and Handlers](./reference/architecture/stream-service-core-kafka-and-handlers/stream-service-core-kafka-and-handlers.md) β€” Debezium CDC ingestion, event enrichment, unified event type mapping - -#### External API Service Core -- [External API Service](./reference/architecture/external-api-service-core/external-api-service-core.md) β€” Public REST interface for third-party integrations - -#### Management Service Core -- [Initializers and Schedulers](./reference/architecture/management-service-core-initializers-and-schedulers/management-service-core-initializers-and-schedulers.md) β€” Startup initializers, ShedLock distributed schedulers - -#### Security Core -- [Security Core and OAuth BFF](./reference/architecture/security-core-and-oauth-bff/security-core-and-oauth-bff.md) β€” PKCE utilities, JWT encoder/decoder, OAuth BFF login flow - -#### Data Layer -- [MongoDB Domain Model](./reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md) β€” Canonical MongoDB documents: User, Organization, Device, Ticket, Tool -- [MongoDB Base Repositories](./reference/architecture/data-mongo-base-repositories/data-mongo-base-repositories.md) β€” Base repository interfaces and patterns -- [MongoDB Query Filters](./reference/architecture/data-mongo-query-filters/data-mongo-query-filters.md) β€” MongoDB query filter construction -- [MongoDB Sync Config and Custom Repositories](./reference/architecture/data-mongo-sync-config-and-custom-repositories/data-mongo-sync-config-and-custom-repositories.md) β€” Synchronous repositories, index configuration, custom queries -- [MongoDB Reactive Repositories](./reference/architecture/data-mongo-reactive-repositories/data-mongo-reactive-repositories.md) β€” Reactive MongoDB repository layer -- [Redis Cache](./reference/architecture/data-redis-cache/data-redis-cache.md) β€” Tenant-aware cache key prefixing, Spring Cache integration -- [Kafka Configuration and Retry](./reference/architecture/data-kafka-configuration-and-retry/data-kafka-configuration-and-retry.md) β€” Multi-tenant Kafka config, topic provisioning, retry handling -- [NATS Notifications](./reference/architecture/data-nats-notifications/data-nats-notifications.md) β€” Persist-first notification strategy, read-state tracking, NATS publishing -- [Pinot Repositories](./reference/architecture/data-pinot-repositories/data-pinot-repositories.md) β€” Apache Pinot analytics queries for logs and device facets +## Reference Architecture + +Detailed technical documentation for every major module, generated from source code analysis. + +### API Layer + +| Document | Description | +|----------|-------------| +| [API Lib β€” DTO Contracts](./reference/architecture/api-lib-dto-contracts/api-lib-dto-contracts.md) | Stable transport contracts shared between REST, GraphQL, and services | +| [API Lib β€” Mapping & Domain Services](./reference/architecture/api-lib-mapping-and-domain-services/api-lib-mapping-and-domain-services.md) | Mapping layer between DTOs, domain documents, and repositories | +| [API Service Core β€” REST Controllers](./reference/architecture/api-service-core-rest-controllers/api-service-core-rest-controllers.md) | Thin HTTP controllers for organizations, devices, API keys, users | +| [API Service Core β€” GraphQL Layer](./reference/architecture/api-service-core-graphql-layer/api-service-core-graphql-layer.md) | Relay-compliant GraphQL with cursor pagination and mutations | +| [API Service Core β€” GraphQL DataLoaders](./reference/architecture/api-service-core-graphql-dataloaders/api-service-core-graphql-dataloaders.md) | Batched DataLoaders preventing N+1 queries | +| [API Service Core β€” DTOs](./reference/architecture/api-service-core-dtos/api-service-core-dtos.md) | REST and GraphQL DTOs for SSO, OAuth, invitations, and notifications | +| [API Service Core β€” Business Services](./reference/architecture/api-service-core-business-services/api-service-core-business-services.md) | Core domain orchestration: users, SSO, domains, extension processors | +| [API Service Core β€” Config & Security](./reference/architecture/api-service-core-config-and-security/api-service-core-config-and-security.md) | JWT resource server, OAuth integration, custom GraphQL scalars | + +### Security & Authorization + +| Document | Description | +|----------|-------------| +| [Authorization Service Core](./reference/architecture/authorization-service-core/authorization-service-core.md) | Full OAuth2/OIDC server: multi-tenant JWT issuers, PKCE, SSO providers | +| [Security β€” OAuth & JWT](./reference/architecture/security-oauth-and-jwt/security-oauth-and-jwt.md) | JWT encoder/decoder, PKCE utilities, OAuth BFF controller | + +### Gateway + +| Document | Description | +|----------|-------------| +| [Gateway Service Core](./reference/architecture/gateway-service-core/gateway-service-core.md) | Reactive Spring Cloud Gateway: JWT validation, API key auth, WebSocket proxy | + +### Persistence + +| Document | Description | +|----------|-------------| +| [Data Model & Repositories (Mongo)](./reference/architecture/data-model-and-repositories-mongo/data-model-and-repositories-mongo.md) | MongoDB documents, query filters, base repositories, tenant ID provider | +| [Data Access β€” Mongo Sync](./reference/architecture/data-access-mongo-sync/data-access-mongo-sync.md) | MongoTemplate-based repositories: pagination, aggregations, optimistic locking | + +### Messaging & Events + +| Document | Description | +|----------|-------------| +| [Eventing & Messaging β€” Kafka & NATS](./reference/architecture/eventing-and-messaging-kafka-nats/eventing-and-messaging-kafka-nats.md) | Hybrid messaging: Kafka durable streaming, NATS real-time, Debezium CDC | + +### Stream Processing & Analytics + +| Document | Description | +|----------|-------------| +| [Stream Processing Core](./reference/architecture/stream-processing-core/stream-processing-core.md) | Real-time event enrichment: Kafka Streams, Pinot ingestion, unified taxonomy | +| [Analytics β€” Pinot](./reference/architecture/analytics-pinot/analytics-pinot.md) | High-performance read layer: device filtering, log search, cursor pagination | + +### Agent & Management + +| Document | Description | +|----------|-------------| +| [Client Core β€” Agent Ingress](./reference/architecture/client-core-agent-ingress/client-core-agent-ingress.md) | Agent registration, OAuth token issuance, heartbeat, JetStream listeners | +| [Management Service Core](./reference/architecture/management-service-core/management-service-core.md) | NATS stream init, tool lifecycle, schedulers, Mongock migrations | + +### Integrations + +| Document | Description | +|----------|-------------| +| [Integrations SDKs](./reference/architecture/integrations-sdks/integrations-sdks.md) | Typed SDK abstractions for Tactical RMM and Fleet MDM | + +### External API + +| Document | Description | +|----------|-------------| +| [External API Service Core](./reference/architecture/external-api-service-core/external-api-service-core.md) | OpenAPI-documented REST API for third-party consumers | --- -### Architecture Diagrams +## Architecture Diagrams -Visual Mermaid diagrams are available for every module under: +Visual documentation is available as Mermaid diagram files in: ```text docs/diagrams/architecture/ ``` -Diagrams cover request flows, data flows, class relationships, and sequence diagrams for all major components. +Key diagrams include: +- `README.mmd` β€” Full end-to-end system overview +- `gateway-service-core.mmd` β€” Gateway routing and authentication flow +- `authorization-service-core.mmd` β€” OAuth2/OIDC authorization flows +- `stream-processing-core.mmd` β€” Kafka Streams event enrichment pipeline +- `security-oauth-and-jwt.mmd` β€” JWT multi-issuer validation flow +- `data-access-mongo-sync.mmd` β€” Repository patterns and cursor pagination +- `client-core-agent-ingress.mmd` β€” Agent registration and command dispatch +- `management-service-core.mmd` β€” Bootstrapping and scheduler lifecycle --- ## πŸ“– Quick Links - [Project README](../README.md) β€” Main project overview and quick start -- [Contributing Guide](../CONTRIBUTING.md) β€” How to contribute to openframe-oss-lib -- [OpenFrame Platform](https://openframe.ai) β€” The OpenFrame MSP platform -- [OpenMSP Community (Slack)](https://www.openmsp.ai/) β€” Community discussions and support -- [Flamingo](https://flamingo.run) β€” The team behind OpenFrame +- [Contributing Guidelines](../CONTRIBUTING.md) β€” How to contribute +- [OpenMSP Community](https://www.openmsp.ai/) β€” Slack community for support and discussions +- [Flamingo Platform](https://flamingo.run) β€” The commercial platform built on OpenFrame +- [OpenFrame](https://openframe.ai) β€” Unified AI-driven MSP platform + +--- + +> πŸ’¬ **Need help?** Join the [OpenMSP Slack community](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) β€” all discussions and support happen there. --- diff --git a/docs/development/README.md b/docs/development/README.md index eb274e770..89fce01a0 100644 --- a/docs/development/README.md +++ b/docs/development/README.md @@ -1,64 +1,41 @@ # Development Documentation -Welcome to the **openframe-oss-lib** development documentation. This section covers everything you need to contribute to, extend, and maintain the OpenFrame OSS library. +Welcome to the OpenFrame OSS Lib development documentation. This section covers everything you need to set up, build, test, and contribute to the library. --- -## Overview - -**openframe-oss-lib** is a Java 21 / Spring Boot 3.3 multi-module Maven library. It is the shared backend infrastructure stack for the OpenFrame MSP platform. The development section explains how to set up your environment, understand the architecture, write tests, secure your code, and contribute effectively. - ---- - -## Documentation Index +## Quick Navigation | Document | Description | |----------|-------------| -| [Environment Setup](./setup/environment.md) | IDE configuration, extensions, and dev tools | -| [Local Development](./setup/local-development.md) | Clone, build, run, and debug locally | -| [Architecture Overview](./architecture/README.md) | System design, module relationships, data flows | -| [Security Best Practices](./security/README.md) | Auth patterns, secrets management, input validation | -| [Testing Overview](./testing/README.md) | Test structure, running tests, writing new tests | -| [Contributing Guidelines](./contributing/guidelines.md) | Code style, PR process, commit conventions | +| [Environment Setup](setup/environment.md) | IDE configuration, editor plugins, and dev tools | +| [Local Development](setup/local-development.md) | Clone, build, run, and debug locally | +| [Architecture Overview](architecture/README.md) | High-level system design and module relationships | +| [Security Guide](security/README.md) | Authentication, JWT, secrets management | +| [Testing Guide](testing/README.md) | Unit tests, integration tests, test utilities | +| [Contributing Guidelines](contributing/guidelines.md) | Code style, branch naming, PR process | --- -## Quick Navigation - -### I want to... - -**Set up my local environment** -β†’ Start with [Environment Setup](./setup/environment.md), then [Local Development](./setup/local-development.md) - -**Understand how the system works** -β†’ Read the [Architecture Overview](./architecture/README.md) - -**Add a new feature or fix a bug** -β†’ Follow the [Contributing Guidelines](./contributing/guidelines.md) and review [Local Development](./setup/local-development.md) - -**Write tests for my changes** -β†’ See [Testing Overview](./testing/README.md) - -**Understand security requirements** -β†’ Read [Security Best Practices](./security/README.md) - ---- - -## Tech Stack at a Glance - -| Layer | Technology | -|-------|-----------| -| Language | Java 21 | -| Framework | Spring Boot 3.3 | -| Build Tool | Apache Maven 3.9+ | -| Multi-tenancy | Thread-local tenant context, per-tenant RSA keys | -| Auth | Spring Authorization Server, Spring Security OAuth2 | -| Persistence | MongoDB (sync + reactive), Redis, Cassandra, Apache Pinot | -| Messaging | Kafka / Debezium CDC, NATS JetStream | -| Gateway | Spring Cloud Gateway + WebFlux + Netty | -| API | Relay-compliant GraphQL (Netflix DGS), REST | -| Testing | JUnit 5, Testcontainers, RestAssured | -| Distributed Locking | ShedLock + Redis | +## Technology Stack + +`openframe-oss-lib` is built on: + +| Technology | Version | Role | +|------------|---------|------| +| **Java** | 21 | Primary language (Virtual Threads, Records, Sealed Classes) | +| **Spring Boot** | 3.3.0 | Application framework | +| **Spring Cloud Gateway** | 2023.0.3 | Reactive API gateway | +| **Spring Authorization Server** | 1.3.1 | OAuth2/OIDC server | +| **Spring Data MongoDB** | 4.2.0 | MongoDB persistence | +| **Netflix DGS** | 9.0.3 | GraphQL framework | +| **Apache Kafka** | via Spring Cloud | Event streaming | +| **NATS** | 0.6.2+3.5 | Real-time messaging | +| **Apache Pinot** | 1.2.0 | Analytics query engine | +| **Lombok** | 1.18.30 | Code generation | +| **JJWT** | 0.11.5 | JWT utilities | +| **gRPC** | 1.58.0 | Internal service communication | +| **Testcontainers** | 1.21.4 | Integration test infrastructure | --- @@ -66,39 +43,100 @@ Welcome to the **openframe-oss-lib** development documentation. This section cov ```text openframe-oss-lib/ -β”œβ”€β”€ pom.xml # Parent POM (unified versioning) -β”œβ”€β”€ openframe-core/ # Core utilities +β”œβ”€β”€ pom.xml # Parent POM (BOM + shared config) β”œβ”€β”€ openframe-exception/ # Exception hierarchy -β”œβ”€β”€ openframe-core-crypto/ # Encryption -β”œβ”€β”€ openframe-security-core/ # JWT, PKCE, cookies -β”œβ”€β”€ openframe-security-oauth/ # OAuth2 BFF -β”œβ”€β”€ openframe-authorization-service-core/# Multi-tenant auth server -β”œβ”€β”€ openframe-api-lib/ # API contracts, DTOs +β”œβ”€β”€ openframe-core/ # Core utilities +β”œβ”€β”€ openframe-core-crypto/ # Encryption services +β”œβ”€β”€ openframe-security-core/ # JWT infrastructure +β”œβ”€β”€ openframe-security-oauth/ # OAuth BFF layer +β”œβ”€β”€ openframe-api-lib/ # Shared DTO contracts β”œβ”€β”€ openframe-api-service-core/ # REST + GraphQL API +β”œβ”€β”€ openframe-authorization-service-core/# OAuth2/OIDC server β”œβ”€β”€ openframe-gateway-service-core/ # Reactive gateway -β”œβ”€β”€ openframe-client-core/ # Agent/client endpoints -β”œβ”€β”€ openframe-data-mongo-common/ # MongoDB documents -β”œβ”€β”€ openframe-data-mongo-sync/ # Sync repositories +β”œβ”€β”€ openframe-client-core/ # Agent ingress +β”œβ”€β”€ openframe-management-service-core/ # Management + schedulers +β”œβ”€β”€ openframe-data-mongo-common/ # Domain documents +β”œβ”€β”€ openframe-data-mongo-sync/ # MongoDB repositories β”œβ”€β”€ openframe-data-mongo-reactive/ # Reactive repositories -β”œβ”€β”€ openframe-data-redis/ # Redis cache -β”œβ”€β”€ openframe-data-kafka/ # Kafka configuration -β”œβ”€β”€ openframe-data-nats/ # NATS messaging -β”œβ”€β”€ openframe-data-cassandra/ # Cassandra storage -β”œβ”€β”€ openframe-data-pinot/ # Pinot analytics -β”œβ”€β”€ openframe-management-service-core/ # Schedulers, initializers -β”œβ”€β”€ openframe-stream-service-core/ # Kafka streams +β”œβ”€β”€ openframe-data-redis/ # Redis support +β”œβ”€β”€ openframe-data-kafka/ # Kafka support +β”œβ”€β”€ openframe-data-nats/ # NATS support +β”œβ”€β”€ openframe-data-pinot/ # Pinot support +β”œβ”€β”€ openframe-data-cassandra/ # Cassandra support +β”œβ”€β”€ openframe-data-device-aspect/ # Device event aspects +β”œβ”€β”€ openframe-stream-service-core/ # Stream processing β”œβ”€β”€ openframe-external-api-service-core/ # External REST API -β”œβ”€β”€ openframe-test-service-core/ # Integration test utilities +β”œβ”€β”€ openframe-debezium-initializer/ # Debezium CDC +β”œβ”€β”€ openframe-pinot-initializer/ # Pinot schema init +β”œβ”€β”€ openframe-notification-mail/ # Email notifications +β”œβ”€β”€ openframe-config-core/ # Config server +β”œβ”€β”€ openframe-fe-feature-flags/ # Feature flags +β”œβ”€β”€ openframe-test-service-core/ # Test utilities β”œβ”€β”€ sdk/ β”‚ β”œβ”€β”€ fleetmdm/ # Fleet MDM SDK β”‚ └── tacticalrmm/ # Tactical RMM SDK -└── ... +└── openframe-frontend-core/ # Shared frontend components (React) +``` + +--- + +## Module Dependency Layers + +```mermaid +graph LR + subgraph Layer1["Layer 1 - Foundation"] + E["openframe-exception"] + C["openframe-core"] + end + + subgraph Layer2["Layer 2 - Data & Security"] + M["openframe-data-mongo-common"] + SEC["openframe-security-core"] + CRYPTO["openframe-core-crypto"] + end + + subgraph Layer3["Layer 3 - Repositories & Messaging"] + MS["openframe-data-mongo-sync"] + MR["openframe-data-mongo-reactive"] + KF["openframe-data-kafka"] + NT["openframe-data-nats"] + end + + subgraph Layer4["Layer 4 - Services"] + API["openframe-api-service-core"] + AUTH["openframe-authorization-service-core"] + GW["openframe-gateway-service-core"] + MGMT["openframe-management-service-core"] + CLI["openframe-client-core"] + end + + E --> C + C --> CRYPTO + CRYPTO --> M + M --> SEC + M --> MS + M --> MR + MS --> API + SEC --> AUTH + API --> GW + API --> MGMT + API --> CLI ``` --- -## Community +## Getting Started + +If you haven't already: + +1. Review [Prerequisites](../getting-started/prerequisites.md) +2. Follow [Local Development](setup/local-development.md) to set up your environment +3. Read [Architecture Overview](architecture/README.md) before writing code + +--- -All development discussions happen in the [OpenMSP Slack Community](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA). We do not use GitHub Issues or GitHub Discussions. +## Community Support -[![OpenFrame v0.5.2: Autonomous AI Agent Architecture for MSPs](https://img.youtube.com/vi/PexpoNdZtUk/maxresdefault.jpg)](https://www.youtube.com/watch?v=PexpoNdZtUk) +> All development discussions happen on **OpenMSP Slack** β€” no GitHub Issues or Discussions. +> +> πŸ’¬ [https://www.openmsp.ai/](https://www.openmsp.ai/) diff --git a/docs/development/architecture/README.md b/docs/development/architecture/README.md index e03e09358..2efc372ca 100644 --- a/docs/development/architecture/README.md +++ b/docs/development/architecture/README.md @@ -1,6 +1,6 @@ # Architecture Overview -**openframe-oss-lib** implements a layered, multi-tenant service-oriented architecture. This document provides a high-level overview of the system design, component relationships, and key data flows. +`openframe-oss-lib` implements a **layered, multi-tenant, event-driven architecture** built on Spring Boot 3 and Java 21. This document provides a high-level overview of how all components interact. --- @@ -8,192 +8,227 @@ ```mermaid flowchart TD - Client["Client / Browser / Agent"] --> Gateway["Gateway Service Core\n(Spring Cloud Gateway + WebFlux)"] - Gateway --> ExternalAPI["External API Service Core\n(REST + API Keys)"] - Gateway --> ApiCore["API Service Core\n(REST + GraphQL)"] - - ApiCore --> Authz["Authorization Service Core\n(OAuth2 / JWT)"] - ApiCore --> Stream["Stream Service Core\n(Kafka / Debezium)"] - ApiCore --> Management["Management Service Core\n(Schedulers / Initializers)"] - ApiCore --> ClientSvc["Client Core\n(Agent Registration)"] - - Authz --> Mongo["MongoDB"] - ApiCore --> Mongo - ApiCore --> Redis["Redis"] - Stream --> Kafka["Apache Kafka"] - Stream --> Pinot["Apache Pinot"] - Stream --> Cassandra["Apache Cassandra"] - Management --> NATS["NATS JetStream"] - Management --> Mongo -``` + subgraph Edge["Edge Clients"] + Browser["Browser / SPA"] + Agent["Endpoint Agent"] + External["External API Consumer"] + end ---- + subgraph GatewayLayer["Gateway Service Core (Reactive)"] + Gateway["Spring Cloud Gateway"] + WsProxy["WebSocket Proxy"] + RateLimit["Rate Limiter"] + end -## Core Modules - -| Module | Responsibility | Key Technologies | -|--------|---------------|-----------------| -| `openframe-gateway-service-core` | Edge layer: JWT auth, API key rate limiting, WebSocket proxying, tool routing | Spring Cloud Gateway, WebFlux, Netty | -| `openframe-authorization-service-core` | Multi-tenant OAuth2 auth server: JWT issuance, SSO, per-tenant keys | Spring Authorization Server, Spring Security | -| `openframe-security-core` | JWT encoding/decoding, PKCE, cookie management | Nimbus JOSE, Spring Security | -| `openframe-security-oauth` | OAuth2 BFF: browser-facing login/callback/refresh/logout endpoints | Spring Security OAuth2 | -| `openframe-api-service-core` | Internal API: GraphQL (Relay) + REST controllers | Netflix DGS, Spring MVC | -| `openframe-api-lib` | API contracts: filter DTOs, cursor pagination, mutation types | Spring, Jackson | -| `openframe-external-api-service-core` | External API: API key–authenticated REST endpoints for integrations | Spring MVC, OpenAPI | -| `openframe-client-core` | Agent/device registration, tool agent endpoints | Spring MVC, NATS | -| `openframe-stream-service-core` | Kafka/Debezium CDC: event ingestion, normalization, enrichment | Kafka Streams, Spring Kafka | -| `openframe-management-service-core` | Startup initializers, distributed schedulers, tool orchestration | ShedLock, Spring Retry | -| `openframe-data-mongo-common` | MongoDB domain documents: canonical persistence model | Spring Data MongoDB | -| `openframe-data-mongo-sync` | Synchronous MongoDB repositories + index configuration | Spring Data MongoDB | -| `openframe-data-mongo-reactive` | Reactive MongoDB repositories | Spring Data MongoDB Reactive | -| `openframe-data-redis` | Tenant-aware Redis cache, reactive repositories | Spring Data Redis | -| `openframe-data-kafka` | Multi-tenant Kafka configuration, topic provisioning, retry | Spring Kafka | -| `openframe-data-nats` | NATS JetStream publishers, notification broadcasting | NATS Spring Cloud Stream | -| `openframe-data-cassandra` | Tenant-scoped Cassandra log storage | Spring Data Cassandra | -| `openframe-data-pinot` | Apache Pinot analytics queries | Pinot Java Client | -| `sdk/fleetmdm` | Fleet MDM Java client | Spring WebClient | -| `sdk/tacticalrmm` | Tactical RMM Java client | Spring WebClient | + subgraph AuthLayer["Authorization & Security"] + AuthServer["OAuth2/OIDC Server"] + SecurityOAuth["OAuth BFF Controller"] + JwtCore["JWT Encoder/Decoder"] + end ---- + subgraph ApiLayer["API Layer"] + RestAPI["REST Controllers"] + GraphQL["GraphQL Data Fetchers"] + DataLoaders["GraphQL DataLoaders"] + ExtAPI["External REST API"] + end -## Data Flow: Request Processing + subgraph DomainLayer["Domain & Business Services"] + BizServices["Business Services"] + Mapping["DTO Mappers"] + end -```mermaid -sequenceDiagram - participant C as Client - participant GW as Gateway - participant Auth as Auth Service - participant API as API Service - participant DB as MongoDB - - C->>GW: HTTP Request (with Bearer token or API key) - GW->>GW: Validate JWT (multi-issuer) / API key - GW->>API: Forwarded request + X-User-Id header - API->>DB: Query data (via Spring Data repositories) - DB-->>API: Domain documents - API-->>GW: Response - GW-->>C: HTTP Response + subgraph PersistenceLayer["Persistence"] + MongoSync["MongoDB Sync Repositories"] + MongoReactive["MongoDB Reactive Repositories"] + Redis["Redis Cache"] + Cassandra["Cassandra Audit Logs"] + end + + subgraph MessagingLayer["Messaging"] + Kafka["Apache Kafka"] + NATS["NATS JetStream"] + end + + subgraph StreamLayer["Analytics & Stream Processing"] + KafkaStreams["Kafka Streams"] + Debezium["Debezium CDC"] + Pinot["Apache Pinot"] + end + + subgraph AgentIngress["Agent Ingress"] + AgentReg["Agent Registration"] + AgentAuth["Agent OAuth Tokens"] + NatsListeners["NATS Listeners"] + end + + subgraph Management["Management & Operations"] + Initializers["App Initializers"] + Schedulers["Distributed Schedulers"] + Migrations["Mongock Migrations"] + end + + Browser --> Gateway + Agent --> Gateway + External --> Gateway + + Gateway --> AuthServer + Gateway --> RestAPI + Gateway --> GraphQL + Gateway --> ExtAPI + Gateway --> WsProxy + + AuthServer --> JwtCore + AuthServer --> MongoSync + SecurityOAuth --> AuthServer + + RestAPI --> BizServices + GraphQL --> DataLoaders + DataLoaders --> BizServices + ExtAPI --> BizServices + + BizServices --> Mapping + Mapping --> MongoSync + Mapping --> MongoReactive + + BizServices --> Kafka + BizServices --> NATS + + NATS --> AgentReg + NATS --> NatsListeners + AgentReg --> MongoSync + + Kafka --> KafkaStreams + Debezium --> KafkaStreams + KafkaStreams --> Pinot + + MongoSync --> Redis + Schedulers --> Redis + + Initializers --> MongoSync + Migrations --> MongoSync ``` --- -## Data Flow: Authentication (OAuth2) +## Core Components + +| Component | Module | Description | +|-----------|--------|-------------| +| **API Gateway** | `openframe-gateway-service-core` | Reactive edge layer: JWT validation, API key auth, WebSocket proxy, rate limiting | +| **Authorization Server** | `openframe-authorization-service-core` | Full OAuth2/OIDC server with multi-tenant JWT issuers and SSO | +| **Security OAuth/JWT** | `openframe-security-core`, `openframe-security-oauth` | RSA JWT infrastructure and OAuth BFF layer | +| **REST API** | `openframe-api-service-core` | REST controllers for devices, organizations, users, invitations | +| **GraphQL API** | `openframe-api-service-core` | Netflix DGS-based GraphQL with Relay cursor pagination | +| **External API** | `openframe-external-api-service-core` | OpenAPI-documented REST API for external consumers | +| **Business Services** | `openframe-api-lib` | Domain service interfaces (Device, Ticket, Org, Script, etc.) | +| **Domain Documents** | `openframe-data-mongo-common` | Canonical MongoDB documents for all aggregates | +| **Repositories** | `openframe-data-mongo-sync` | Spring Data MongoDB implementation with custom queries | +| **Agent Ingress** | `openframe-client-core` | Agent registration, authentication, and NATS event handling | +| **Management** | `openframe-management-service-core` | System bootstrapping, schedulers, and data migrations | +| **Kafka** | `openframe-data-kafka` | Kafka producers, topic config, retry producers | +| **NATS** | `openframe-data-nats` | NATS publishers for notifications, tool installation, commands | +| **Stream Processing** | `openframe-stream-service-core` | Kafka Streams topology with Debezium CDC enrichment | +| **Analytics** | `openframe-data-pinot` | Apache Pinot query client for logs and device data | +| **Tool SDKs** | `sdk/tacticalrmm`, `sdk/fleetmdm` | Type-safe SDK wrappers for external RMM and MDM tools | + +--- + +## Data Flow: User Request β†’ Response + +The following sequence shows a typical API request from a browser to a data response: ```mermaid sequenceDiagram - participant B as Browser - participant BFF as OAuth BFF - participant AuthSrv as Authorization Server - participant KS as TenantKeyService - participant DB as MongoDB - - B->>BFF: GET /oauth/login - BFF->>AuthSrv: Redirect (PKCE + state) - AuthSrv->>B: Login UI - B->>AuthSrv: Credentials - AuthSrv->>KS: Get tenant RSA key pair - KS->>DB: Load/create TenantKey - AuthSrv->>BFF: callback?code=... - BFF->>AuthSrv: Exchange code β†’ tokens - AuthSrv-->>BFF: JWT (access + refresh) - BFF->>B: Set HttpOnly cookies + participant Browser + participant Gateway as "Gateway Service Core" + participant AuthServer as "Authorization Server" + participant API as "API Service Core" + participant Services as "Business Services" + participant MongoDB + + Browser->>Gateway: HTTP Request + JWT Cookie + Gateway->>Gateway: Extract JWT from Cookie/Header + Gateway->>Gateway: Validate JWT (multi-issuer cache) + Gateway->>API: Forwarded request + Authorization header + API->>API: Resolve AuthPrincipal from JWT + API->>Services: Invoke business service + Services->>MongoDB: Query (tenant-scoped) + MongoDB-->>Services: Documents + Services-->>API: Domain objects + API-->>Gateway: HTTP Response + Gateway-->>Browser: HTTP Response ``` --- -## Data Flow: Event Streaming (Kafka / Debezium) +## Multi-Tenancy Model + +Multi-tenancy is a first-class concern at every layer: ```mermaid flowchart LR - subgraph Tools["Integrated Tools"] - T1["Tactical RMM"] - T2["Fleet MDM"] - T3["MeshCentral"] + subgraph JWTLayer["JWT Token"] + TenantClaim["tenant_id claim"] end - subgraph Processing["Stream Service Core"] - L["Kafka Listener"] - D["Tool Deserializer"] - E["Enrichment Service"] - M["EventTypeMapper"] - H["Message Handler"] - end - subgraph Storage["Storage"] - C["Cassandra\n(UnifiedLogEvent)"] - K["Kafka\n(Enriched Topics)"] - end - - T1 --> L - T2 --> L - T3 --> L - L --> D - D --> E - E --> M - M --> H - H --> C - H --> K -``` ---- + subgraph GatewayLayer["Gateway"] + IssuerValidation["Per-tenant issuer validation"] + end -## Multi-Tenancy Design + subgraph AuthLayer["Auth Server"] + TenantKeys["Per-tenant RSA signing keys"] + TenantContext["TenantContextFilter (ThreadLocal)"] + end -Every module implements strict tenant isolation: + subgraph DataLayer["Data Layer"] + TenantScoped["TenantScoped documents"] + TenantProvider["TenantIdProvider"] + end -```mermaid -flowchart TD - Request["HTTP Request"] --> TenantFilter["TenantContextFilter\n(Extracts tenant ID)"] - TenantFilter --> ThreadLocal["TenantContext\n(ThreadLocal)"] - ThreadLocal --> KeyService["TenantKeyService\n(Per-tenant RSA keys)"] - ThreadLocal --> Repo["Tenant-Scoped Repositories"] - ThreadLocal --> LockKey["ShedLock Key\nof:{tenantId}:job-lock:..."] - ThreadLocal --> CacheKey["Redis Key\nof:{tenantId}:..."] + TenantClaim --> IssuerValidation + IssuerValidation --> TenantContext + TenantContext --> TenantKeys + TenantKeys --> TenantScoped + TenantProvider --> TenantScoped ``` -Key multi-tenancy patterns: - -| Pattern | Implementation | -|---------|---------------| -| Tenant context propagation | `TenantContext` ThreadLocal, set by `TenantContextFilter` | -| Per-tenant JWT signing keys | `TenantKeyService` with RSA key pairs stored in MongoDB | -| Tenant-scoped cache keys | `OpenframeRedisKeyBuilder` prefixes every key with tenant ID | -| Tenant-scoped scheduler locks | ShedLock keys include `tenantId` and `environment` | -| JWT claim injection | `tenant_id`, `userId`, `roles` are embedded in every access token | +**Key rules:** +- Every MongoDB document has a `tenantId` field with an index +- JWTs include `tenant_id`, `roles`, and `userId` claims +- The `TenantIdProvider` resolves the current tenant from the security context +- In OSS (single-tenant) mode, `TENANT_ID=oss` is used as the default --- ## Key Design Decisions -### 1. Reactive Gateway, Blocking Services - -The gateway (`openframe-gateway-service-core`) is fully reactive (WebFlux + Netty). Internal services use Spring MVC (blocking), keeping service-level logic simple while the gateway handles high concurrency. +### 1. Gateway-Enforced Security +The **Gateway Service Core** handles all authentication and authorization. Downstream API services operate as OAuth2 resource servers only for principal resolution β€” actual enforcement is at the gateway. -### 2. GraphQL + REST Coexistence +### 2. Processor Pattern for Extensibility +Every lifecycle event has a corresponding `*Processor` interface (e.g., `AgentRegistrationProcessor`, `InvitationProcessor`). This allows downstream services to inject custom behavior without modifying the core library. -- **Internal API:** Relay-compliant GraphQL (Netflix DGS) with DataLoaders for N+1 prevention -- **External API:** REST with OpenAPI documentation and API key authentication -- Both APIs share the same domain services and repositories +### 3. Hybrid Messaging Model +- **Kafka** is used for durable, ordered, high-throughput event streaming (Debezium CDC, device telemetry) +- **NATS** is used for low-latency, real-time agent communication (heartbeats, command dispatch, tool installation) -### 3. Event-Driven Normalization +### 4. Relay-Compliant GraphQL +The GraphQL layer implements the Relay pagination specification with cursor-based pagination, `edges/nodes` pattern, and `PageInfo`. This ensures frontend compatibility with Relay or Apollo Client. -Integrated tools (Tactical RMM, Fleet MDM, MeshCentral) emit raw Debezium CDC events into Kafka. The Stream Service normalizes these into a unified `UnifiedEventType` before persisting to Cassandra or re-publishing. - -### 4. Startup Orchestration - -All bootstrapping logic is centralized in `openframe-management-service-core` using Spring `ApplicationRunner`. This ensures consistent initialization order across deployments. +### 5. ShedLock for Distributed Scheduling +All scheduled jobs use **ShedLock with Redis** to prevent duplicate execution in clustered environments. Lock keys are namespaced per tenant and environment. --- ## Reference Documentation -Detailed architecture documentation is available for each module: - -- [Gateway Service Core](./reference/architecture/gateway-service-core-security-and-routing/gateway-service-core-security-and-routing.md) -- [Authorization Service Core](./reference/architecture/authorization-service-core-server-and-tenant/authorization-service-core-server-and-tenant.md) -- [Stream Service Core](./reference/architecture/stream-service-core-kafka-and-handlers/stream-service-core-kafka-and-handlers.md) -- [Management Service Core](./reference/architecture/management-service-core-initializers-and-schedulers/management-service-core-initializers-and-schedulers.md) -- [External API Service Core](./reference/architecture/external-api-service-core/external-api-service-core.md) -- [Data Mongo Domain Model](./reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md) -- [Data Kafka Configuration](./reference/architecture/data-kafka-configuration-and-retry/data-kafka-configuration-and-retry.md) -- [Security Core and OAuth BFF](./reference/architecture/security-core-and-oauth-bff/security-core-and-oauth-bff.md) +For detailed documentation on each module, see the reference docs: -[![Watch What's New in OpenFrame 0.7.8](https://img.youtube.com/vi/BQAjDB4ED2Y/maxresdefault.jpg)](https://www.youtube.com/watch?v=BQAjDB4ED2Y) +- [API Service Core (Config & Security)](../../reference/architecture/api-service-core-config-and-security/api-service-core-config-and-security.md) +- [Authorization Service Core](../../reference/architecture/authorization-service-core/authorization-service-core.md) +- [Gateway Service Core](../../reference/architecture/gateway-service-core/gateway-service-core.md) +- [Data Model & Repositories (Mongo)](../../reference/architecture/data-model-and-repositories-mongo/data-model-and-repositories-mongo.md) +- [Security OAuth & JWT](../../reference/architecture/security-oauth-and-jwt/security-oauth-and-jwt.md) +- [Management Service Core](../../reference/architecture/management-service-core/management-service-core.md) +- [Integrations SDKs](../../reference/architecture/integrations-sdks/integrations-sdks.md) diff --git a/docs/development/contributing/guidelines.md b/docs/development/contributing/guidelines.md index d12792a64..ee5afbb76 100644 --- a/docs/development/contributing/guidelines.md +++ b/docs/development/contributing/guidelines.md @@ -1,122 +1,111 @@ # Contributing Guidelines -Thank you for contributing to **openframe-oss-lib**! This guide covers code style, branching strategy, commit conventions, and the pull request process. +Thank you for contributing to `openframe-oss-lib`! This document outlines the code style, branching strategy, commit conventions, and PR review process. --- ## Community First -All contribution discussions happen on the [OpenMSP Slack Community](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA). We do **not** use GitHub Issues or GitHub Discussions. +> **All project discussion, support, and contributions are coordinated through the OpenMSP Slack community.** +> We do not use GitHub Issues or GitHub Discussions. +> +> πŸ’¬ Join: [https://www.openmsp.ai/](https://www.openmsp.ai/) +> +> πŸ”— Direct invite: [https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) -Before starting a large contribution: -1. Join the [OpenMSP Slack](https://www.openmsp.ai/) -2. Describe what you're planning in the `#openframe-dev` channel -3. Get feedback from maintainers before investing significant effort +Before opening a pull request for a significant change, discuss it in Slack to align on the approach. --- ## Code Style and Conventions -### Java Style +### Java Conventions -The project follows standard Java conventions with the following specifics: +The project follows standard Spring Boot and Java conventions: | Convention | Rule | |-----------|------| -| Indentation | 4 spaces (no tabs) | -| Line length | Max 120 characters | -| Imports | No wildcard imports; organize by static β†’ java β†’ jakarta β†’ spring β†’ other | -| Braces | Allman-adjacent style (opening brace on same line) | -| Naming | `camelCase` methods/fields, `PascalCase` classes, `UPPER_SNAKE` constants | +| **Indentation** | 4 spaces (no tabs) | +| **Line length** | 120 characters maximum | +| **Imports** | No wildcard imports (`import com.example.*`) | +| **Naming** | Standard Java naming (camelCase for methods/fields, PascalCase for classes) | +| **Access modifiers** | Use the most restrictive access level possible | +| **Final** | Use `final` for fields that are not reassigned | +| **Null handling** | Use `Optional` for nullable return values; avoid `@Nullable` in service layers | ### Lombok Usage -Use Lombok to reduce boilerplate. Preferred annotations: +This project uses Lombok to reduce boilerplate. Prefer: ```java -// Prefer these -@Data // getters, setters, equals, hashCode, toString -@Value // immutable value objects -@Builder // fluent builders -@RequiredArgsConstructor // constructor injection -@Slf4j // logging - -// Avoid manual getters/setters when @Data or @Value apply -``` - -### Spring Boot Conventions +// βœ… Use @Value for immutable DTOs +@Value +public class AgentRegistrationResponse { + String machineId; + String clientSecret; +} -- Use constructor injection (via `@RequiredArgsConstructor`) over field injection (`@Autowired`) -- Prefer `@ConfigurationProperties` over `@Value` for configuration -- Use `@Service`, `@Repository`, `@Component`, `@Controller` consistently -- Configuration classes should be annotated with `@Configuration` +// βœ… Use @Data for mutable domain classes (sparingly) +@Data +@Document(collection = "devices") +public class Device implements TenantScoped { + @Id + private String id; + private String tenantId; +} -```java -// Correct: Constructor injection -@Service -@RequiredArgsConstructor -public class MyService { - private final MyRepository repository; - private final AnotherService anotherService; +// βœ… Use @Builder for complex object construction +@Builder +public class NotificationMessage { + private String title; + private String description; + private NotificationSeverity severity; } -// Avoid: Field injection +// βœ… Use @RequiredArgsConstructor for constructor injection @Service -public class MyService { - @Autowired - private MyRepository repository; +@RequiredArgsConstructor +public class DeviceService { + private final MachineRepository machineRepository; + private final TagService tagService; } ``` -### Multi-Tenancy Requirements +### Spring Conventions -Every service and repository method that accesses tenant-scoped data **must** include `tenantId` in queries: - -```java -// Correct: Always scope to tenant -Optional findByIdAndTenantId(String id, String tenantId); - -// Wrong: Missing tenant scope -Optional findById(String id); // Never use for tenant-scoped data -``` - -### Exception Handling - -Use the standard exception hierarchy from `openframe-exception`: - -```java -// Use specific exception types -throw new NotFoundException("Organization not found: " + id); -throw new BadRequestException("Invalid email format"); -throw new ForbiddenException("Access denied for tenant: " + tenantId); -throw new ConflictException("Email already exists"); -throw new ValidationException("Required field missing: name"); -``` - -Never throw `RuntimeException` or `Exception` directly. +- **Constructor injection** is preferred over field injection +- **`@RequiredArgsConstructor` + `final` fields** is the recommended pattern for Spring components +- Avoid `@Autowired` on fields +- Keep controllers thin: delegate all business logic to service classes +- Controllers should return DTOs, not domain documents --- ## Branch Naming -Use descriptive branch names with a type prefix: +Use the following conventions for branch names: + +| Type | Format | Example | +|------|--------|---------| +| Feature | `feature/short-description` | `feature/add-notification-webhooks` | +| Bug fix | `fix/short-description` | `fix/agent-registration-timeout` | +| Refactor | `refactor/short-description` | `refactor/simplify-tenant-resolution` | +| Documentation | `docs/short-description` | `docs/update-gateway-readme` | +| Chore | `chore/short-description` | `chore/upgrade-spring-boot-3.4` | -| Type | Pattern | Example | -|------|---------|---------| -| Feature | `feature/` | `feature/add-nats-retry-logic` | -| Bug fix | `fix/` | `fix/tenant-context-not-cleared` | -| Refactor | `refactor/` | `refactor/notification-repository` | -| Documentation | `docs/` | `docs/update-kafka-readme` | -| Dependency updates | `deps/` | `deps/upgrade-spring-boot-3.4` | +**Rules:** +- Use lowercase and hyphens, no spaces or underscores +- Keep names short but descriptive (3-6 words) +- Branch from `main` (or the current release branch) unless told otherwise --- ## Commit Message Format -Follow the [Conventional Commits](https://www.conventionalcommits.org/) specification: +Use the **Conventional Commits** specification: ```text -(): +(): [optional body] @@ -125,45 +114,45 @@ Follow the [Conventional Commits](https://www.conventionalcommits.org/) specific ### Types -| Type | When to Use | +| Type | When to use | |------|------------| -| `feat` | New feature or capability | -| `fix` | Bug fix | -| `refactor` | Code change that neither fixes a bug nor adds a feature | +| `feat` | A new feature or capability | +| `fix` | A bug fix | +| `refactor` | Code restructuring without behavior change | | `test` | Adding or updating tests | | `docs` | Documentation only changes | -| `chore` | Build system, dependency updates, CI changes | +| `chore` | Build system, dependency updates, CI config | | `perf` | Performance improvements | +| `style` | Formatting, missing semicolons (no behavior change) | -### Scope +### Scope Examples -Use the module name (without `openframe-` prefix) as scope: +Use the module name (without `openframe-` prefix) as the scope: ```text -feat(security-core): add PKCE utility for code challenge generation -fix(data-mongo-sync): resolve tenant context leak in batch operations -refactor(gateway-service-core): extract rate limit logic into service -test(data-nats): add integration test for notification broadcast -docs(api-service-core): update GraphQL data fetcher documentation -chore(deps): upgrade spring-boot to 3.3.2 +feat(api-service-core): add bulk device status update endpoint +fix(gateway-service-core): resolve rate limit header not being set on 429 response +refactor(data-mongo-sync): simplify cursor pagination in notification repository +test(authorization-service-core): add SSO invitation acceptance integration test +chore: upgrade spring-boot to 3.3.1 ``` -### Examples +### Commit Message Examples ```text -feat(authorization-service-core): add Microsoft SSO provider strategy +feat(client-core): add virtual thread executor for tool installation tasks -Implements MicrosoftClientRegistrationStrategy to support Microsoft -Entra ID (Azure AD) authentication flows alongside existing Google SSO. +Tool installation tasks can be I/O bound. Using virtual threads +allows higher concurrency without increased memory overhead. -Closes: #discussion in #openframe-dev slack +Closes #123 (if you are using a ticket or task reference) ``` ```text -fix(stream-service-core): handle null tenant_id in debezium enrichment +fix(data-nats): retry NATS publish on transient connection failure -When a Debezium event is missing the tenant header, the enrichment service -now falls back to domain-based tenant resolution instead of throwing NPE. +The publisher now retries up to 3 times with exponential backoff +when encountering a NatsException due to temporary disconnection. ``` --- @@ -172,93 +161,99 @@ now falls back to domain-based tenant resolution instead of throwing NPE. ### Before Opening a PR -1. **Build passes:** `mvn install -DskipTests` -2. **Tests pass:** `mvn test -pl ` -3. **No secrets committed:** Review all changed files -4. **Follows code style:** Check Lombok, constructor injection, tenant scoping -5. **Covers edge cases:** Add unit tests for new logic +1. **Discuss in Slack** β€” For significant changes, align with maintainers first +2. **Write tests** β€” New features require unit tests; bug fixes require a regression test +3. **Check existing tests pass** β€” Run `mvn test` locally before pushing +4. **Follow code style** β€” Run your IDE formatter before committing ### PR Title Use the same format as commit messages: ```text -feat(security-core): add PKCE utility for authorization flows -fix(data-mongo-sync): resolve tenant context leak +feat(api-service-core): add GraphQL subscription for device status changes ``` ### PR Description Template ```markdown -## What does this PR do? +## Summary +Brief description of what this PR does. -Brief description of the change. +## Changes +- List key changes made +- Include any migration steps if needed -## Why? +## Testing +- Describe how you tested the changes +- List new tests added -Motivation for the change. +## Related +- Link to Slack thread or task (if applicable) +``` -## How was it tested? +### Review Process -- [ ] Unit tests added/updated -- [ ] Integration tests added/updated -- [ ] Manual testing performed +1. At least **one maintainer approval** is required to merge +2. All CI checks (build + tests) must pass +3. Resolve all review comments before requesting re-review +4. Squash commits if the history is noisy (maintainer may do this at merge) -## Checklist +--- -- [ ] No secrets in code or tests -- [ ] All new endpoints have authorization rules -- [ ] New DB queries are tenant-scoped -- [ ] Input validation on all new DTOs -- [ ] No breaking changes (or breaking changes are documented) -``` +## Review Checklist -### Review Checklist (for Reviewers) +Use this checklist when reviewing or self-reviewing a PR: -- [ ] Code follows established patterns (constructor injection, Lombok, multi-tenancy) -- [ ] New functionality is tested -- [ ] Security considerations are addressed (tenant scope, input validation, no secrets) -- [ ] Error handling uses the standard exception hierarchy -- [ ] No performance regressions (N+1 queries, missing indexes) +### Correctness +- [ ] The change does what the PR description says +- [ ] Edge cases are handled (null inputs, empty collections, concurrent access) +- [ ] Error handling is appropriate (no swallowed exceptions) ---- +### Security +- [ ] New MongoDB queries always filter by `tenantId` +- [ ] New API endpoints are covered by Gateway authorization rules +- [ ] Sensitive data uses `EncryptionService` for at-rest protection +- [ ] No secrets or credentials are hardcoded or logged -## Versioning +### Testing +- [ ] Unit tests cover the happy path and key error scenarios +- [ ] Integration tests are added for new repository methods +- [ ] Existing tests still pass -All modules are versioned together using `${revision}` in the parent POM. Version bumps are managed by the maintainers. Contributors do not need to update the version number in PRs. +### Code Quality +- [ ] No unnecessary complexity added +- [ ] Follows the processor/extension pattern for lifecycle hooks +- [ ] Lombok annotations are used appropriately +- [ ] Constructor injection is used (not field injection) +- [ ] DTOs are used in controllers (not domain documents) -The version follows [Semantic Versioning](https://semver.org/): -- **Major:** Breaking API changes -- **Minor:** New backward-compatible features -- **Patch:** Backward-compatible bug fixes +### Documentation +- [ ] Public interfaces and complex methods have Javadoc +- [ ] Configuration properties are documented (in-class or via `@ConfigurationProperties` description) --- ## Adding a New Module -When adding a new module to the library: - -1. Create the module directory following the existing naming convention (`openframe-/`) -2. Add a `pom.xml` that inherits from the parent -3. Add the module to the parent `pom.xml` `` section -4. Add the module to `` in the parent with `${revision}` -5. Write unit tests before submitting -6. Update the README and architecture documentation - -```xml - -openframe-my-new-module - - - - com.openframe.oss - openframe-my-new-module - ${revision} - -``` +If your contribution requires a new module: + +1. Follow the existing module structure conventions +2. Add the module to the root `pom.xml` `` section +3. Add it to `` with `${revision}` version +4. Use the `com.openframe.oss` groupId and follow the `openframe-*` artifact naming convention +5. Ensure the module has a meaningful `` in its `pom.xml` +6. Add appropriate unit and integration tests before the PR --- -## Getting Help +## Release Process + +The project uses **Flat Maven versioning** via the `${revision}` property in the parent POM. The current version is `6.0.10`. + +Versions are published to GitHub Packages at: +```text +https://maven.pkg.github.com/flamingo-stack/openframe-oss-lib +``` -Stuck on a contribution? Reach out on the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) in `#openframe-dev`. +Release coordination happens through the OpenMSP Slack `#engineering` channel. diff --git a/docs/development/security/README.md b/docs/development/security/README.md index 36431181f..b4d28d7f5 100644 --- a/docs/development/security/README.md +++ b/docs/development/security/README.md @@ -1,153 +1,215 @@ # Security Best Practices -This guide covers the security patterns, conventions, and requirements for developing with and contributing to **openframe-oss-lib**. +This guide covers security patterns, authentication flows, secrets management, and security testing practices for `openframe-oss-lib`. --- -## Authentication and Authorization Patterns +## Overview -### JWT-Based Authentication +OpenFrame OSS Lib implements a defense-in-depth security model with multiple layers: -Every service in OpenFrame uses JWT bearer tokens for authentication. The library provides: +```mermaid +flowchart TD + subgraph Layer1["Layer 1 - Network Edge"] + CORS["CORS Policy"] + RateLimit["Rate Limiting"] + OriginSanitize["Origin Sanitizer"] + end + + subgraph Layer2["Layer 2 - Authentication"] + JWT["JWT Validation (multi-issuer)"] + ApiKey["API Key Authentication"] + PKCE["PKCE + OAuth2"] + end + + subgraph Layer3["Layer 3 - Authorization"] + RoleBased["Role-Based Access Control"] + TenantScope["Tenant Scoping"] + end + + subgraph Layer4["Layer 4 - Data"] + Encryption["Field Encryption (AES)"] + HashPasswords["BCrypt Password Hashing"] + TenantFilter["Tenant-Scoped Queries"] + end + + Layer1 --> Layer2 + Layer2 --> Layer3 + Layer3 --> Layer4 +``` -- **`openframe-security-core`** β€” JWT encoder/decoder beans, RSA key loading -- **`openframe-authorization-service-core`** β€” Multi-tenant OAuth2 Authorization Server -- **`openframe-gateway-service-core`** β€” Multi-issuer JWT validation at the edge +--- -**Key principle:** JWTs are issued per-tenant with tenant-scoped RSA key pairs. Never use a shared signing key across tenants. +## Authentication Architecture -#### JWT Claims Structure +### JWT-Based Authentication -Every access token must contain: +All API requests are authenticated via **RS256 JWT tokens** issued by the Authorization Service Core. -| Claim | Description | -|-------|-------------| -| `tenant_id` | Tenant identifier for multi-tenancy | -| `userId` | Authenticated user's ID | -| `roles` | User roles (`ADMIN`, `OWNER`, `AGENT`) | -| `iss` | Issuer URL (tenant-specific) | -| `exp` | Expiration timestamp | +**JWT Claims structure:** -```java -// Correct: Extract tenant from JWT principal -@GetMapping("/me") -public ResponseEntity getCurrentUser(@AuthenticationPrincipal Jwt jwt) { - String tenantId = jwt.getClaimAsString("tenant_id"); - String userId = jwt.getClaimAsString("userId"); - // ... +```json +{ + "sub": "user-id", + "tenant_id": "my-tenant", + "roles": ["ADMIN"], + "iss": "https://auth.yourdomain.com/my-tenant", + "exp": 1234567890 } ``` -### Multi-Tenant Key Isolation - -Each tenant has its own RSA key pair managed by `TenantKeyService`: - -```mermaid -flowchart LR - Token["JWT Signing Request"] --> CTX["TenantContext.getTenantId()"] - CTX --> KS["TenantKeyService.getOrCreateActiveKey()"] - KS --> DB["MongoDB: TenantKey collection"] - KS --> RSA["RSA Key Pair"] - RSA --> JWT["Signed JWT"] -``` +**Key security properties:** -> **Never** share RSA key material between tenants or inject keys via environment variables in production. Always use the `TenantKeyService` for key lifecycle management. +| Property | Value | +|----------|-------| +| Algorithm | RS256 (RSA + SHA-256) | +| Key Type | Per-tenant RSA key pairs | +| Key Storage | Encrypted in MongoDB (`tenant_keys` collection) | +| JWKS Exposure | Per-tenant `.well-known/jwks.json` endpoint | +| Token Validation | Strict issuer + signature + expiry validation | -### Role-Based Access Control +### Multi-Issuer JWT Validation -Gateway-level role enforcement is configured in `GatewaySecurityConfig`: +The gateway maintains a **Caffeine cache** of authentication managers keyed by issuer URL. This means: -| Path Pattern | Required Role | -|-------------|--------------| -| `/api/**` | `ADMIN` | -| `/tools/agent/**` | `AGENT` | -| `/ws/tools/agent/**` | `AGENT` | -| `/ws/nats` | `ADMIN` or `AGENT` | -| `/external-api/**` | API key (no JWT required) | +- Each tenant has a distinct issuer URL +- Tokens from different tenants are validated against the correct signing keys +- Invalid issuer URLs are immediately rejected (not cached) -When adding new routes, always explicitly define authorization rules. Never rely on implicit `permitAll()` for sensitive endpoints. +```mermaid +flowchart TD + Token["JWT Token"] --> Extract["Extract iss claim"] + Extract --> Allowed{"Is issuer allowed?"} + Allowed -->|No| Reject["401 Unauthorized"] + Allowed -->|Yes| Cache{"Cache hit?"} + Cache -->|Yes| Validate["Validate Signature"] + Cache -->|No| LoadKey["Load Public Key via JWKS"] + LoadKey --> CacheKey["Cache Auth Manager"] + CacheKey --> Validate + Validate --> Success["Request proceeds"] + Validate -->|Invalid| Reject +``` --- -## OAuth2 Best Practices +## API Key Authentication -### PKCE (Proof Key for Code Exchange) +The External API uses **API key authentication** instead of JWT for machine-to-machine access. -All browser-initiated authorization flows **must** use PKCE. The `PKCEUtils` class provides: +### API Key Format -```java -// Always use PKCE for browser flows -String codeVerifier = PKCEUtils.generateCodeVerifier(); -String codeChallenge = PKCEUtils.generateCodeChallenge(codeVerifier); -String state = PKCEUtils.generateState(); -``` +API keys are stored in MongoDB with the following security controls: -Never implement custom PKCE logic β€” use the provided utility class. +- Keys are **hashed** before storage (BCrypt) +- Raw key value is only returned once at creation time +- Keys support per-minute, per-hour, and per-day rate limits +- API key ID is sent as a request header (`X-API-KEY-ID`) for tracing -### Token Storage - -Tokens are stored as **HTTP-only cookies** by the OAuth BFF controller. This pattern prevents XSS-based token theft: +### Rate Limiting ```text -Set-Cookie: access_token=...; HttpOnly; Secure; SameSite=Strict -Set-Cookie: refresh_token=...; HttpOnly; Secure; SameSite=Strict +Default rate limits per API key: +- Per minute: 60 requests +- Per hour: 1,000 requests +- Per day: 10,000 requests ``` -> **Never** expose tokens in JavaScript-accessible storage (localStorage, sessionStorage) or URL parameters. +Rate limit state is stored in **Redis** for cross-instance consistency. -### API Keys +--- -External API consumers authenticate via `X-API-Key` header. API keys follow a two-part format: +## OAuth2 / OIDC Security -```text -X-API-Key: . +### PKCE Enforcement + +All OAuth2 authorization code flows **require PKCE** (Proof Key for Code Exchange): + +```mermaid +sequenceDiagram + participant Client + participant BFF as "OAuth BFF Controller" + participant AuthServer + + Client->>BFF: GET /oauth/login + BFF->>BFF: Generate code_verifier + code_challenge (SHA-256) + BFF->>BFF: Generate state (SecureRandom) + BFF->>AuthServer: Redirect with code_challenge + state + AuthServer->>Client: Login form + Client->>AuthServer: Submit credentials + AuthServer->>BFF: Authorization code + state + BFF->>BFF: Validate state (from secure cookie) + BFF->>AuthServer: Exchange code + code_verifier for tokens + AuthServer->>BFF: Access + Refresh tokens + BFF->>Client: Set HttpOnly cookies ``` -Key security requirements: +**Security controls:** +- `code_verifier` is 256-bit random, Base64URL-encoded +- `code_challenge` = `BASE64URL(SHA256(code_verifier))` +- `state` is a signed JWT stored in a secure cookie to prevent CSRF + +### Token Storage -- Store only the **hashed** secret in MongoDB (`ApiKey.secretHash`) -- Never log raw API key values -- Apply rate limiting via `RateLimitService` for all API key–authenticated routes -- Rotate API keys on suspected compromise +Tokens are stored in **HttpOnly, Secure cookies** to prevent XSS access: + +| Cookie | Content | Attributes | +|--------|---------|-----------| +| `access_token` | JWT access token | HttpOnly, Secure, SameSite=Strict | +| `refresh_token` | Opaque refresh token | HttpOnly, Secure, SameSite=Strict | --- -## Data Encryption and Secure Storage +## Role-Based Access Control (RBAC) -### Encryption Service +### Roles -The `openframe-core-crypto` module provides symmetric encryption for sensitive fields stored in MongoDB: +| Role | Description | Scope | +|------|-------------|-------| +| `OWNER` | Full tenant access; implicitly includes ADMIN | Tenant-level | +| `ADMIN` | Full administrative access to tenant resources | Tenant-level | +| `AGENT` | Machine/agent authentication only | Device-level | -```java -@Autowired -private EncryptionService encryptionService; +### Path Authorization -// Encrypt before storing -String encrypted = encryptionService.encrypt(plainText); +The gateway enforces role requirements per path: -// Decrypt on read -String plain = encryptionService.decrypt(encrypted); -``` +| Path Pattern | Required Role | +|-------------|--------------| +| `/api/**` | `ADMIN` | +| `/tools/agent/**` | `AGENT` | +| `/ws/tools/agent/**` | `AGENT` | +| `/ws/nats` | `AGENT` or `ADMIN` | +| `/content/**` | `ADMIN` | +| `/external-api/**` | Valid API Key | + +--- -Fields that should always be encrypted in MongoDB: +## Secrets Management -- Tool credentials (`ToolCredentials`) -- API key secrets (`ApiKey.secretHash` β€” hashed, not encrypted) -- SSO provider client secrets (`SSOConfig`) +### Environment Variables for Secrets -### Password Hashing +Never hardcode secrets. Use environment variables or a secrets manager: -Passwords are hashed using BCrypt via the `PasswordEncoder` bean provided by `ManagementConfiguration`: +```bash +# JWT RSA Keys (load from secrets manager in production) +export JWT_PUBLIC_KEY="-----BEGIN PUBLIC KEY-----..." +export JWT_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----..." -```java -// Correct: Always hash passwords before storing -String hashed = passwordEncoder.encode(rawPassword); +# OAuth client credentials +export OAUTH_CLIENT_DEFAULT_ID="your-client-id" +export OAUTH_CLIENT_DEFAULT_SECRET="your-client-secret" -// Correct: Verify passwords using the encoder -boolean matches = passwordEncoder.matches(rawPassword, hashed); +# Database credentials +export SPRING_DATA_MONGODB_URI="mongodb+srv://user:pass@host/db" ``` -Never store raw passwords in any form β€” not in logs, databases, or environment variables. +### Key Rotation + +Per-tenant RSA signing keys are stored encrypted in MongoDB. The `TenantKeyService` supports: + +- Generating new RSA key pairs per tenant +- Encrypting private keys at rest using `EncryptionService` +- Rotating keys without disrupting active sessions (old keys remain in JWKS until expired) --- @@ -155,138 +217,107 @@ Never store raw passwords in any form β€” not in logs, databases, or environment ### Bean Validation -All request DTOs must use Jakarta Bean Validation annotations: +All DTOs use Jakarta Bean Validation (`@Valid`, `@NotNull`, `@Email`, etc.): ```java -@NotNull -@ValidEmail // Custom OpenFrame validator -private String email; +// Example: validated DTO +public class CreateOrganizationRequest { + @NotBlank + private String name; -@NotBlank -@TenantDomain // Custom OpenFrame validator -private String tenantDomain; + @ValidEmail // Custom validator in openframe-core + private String email; +} ``` -Custom validators in `openframe-core`: - -| Annotation | Validates | -|-----------|----------| -| `@ValidEmail` | Email format and domain | -| `@TenantDomain` | Tenant domain slug format | - -### SQL / NoSQL Injection Prevention - -MongoDB queries are always executed via Spring Data repositories or `MongoTemplate` with typed objects β€” never with raw string interpolation: +### Custom Validators -```java -// Correct: Use typed Spring Data query method -List users = userRepository.findByTenantIdAndEmail(tenantId, email); +The `openframe-core` module provides reusable custom validators: -// Wrong: Never build raw query strings -// mongoTemplate.find(Query.query(Criteria.where("email").is("' OR '1'='1")), User.class); -``` +| Annotation | Description | +|-----------|-------------| +| `@ValidEmail` | Email format validation with normalization | +| `@TenantDomain` | Tenant subdomain format validation | ---- - -## Common Security Vulnerabilities and Mitigations +### GraphQL Input Validation -| Vulnerability | Risk | Mitigation in openframe-oss-lib | -|-------------|------|-------------------------------| -| Cross-Tenant Data Access | CRITICAL | TenantContext + tenant-scoped repositories; always include `tenantId` in queries | -| JWT Token Forgery | HIGH | Per-tenant RSA keys; multi-issuer validation; short expiry | -| CSRF | MEDIUM | OAuth2 state parameter + PKCE; HTTP-only cookies with SameSite | -| API Key Exposure | HIGH | Keys hashed at rest; rate limiting; `X-API-Key` header only | -| XSS Token Theft | HIGH | HTTP-only cookies; CSP headers enforced at gateway | -| Mass Assignment | MEDIUM | Explicit DTO mapping; never expose domain documents directly | -| Insecure Direct Object Reference | HIGH | Always scope queries with `tenantId` + authorization checks | +GraphQL mutations validate inputs before reaching the service layer. Validation errors are returned as structured `MutationError` objects (not HTTP errors). --- -## Secrets Management - -### Local Development - -For local development, use `application-local.yml` (never committed) to override sensitive properties: - -```yaml -# application-local.yml (gitignored) -jwt: - private-key: classpath:keys/local-private.pem - public-key: classpath:keys/local-public.pem -spring: - data: - mongodb: - uri: mongodb://localhost:27017/openframe-local -``` +## Data Encryption -### Environment Variable Conventions +### Field-Level Encryption -In production deployments, secrets must be injected as environment variables, never hardcoded: +The `openframe-core-crypto` module provides `EncryptionService` for encrypting sensitive fields at rest: -```bash -# JWT keys -JWT_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----..." -JWT_PUBLIC_KEY="-----BEGIN PUBLIC KEY-----..." - -# MongoDB -SPRING_DATA_MONGODB_URI="mongodb+srv://user:pass@cluster..." - -# Redis -SPRING_DATA_REDIS_HOST="redis.internal" -``` - -> Use your platform's secret manager (AWS Secrets Manager, HashiCorp Vault, Kubernetes Secrets) to inject these values. Never commit secrets to version control. - -### CI/CD Secret Handling +```java +@Autowired +private EncryptionService encryptionService; -GitHub Actions secrets are referenced via: +// Encrypt before saving +String encrypted = encryptionService.encrypt(sensitiveValue); -```yaml -env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} +// Decrypt when reading +String plaintext = encryptionService.decrypt(encrypted); ``` -Never echo, log, or print secrets in CI steps. +This is used for: +- Tool credentials (API keys for Tactical RMM, Fleet MDM) +- Per-tenant RSA private keys +- Agent registration secrets --- -## Security Testing and Code Review Guidelines - -### Pre-Commit Checklist +## Common Security Vulnerabilities and Mitigations -Before opening a Pull Request, verify: +| Vulnerability | Mitigation | +|--------------|-----------| +| **JWT Forgery** | RS256 asymmetric keys; private key never leaves Authorization Server | +| **CSRF** | PKCE `state` parameter; SameSite cookie policy | +| **XSS token theft** | HttpOnly cookies; tokens never in JavaScript-accessible storage | +| **SQL/NoSQL Injection** | Spring Data MongoTemplate with typed queries; no raw string interpolation | +| **Authorization Bypass** | Gateway-level enforcement; tenant-scoped queries at data layer | +| **Replay Attacks** | Short-lived access tokens (configurable TTL); refresh token rotation | +| **Brute Force** | Rate limiting at Gateway (Redis-backed per API key) | +| **Insecure Direct Object Reference** | Tenant-scoped queries; all queries include `tenantId` filter | -- [ ] No secrets, tokens, or keys in code or test fixtures -- [ ] All new endpoints have explicit authorization rules -- [ ] New database queries include `tenantId` scope -- [ ] Input validation annotations on all request DTOs -- [ ] Sensitive fields are encrypted at rest -- [ ] No raw SQL/NoSQL string interpolation +--- -### Integration Test Security +## Security Testing Guidelines -Integration tests should: +### Testing Authentication -- Use randomly generated test data (not hardcoded UUIDs matching production patterns) -- Clean up test data after each test -- Never use production URLs or credentials +The `openframe-test-service-core` module provides authentication helpers for integration tests: ---- - -## Origin Sanitization +```java +// Obtain an auth token for tests +AuthFlow authFlow = new AuthFlowOSS(environmentConfig); +AuthParts auth = authFlow.login("user@example.com", password); + +// Use token in test requests +RequestSpecHelper.withAuth(auth) + .get("/api/devices") + .then() + .statusCode(200); +``` -The `OriginSanitizerFilter` in the gateway sanitizes the `Origin` header to prevent header injection attacks. Never bypass this filter for external-facing routes. +### Testing Authorization -```mermaid -flowchart LR - Request["Incoming Request"] --> OSF["OriginSanitizerFilter"] - OSF --> AHF["AddAuthorizationHeaderFilter"] - AHF --> JWT["JWT Validation"] - JWT --> Route["Route Handler"] -``` +Always test: +1. **Authenticated access** β†’ returns expected data +2. **Unauthenticated access** β†’ returns 401 +3. **Wrong tenant access** β†’ returns 403 or empty data +4. **Insufficient role** β†’ returns 403 ---- +### Security Code Review Checklist -## Reporting Security Issues +When reviewing security-relevant code, verify: -For security vulnerabilities, do **not** open a public GitHub issue. Contact the team via the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) in a direct message to the maintainers, or email the security contact listed on [flamingo.run](https://flamingo.run). +- [ ] All new DTOs have `@Valid` annotations on request-body parameters +- [ ] New MongoDB queries always filter by `tenantId` +- [ ] New API endpoints are covered by Gateway path authorization rules +- [ ] Sensitive fields use `EncryptionService` for at-rest protection +- [ ] No secrets or keys are logged (check log statements in new code) +- [ ] New OAuth flows use PKCE +- [ ] New external integrations use stored, encrypted credentials diff --git a/docs/development/setup/environment.md b/docs/development/setup/environment.md index c2c95cf53..675b04fe1 100644 --- a/docs/development/setup/environment.md +++ b/docs/development/setup/environment.md @@ -1,6 +1,6 @@ # Development Environment Setup -This guide walks you through setting up a complete development environment for **openframe-oss-lib**. +This guide walks you through setting up a complete development environment for working on `openframe-oss-lib`. --- @@ -8,202 +8,220 @@ This guide walks you through setting up a complete development environment for * ### IntelliJ IDEA (Recommended) -IntelliJ IDEA Community or Ultimate is the recommended IDE for this project. It provides the best support for: +IntelliJ IDEA provides the best developer experience for this Spring Boot multi-module project. -- Maven multi-module projects -- Spring Boot auto-configuration detection -- Lombok annotation processing -- Java 21 features (records, virtual threads, sealed classes) +**Setup steps:** -**Download:** [https://www.jetbrains.com/idea/](https://www.jetbrains.com/idea/) +1. **Install IntelliJ IDEA** β€” Community Edition is sufficient; Ultimate adds Spring Boot tooling. +2. **Open the project** β€” Select `File β†’ Open` and choose the root `pom.xml`. +3. **Enable annotation processing** β€” Required for Lombok: + - Go to `Settings β†’ Build, Execution, Deployment β†’ Compiler β†’ Annotation Processors` + - Check **Enable annotation processing** +4. **Set Project SDK to Java 21**: + - `File β†’ Project Structure β†’ Project SDK β†’ Java 21` +5. **Install the Lombok plugin** (usually pre-installed in recent versions): + - `Settings β†’ Plugins β†’ Search "Lombok" β†’ Install` -### VS Code (Alternative) +**Recommended IntelliJ plugins:** -VS Code with the **Extension Pack for Java** is a lighter-weight alternative. - -Required extensions: - -- Extension Pack for Java (Microsoft) -- Spring Boot Extension Pack (VMware) -- Lombok Annotations Support +| Plugin | Purpose | +|--------|---------| +| Lombok | Reduces boilerplate code annotation support | +| Spring Boot | Run configurations, Spring-aware navigation | +| MongoDB | Database introspection and query editing | +| GraphQL | Schema-aware editing for `.graphqls` files | +| Docker | Docker Compose integration | +| SonarLint | Real-time code quality analysis | --- -## IntelliJ IDEA Setup - -### Step 1 β€” Import the Project - -1. Open IntelliJ IDEA -2. Select **File β†’ Open** -3. Navigate to the cloned `openframe-oss-lib` directory -4. Select the root `pom.xml` β†’ click **Open as Project** -5. Wait for Maven to import all 30+ modules +### VS Code -### Step 2 β€” Configure Project SDK +VS Code is a viable alternative with the right extensions. -1. Open **File β†’ Project Structure β†’ Project** -2. Set **SDK** to Java 21 -3. Set **Language Level** to `21` +**Required extensions:** -### Step 3 β€” Enable Annotation Processors (Lombok) +| Extension | Publisher | Purpose | +|-----------|-----------|---------| +| Extension Pack for Java | Microsoft | Full Java language support | +| Spring Boot Extension Pack | VMware/Pivotal | Spring Boot tooling | +| Lombok Annotations Support | GabrielBB | Lombok support | +| Docker | Microsoft | Docker Compose integration | +| GraphQL: Language Feature Support | GraphQL Foundation | GraphQL schema editing | -1. Open **Settings β†’ Build, Execution, Deployment β†’ Compiler β†’ Annotation Processors** -2. Check **Enable annotation processing** -3. Select **Obtain processors from project classpath** +**Setup steps:** -Without this step, Lombok-generated code will show errors. +```bash +# Install Java extensions from command line +code --install-extension vscjava.vscode-java-pack +code --install-extension vmware.vscode-spring-boot +code --install-extension GabrielBB.vscode-lombok +``` -### Step 4 β€” Maven Delegate (Recommended) +--- -1. Open **Settings β†’ Build, Execution, Deployment β†’ Build Tools β†’ Maven β†’ Runner** -2. Check **Delegate IDE build/run actions to Maven** +## Required Development Tools -This ensures builds use Maven directly rather than IntelliJ's internal compiler, avoiding configuration drift. +### Java 21 -### Step 5 β€” Increase Memory (Optional but Recommended) +```bash +# Using SDKMAN (recommended for version management) +sdk install java 21.0.3-tem -Edit `Help β†’ Change Memory Settings` and increase to at least: +# Or using Homebrew on macOS +brew install --cask temurin@21 -```text -Xmx: 4096 MB +# Verify +java -version ``` ---- - -## VS Code Setup - -Install the Extension Pack for Java: +### Apache Maven 3.9+ ```bash -code --install-extension vscjava.vscode-java-pack -code --install-extension vmware.vscode-spring-boot -``` +# Using SDKMAN +sdk install maven 3.9.9 -Add to `.vscode/settings.json` in your workspace: - -```json -{ - "java.configuration.runtimes": [ - { - "name": "JavaSE-21", - "path": "/path/to/jdk-21" - } - ], - "java.compile.nullAnalysis.mode": "disabled", - "maven.executable.path": "/path/to/mvn" -} +# Or using Homebrew on macOS +brew install maven + +# Verify +mvn -version ``` ---- +### Docker & Docker Compose -## Required Development Tools +```bash +# Install Docker Desktop (macOS/Windows) +# https://www.docker.com/products/docker-desktop/ -| Tool | Installation | -|------|-------------| -| Java 21 JDK | [Adoptium Temurin 21](https://adoptium.net/) | -| Maven 3.9+ | [https://maven.apache.org/download.cgi](https://maven.apache.org/download.cgi) | -| Docker Desktop | [https://www.docker.com/products/docker-desktop/](https://www.docker.com/products/docker-desktop/) | -| Git | System package manager or [https://git-scm.com/](https://git-scm.com/) | +# Verify +docker --version +docker compose version +``` --- ## Environment Variables for Development -Set these in your shell profile (`~/.bashrc`, `~/.zshrc`, or equivalent): +Create a `.env` file or configure your shell profile with development defaults: ```bash -# Java 21 home (adjust path for your OS and distribution) -export JAVA_HOME=/path/to/jdk-21 -export PATH="$JAVA_HOME/bin:$PATH" +# Core tenant configuration (OSS single-tenant mode) +export TENANT_ID=oss -# Increase Maven heap for large multi-module builds -export MAVEN_OPTS="-Xmx4g -XX:MaxMetaspaceSize=512m" +# MongoDB connection +export SPRING_DATA_MONGODB_URI=mongodb://localhost:27017/openframe -# GitHub Packages credentials (required for dependency resolution) -export GITHUB_ACTOR="your-github-username" -export GITHUB_TOKEN="your-github-pat" -``` +# Redis +export SPRING_REDIS_HOST=localhost +export SPRING_REDIS_PORT=6379 -> Note: `$GITHUB_TOKEN` must have `read:packages` permission to resolve OSS library dependencies. +# NATS +export NATS_SERVER=nats://localhost:4222 ---- +# Kafka +export SPRING_KAFKA_BOOTSTRAP_SERVERS=localhost:9092 -## Git Configuration - -Configure your Git identity: - -```bash -git config --global user.name "Your Name" -git config --global user.email "your-email@example.com" +# JWT config (for local dev, keys are generated from a dev keystore) +# In production, these are managed by your secrets provider +export jwt__issuer=http://localhost:8080/oss ``` -Configure line endings (important for cross-platform teams): +> **Tip:** Use [direnv](https://direnv.net/) to automatically load `.env` files per project directory. -```bash -# macOS / Linux -git config --global core.autocrlf input +--- -# Windows -git config --global core.autocrlf true +## Maven Settings + +To resolve dependencies from GitHub Packages, configure `~/.m2/settings.xml`: + +```xml + + + + + github + YOUR_GITHUB_USERNAME + YOUR_GITHUB_PAT_WITH_READ_PACKAGES + + + ``` --- -## Useful Maven Commands +## Code Style Configuration -| Command | Purpose | -|---------|---------| -| `mvn install -DskipTests` | Build all modules, skip tests | -| `mvn test -pl openframe-core` | Run unit tests for a specific module | -| `mvn verify -pl openframe-data-mongo-sync` | Run integration tests for a module | -| `mvn clean install -DskipTests` | Clean build all modules | -| `mvn dependency:tree -pl openframe-api-service-core` | View dependency tree | -| `mvn versions:display-dependency-updates` | Check for dependency updates | -| `mvn flatten:flatten` | Apply CI-friendly version flattening | +The project follows standard Java conventions aligned with Spring Boot coding style. Configure your IDE with: ---- +### IntelliJ Code Style -## Checkstyle and Code Quality (Optional) +1. Import the Google Java Format or use the default IntelliJ style +2. Enable **"Reformat code on save"**: `Settings β†’ Tools β†’ Actions on Save β†’ Reformat code` +3. Enable **"Organize imports on save"** -The project uses standard Spring Boot conventions. For consistent code style: +### EditorConfig -- Java files follow standard Java conventions -- Lombok reduces boilerplate (avoid raw getters/setters when Lombok `@Data`, `@Value`, etc. apply) -- Import ordering follows IntelliJ defaults +If an `.editorconfig` file is present in the repository root, your IDE will automatically use it for consistent formatting. --- -## Docker Configuration +## Git Configuration -Docker is required for integration tests via Testcontainers. Ensure: +Ensure your Git author information is set correctly before committing: ```bash -# Docker daemon is running -docker info +git config --global user.name "Your Name" +git config --global user.email "your.email@example.com" -# Pull commonly used images in advance for faster test runs -docker pull mongo:7 -docker pull nats:2 +# Optional: set default branch name to main +git config --global init.defaultBranch main ``` -Testcontainers will automatically manage container lifecycle during tests. +### Recommended Git Aliases + +```bash +git config --global alias.co checkout +git config --global alias.br branch +git config --global alias.st status +git config --global alias.lg "log --oneline --graph --decorate --all" +``` --- -## Verification +## Verifying Your Setup -After completing setup, verify everything works: +Run this checklist to confirm everything is ready: ```bash -# Full build with tests skipped -mvn install -DskipTests - -# Run unit tests for core module -mvn test -pl openframe-core - -# Check Java version is 21 +# 1. Java 21 java -version -# Check Maven version is 3.9+ +# 2. Maven mvn -version + +# 3. Docker +docker info + +# 4. Clone and compile +git clone https://github.com/flamingo-stack/openframe-oss-lib.git +cd openframe-oss-lib +mvn compile -DskipTests + +# 5. Run unit tests for core module +mvn test -pl openframe-core ``` + +All commands should complete without errors. + +--- + +## Troubleshooting Common Issues + +| Issue | Solution | +|-------|---------| +| `Could not resolve dependencies` | Verify GitHub Packages credentials in `~/.m2/settings.xml` | +| `Lombok annotations not processed` | Enable annotation processing in IDE settings | +| `Java version mismatch` | Set project SDK to Java 21 in IDE project settings | +| `Docker not running` | Start Docker Desktop and run `docker info` to verify | +| `Port already in use` | Stop conflicting services or change ports in `application.yml` | diff --git a/docs/development/setup/local-development.md b/docs/development/setup/local-development.md index 6fa2f1815..729dbc87c 100644 --- a/docs/development/setup/local-development.md +++ b/docs/development/setup/local-development.md @@ -1,73 +1,110 @@ # Local Development Guide -This guide covers everything you need to work with **openframe-oss-lib** locally: cloning, building, iterating, and debugging. +This guide covers cloning the repository, running it locally, working with hot reload, and configuring debug sessions. --- ## Clone and Initial Setup ```bash -# 1. Clone the repository +# Clone the repository git clone https://github.com/flamingo-stack/openframe-oss-lib.git cd openframe-oss-lib -# 2. Verify Java 21 is active +# Verify Java 21 is active java -version -# 3. Configure GitHub Packages (if not already done) -# See prerequisites.md for settings.xml configuration - -# 4. Build all modules (skip tests for speed) +# Build all modules (skip tests for faster setup) mvn install -DskipTests ``` --- -## Understanding the Multi-Module Build +## Project Structure for Local Development -This is a Maven multi-module project. Key concepts: +This is a **library project**, not a standalone runnable application. Each module compiles to a JAR that is consumed by downstream OpenFrame services. -- The **root `pom.xml`** is the **parent POM** β€” it defines shared dependencies, plugin versions, and the module list -- Each module has its own `pom.xml` that inherits from the parent -- **Unified versioning** β€” all modules share the same version via `${revision}` (currently `5.79.3`) -- The `flatten-maven-plugin` resolves `${revision}` at build time +The recommended local development workflow is: -### Building a Single Module +1. **Make changes** in the `openframe-oss-lib` module you're working on +2. **Build and install** the module to your local Maven repository +3. **Run your downstream service** that depends on it (with the local snapshot version) ```bash -# Build only openframe-core and its dependencies -mvn install -pl openframe-core -am -DskipTests +# Install a single module and its dependencies to local repo +mvn install -pl openframe-api-service-core -am -DskipTests + +# The JAR is now at: +# ~/.m2/repository/com/openframe/oss/openframe-api-service-core/6.0.10/ +``` + +--- + +## Starting Local Infrastructure -# Build only the security modules -mvn install -pl openframe-security-core,openframe-security-oauth -am -DskipTests +Some modules have integration tests that require running infrastructure. Start the minimum required services using Docker: + +### MongoDB (Required for Most Modules) + +```bash +# Start MongoDB for integration tests +cd openframe-data-mongo-sync/src/test/docker +docker compose up -d +cd - ``` -The `-am` flag (`--also-make`) ensures upstream dependencies are built first. +### Full Local Stack (Manual Docker) + +For a more complete local environment, run the required services individually: + +```bash +# MongoDB +docker run -d --name openframe-mongo \ + -p 27017:27017 \ + mongo:7 + +# Redis +docker run -d --name openframe-redis \ + -p 6379:6379 \ + redis:7-alpine + +# NATS with JetStream +docker run -d --name openframe-nats \ + -p 4222:4222 \ + nats:2-alpine -js + +# Kafka (using Confluent's image with KRaft mode) +docker run -d --name openframe-kafka \ + -p 9092:9092 \ + -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 \ + confluentinc/cp-kafka:latest +``` --- -## Running Tests +## Running Module Tests Locally ### Unit Tests -Unit tests follow the naming conventions `*Test.java` and `*Tests.java`: +Unit tests use JUnit 5 and Mockito. No infrastructure is required: ```bash +# Run all unit tests +mvn test + # Run unit tests for a specific module -mvn test -pl openframe-core +mvn test -pl openframe-api-service-core -# Run all unit tests across the project -mvn test +# Run a specific test class +mvn test -pl openframe-api-service-core \ + -Dtest=CommandDispatchServiceTest ``` ### Integration Tests -Integration tests are named `*IT.java` and require Docker (Testcontainers): +Integration tests use **Testcontainers** (version 1.21.4) to spin up Docker containers automatically: ```bash -# Ensure Docker is running first -docker info - # Run integration tests for MongoDB sync module mvn verify -pl openframe-data-mongo-sync @@ -75,144 +112,115 @@ mvn verify -pl openframe-data-mongo-sync mvn verify -pl openframe-data-nats ``` -> Integration tests spin up real service containers (MongoDB, NATS) via Testcontainers. They run during the `verify` phase. - -### Running a Specific Test - -```bash -# Run a specific test class -mvn test -pl openframe-data-mongo-sync -Dtest=NotificationReadStateServiceIT - -# Run a specific test method -mvn test -pl openframe-data-mongo-sync -Dtest="NotificationReadStateServiceIT#shouldMarkNotificationAsRead" -``` +Testcontainers will pull the required Docker images on first run. --- -## Development Workflow +## Working with Maven Multi-Module -### Typical Feature Development Flow +### Building Only Changed Modules -```mermaid -graph TD - A["Create feature branch"] --> B["Implement changes"] - B --> C["Run unit tests: mvn test -pl "] - C --> D["Run integration tests: mvn verify -pl "] - D --> E["Build full project: mvn install -DskipTests"] - E --> F["Open PR to main branch"] -``` +Use Maven's `-pl` (project list) and `-am` (also make dependencies) flags: -### Watch Mode / Hot Reload +```bash +# Build the gateway module and everything it depends on +mvn install -pl openframe-gateway-service-core -am -DskipTests -openframe-oss-lib is a **library**, not a standalone application. There is no hot-reload in the traditional sense. Instead, iterate by: +# Build multiple specific modules +mvn install -pl openframe-core,openframe-data-mongo-common -DskipTests +``` -1. Making code changes in the module -2. Running `mvn install -pl -DskipTests` to install to local `.m2` -3. Your downstream service (which depends on this library) picks up the new version +### Skipping Test Compilation -For faster iteration in the downstream service: +For the fastest possible build cycle: ```bash -# Install specific module to local repo quickly -mvn install -pl openframe-security-core -DskipTests -q +mvn install -DskipTests -Dmaven.test.skip=true ``` --- -## Debugging - -### IntelliJ Remote Debug (for downstream services) +## IntelliJ IDEA Local Run Configuration -When running a downstream Spring Boot service that uses these library modules, attach the IntelliJ debugger: +If you have a downstream Spring Boot service that depends on this library, configure IntelliJ to use local snapshots: -1. Run the service with: `mvn spring-boot:run -Dspring-boot.run.jvmArguments="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"` -2. In IntelliJ: **Run β†’ Edit Configurations β†’ Remote JVM Debug** -3. Set host to `localhost`, port to `5005` -4. Click **Debug** - -### IntelliJ Test Debugging - -Right-click any test class or method β†’ **Debug 'TestName'**. IntelliJ uses Maven's test infrastructure with Lombok annotation processing enabled. +1. **Open the downstream project** in IntelliJ +2. In `pom.xml`, update the version to match your local build (e.g., `6.0.10`) +3. Run `mvn install -DskipTests` in `openframe-oss-lib` to refresh your local `.m2` +4. **Reload Maven** in the downstream project (`Right-click pom.xml β†’ Maven β†’ Reload project`) +5. Run the downstream service with your IDE's Spring Boot run configuration --- -## Local Docker-Compose for Integration Tests - -A `docker-compose.yml` exists for the MongoDB sync integration test environment: +## Useful Maven Commands Reference -```bash -# Start MongoDB for manual integration testing -cd openframe-data-mongo-sync/src/test/docker -docker-compose up -d - -# Run integration tests against the running container -mvn verify -pl openframe-data-mongo-sync -``` - -> For most use cases, Testcontainers handles container lifecycle automatically. The docker-compose file is useful for persistent debugging sessions. +| Command | Description | +|---------|-------------| +| `mvn compile` | Compile all sources | +| `mvn test` | Run unit tests | +| `mvn verify` | Run unit + integration tests | +| `mvn install -DskipTests` | Build and install to local repo (no tests) | +| `mvn install -pl MODULE -am` | Build a module with its dependencies | +| `mvn dependency:tree` | Show full dependency tree | +| `mvn dependency:tree -pl MODULE` | Show dependency tree for one module | +| `mvn versions:display-dependency-updates` | Check for dependency version updates | +| `mvn clean` | Remove all build artifacts | --- -## Dependency Management +## Hot Reload / Watch Mode -### Adding a New Dependency +Since this is a library (not a runnable application), there is no traditional hot reload. However, you can achieve a fast feedback loop: -1. Add the version property to the root `pom.xml` `` section (if it's a new dependency) -2. Add the `` entry to `` in the root POM -3. Reference the dependency in the module's `pom.xml` **without a version** +### Option 1: IDE Auto-Build -Example β€” adding a new library: +In IntelliJ IDEA: +- Enable `Settings β†’ Build β†’ Build project automatically` +- Use `Ctrl+Shift+F9` (or `Cmd+Shift+F9`) to recompile specific files -```xml - -1.2.3 +### Option 2: Maven Daemon - - - com.example - my-library - ${my.library.version} - +Use the Maven Daemon for faster builds: - - - com.example - my-library - +```bash +# Install mvnd (Maven Daemon) +brew install mvnd # macOS + +# Use mvnd instead of mvn for faster incremental builds +mvnd install -pl openframe-api-service-core -am -DskipTests ``` --- -## Common Issues +## Debug Configuration -| Issue | Solution | -|-------|---------| -| `Cannot resolve symbol` (Lombok) | Enable annotation processing in IDE settings | -| Tests fail with `Connection refused` | Start Docker before running integration tests | -| `${revision}` not resolved | Run `mvn flatten:flatten` or upgrade Maven to 3.9+ | -| Slow builds | Use `-DskipTests`, `-T4` (parallel builds), or `-pl module -am` | -| `dependency:resolve` fails | Check `~/.m2/settings.xml` for GitHub Packages credentials | +### Debugging a Specific Test ---- +In IntelliJ IDEA: +1. Open the test class +2. Set breakpoints +3. Right-click the test method β†’ `Debug 'testMethodName'` -## Useful Development Aliases +### Remote Debugging a Downstream Service -Add to your shell profile for convenience: +If you're testing changes in a downstream service, add debug JVM args: ```bash -# Build specific module quickly -alias mbi='mvn install -DskipTests' -alias mbt='mvn test' -alias mbv='mvn verify' - -# Build and install a specific module -function mbm() { - mvn install -pl "$1" -am -DskipTests -} +# Start your downstream Spring Boot service with remote debug +java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 \ + -jar your-service.jar ``` -Usage: +Then in IntelliJ: `Run β†’ Edit Configurations β†’ Add β†’ Remote JVM Debug β†’ Port 5005` -```bash -mbm openframe-security-core -``` +--- + +## Common Errors and Fixes + +| Error | Cause | Fix | +|-------|-------|-----| +| `Could not find artifact com.openframe.oss:*` | Module not installed locally | Run `mvn install -DskipTests` in `openframe-oss-lib` | +| `Connection refused: localhost:27017` | MongoDB not running | Start MongoDB Docker container | +| `java.lang.UnsupportedClassVersionError` | Wrong Java version | Switch to Java 21 | +| `BeanCreationException: Unsatisfied dependency` | Missing Spring bean | Check if the required module is on the classpath | +| `Testcontainers failed to start` | Docker not running | Start Docker Desktop | diff --git a/docs/development/testing/README.md b/docs/development/testing/README.md index 6bd092b81..f12a5e5bd 100644 --- a/docs/development/testing/README.md +++ b/docs/development/testing/README.md @@ -1,85 +1,126 @@ # Testing Overview -This guide describes the testing strategy, structure, and conventions used across **openframe-oss-lib**. +`openframe-oss-lib` uses a comprehensive testing strategy with unit tests, integration tests, and end-to-end test utilities. This document explains the test structure, how to run tests, and how to write new tests. --- -## Test Structure and Organization +## Test Strategy + +```mermaid +graph TD + subgraph UnitTests["Unit Tests (Fast, No Infrastructure)"] + JUnit["JUnit 5"] + Mockito["Mockito"] + AssertJ["AssertJ"] + end + + subgraph IntegrationTests["Integration Tests (With Infrastructure)"] + Testcontainers["Testcontainers (Docker)"] + SpringTest["Spring Boot Test"] + MongoDB["Real MongoDB"] + NATS["Real NATS"] + end + + subgraph E2ETests["End-to-End Test Framework"] + TestCore["openframe-test-service-core"] + RestAssured["REST Assured"] + Playwright["Playwright (UI Tests)"] + end +``` + +--- -Tests are co-located with their source code in the standard Maven layout: +## Test Structure by Module + +Most modules follow this convention: ```text -openframe-/ -β”œβ”€β”€ src/ -β”‚ β”œβ”€β”€ main/java/com/openframe/... # Production code -β”‚ └── test/java/com/openframe/... # Test code -β”‚ β”œβ”€β”€ com/openframe/.../FooTest.java # Unit test -β”‚ └── com/openframe/.../FooIT.java # Integration test +src/ +β”œβ”€β”€ main/ +β”‚ └── java/ +β”‚ └── com/openframe/... +└── test/ + └── java/ + └── com/openframe/ + β”œβ”€β”€ unit/ # Unit tests (fast, mocked) + β”œβ”€β”€ integration/ # Integration tests (require Docker) + └── support/ # Test utilities and fixtures ``` ### Naming Conventions -| Convention | Type | Lifecycle Phase | -|-----------|------|----------------| -| `*Test.java` | Unit test | `mvn test` (Surefire) | -| `*Tests.java` | Unit test | `mvn test` (Surefire) | -| `*TestCase.java` | Unit test | `mvn test` (Surefire) | -| `*IT.java` | Integration test | `mvn verify` (Failsafe) | - -The Maven Surefire plugin in the parent POM is configured to include all four patterns: - -```xml - - **/Test*.java - **/*Test.java - **/*Tests.java - **/*TestCase.java - **/*IT.java - -``` +| Test Type | File Suffix | Example | +|-----------|------------|---------| +| Unit Test | `*Test.java` | `CommandDispatchServiceTest` | +| Integration Test | `*IT.java` | `NotificationServiceIT` | +| UI Test | `*UITest.java` | `DeviceRemoteTest` | + +The Maven Surefire plugin is configured to pick up all patterns: + +- `**/Test*.java` +- `**/*Test.java` +- `**/*Tests.java` +- `**/*TestCase.java` +- `**/*IT.java` --- ## Running Tests -### All Unit Tests +### Run All Unit Tests ```bash -# Run all unit tests across the entire project mvn test +``` + +Unit tests run without Docker or external services. They should complete in seconds. + +### Run Unit Tests for a Specific Module -# Run unit tests for a specific module -mvn test -pl openframe-core -mvn test -pl openframe-data-nats -mvn test -pl openframe-data-pinot +```bash +# API service core unit tests +mvn test -pl openframe-api-service-core + +# Data-mongo-sync unit tests +mvn test -pl openframe-data-mongo-sync ``` -### Integration Tests +### Run Integration Tests -Integration tests require a running Docker daemon (Testcontainers): +Integration tests use **Testcontainers** to automatically pull and start Docker containers: ```bash +# Run all tests including integration tests +mvn verify + # Run integration tests for a specific module mvn verify -pl openframe-data-mongo-sync -# Run integration + unit tests for a module -mvn verify -pl openframe-api-service-core +# Run integration tests for NATS module +mvn verify -pl openframe-data-nats +``` + +> **Prerequisite:** Docker must be running. Testcontainers pulls images automatically on first run. -# Run all integration tests (may be slow β€” requires Docker) -mvn verify +### Run a Specific Test Class + +```bash +mvn test -pl openframe-api-service-core \ + -Dtest=CommandDispatchServiceTest + +# Run a specific test method +mvn test -pl openframe-api-service-core \ + -Dtest=CommandDispatchServiceTest#shouldDispatchCommand ``` -### Skipping Tests +### Skip Tests ```bash -# Skip all tests during build +# Skip test execution (but still compile tests) mvn install -DskipTests -# Skip integration tests only -mvn install -DskipITs - -# Run only unit tests (skip integration) -mvn test -DskipITs +# Skip test compilation entirely +mvn install -Dmaven.test.skip=true ``` --- @@ -88,208 +129,220 @@ mvn test -DskipITs ### Testcontainers -Integration tests use [Testcontainers](https://testcontainers.com/) for infrastructure dependencies. Containers are automatically started and stopped per test class or suite. - -**MongoDB Integration Tests:** +Integration tests use `@Testcontainers` with module-specific base classes: ```java -// Base class pattern used in openframe-data-mongo-sync -@SpringBootTest(classes = IntegrationTestApplication.class) -@ActiveProfiles("integration") +// Example: MongoDB integration test base class pattern +// (from openframe-data-mongo-sync) +@SpringBootTest +@Testcontainers public abstract class BaseMongoIntegrationTest { - // Testcontainers manages MongoDB lifecycle + @Container + static MongoDBContainer mongoDBContainer = new MongoDBContainer("mongo:7"); + + @DynamicPropertySource + static void mongoProperties(DynamicPropertyRegistry registry) { + registry.add("spring.data.mongodb.uri", + mongoDBContainer::getReplicaSetUrl); + } } ``` -**NATS Integration Tests:** +### NATS Integration Tests + +NATS integration tests use a similar pattern: ```java -// Base class pattern in openframe-data-nats -@SpringBootTest(classes = PublisherIntegrationTestApplication.class) +// Example: NATS integration test base class pattern +// (from openframe-data-nats) +@SpringBootTest +@Testcontainers public abstract class BaseIntegrationTest { - // Testcontainers manages NATS lifecycle + @Container + static GenericContainer natsContainer = + new GenericContainer<>("nats:2-alpine") + .withCommand("-js"); } ``` -### Docker Compose (Alternative) - -For manual testing sessions, a Docker Compose file is available for MongoDB: - -```bash -cd openframe-data-mongo-sync/src/test/docker -docker-compose up -d -``` - --- -## Test Utilities +## Writing Unit Tests -### `openframe-test-service-core` +### Test Structure (Arrange-Act-Assert) -This dedicated module provides reusable test infrastructure for **end-to-end and integration testing** of OpenFrame services: +```java +@ExtendWith(MockitoExtension.class) +class CommandDispatchServiceTest { -| Class | Purpose | -|-------|---------| -| `AuthHelper` | Authentication flow helpers | -| `RequestSpecHelper` | REST-Assured request specification builders | -| `AuthGenerator` | Generate test authentication tokens | -| `OrganizationGenerator` | Generate test organization data | -| `DeviceGenerator` | Generate test device data | -| `TicketGenerator` | Generate test ticket data | -| `NotificationFixtures` | Pre-built notification test data | + @Mock + private CommandNatsPublisher natsPublisher; -Example usage: + @Mock + private MachineRepository machineRepository; -```java -@Autowired -private OrganizationGenerator organizationGenerator; + @InjectMocks + private CommandDispatchService commandDispatchService; -@Test -void shouldCreateOrganization() { - var org = organizationGenerator.createOrganization("Test Org"); - assertThat(org.getId()).isNotNull(); + @Test + void shouldDispatchCommandToAgent() { + // Arrange + var machine = aMachine().withMachineId("machine-123").build(); + var input = new RunCommandInput("machine-123", "echo hello"); + when(machineRepository.findByMachineId("machine-123")) + .thenReturn(Optional.of(machine)); + + // Act + var result = commandDispatchService.dispatch(input); + + // Assert + assertThat(result.isSuccess()).isTrue(); + verify(natsPublisher).publish(any(CommandMessage.class)); + } } ``` -### GraphQL Test Helpers +### Key Testing Libraries -GraphQL integration tests use pre-built query helpers: - -```java -// Available in openframe-test-service-core -DeviceQueries.listDevices(filterInput) -OrganizationQueries.listOrganizations(filterInput) -TicketQueries.listTickets(filterInput) -``` +| Library | Use Case | +|---------|---------| +| **JUnit 5** (`junit-jupiter`) | Test framework, lifecycle, parameterized tests | +| **Mockito** | Mock external dependencies | +| **AssertJ** | Fluent, readable assertions | +| **Spring Boot Test** | Full application context loading for integration tests | +| **Testcontainers** | Real infrastructure in Docker for integration tests | +| **REST Assured** | HTTP API testing with fluent DSL | --- -## Writing New Tests +## Writing Integration Tests -### Unit Test Template +### MongoDB Integration Test Pattern ```java -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.extension.ExtendWith; -import org.mockito.InjectMocks; -import org.mockito.Mock; -import org.mockito.junit.jupiter.MockitoExtension; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.when; - -@ExtendWith(MockitoExtension.class) -class MyServiceTest { +@SpringBootTest(classes = IntegrationTestApplication.class) +@Testcontainers +class NotificationServiceIT extends BaseMongoIntegrationTest { - @Mock - private MyRepository repository; + @Autowired + private NotificationService notificationService; - @InjectMocks - private MyService service; + @Autowired + private NotificationRepository notificationRepository; @Test - void shouldReturnExpectedResult() { - // Given - when(repository.findById("id-1")).thenReturn(Optional.of(new MyEntity("id-1"))); + void shouldCreateAndFetchNotification() { + // Arrange + var notification = NotificationFixtures.aNotification(); - // When - var result = service.getById("id-1"); + // Act + notificationService.create(notification); + var found = notificationRepository.findById(notification.getId()); - // Then - assertThat(result).isPresent(); - assertThat(result.get().getId()).isEqualTo("id-1"); + // Assert + assertThat(found).isPresent(); + assertThat(found.get().getTitle()).isEqualTo(notification.getTitle()); } } ``` -### Integration Test Template (MongoDB) +### Test Fixtures + +The `support` packages in each module provide fixture builders. For example, the `NotificationFixtures` class in `openframe-data-nats`: ```java -import org.junit.jupiter.api.AfterEach; -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.annotation.Autowired; +// Fixture usage pattern +var notification = NotificationFixtures.aNotification() + .withTitle("Test Notification") + .withSeverity(NotificationSeverity.INFO) + .build(); +``` -class MyRepositoryIT extends BaseMongoIntegrationTest { +--- - @Autowired - private MyRepository repository; +## End-to-End Test Framework - @AfterEach - void cleanup() { - repository.deleteAll(); - } +The `openframe-test-service-core` module provides a complete E2E test framework. + +### Available Test APIs + +| Class | Purpose | +|-------|---------| +| `AuthApi` | Authentication flows (login, registration, token refresh) | +| `DeviceApi` | Device management (list, filter, update status) | +| `OrganizationApi` | Organization CRUD | +| `UserApi` | User management and invitations | +| `TicketApi` | Ticket lifecycle (create, update, notes, attachments) | +| `KnowledgeBaseApi` | Knowledge base articles and folders | +| `ScriptApi` | Script management (create, update, run) | +| `LogsApi` | Audit log queries | + +### E2E Test Example + +```java +class DevicesTest extends BaseTest { @Test - void shouldPersistAndRetrieveEntity() { - // Given - var entity = new MyEntity("test-id", "test-name"); - repository.save(entity); + void shouldReturnDevicesForTenant() { + // Authenticate + AuthParts auth = authFlow.login(); - // When - var found = repository.findById("test-id"); + // Call the devices API + var response = deviceApi.listDevices(auth); - // Then - assertThat(found).isPresent(); - assertThat(found.get().getName()).isEqualTo("test-name"); + // Assert + assertThat(response.statusCode()).isEqualTo(200); + assertThat(response.body().jsonPath().getList("data")).isNotEmpty(); } } ``` -### Naming and Structure Conventions - -- Use **Given / When / Then** structure in test methods -- Test method names should describe behavior: `shouldReturnNotFoundWhenEntityDoesNotExist` -- One assertion concept per test where possible -- Always clean up test data in `@AfterEach` for integration tests -- Use `@DisplayName` for complex test scenarios +### UI Test Framework ---- +For UI tests, the framework uses **Playwright** via `openframe-test-service-core`: -## Test Coverage +```java +class DeviceRemoteTest extends BaseUITest { -The project does not enforce a specific line coverage percentage, but the following guidelines apply: + @Test + void shouldOpenRemoteShellForDevice() { + UILoginFlow login = new UILoginFlow(page); + login.login(); -| Component Type | Expected Coverage | -|---------------|------------------| -| Core utilities (`openframe-core`, `openframe-exception`) | High (>80%) | -| Domain services | Medium-High (>70%) | -| Configuration classes | Low (Spring-managed beans) | -| Repository implementations | Covered by integration tests | + DevicesPage devices = new DevicesPage(page); + devices.openFirstDevice(); -Focus on **behavioral coverage** (testing what the code does) over line coverage metrics. + RemoteShellPage shell = new RemoteShellPage(page); + assertThat(shell.isVisible()).isTrue(); + } +} +``` --- -## Notification Integration Tests +## Coverage Requirements -The notification subsystem has particularly thorough integration test coverage in `openframe-data-mongo-sync`: +While there is no enforced coverage threshold in this library, follow these guidelines: -| Test Class | What It Tests | -|-----------|--------------| -| `NotificationContextDispatchIT` | Notification dispatch with custom context | -| `NotificationReadStateIndexesIT` | MongoDB index usage for read state | -| `NotificationLoadTestIT` | Load and performance testing | -| `NotificationReadStateIndexUsageIT` | Query plan analysis | -| `CustomNotificationRepositoryPaginationIT` | Cursor-based notification pagination | +| Test Type | Recommended Coverage | +|-----------|---------------------| +| Unit tests | β‰₯ 80% for service and utility classes | +| Integration tests | All repository methods with real infrastructure | +| E2E tests | All critical user flows (auth, device, ticket, org) | -Run them with: - -```bash -mvn verify -pl openframe-data-mongo-sync -Dtest=Notification*IT -``` +Focus coverage on: +- Business logic in `*Service` classes +- Repository query methods +- Mapper/converter classes +- Security-sensitive code (authentication, authorization) --- -## Pinot Repository Tests - -The `openframe-data-pinot` module contains unit tests for query building: +## CI/CD Test Execution -```bash -mvn test -pl openframe-data-pinot -``` +Tests are run in CI in two phases: -Key test classes: +1. **Unit tests** β€” `mvn test` (no Docker required, fast) +2. **Integration tests** β€” `mvn verify` (requires Docker, slower) -- `PinotQueryBuilderTest` β€” Validates SQL query generation -- `PinotClientDeviceRepositoryTest` β€” Device query logic -- `PinotClientLogRepositoryTest` β€” Log query logic +Separate PR checks may exist for each phase. Check the CI pipeline configuration in your deployment environment for exact commands. diff --git a/docs/diagrams/architecture/README-2.mmd b/docs/diagrams/architecture/README-2.mmd index f0028d3df..0aab9de9e 100644 --- a/docs/diagrams/architecture/README-2.mmd +++ b/docs/diagrams/architecture/README-2.mmd @@ -1,7 +1,5 @@ -flowchart TD - Browser --> AuthController["Auth Controllers"] - AuthController --> TenantContext["TenantContextFilter"] - TenantContext --> AuthServer["AuthorizationServerConfig"] - AuthServer --> TenantKeyService["TenantKeyService"] - TenantKeyService --> Mongo - AuthServer --> JWT["Signed JWT"] +flowchart LR + User["User"] --> Auth["Authorization Server"] + Auth --> IdP["Google / Microsoft"] + Auth --> JWT["Tenant Scoped JWT"] + JWT --> Gateway diff --git a/docs/diagrams/architecture/README-3.mmd b/docs/diagrams/architecture/README-3.mmd index 8873c03c9..edd4a8be5 100644 --- a/docs/diagrams/architecture/README-3.mmd +++ b/docs/diagrams/architecture/README-3.mmd @@ -1,7 +1,6 @@ flowchart TD - Request --> OriginSanitizer - OriginSanitizer --> JwtAuth - JwtAuth --> RoleCheck - RoleCheck --> RouteDecision - RouteDecision --> ToolResolver - ToolResolver --> UpstreamTool + Kafka --> Deserializer + Deserializer --> Enrichment + Enrichment --> UnifiedEvent + UnifiedEvent --> Cassandra + UnifiedEvent --> Pinot diff --git a/docs/diagrams/architecture/README-4.mmd b/docs/diagrams/architecture/README-4.mmd index 36a3b850e..5cd40666d 100644 --- a/docs/diagrams/architecture/README-4.mmd +++ b/docs/diagrams/architecture/README-4.mmd @@ -1,6 +1,17 @@ -flowchart TD - DebeziumEvent --> Deserializer - Deserializer --> Enrichment - Enrichment --> EventTypeMapper - EventTypeMapper --> Cassandra - EventTypeMapper --> KafkaOut +sequenceDiagram + participant Browser + participant Gateway + participant Auth + participant API + participant Mongo + participant Kafka + participant Pinot + + Browser->>Gateway: GraphQL Query + Gateway->>Auth: Validate JWT + Gateway->>API: Forward request + API->>Mongo: Fetch domain data + API-->>Browser: Response + + API->>Kafka: Publish event + Kafka->>Pinot: Analytics ingestion diff --git a/docs/diagrams/architecture/README-5.mmd b/docs/diagrams/architecture/README-5.mmd deleted file mode 100644 index 259b45977..000000000 --- a/docs/diagrams/architecture/README-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - StreamService --> PinotCluster - PinotCluster --> PinotRepositories - PinotRepositories --> ApiService diff --git a/docs/diagrams/architecture/README-6.mmd b/docs/diagrams/architecture/README-6.mmd deleted file mode 100644 index ef6842c36..000000000 --- a/docs/diagrams/architecture/README-6.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Startup --> Initializers - Initializers --> StreamsReady - StreamsReady --> Schedulers - Schedulers --> ExternalSystems diff --git a/docs/diagrams/architecture/README-7.mmd b/docs/diagrams/architecture/README-7.mmd deleted file mode 100644 index 2f8a49559..000000000 --- a/docs/diagrams/architecture/README-7.mmd +++ /dev/null @@ -1,10 +0,0 @@ -sequenceDiagram - participant Browser - participant BFF - participant AuthServer - - Browser->>BFF: GET /oauth/login - BFF->>AuthServer: Redirect with PKCE - AuthServer->>BFF: Callback with code - BFF->>AuthServer: Exchange code - BFF->>Browser: Set cookies diff --git a/docs/diagrams/architecture/README-8.mmd b/docs/diagrams/architecture/README-8.mmd deleted file mode 100644 index 05e53be6c..000000000 --- a/docs/diagrams/architecture/README-8.mmd +++ /dev/null @@ -1,19 +0,0 @@ -flowchart TD - Client --> Gateway - Gateway --> ExternalAPI - Gateway --> ApiCore - - ApiCore --> Authz - ApiCore --> Mongo - ApiCore --> Redis - ApiCore --> Pinot - - Authz --> Mongo - Authz --> JWT - - Stream --> Kafka - Stream --> Cassandra - Stream --> Pinot - - Management --> Redis - Management --> NATS diff --git a/docs/diagrams/architecture/README.mmd b/docs/diagrams/architecture/README.mmd index 7f3f41a75..9a39ff99e 100644 --- a/docs/diagrams/architecture/README.mmd +++ b/docs/diagrams/architecture/README.mmd @@ -1,15 +1,73 @@ flowchart TD - Client["Client / Browser / Agent"] --> Gateway["Gateway Service Core"] - Gateway --> ExternalAPI["External API Service"] - Gateway --> ApiCore["API Service Core"] - - ApiCore --> Authz["Authorization Service Core"] - ApiCore --> Stream["Stream Service Core"] - ApiCore --> Management["Management Service Core"] - - Authz --> Mongo["MongoDB"] - ApiCore --> Mongo - Stream --> Kafka["Kafka"] - Stream --> Pinot["Pinot"] - ApiCore --> Redis["Redis"] - Management --> NATS["NATS"] + + subgraph Edge + Client["Browser / Agent"] + end + + subgraph GatewayLayer["Gateway Service Core"] + Gateway["Reactive API Gateway"] + end + + subgraph SecurityLayer["Authorization & Security"] + AuthServer["Authorization Service Core"] + SecurityCore["Security OAuth & JWT"] + end + + subgraph ApiLayer["API Layer"] + Rest["REST Controllers"] + GraphQL["GraphQL Layer"] + DataLoaders["GraphQL DataLoaders"] + ApiDtos["API DTOs"] + end + + subgraph DomainLayer["Business & Mapping"] + Services["Business Services"] + Mapping["Mapping & Domain Services"] + end + + subgraph PersistenceLayer["Persistence"] + MongoDocs["Mongo Documents"] + MongoSync["Mongo Sync Repositories"] + end + + subgraph MessagingLayer["Messaging"] + Kafka["Kafka"] + Nats["NATS"] + end + + subgraph StreamLayer["Stream Processing"] + StreamCore["Stream Processing Core"] + end + + subgraph AnalyticsLayer["Analytics"] + Pinot["Analytics Pinot"] + end + + subgraph AgentIngress["Client Core Agent Ingress"] + AgentAPI["Agent Registration & Auth"] + AgentListeners["NATS & JetStream Listeners"] + end + + Client --> Gateway + Gateway --> Rest + Gateway --> GraphQL + Gateway --> AuthServer + + AuthServer --> SecurityCore + + Rest --> Services + GraphQL --> DataLoaders + DataLoaders --> Services + + Services --> Mapping + Mapping --> MongoSync + MongoSync --> MongoDocs + + Services --> Kafka + Services --> Nats + + Kafka --> StreamCore + StreamCore --> Pinot + + Nats --> AgentListeners + AgentListeners --> Services diff --git a/docs/diagrams/architecture/analytics-pinot-2.mmd b/docs/diagrams/architecture/analytics-pinot-2.mmd new file mode 100644 index 000000000..b871c0441 --- /dev/null +++ b/docs/diagrams/architecture/analytics-pinot-2.mmd @@ -0,0 +1,8 @@ +flowchart TD + Start["Start Device Query"] --> ExcludeDeleted["Exclude DELETED Status"] + ExcludeDeleted --> ApplyStatus["Apply Status Filters"] + ApplyStatus --> ApplyType["Apply Device Type Filters"] + ApplyType --> ApplyOs["Apply OS Type Filters"] + ApplyOs --> ApplyOrg["Apply Organization Filters"] + ApplyOrg --> ApplyTags["Apply Tag Filters"] + ApplyTags --> Execute["Execute Pinot Query"] diff --git a/docs/diagrams/architecture/analytics-pinot-3.mmd b/docs/diagrams/architecture/analytics-pinot-3.mmd new file mode 100644 index 000000000..6e7049304 --- /dev/null +++ b/docs/diagrams/architecture/analytics-pinot-3.mmd @@ -0,0 +1,9 @@ +flowchart TD + Input["Log Query Request"] --> Build["Build PinotQueryBuilder"] + Build --> DateRange["Apply Date Range"] + DateRange --> Filters["Apply Tool / Event / Severity / Org Filters"] + Filters --> Search["Apply Relevance Search (optional)"] + Search --> Cursor["Apply Cursor"] + Cursor --> Sort["Validate & Apply Sort"] + Sort --> Limit["Apply Limit"] + Limit --> Execute["Execute Pinot Query"] diff --git a/docs/diagrams/architecture/analytics-pinot-4.mmd b/docs/diagrams/architecture/analytics-pinot-4.mmd new file mode 100644 index 000000000..85816f5c1 --- /dev/null +++ b/docs/diagrams/architecture/analytics-pinot-4.mmd @@ -0,0 +1,4 @@ +flowchart LR + Request["Tenant Scoped Request"] --> Builder["PinotQueryBuilder(tenantId)"] + Builder --> Query["SELECT ... WHERE tenantId = X"] + Query --> Pinot[("Pinot Cluster")] diff --git a/docs/diagrams/architecture/analytics-pinot-5.mmd b/docs/diagrams/architecture/analytics-pinot-5.mmd new file mode 100644 index 000000000..dd745836c --- /dev/null +++ b/docs/diagrams/architecture/analytics-pinot-5.mmd @@ -0,0 +1,6 @@ +flowchart LR + Producers["Devices / Tools / Stream Processors"] --> Kafka["Kafka / Streaming"] + Kafka --> PinotIngest["Pinot Real-Time Tables"] + PinotIngest --> Analytics["Analytics Pinot Module"] + Analytics --> ApiLayer["API / GraphQL"] + ApiLayer --> UI["Dashboard / Client"] diff --git a/docs/diagrams/architecture/analytics-pinot.mmd b/docs/diagrams/architecture/analytics-pinot.mmd new file mode 100644 index 000000000..14b0560c6 --- /dev/null +++ b/docs/diagrams/architecture/analytics-pinot.mmd @@ -0,0 +1,13 @@ +flowchart LR + Client["API Layer / GraphQL / External API"] --> DeviceRepo["PinotClientDeviceRepository"] + Client --> LogRepo["PinotClientLogRepository"] + + DeviceRepo --> QueryBuilder["PinotQueryBuilder"] + LogRepo --> QueryBuilder + + QueryBuilder --> BrokerConn["Pinot Broker Connection"] + BrokerConn --> PinotCluster[("Apache Pinot Cluster")] + + ControllerConn["Pinot Controller Connection"] --> PinotCluster + + Warmup["PinotBrokerWarmup"] --> BrokerConn diff --git a/docs/diagrams/architecture/api-contracts-and-pagination-2.mmd b/docs/diagrams/architecture/api-contracts-and-pagination-2.mmd deleted file mode 100644 index 171348e9c..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination-2.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - Request["Query Request with Filters"] --> Service["Query Service"] - Service --> Repo["Custom Repository"] - Repo --> Result["CountedGenericQueryResult"] - Result --> API["Serialized API Response"] diff --git a/docs/diagrams/architecture/api-contracts-and-pagination-3.mmd b/docs/diagrams/architecture/api-contracts-and-pagination-3.mmd deleted file mode 100644 index 76d425bb5..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Client["Client"] --> Args["ConnectionArgs"] - Args --> Decode["CursorCodec.decode()"] - Decode --> Query["Repository Query with Limit"] - Query --> Encode["CursorCodec.encode()"] - Encode --> Response["Connection Response with Edges"] diff --git a/docs/diagrams/architecture/api-contracts-and-pagination-4.mmd b/docs/diagrams/architecture/api-contracts-and-pagination-4.mmd deleted file mode 100644 index 0bba98fe8..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination-4.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart LR - Raw["Raw Internal Cursor"] --> Encode["Base64 Encode"] - Encode --> Opaque["Opaque Cursor String"] diff --git a/docs/diagrams/architecture/api-contracts-and-pagination-5.mmd b/docs/diagrams/architecture/api-contracts-and-pagination-5.mmd deleted file mode 100644 index 293cd83af..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination-5.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart LR - Opaque["Opaque Cursor"] --> Decode["Base64 Decode"] - Decode --> Raw["Raw Internal Cursor"] diff --git a/docs/diagrams/architecture/api-contracts-and-pagination-6.mmd b/docs/diagrams/architecture/api-contracts-and-pagination-6.mmd deleted file mode 100644 index 91f5ede92..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination-6.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Client["Client Mutation"] --> Input["MutationDeleteInput"] - Input --> Validate["Validation Layer"] - Validate --> Service["Domain Service"] - Service --> Repo["Repository Delete by ID"] - Repo --> Response["Mutation Result"] diff --git a/docs/diagrams/architecture/api-contracts-and-pagination-7.mmd b/docs/diagrams/architecture/api-contracts-and-pagination-7.mmd deleted file mode 100644 index 62c5f782c..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination-7.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Step1["Client Requests first 20 after Cursor"] --> Step2["ConnectionArgs Validated"] - Step2 --> Step3["CursorCodec.decode()"] - Step3 --> Step4["Repository Query with Limit 21"] - Step4 --> Step5["Determine hasNextPage"] - Step5 --> Step6["CursorCodec.encode() for Edges"] - Step6 --> Step7["Return Connection Response"] diff --git a/docs/diagrams/architecture/api-contracts-and-pagination.mmd b/docs/diagrams/architecture/api-contracts-and-pagination.mmd deleted file mode 100644 index 2e19bb2b8..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Client["Client Application"] --> API["REST Controllers / GraphQL DataFetchers"] - API --> Contracts["Api Contracts And Pagination"] - Contracts --> Services["Domain Query Services"] - Services --> Repositories["Mongo Repositories"] - Repositories --> Database[("MongoDB")] diff --git a/docs/diagrams/architecture/api-domain-filters-dtos-2.mmd b/docs/diagrams/architecture/api-domain-filters-dtos-2.mmd deleted file mode 100644 index 238250c63..000000000 --- a/docs/diagrams/architecture/api-domain-filters-dtos-2.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - LogFilterCriteria["LogFilterCriteria"] --> DateRange["Date Range"] - LogFilterCriteria --> EventTypes["Event Types"] - LogFilterCriteria --> ToolTypes["Tool Types"] - LogFilterCriteria --> Severities["Severities"] - LogFilterCriteria --> Organizations["Organization IDs"] - LogFilterCriteria --> Device["Device ID"] diff --git a/docs/diagrams/architecture/api-domain-filters-dtos-3.mmd b/docs/diagrams/architecture/api-domain-filters-dtos-3.mmd deleted file mode 100644 index feae0ce16..000000000 --- a/docs/diagrams/architecture/api-domain-filters-dtos-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - DeviceFilterCriteria["DeviceFilterCriteria"] --> Statuses["Device Statuses"] - DeviceFilterCriteria --> Types["Device Types"] - DeviceFilterCriteria --> OsTypes["OS Types"] - DeviceFilterCriteria --> Orgs["Organization IDs"] - DeviceFilterCriteria --> Tags["Tag Keys / Values"] diff --git a/docs/diagrams/architecture/api-domain-filters-dtos-4.mmd b/docs/diagrams/architecture/api-domain-filters-dtos-4.mmd deleted file mode 100644 index 278d589a8..000000000 --- a/docs/diagrams/architecture/api-domain-filters-dtos-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Organization["Organization (Mongo Document)"] --> OrganizationResponse["OrganizationResponse DTO"] - OrganizationResponse --> RestAPI["REST API"] - OrganizationResponse --> GraphQL["GraphQL API"] diff --git a/docs/diagrams/architecture/api-domain-filters-dtos-5.mmd b/docs/diagrams/architecture/api-domain-filters-dtos-5.mmd deleted file mode 100644 index 779b2cc46..000000000 --- a/docs/diagrams/architecture/api-domain-filters-dtos-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - GraphQLInput["GraphQL Filter Input"] --> Criteria["FilterCriteria DTO"] - Criteria --> Service["Query Service"] - Service --> MongoFilter["Mongo Query Filter"] diff --git a/docs/diagrams/architecture/api-domain-filters-dtos-6.mmd b/docs/diagrams/architecture/api-domain-filters-dtos-6.mmd deleted file mode 100644 index ced28036e..000000000 --- a/docs/diagrams/architecture/api-domain-filters-dtos-6.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - DeviceFilterCriteria --> DeviceQueryFilter["DeviceQueryFilter"] - EventFilterCriteria --> EventQueryFilter["EventQueryFilter"] - ToolFilterCriteria --> ToolQueryFilter["ToolQueryFilter"] diff --git a/docs/diagrams/architecture/api-domain-filters-dtos.mmd b/docs/diagrams/architecture/api-domain-filters-dtos.mmd deleted file mode 100644 index 5a45aaf4d..000000000 --- a/docs/diagrams/architecture/api-domain-filters-dtos.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - Client["Client (REST or GraphQL)"] --> ApiInput["GraphQL Input / REST Request"] - ApiInput --> DomainFilter["Api Domain Filters Dtos"] - DomainFilter --> QueryLayer["Mongo Query Filters"] - QueryLayer --> Repository["Mongo Repositories"] - Repository --> Database[("MongoDB")] diff --git a/docs/diagrams/architecture/api-lib-core-services-4.mmd b/docs/diagrams/architecture/api-lib-core-services-4.mmd deleted file mode 100644 index 45ae4bcf6..000000000 --- a/docs/diagrams/architecture/api-lib-core-services-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - API["API Layer"] --> TicketService["TicketQueryService"] - TicketService --> QueryFilter["TicketQueryFilter"] - TicketService --> Repo["TicketRepository"] - Repo --> Mongo[("MongoDB")] diff --git a/docs/diagrams/architecture/api-lib-core-services-5.mmd b/docs/diagrams/architecture/api-lib-core-services-5.mmd deleted file mode 100644 index d07ad4a9b..000000000 --- a/docs/diagrams/architecture/api-lib-core-services-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Save["Save KnowledgeBaseItem"] --> MongoEvent["BeforeConvertEvent"] - MongoEvent --> Listener["KnowledgeBasePublishLifecycleListener"] - Listener --> Stamp["Set publishedAt if null"] - Stamp --> Persist[("MongoDB")] diff --git a/docs/diagrams/architecture/api-lib-core-services-6.mmd b/docs/diagrams/architecture/api-lib-core-services-6.mmd deleted file mode 100644 index 7181e803f..000000000 --- a/docs/diagrams/architecture/api-lib-core-services-6.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart LR - DeviceStatusUpdate["Machine Status Updated"] --> Processor["DeviceStatusProcessor"] - Processor --> DefaultImpl["DefaultDeviceStatusProcessor"] diff --git a/docs/diagrams/architecture/api-lib-core-services-7.mmd b/docs/diagrams/architecture/api-lib-core-services-7.mmd deleted file mode 100644 index 91e5a1f40..000000000 --- a/docs/diagrams/architecture/api-lib-core-services-7.mmd +++ /dev/null @@ -1,13 +0,0 @@ -flowchart TD - Client["GraphQL Client"] --> DataFetcher["MachineDataFetcher"] - DataFetcher --> AgentLoader["InstalledAgentDataLoader"] - DataFetcher --> ToolLoader["ToolConnectionDataLoader"] - - AgentLoader --> AgentService["InstalledAgentService"] - ToolLoader --> ToolService["ToolConnectionService"] - - AgentService --> AgentRepo["InstalledAgentRepository"] - ToolService --> ToolRepo["ToolConnectionRepository"] - - AgentRepo --> Mongo[("MongoDB")] - ToolRepo --> Mongo diff --git a/docs/diagrams/architecture/api-lib-core-services.mmd b/docs/diagrams/architecture/api-lib-core-services.mmd deleted file mode 100644 index 86fb571e2..000000000 --- a/docs/diagrams/architecture/api-lib-core-services.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Controllers["REST Controllers"] --> Services["Api Lib Core Services"] - DataFetchers["GraphQL Data Fetchers"] --> Services - Services --> Repositories["Mongo Repositories"] - Repositories --> MongoDB[("MongoDB")] - - Services --> Processors["Domain Processors"] diff --git a/docs/diagrams/architecture/api-lib-dto-contracts-2.mmd b/docs/diagrams/architecture/api-lib-dto-contracts-2.mmd new file mode 100644 index 000000000..218c4ea81 --- /dev/null +++ b/docs/diagrams/architecture/api-lib-dto-contracts-2.mmd @@ -0,0 +1,10 @@ +flowchart TD + Root["Api Lib Dto Contracts"] --> Audit["Audit & Logs"] + Root --> Device["Device"] + Root --> Event["Event"] + Root --> Knowledge["Knowledge Base"] + Root --> Organization["Organization"] + Root --> Script["Script"] + Root --> Tool["Tool"] + Root --> Command["Command Dispatch"] + Root --> Shared["Shared & Pagination"] diff --git a/docs/diagrams/architecture/api-lib-dto-contracts-3.mmd b/docs/diagrams/architecture/api-lib-dto-contracts-3.mmd new file mode 100644 index 000000000..cd6f72419 --- /dev/null +++ b/docs/diagrams/architecture/api-lib-dto-contracts-3.mmd @@ -0,0 +1,4 @@ +flowchart LR + Criteria["DeviceFilterCriteria"] --> Query["Repository Query"] + Query --> Result["DeviceFilters"] + Result --> Options["DeviceFilterOption"] diff --git a/docs/diagrams/architecture/api-lib-dto-contracts-4.mmd b/docs/diagrams/architecture/api-lib-dto-contracts-4.mmd new file mode 100644 index 000000000..4c5fe3116 --- /dev/null +++ b/docs/diagrams/architecture/api-lib-dto-contracts-4.mmd @@ -0,0 +1,4 @@ +flowchart TD + CreateInput["CreateScriptInput"] --> Service["Script Service"] + Service --> Stored["Script Document"] + Stored --> Response["ScriptResponse"] diff --git a/docs/diagrams/architecture/api-lib-dto-contracts-5.mmd b/docs/diagrams/architecture/api-lib-dto-contracts-5.mmd new file mode 100644 index 000000000..516d5ab2b --- /dev/null +++ b/docs/diagrams/architecture/api-lib-dto-contracts-5.mmd @@ -0,0 +1,13 @@ +sequenceDiagram + participant Client + participant Api + participant Agent + + Client->>Api: RunCommandInput + Api->>Agent: Dispatch Command + Agent-->>Api: ExecutionId + Api-->>Client: CommandDispatchResponse + + Client->>Api: CancelExecutionInput + Api->>Agent: Cancel Command + Api-->>Client: CancelDispatchResponse diff --git a/docs/diagrams/architecture/api-lib-dto-contracts-6.mmd b/docs/diagrams/architecture/api-lib-dto-contracts-6.mmd new file mode 100644 index 000000000..c1294b7a0 --- /dev/null +++ b/docs/diagrams/architecture/api-lib-dto-contracts-6.mmd @@ -0,0 +1,5 @@ +flowchart LR + Raw["Raw Cursor"] --> Encode["Base64 Encode"] + Encode --> Opaque["Opaque Cursor"] + Opaque --> Decode["Base64 Decode"] + Decode --> Raw diff --git a/docs/diagrams/architecture/api-lib-dto-contracts.mmd b/docs/diagrams/architecture/api-lib-dto-contracts.mmd new file mode 100644 index 000000000..f9223cb4a --- /dev/null +++ b/docs/diagrams/architecture/api-lib-dto-contracts.mmd @@ -0,0 +1,10 @@ +flowchart LR + Client["Client Applications"] --> GraphQL["GraphQL Layer"] + Client --> Rest["REST Controllers"] + + GraphQL --> Dto["Api Lib Dto Contracts"] + Rest --> Dto + + Dto --> Services["Business Services"] + Services --> Domain["Domain Models"] + Domain --> Mongo["Mongo Repositories"] diff --git a/docs/diagrams/architecture/api-lib-mapping-and-domain-services-2.mmd b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-2.mmd new file mode 100644 index 000000000..1fdc09217 --- /dev/null +++ b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-2.mmd @@ -0,0 +1,5 @@ +flowchart LR + CreateReq["CreateOrganizationRequest"] --> Mapper["OrganizationMapper"] + Mapper --> Entity["Organization Document"] + Entity --> Mapper + Mapper --> Response["OrganizationResponse"] diff --git a/docs/diagrams/architecture/api-lib-core-services-2.mmd b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-3.mmd similarity index 57% rename from docs/diagrams/architecture/api-lib-core-services-2.mmd rename to docs/diagrams/architecture/api-lib-mapping-and-domain-services-3.mmd index 8f71addba..9c00d11b4 100644 --- a/docs/diagrams/architecture/api-lib-core-services-2.mmd +++ b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-3.mmd @@ -1,4 +1,6 @@ -flowchart LR +flowchart TD DataLoader["InstalledAgentDataLoader"] --> Service["InstalledAgentService"] Service --> Repo["InstalledAgentRepository"] Repo --> Mongo[("MongoDB")] + Service --> Grouping["Group by machineId"] + Grouping --> OrderedResult["List>"] diff --git a/docs/diagrams/architecture/api-lib-core-services-3.mmd b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-4.mmd similarity index 93% rename from docs/diagrams/architecture/api-lib-core-services-3.mmd rename to docs/diagrams/architecture/api-lib-mapping-and-domain-services-4.mmd index 5a2e40ac8..18d51ebee 100644 --- a/docs/diagrams/architecture/api-lib-core-services-3.mmd +++ b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-4.mmd @@ -1,4 +1,4 @@ -flowchart LR +flowchart TD ToolDataLoader["ToolConnectionDataLoader"] --> ToolService["ToolConnectionService"] ToolService --> ToolRepo["ToolConnectionRepository"] ToolRepo --> Mongo[("MongoDB")] diff --git a/docs/diagrams/architecture/api-lib-mapping-and-domain-services-5.mmd b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-5.mmd new file mode 100644 index 000000000..f8d762278 --- /dev/null +++ b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-5.mmd @@ -0,0 +1,5 @@ +flowchart TD + Controller["REST or GraphQL"] --> TicketService["TicketQueryService"] + TicketService --> BuildQuery["buildTicketQuery()"] + BuildQuery --> CursorQuery["findTicketsWithCursor()"] + CursorQuery --> Mongo[("MongoDB")] diff --git a/docs/diagrams/architecture/api-lib-mapping-and-domain-services-6.mmd b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-6.mmd new file mode 100644 index 000000000..451ce4bf8 --- /dev/null +++ b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-6.mmd @@ -0,0 +1,5 @@ +flowchart TD + Save["Save KnowledgeBaseItem"] --> BeforeConvert["BeforeConvertEvent"] + BeforeConvert --> Check["Status == PUBLISHED?"] + Check -->|"Yes and publishedAt null"| Stamp["Set publishedAt"] + Check -->|"Otherwise"| NoChange["No change"] diff --git a/docs/diagrams/architecture/api-lib-mapping-and-domain-services-7.mmd b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-7.mmd new file mode 100644 index 000000000..384801749 --- /dev/null +++ b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-7.mmd @@ -0,0 +1,3 @@ +flowchart LR + Machine["Machine"] --> Processor["DeviceStatusProcessor"] + Processor --> Log["Debug log"] diff --git a/docs/diagrams/architecture/api-lib-mapping-and-domain-services-8.mmd b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-8.mmd new file mode 100644 index 000000000..80c329ca7 --- /dev/null +++ b/docs/diagrams/architecture/api-lib-mapping-and-domain-services-8.mmd @@ -0,0 +1,6 @@ +flowchart TD + Client["Client"] --> Gateway["Gateway Service Core"] + Gateway --> ApiCore["Api Service Core"] + ApiCore --> ApiLib["Api Lib Mapping And Domain Services"] + ApiLib --> MongoLayer["Data Model And Repositories Mongo"] + MongoLayer --> Mongo[("MongoDB")] diff --git a/docs/diagrams/architecture/api-lib-mapping-and-domain-services.mmd b/docs/diagrams/architecture/api-lib-mapping-and-domain-services.mmd new file mode 100644 index 000000000..55f4bdee1 --- /dev/null +++ b/docs/diagrams/architecture/api-lib-mapping-and-domain-services.mmd @@ -0,0 +1,10 @@ +flowchart TD + RestControllers["REST Controllers"] --> ApiServices["Api Lib Mapping And Domain Services"] + GraphQL["GraphQL Data Fetchers"] --> ApiServices + ApiServices --> Repositories["Mongo Repositories"] + ApiServices --> DomainEntities["Domain Documents"] + ApiServices --> DtoContracts["DTO Contracts"] + + subgraph data_layer["Data Layer"] + Repositories --> MongoDB[("MongoDB")] + end diff --git a/docs/diagrams/architecture/api-organization-mapping-2.mmd b/docs/diagrams/architecture/api-organization-mapping-2.mmd deleted file mode 100644 index 7964c3000..000000000 --- a/docs/diagrams/architecture/api-organization-mapping-2.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - Request["CreateOrganizationRequest"] --> MapperCreate["toEntity()"] - MapperCreate --> GenId["Generate UUID"] - MapperCreate --> MapFields["Map Scalar Fields"] - MapperCreate --> MapContact["Map Contact Information"] - MapContact --> MapAddress["Map Address"] - MapContact --> MapContacts["Map Contact Persons"] - GenId --> OrgEntity["Organization Entity"] - MapFields --> OrgEntity - MapContact --> OrgEntity diff --git a/docs/diagrams/architecture/api-organization-mapping-3.mmd b/docs/diagrams/architecture/api-organization-mapping-3.mmd deleted file mode 100644 index 4b4b0d04a..000000000 --- a/docs/diagrams/architecture/api-organization-mapping-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - UpdateRequest["UpdateOrganizationRequest"] --> CheckField["Field != null?"] - CheckField -->|"Yes"| ApplyUpdate["Set Field on Entity"] - CheckField -->|"No"| Skip["Keep Existing Value"] - ApplyUpdate --> OrgEntity["Updated Organization"] - Skip --> OrgEntity diff --git a/docs/diagrams/architecture/api-organization-mapping-4.mmd b/docs/diagrams/architecture/api-organization-mapping-4.mmd deleted file mode 100644 index 24ccc880a..000000000 --- a/docs/diagrams/architecture/api-organization-mapping-4.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart LR - OrgEntity["Organization Entity"] --> MapperResponse["toResponse()"] - MapperResponse --> MapScalars["Map Scalar Fields"] - MapperResponse --> MapStatus["Enum to String"] - MapperResponse --> MapNested["Map Contact Information DTO"] - MapScalars --> Response["OrganizationResponse"] - MapStatus --> Response - MapNested --> Response diff --git a/docs/diagrams/architecture/api-organization-mapping-5.mmd b/docs/diagrams/architecture/api-organization-mapping-5.mmd deleted file mode 100644 index 1a2d95abc..000000000 --- a/docs/diagrams/architecture/api-organization-mapping-5.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Dto["ContactInformationDto"] --> CheckFlag{{"Same As Physical?"}} - CheckFlag -->|"Yes"| CopyAddr["Copy Physical Address"] - CheckFlag -->|"No"| MapMail["Map Mailing Address"] - CopyAddr --> Entity["ContactInformation Entity"] - MapMail --> Entity diff --git a/docs/diagrams/architecture/api-organization-mapping-6.mmd b/docs/diagrams/architecture/api-organization-mapping-6.mmd deleted file mode 100644 index 46c956ee5..000000000 --- a/docs/diagrams/architecture/api-organization-mapping-6.mmd +++ /dev/null @@ -1,11 +0,0 @@ -flowchart LR - Client["Client App"] --> RestAPI["REST or GraphQL API"] - RestAPI --> MapperCreate["OrganizationMapper"] - MapperCreate --> Service["Organization Service"] - Service --> Repo["Mongo Repository"] - Repo --> DB["MongoDB"] - DB --> Repo - Repo --> Service - Service --> MapperResponse["OrganizationMapper"] - MapperResponse --> RestAPI - RestAPI --> Client diff --git a/docs/diagrams/architecture/api-organization-mapping.mmd b/docs/diagrams/architecture/api-organization-mapping.mmd deleted file mode 100644 index 38fda5d45..000000000 --- a/docs/diagrams/architecture/api-organization-mapping.mmd +++ /dev/null @@ -1,23 +0,0 @@ -flowchart LR - subgraph ApiLayer["API Layer"] - RestController["OrganizationController"] - GraphQLFetcher["OrganizationDataFetcher"] - end - - Mapper["OrganizationMapper"] - - subgraph DomainLayer["Domain & Persistence"] - OrgEntity["Organization Entity"] - MongoRepo["Mongo Repository"] - end - - RestController -->|"Create/Update Request"| Mapper - GraphQLFetcher -->|"Mutation Input"| Mapper - - Mapper -->|"Entity"| OrgEntity - OrgEntity --> MongoRepo - - MongoRepo --> OrgEntity - OrgEntity -->|"Response DTO"| Mapper - Mapper -->|"OrganizationResponse"| RestController - Mapper -->|"GraphQL DTO"| GraphQLFetcher diff --git a/docs/diagrams/architecture/api-service-core-business-services-2.mmd b/docs/diagrams/architecture/api-service-core-business-services-2.mmd new file mode 100644 index 000000000..d264ee14c --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-business-services-2.mmd @@ -0,0 +1,6 @@ +flowchart TD + Request["Controller Request"] --> GetUser["Load User from Repository"] + GetUser --> Validate["Apply Business Rules"] + Validate --> Persist["Save User"] + Persist --> PostProcess["UserProcessor Hook"] + PostProcess --> Response["Return UserResponse"] diff --git a/docs/diagrams/architecture/api-service-core-business-services-3.mmd b/docs/diagrams/architecture/api-service-core-business-services-3.mmd new file mode 100644 index 000000000..ae860271f --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-business-services-3.mmd @@ -0,0 +1,10 @@ +flowchart TD + Request["SSOConfigRequest"] --> Normalize["Normalize Domains"] + Normalize --> ValidatePublic["Validate Generic Public Domains"] + ValidatePublic --> ValidateExist["Validate Domain Exists"] + ValidateExist --> ValidateAuto["Validate Auto-Provision Rules"] + ValidateAuto --> MapEntity["Map to SSOConfig Entity"] + MapEntity --> Encrypt["Encrypt Client Secret"] + Encrypt --> Save["Save via Repository"] + Save --> PostProcess["SSOConfigProcessor Hook"] + PostProcess --> Response["SSOConfigResponse"] diff --git a/docs/diagrams/architecture/api-service-core-business-services-4.mmd b/docs/diagrams/architecture/api-service-core-business-services-4.mmd new file mode 100644 index 000000000..97ff70c37 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-business-services-4.mmd @@ -0,0 +1,4 @@ +flowchart LR + Service["Domain Service"] --> ProcessorInterface["Processor Interface"] + ProcessorInterface --> DefaultImpl["Default Implementation"] + ProcessorInterface --> CustomImpl["Custom Deployment Implementation"] diff --git a/docs/diagrams/architecture/api-service-core-business-services-5.mmd b/docs/diagrams/architecture/api-service-core-business-services-5.mmd new file mode 100644 index 000000000..ec0e05d26 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-business-services-5.mmd @@ -0,0 +1,3 @@ +flowchart TD + Domains["List of Domains"] --> Validator["DomainExistenceValidator"] + Validator --> Result["Return false (no blocking)"] diff --git a/docs/diagrams/architecture/api-service-core-business-services.mmd b/docs/diagrams/architecture/api-service-core-business-services.mmd new file mode 100644 index 000000000..38745be8b --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-business-services.mmd @@ -0,0 +1,9 @@ +flowchart TD + Controllers["REST & GraphQL Controllers"] --> Services["Api Service Core Business Services"] + Services --> Repositories["Mongo Repositories"] + Services --> Mappers["DTO Mappers"] + Services --> Processors["Domain Processors (Extension Points)"] + Services --> Encryption["Encryption Service"] + Services --> DomainValidation["Domain Validation Service"] + + Repositories --> Mongo["MongoDB"] diff --git a/docs/diagrams/architecture/api-service-core-config-and-security-2.mmd b/docs/diagrams/architecture/api-service-core-config-and-security-2.mmd index 1d4c2908b..58410114c 100644 --- a/docs/diagrams/architecture/api-service-core-config-and-security-2.mmd +++ b/docs/diagrams/architecture/api-service-core-config-and-security-2.mmd @@ -1,5 +1,5 @@ -flowchart TD - Request["HTTP Request"] --> Security["OAuth2 Resource Server"] - Security --> Controller["Controller Method"] - Controller --> Resolver["AuthPrincipalArgumentResolver"] - Resolver --> Principal["AuthPrincipal Injected"] +flowchart LR + JwtToken["JWT Token"] --> SecurityLayer["Security Filter Chain"] + SecurityLayer --> AuthPrincipal["AuthPrincipal"] + AuthPrincipal --> Resolver["AuthPrincipalArgumentResolver"] + Resolver --> Controller["REST Controller Method"] diff --git a/docs/diagrams/architecture/api-service-core-config-and-security-3.mmd b/docs/diagrams/architecture/api-service-core-config-and-security-3.mmd index 5c7317a24..7bc42b83e 100644 --- a/docs/diagrams/architecture/api-service-core-config-and-security-3.mmd +++ b/docs/diagrams/architecture/api-service-core-config-and-security-3.mmd @@ -1,9 +1,9 @@ flowchart TD - Incoming["Incoming Request with JWT"] --> ExtractIssuer["Extract iss Claim"] - ExtractIssuer --> CacheCheck["Check JwtAuthenticationProvider Cache"] - CacheCheck -->|"Miss"| CreateDecoder["Create JwtDecoder from Issuer"] - CreateDecoder --> StoreCache["Store in Caffeine Cache"] - CacheCheck -->|"Hit"| UseProvider["Reuse Cached Provider"] - StoreCache --> Authenticate["Authenticate JWT"] - UseProvider --> Authenticate - Authenticate --> Principal["SecurityContext Updated"] + Request["Incoming Request"] --> ExtractIssuer["Extract JWT Issuer"] + ExtractIssuer --> CacheCheck["Caffeine Cache Lookup"] + + CacheCheck -->|"Hit"| Provider["JwtAuthenticationProvider"] + CacheCheck -->|"Miss"| CreateDecoder["JwtDecoders.fromIssuerLocation()"] + + CreateDecoder --> Provider + Provider --> Authenticate["Authenticate JWT"] diff --git a/docs/diagrams/architecture/api-service-core-config-and-security-4.mmd b/docs/diagrams/architecture/api-service-core-config-and-security-4.mmd index da7c9a7a1..1c36c6ee6 100644 --- a/docs/diagrams/architecture/api-service-core-config-and-security-4.mmd +++ b/docs/diagrams/architecture/api-service-core-config-and-security-4.mmd @@ -1,7 +1,9 @@ flowchart TD - Startup["Application Startup"] --> LoadProps["Read OAuth Properties"] - LoadProps --> QueryRepo["Find Client by ID"] - QueryRepo -->|"Exists"| CompareSecret["Compare Secrets"] - CompareSecret -->|"Different"| UpdateSecret["Update and Save"] - CompareSecret -->|"Same"| Skip["No Action"] - QueryRepo -->|"Missing"| CreateClient["Create OAuthClient"] + Startup["Application Startup"] --> ReadEnv["Read OAuth Properties"] + ReadEnv --> Lookup["Find OAuth Client"] + + Lookup -->|"Exists"| CompareSecret["Compare Client Secret"] + CompareSecret -->|"Different"| UpdateSecret["Update Secret"] + CompareSecret -->|"Same"| Skip["No Change"] + + Lookup -->|"Missing"| CreateClient["Create OAuth Client"] diff --git a/docs/diagrams/architecture/api-service-core-config-and-security-5.mmd b/docs/diagrams/architecture/api-service-core-config-and-security-5.mmd index e5f850975..b5a944065 100644 --- a/docs/diagrams/architecture/api-service-core-config-and-security-5.mmd +++ b/docs/diagrams/architecture/api-service-core-config-and-security-5.mmd @@ -1,5 +1,8 @@ flowchart LR - GraphQLInput["GraphQL Query Input"] --> Scalar["Custom Scalar Coercing"] - Scalar --> Validation["Format Validation"] - Validation --> Parsed["Java Type (LocalDate / Instant / Long)"] - Parsed --> DataFetcher["Data Fetcher Execution"] + GraphQlSchema["GraphQL Schema"] --> DateScalar["Date Scalar"] + GraphQlSchema --> InstantScalar["Instant Scalar"] + GraphQlSchema --> LongScalar["Long Scalar"] + + DateScalar --> LocalDate["LocalDate"] + InstantScalar --> InstantType["Instant"] + LongScalar --> LongType["Long"] diff --git a/docs/diagrams/architecture/api-service-core-config-and-security-6.mmd b/docs/diagrams/architecture/api-service-core-config-and-security-6.mmd index 265582add..0abdb3226 100644 --- a/docs/diagrams/architecture/api-service-core-config-and-security-6.mmd +++ b/docs/diagrams/architecture/api-service-core-config-and-security-6.mmd @@ -1,14 +1,14 @@ sequenceDiagram participant Client participant Gateway - participant Api as "API Service" + participant Api participant Security as "SecurityConfig" participant Controller - Client->>Gateway: Request with Cookie or Token - Gateway->>Gateway: Validate JWT + Client->>Gateway: HTTP Request with JWT Gateway->>Api: Forward request with Authorization header - Api->>Security: Resolve issuer and authenticate - Security->>Api: Populate SecurityContext - Api->>Controller: Invoke handler - Controller->>Controller: Inject AuthPrincipal + Api->>Security: Resolve issuer and validate JWT + Security->>Controller: Inject AuthPrincipal + Controller->>Api: Return response + Api->>Gateway: HTTP response + Gateway->>Client: Final response diff --git a/docs/diagrams/architecture/api-service-core-config-and-security.mmd b/docs/diagrams/architecture/api-service-core-config-and-security.mmd index da769acb9..3852e20d4 100644 --- a/docs/diagrams/architecture/api-service-core-config-and-security.mmd +++ b/docs/diagrams/architecture/api-service-core-config-and-security.mmd @@ -1,7 +1,19 @@ -flowchart LR - Gateway["Gateway Service"] -->|"JWT Forwarded"| ApiService["API Service Runtime"] - ApiService -->|"Uses"| ConfigModule["Api Service Core Config And Security"] - ApiService --> Controllers["REST Controllers"] - ApiService --> GraphQL["GraphQL Data Fetchers"] - ApiService --> Services["Domain Services"] - Services --> DataLayer["Mongo / Kafka / Redis"] +flowchart TD + Client["Client Application"] --> Gateway["Gateway Service Core"] + Gateway --> ApiService["API Service"] + + subgraph ApiCore["Api Service Core"] + Config["Api Service Core Config And Security"] + RestLayer["REST Controllers"] + GraphQlLayer["GraphQL Layer"] + Business["Business Services"] + end + + ApiService --> Config + ApiService --> RestLayer + ApiService --> GraphQlLayer + RestLayer --> Business + GraphQlLayer --> Business + + Config --> AuthService["Authorization Service Core"] + Business --> DataLayer["Mongo + Repositories"] diff --git a/docs/diagrams/architecture/api-service-core-dataloaders-2.mmd b/docs/diagrams/architecture/api-service-core-dataloaders-2.mmd deleted file mode 100644 index 0d3879f3c..000000000 --- a/docs/diagrams/architecture/api-service-core-dataloaders-2.mmd +++ /dev/null @@ -1,9 +0,0 @@ -sequenceDiagram - participant Resolver as "GraphQL Resolver" - participant Loader as "DataLoader" - participant Service as "Service/Repository" - - Resolver->>Loader: load([id1, id2, id3]) - Loader->>Service: Batch fetch entities - Service-->>Loader: List of entities - Loader-->>Resolver: Ordered results diff --git a/docs/diagrams/architecture/api-service-core-dataloaders-3.mmd b/docs/diagrams/architecture/api-service-core-dataloaders-3.mmd deleted file mode 100644 index 8533426d6..000000000 --- a/docs/diagrams/architecture/api-service-core-dataloaders-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Resolver --> InstalledAgentDataLoader - InstalledAgentDataLoader --> InstalledAgentService - InstalledAgentService --> Mongo diff --git a/docs/diagrams/architecture/api-service-core-dataloaders-4.mmd b/docs/diagrams/architecture/api-service-core-dataloaders-4.mmd deleted file mode 100644 index f66c56913..000000000 --- a/docs/diagrams/architecture/api-service-core-dataloaders-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Resolver --> KnowledgeBaseItemDataLoader - KnowledgeBaseItemDataLoader --> KnowledgeBaseItemRepository - KnowledgeBaseItemRepository --> Mongo diff --git a/docs/diagrams/architecture/api-service-core-dataloaders-5.mmd b/docs/diagrams/architecture/api-service-core-dataloaders-5.mmd deleted file mode 100644 index 85557302f..000000000 --- a/docs/diagrams/architecture/api-service-core-dataloaders-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Resolver --> OrganizationDataLoader - OrganizationDataLoader --> OrganizationRepository - OrganizationRepository --> Mongo diff --git a/docs/diagrams/architecture/api-service-core-dataloaders-6.mmd b/docs/diagrams/architecture/api-service-core-dataloaders-6.mmd deleted file mode 100644 index a624e2aa6..000000000 --- a/docs/diagrams/architecture/api-service-core-dataloaders-6.mmd +++ /dev/null @@ -1,20 +0,0 @@ -flowchart TD - subgraph graphql_layer["GraphQL Layer"] - DF["Data Fetchers"] - DL["DataLoaders"] - end - - subgraph service_layer["Service Layer"] - SVC["Domain Services"] - end - - subgraph data_layer["Data Layer"] - REPO["Mongo Repositories"] - DB[("MongoDB")] - end - - DF --> DL - DL --> SVC - DL --> REPO - SVC --> DB - REPO --> DB diff --git a/docs/diagrams/architecture/api-service-core-dataloaders.mmd b/docs/diagrams/architecture/api-service-core-dataloaders.mmd deleted file mode 100644 index bc6df6f13..000000000 --- a/docs/diagrams/architecture/api-service-core-dataloaders.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Client["GraphQL Client"] --> GQL["GraphQL Data Fetchers"] - GQL --> DL["Api Service Core Dataloaders"] - DL --> Service["Domain Services"] - DL --> Repo["Mongo Repositories"] - Service --> Mongo[("MongoDB")] - Repo --> Mongo diff --git a/docs/diagrams/architecture/api-service-core-dtos-2.mmd b/docs/diagrams/architecture/api-service-core-dtos-2.mmd new file mode 100644 index 000000000..31872a386 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-dtos-2.mmd @@ -0,0 +1,5 @@ +flowchart TD + Connection["CountedGenericConnection"] --> Edge["GenericEdge"] + Edge --> Node["Domain Node"] + Connection --> PageInfo["PageInfo"] + Connection --> FilteredCount["filteredCount"] diff --git a/docs/diagrams/architecture/api-service-core-dtos-3.mmd b/docs/diagrams/architecture/api-service-core-dtos-3.mmd new file mode 100644 index 000000000..712b8be94 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-dtos-3.mmd @@ -0,0 +1,5 @@ +flowchart TD + Admin["Admin User"] --> Req["SSOConfigRequest"] + Req --> Service["SSOConfigService"] + Service --> Resp["SSOConfigResponse"] + Service --> Status["SSOConfigStatusResponse"] diff --git a/docs/diagrams/architecture/api-service-core-dtos-4.mmd b/docs/diagrams/architecture/api-service-core-dtos-4.mmd new file mode 100644 index 000000000..205c74dd8 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-dtos-4.mmd @@ -0,0 +1,4 @@ +flowchart TD + Admin["Admin"] --> CreateInvite["CreateInvitationRequest"] + CreateInvite --> Processor["Invitation Processor"] + Processor --> PageResp["InvitationPageResponse"] diff --git a/docs/diagrams/architecture/api-service-core-dtos-5.mmd b/docs/diagrams/architecture/api-service-core-dtos-5.mmd new file mode 100644 index 000000000..af8822505 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-dtos-5.mmd @@ -0,0 +1,4 @@ +flowchart LR + FilterInput["FilterInput DTO"] --> DataFetcher["GraphQL DataFetcher"] + DataFetcher --> QueryService["Query Service"] + QueryService --> Repository["Mongo Repository"] diff --git a/docs/diagrams/architecture/api-service-core-dtos-6.mmd b/docs/diagrams/architecture/api-service-core-dtos-6.mmd new file mode 100644 index 000000000..0e06975c5 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-dtos-6.mmd @@ -0,0 +1,5 @@ +flowchart TD + Admin["Admin"] --> Request["ForceToolAgentUpdateRequest"] + Request --> Controller["ForceAgentController"] + Controller --> Service["Update Processor"] + Service --> Messaging["Kafka or NATS"] diff --git a/docs/diagrams/architecture/api-service-core-dtos-7.mmd b/docs/diagrams/architecture/api-service-core-dtos-7.mmd new file mode 100644 index 000000000..cb6f77697 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-dtos-7.mmd @@ -0,0 +1,11 @@ +flowchart TD + Client["Frontend or Agent"] --> Controllers["REST Controllers"] + Client --> GraphQL["GraphQL Layer"] + + Controllers --> DTOs["Api Service Core Dtos"] + GraphQL --> DTOs + + DTOs --> Business["Business Services"] + Business --> Data["Mongo Data Layer"] + Business --> Auth["Authorization Service"] + Business --> Streams["Stream Processing"] diff --git a/docs/diagrams/architecture/api-service-core-dtos.mmd b/docs/diagrams/architecture/api-service-core-dtos.mmd new file mode 100644 index 000000000..3cba307b2 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-dtos.mmd @@ -0,0 +1,10 @@ +flowchart LR + Client["Client Applications"] --> Rest["REST Controllers"] + Client --> GraphQL["GraphQL Layer"] + + Rest --> DTOs["Api Service Core Dtos"] + GraphQL --> DTOs + + DTOs --> Services["Business Services"] + Services --> Repos["Mongo Repositories"] + Services --> Auth["Authorization Service Core"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-2.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-2.mmd deleted file mode 100644 index 75dcb15cb..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-2.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Query["GraphQL Query"] --> Args["ConnectionArgs"] - Args --> Pagination["CursorPaginationCriteria"] - Pagination --> ServiceQuery["Service.query(...)"] - ServiceQuery --> Result["QueryResult"] - Result --> Mapper["GraphQL Mapper"] - Mapper --> Connection["Relay Connection"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-3.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-3.mmd deleted file mode 100644 index 39727fc93..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-3.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart LR - Query["devices query"] --> DeviceFetcher - DeviceFetcher --> DeviceService - DeviceFetcher --> MachineNode["Machine"] - MachineNode -->|"tags"| TagLoader - MachineNode -->|"installedAgents"| AgentLoader - TagLoader --> TagService - AgentLoader --> InstalledAgentService diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-4.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-4.mmd deleted file mode 100644 index 679e3471f..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Mutation["createArticle"] --> GetUser["Extract JWT User"] - GetUser --> Mapper - Mapper --> Service["KnowledgeBaseService"] - Service --> DB["Mongo Repository"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-5.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-5.mmd deleted file mode 100644 index 7b60b20bd..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-5.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - NodeQuery["node(id)"] --> Decode["Relay.fromGlobalId"] - Decode --> TypeSwitch["NodeType enum"] - TypeSwitch --> DeviceService - TypeSwitch --> OrganizationService - TypeSwitch --> EventService - TypeSwitch --> ToolService - TypeSwitch --> TagService diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-6.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-6.mmd deleted file mode 100644 index cf844476a..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-6.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Query["notifications"] --> CurrentRecipient - CurrentRecipient --> Principal["AuthPrincipal"] - Principal --> NotificationService - NotificationService --> ReadStateService diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-7.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-7.mmd deleted file mode 100644 index 24a82acbb..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-7.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - NotificationContext --> Resolver - Resolver -->|"type discriminator"| TypeMap - TypeMap --> GraphQLType diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-8.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-8.mmd deleted file mode 100644 index cf75e6de7..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-8.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - GraphQL["GraphQL DataFetchers"] --> Services - Services --> Domain["Domain Models"] - Services --> Repositories - Services --> Messaging["NATS / Kafka"] - Services --> Cache["Redis"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers.mmd deleted file mode 100644 index fe7cec603..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers.mmd +++ /dev/null @@ -1,11 +0,0 @@ -flowchart LR - Client["GraphQL Client"] --> Schema["GraphQL Schema"] - Schema --> DataFetchers["DGS DataFetchers"] - DataFetchers --> Mappers["GraphQL Mappers"] - DataFetchers --> Services["Application Services"] - Services --> Repositories["Mongo / Pinot / Cassandra Repositories"] - - DataFetchers --> DataLoaders["DataLoaders (Batching)"] - DataLoaders --> Services - - DataFetchers --> Security["SecurityContext + JWT"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-dataloaders-2.mmd b/docs/diagrams/architecture/api-service-core-graphql-dataloaders-2.mmd new file mode 100644 index 000000000..6c5240895 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-graphql-dataloaders-2.mmd @@ -0,0 +1,4 @@ +flowchart LR + GraphQL["Machine.installedAgents"] --> Loader["InstalledAgentDataLoader"] + Loader --> Service["InstalledAgentService"] + Service --> Repo["InstalledAgent Repository"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-dataloaders-3.mmd b/docs/diagrams/architecture/api-service-core-graphql-dataloaders-3.mmd new file mode 100644 index 000000000..b9c631547 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-graphql-dataloaders-3.mmd @@ -0,0 +1,5 @@ +flowchart TD + Machines["Machine List"] --> OrgField["Machine.organization"] + OrgField --> OrgLoader["OrganizationDataLoader"] + OrgLoader --> Repo["OrganizationRepository.findByOrganizationIdIn"] + Repo --> Filter["Filter ACTIVE Only"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-dataloaders-4.mmd b/docs/diagrams/architecture/api-service-core-graphql-dataloaders-4.mmd new file mode 100644 index 000000000..e713b1773 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-graphql-dataloaders-4.mmd @@ -0,0 +1,5 @@ +flowchart LR + KBItem["KnowledgeBaseItem.author"] --> UserLoader["UserDataLoader"] + UserLoader --> UserService["UserService"] + UserService --> Processor["UserProcessor Enrichment"] + Processor --> Repository["User Repository"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-dataloaders-5.mmd b/docs/diagrams/architecture/api-service-core-graphql-dataloaders-5.mmd new file mode 100644 index 000000000..3f7ea3366 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-graphql-dataloaders-5.mmd @@ -0,0 +1,7 @@ +flowchart TD + A["Collect Keys"] --> B["Filter Null Values"] + B --> C["Deduplicate IDs"] + C --> D["Bulk Repository/Service Call"] + D --> E["Map Results by ID"] + E --> F["Rebuild Output List in Input Order"] + F --> G["Return CompletableFuture"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-dataloaders.mmd b/docs/diagrams/architecture/api-service-core-graphql-dataloaders.mmd new file mode 100644 index 000000000..25b75b4a6 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-graphql-dataloaders.mmd @@ -0,0 +1,7 @@ +flowchart TD + Client["GraphQL Client"] --> GraphQL["GraphQL DataFetchers"] + GraphQL --> DataLoaders["Api Service Core Graphql Dataloaders"] + DataLoaders --> Services["Business Services"] + DataLoaders --> Repositories["Mongo Repositories"] + Services --> Repositories + Repositories --> MongoDB[("MongoDB")] diff --git a/docs/diagrams/architecture/api-service-core-graphql-dtos-2.mmd b/docs/diagrams/architecture/api-service-core-graphql-dtos-2.mmd deleted file mode 100644 index 149360786..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-dtos-2.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart LR - Query["GraphQL Query"] --> Args["Connection Arguments"] - Args --> Fetcher["Data Fetcher"] - Fetcher --> Service["Service Layer"] - Service --> Edge["GenericEdge"] - Edge --> Connection["CountedGenericConnection"] - Connection --> Client["GraphQL Client"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-dtos-3.mmd b/docs/diagrams/architecture/api-service-core-graphql-dtos-3.mmd deleted file mode 100644 index 402106ef8..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-dtos-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - GraphQLInput["GraphQL Filter Input"] --> DataFetcher["Data Fetcher"] - DataFetcher --> Criteria["Domain Filter Criteria"] - Criteria --> Repository["Custom Repository"] - Repository --> Results["Domain Documents"] - Results --> Connection["Connection DTO"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-dtos-4.mmd b/docs/diagrams/architecture/api-service-core-graphql-dtos-4.mmd deleted file mode 100644 index c3ebeb57e..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-dtos-4.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Client["GraphQL Mutation"] --> Input["Mutation Input DTO"] - Input --> Validation["Bean Validation"] - Validation --> Processor["Service or Processor"] - Processor --> Repository["Mongo Repository"] - Repository --> Response["Response DTO"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-dtos-5.mmd b/docs/diagrams/architecture/api-service-core-graphql-dtos-5.mmd deleted file mode 100644 index fb0a10865..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-dtos-5.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - UI["Frontend UI"] --> Query["GraphQL Query or Mutation"] - Query --> DTOIn["Input DTO"] - DTOIn --> Fetcher["Data Fetcher"] - Fetcher --> Service["Core Service"] - Service --> Repo["Mongo Repository"] - Repo --> Domain["Domain Document"] - Domain --> DTOOut["Edge or Response DTO"] - DTOOut --> Connection["CountedGenericConnection"] - Connection --> UI diff --git a/docs/diagrams/architecture/api-service-core-graphql-dtos.mmd b/docs/diagrams/architecture/api-service-core-graphql-dtos.mmd deleted file mode 100644 index 0a767217f..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-dtos.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Client["GraphQL Client"] --> Schema["GraphQL Schema"] - Schema --> DataFetcher["GraphQL Data Fetchers"] - DataFetcher --> Dtos["Api Service Core Graphql Dtos"] - Dtos --> Services["Core Services"] - Services --> Repositories["Mongo Repositories"] - Repositories --> Database[("Database")] diff --git a/docs/diagrams/architecture/api-service-core-graphql-layer-2.mmd b/docs/diagrams/architecture/api-service-core-graphql-layer-2.mmd new file mode 100644 index 000000000..99dbac636 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-graphql-layer-2.mmd @@ -0,0 +1,6 @@ +flowchart LR + GID["Global ID"] --> Decode["Relay.fromGlobalId"] + Decode --> Resolve["NodeDataFetcher.resolveNode"] + Resolve --> Service["Domain Service"] + Service --> Entity["Domain Entity"] + Entity --> Encode["Relay.toGlobalId"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-layer-3.mmd b/docs/diagrams/architecture/api-service-core-graphql-layer-3.mmd new file mode 100644 index 000000000..ba0df5b95 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-graphql-layer-3.mmd @@ -0,0 +1,15 @@ +sequenceDiagram + participant Client + participant GraphQL + participant DataFetcher + participant Service + participant Repository + + Client->>GraphQL: Query devices(first: 20) + GraphQL->>DataFetcher: devices() + DataFetcher->>Service: queryDevices() + Service->>Repository: findByFilter() + Repository-->>Service: Device list + Service-->>DataFetcher: QueryResult + DataFetcher-->>GraphQL: Connection + GraphQL-->>Client: JSON response diff --git a/docs/diagrams/architecture/api-service-core-graphql-layer.mmd b/docs/diagrams/architecture/api-service-core-graphql-layer.mmd new file mode 100644 index 000000000..9bb50371b --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-graphql-layer.mmd @@ -0,0 +1,30 @@ +flowchart TD + Client["Frontend / API Client"] -->|"GraphQL HTTP"| DGS["DGS GraphQL Engine"] + + subgraph graphql_layer["Api Service Core Graphql Layer"] + DF["DataFetchers"] + TR["Type Resolvers"] + NodeQ["Node & Relay Support"] + end + + subgraph dataloaders["GraphQL DataLoaders"] + DL1["Entity Batch Loaders"] + end + + subgraph services["Business Services"] + S1["Domain Services"] + end + + subgraph persistence["Mongo Repositories"] + DB[("MongoDB")] + end + + Client --> DGS + DGS --> DF + DF --> S1 + DF --> DL1 + DL1 --> S1 + S1 --> DB + + DF --> TR + DF --> NodeQ diff --git a/docs/diagrams/architecture/api-service-core-relay-type-resolution-2.mmd b/docs/diagrams/architecture/api-service-core-relay-type-resolution-2.mmd deleted file mode 100644 index 4e1ad0da6..000000000 --- a/docs/diagrams/architecture/api-service-core-relay-type-resolution-2.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - Input["AssignableTarget Object"] --> CheckOrg{"Organization?"} - CheckOrg -->|"Yes"| Org["Return Organization"] - CheckOrg -->|"No"| CheckMachine{"Machine?"} - CheckMachine -->|"Yes"| Machine["Return Machine"] - CheckMachine -->|"No"| CheckTicket{"Ticket?"} - CheckTicket -->|"Yes"| Ticket["Return Ticket"] - CheckTicket -->|"No"| CheckKB{"KnowledgeBaseItem?"} - CheckKB -->|"Yes"| KB["Return KnowledgeBaseItem"] - CheckKB -->|"No"| Error["Throw IllegalArgumentException"] diff --git a/docs/diagrams/architecture/api-service-core-relay-type-resolution-3.mmd b/docs/diagrams/architecture/api-service-core-relay-type-resolution-3.mmd deleted file mode 100644 index 5341a0d37..000000000 --- a/docs/diagrams/architecture/api-service-core-relay-type-resolution-3.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - NodeInput["Node Object"] --> CheckMachine{"Machine?"} - CheckMachine -->|"Yes"| ReturnMachine["Return Machine"] - CheckMachine -->|"No"| CheckOrg{"Organization?"} - CheckOrg -->|"Yes"| ReturnOrg["Return Organization"] - CheckOrg -->|"No"| CheckEvent{"Event?"} - CheckEvent -->|"Yes"| ReturnEvent["Return Event"] - CheckEvent -->|"No"| Continue["Other instanceof checks"] - Continue --> ReturnType["Return Matching Type"] diff --git a/docs/diagrams/architecture/api-service-core-relay-type-resolution-4.mmd b/docs/diagrams/architecture/api-service-core-relay-type-resolution-4.mmd deleted file mode 100644 index d122b76ab..000000000 --- a/docs/diagrams/architecture/api-service-core-relay-type-resolution-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - DataFetcher["DataFetcher"] --> Domain["Domain Object"] - Domain --> Resolver["Node or AssignableTarget Resolver"] - Resolver --> SchemaType["GraphQL Schema Type"] - SchemaType --> Response["GraphQL Response"] diff --git a/docs/diagrams/architecture/api-service-core-relay-type-resolution-5.mmd b/docs/diagrams/architecture/api-service-core-relay-type-resolution-5.mmd deleted file mode 100644 index 60885d35a..000000000 --- a/docs/diagrams/architecture/api-service-core-relay-type-resolution-5.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart TB - DomainModel["Mongo Domain Model"] --> RelayResolver["Relay Type Resolver"] - RelayResolver --> GraphQLSchema["GraphQL Schema Types"] diff --git a/docs/diagrams/architecture/api-service-core-relay-type-resolution.mmd b/docs/diagrams/architecture/api-service-core-relay-type-resolution.mmd deleted file mode 100644 index df97025e3..000000000 --- a/docs/diagrams/architecture/api-service-core-relay-type-resolution.mmd +++ /dev/null @@ -1,14 +0,0 @@ -flowchart TD - Client["GraphQL Client"] --> Query["GraphQL Query"] - Query --> DGS["DGS Runtime"] - DGS --> Resolver["Type Resolver"] - Resolver --> DomainObj["Domain Object"] - DomainObj --> GraphQLType["GraphQL Type Name"] - - subgraph relay_layer["Relay Type Resolution Layer"] - Resolver - end - - subgraph domain_layer["Mongo Domain Model"] - DomainObj - end diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-10.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-10.mmd deleted file mode 100644 index 93e3f4c68..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-10.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Admin["Admin"] --> Controller["UserController"] - Controller --> Service["UserService"] - Service --> Repo["User Repository"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-11.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-11.mmd deleted file mode 100644 index d45884088..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-11.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Request["Incoming HTTP Request"] --> Filter["JWT Filter"] - Filter --> Principal["AuthPrincipal"] - Principal --> Controller["REST Controller"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-2.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-2.mmd index b7227563f..9e7f09173 100644 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-2.mmd +++ b/docs/diagrams/architecture/api-service-core-rest-controllers-2.mmd @@ -1,9 +1,5 @@ -sequenceDiagram - participant Admin - participant Controller as AgentRegistrationSecretController - participant Service as AgentRegistrationSecretService - - Admin->>Controller: POST /agent/registration-secret/generate - Controller->>Service: generateNewSecret() - Service-->>Controller: AgentRegistrationSecretResponse - Controller-->>Admin: 201 Created +flowchart LR + Admin["Admin Request"] --> Controller["AgentRegistrationSecretController"] + Controller --> Service["AgentRegistrationSecretService"] + Service --> Processor["DefaultAgentRegistrationSecretProcessor"] + Processor --> Repo["BaseApiKeyRepository / Tenant Repository"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-3.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-3.mmd index 38721fa21..fb0971901 100644 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-3.mmd +++ b/docs/diagrams/architecture/api-service-core-rest-controllers-3.mmd @@ -1,4 +1,9 @@ -flowchart LR - User["Authenticated User"] --> Controller["ApiKeyController"] - Controller --> Service["ApiKeyService"] - Service --> Repo["BaseApiKeyRepository"] +flowchart TD + Admin["Admin UI"] --> ForceController["ForceAgentController"] + ForceController --> InstallService["ForceToolInstallationService"] + ForceController --> UpdateService["ForceToolAgentUpdateService"] + ForceController --> ClientService["ForceClientUpdateService"] + + InstallService --> Messaging["NATS Publisher"] + UpdateService --> Messaging + ClientService --> Messaging diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-4.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-4.mmd index 8f96fb140..d98a1ece5 100644 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-4.mmd +++ b/docs/diagrams/architecture/api-service-core-rest-controllers-4.mmd @@ -1,4 +1,9 @@ -flowchart TD - Agent["Agent"] --> Controller["DeviceController"] - Controller --> Service["DeviceService"] - Service --> DeviceDoc["Device Document"] +sequenceDiagram + participant Client + participant Controller as Api Service + participant Security as JwtSecurityConfig + + Client->>Controller: GET /me with Bearer token + Controller->>Security: Resolve AuthPrincipal + Security-->>Controller: AuthPrincipal + Controller-->>Client: JSON user payload diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-5.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-5.mmd index 41904c5b9..05ffbf21a 100644 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-5.mmd +++ b/docs/diagrams/architecture/api-service-core-rest-controllers-5.mmd @@ -1,9 +1,5 @@ flowchart TD - Admin["Admin Action"] --> Controller["ForceAgentController"] - Controller --> InstallSvc["ForceToolInstallationService"] - Controller --> UpdateSvc["ForceToolAgentUpdateService"] - Controller --> ClientSvc["ForceClientUpdateService"] - - InstallSvc --> Kafka["Kafka Event"] - UpdateSvc --> Kafka - ClientSvc --> Kafka + Request["PATCH status=ARCHIVED"] --> Service["OrganizationCommandService"] + Service --> Check["canArchiveOrganization()"] + Check -->|"false"| Conflict["409 Conflict"] + Check -->|"true"| Update["Update status to ARCHIVED"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-6.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-6.mmd deleted file mode 100644 index c5f8a4036..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-6.mmd +++ /dev/null @@ -1,9 +0,0 @@ -sequenceDiagram - participant Admin - participant Controller as InvitationController - participant Service as InvitationService - - Admin->>Controller: POST /invitations - Controller->>Service: createInvitation(request) - Service-->>Controller: InvitationResponse - Controller-->>Admin: 201 Created diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-7.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-7.mmd deleted file mode 100644 index bb0eb550e..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-7.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Request["GET /me"] --> AuthCheck["AuthPrincipal Present?"] - AuthCheck -->|"Yes"| Response["Return User Info"] - AuthCheck -->|"No"| Unauthorized["401 Unauthorized"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-8.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-8.mmd deleted file mode 100644 index 5270d4876..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-8.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Admin["Admin"] --> Controller["OrganizationController"] - Controller --> CommandSvc["OrganizationCommandService"] - Controller --> DomainSvc["OrganizationService"] - CommandSvc --> Repo["Organization Repository"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-9.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-9.mmd deleted file mode 100644 index 845810ee8..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-9.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Admin["Admin"] --> Controller["SSOConfigController"] - Controller --> Service["SSOConfigService"] - Service --> Strategy["Provider Strategy"] - Strategy --> Provider["Google / Microsoft"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers.mmd index 6feac1f96..da03c2141 100644 --- a/docs/diagrams/architecture/api-service-core-rest-controllers.mmd +++ b/docs/diagrams/architecture/api-service-core-rest-controllers.mmd @@ -1,29 +1,15 @@ flowchart TD - Client["Client / UI / Agent"] --> Gateway["Gateway Service"] - Gateway --> ApiCore["API Service Core"] + Client["Web UI / Internal Service"] --> Gateway["Gateway Service Core"] + Gateway --> RestControllers["Api Service Core Rest Controllers"] - subgraph rest_layer["REST Controller Layer"] - Controllers["Api Service Core Rest Controllers"] - end + RestControllers --> BusinessServices["Api Service Core Business Services"] + RestControllers --> MappingLayer["Api Lib Mapping and Domain Services"] + RestControllers --> Dtos["Api Service Core Dtos"] - subgraph service_layer["Service Layer"] - CommandServices["Command Services"] - QueryServices["Query Services"] - DomainProcessors["Domain Processors"] - end + BusinessServices --> Repositories["Data Model and Repositories Mongo"] + BusinessServices --> Messaging["Eventing and Messaging Kafka NATS"] + BusinessServices --> Authz["Authorization Service Core"] - subgraph data_layer["Data Layer"] - Mongo["Mongo Repositories"] - Redis["Redis Cache"] - Kafka["Kafka / Events"] - end - - ApiCore --> Controllers - Controllers --> CommandServices - Controllers --> QueryServices - Controllers --> DomainProcessors - - CommandServices --> Mongo - QueryServices --> Mongo - DomainProcessors --> Kafka - DomainProcessors --> Redis + Repositories --> Mongo[("MongoDB")] + Messaging --> Kafka[("Kafka")] + Messaging --> Nats[("NATS")] diff --git a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-2.mmd b/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-2.mmd deleted file mode 100644 index 9eb324363..000000000 --- a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-2.mmd +++ /dev/null @@ -1,11 +0,0 @@ -flowchart TD - Request["SSOConfigRequest"] --> Normalize["Normalize Domains"] - Normalize --> GenericCheck["validateGenericPublicDomain()"] - GenericCheck --> ExistsCheck["validateExists()"] - ExistsCheck --> Decision{"Auto Provision?"} - Decision -->|"No"| Save["Save Config"] - Decision -->|"Yes"| RequireDomain["Require At Least One Domain"] - RequireDomain --> MicrosoftCheck{"Microsoft?"} - MicrosoftCheck -->|"Yes"| RequireTenant["Require msTenantId"] - MicrosoftCheck -->|"No"| Save - RequireTenant --> Save diff --git a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-3.mmd b/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-3.mmd deleted file mode 100644 index 025f8d4a5..000000000 --- a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-3.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - ListUsers["listUsers()"] --> FetchPage["UserRepository.findAll()"] - FetchPage --> MapResponse["UserMapper.toResponse()"] - MapResponse --> PostProcess["UserProcessor.postProcessUserGet()"] - - DeleteUser["softDeleteUser()"] --> ValidateSelf["Prevent Self Delete"] - ValidateSelf --> ValidateOwner["Prevent OWNER Delete"] - ValidateOwner --> MarkDeleted["Set Status DELETED"] - MarkDeleted --> SaveUser["UserRepository.save()"] - SaveUser --> PostDelete["UserProcessor.postProcessUserDeleted()"] diff --git a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-4.mmd b/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-4.mmd deleted file mode 100644 index 8c3ba9978..000000000 --- a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-4.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - Upsert["upsertConfig()"] --> Exists{"Config Exists?"} - Exists -->|"Yes"| Update["Update Existing"] - Exists -->|"No"| Create["Create New"] - Update --> Encrypt["Encrypt Client Secret"] - Create --> Encrypt - Encrypt --> Save["SSOConfigRepository.save()"] - Save --> PostProcess["SSOConfigProcessor.postProcessConfigSaved()"] diff --git a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors.mmd b/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors.mmd deleted file mode 100644 index 5d8c79678..000000000 --- a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors.mmd +++ /dev/null @@ -1,18 +0,0 @@ -flowchart TD - Controller["REST / GraphQL Controllers"] --> UserService["UserService"] - Controller --> SSOService["SSOConfigService"] - - UserService --> UserRepo["UserRepository"] - UserService --> UserProcessor["UserProcessor"] - - SSOService --> SSORepo["SSOConfigRepository"] - SSOService --> Encryption["EncryptionService"] - SSOService --> DomainValidation["DomainValidationService"] - SSOService --> SSOProcessor["SSOConfigProcessor"] - - subgraph processors_layer["Post-Processing Layer"] - UserProcessor --> DefaultUserProcessor["DefaultUserProcessor"] - SSOProcessor --> DefaultSSOProcessor["DefaultSSOConfigProcessor"] - InvitationProcessor --> DefaultInvitationProcessor["DefaultInvitationProcessor"] - AgentSecretProcessor --> DefaultAgentProcessor["DefaultAgentRegistrationSecretProcessor"] - end diff --git a/docs/diagrams/architecture/authorization-service-core-2.mmd b/docs/diagrams/architecture/authorization-service-core-2.mmd new file mode 100644 index 000000000..252984245 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-2.mmd @@ -0,0 +1,5 @@ +flowchart TD + Request["Incoming HTTP Request"] --> Filter["TenantContextFilter"] + Filter -->|"Extract tenantId"| Context["TenantContext (ThreadLocal)"] + Context --> AuthLogic["Auth & Token Logic"] + AuthLogic --> Clear["Clear Context After Request"] diff --git a/docs/diagrams/architecture/authorization-service-core-3.mmd b/docs/diagrams/architecture/authorization-service-core-3.mmd new file mode 100644 index 000000000..7b9123532 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-3.mmd @@ -0,0 +1,6 @@ +flowchart TD + TokenRequest["Token Request"] --> JwkSource["JWKSource"] + JwkSource -->|"Resolve tenantId"| TenantKeyService + TenantKeyService -->|"Load or Create Key"| MongoKeys[("TenantKey Collection")] + TenantKeyService --> RSAKey["RSAKey (with kid)"] + RSAKey --> JwtEncoder["NimbusJwtEncoder"] diff --git a/docs/diagrams/architecture/authorization-service-core-4.mmd b/docs/diagrams/architecture/authorization-service-core-4.mmd new file mode 100644 index 000000000..736d7eabe --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-4.mmd @@ -0,0 +1,5 @@ +flowchart LR + Auth["Authenticated Principal"] --> Customizer["Token Customizer"] + Customizer --> Claims["Add Claims"] + Claims -->|"tenant_id"| Jwt["JWT Access Token"] + Claims -->|"roles"| Jwt diff --git a/docs/diagrams/architecture/authorization-service-core-5.mmd b/docs/diagrams/architecture/authorization-service-core-5.mmd new file mode 100644 index 000000000..0302900b8 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-5.mmd @@ -0,0 +1,11 @@ +sequenceDiagram + participant Browser + participant AuthServer as "Authorization Service" + participant IdP as "OIDC Provider" + + Browser->>AuthServer: GET /oauth2/authorization/{provider} + AuthServer->>IdP: Redirect with state + PKCE + IdP-->>AuthServer: Authorization code + AuthServer->>AuthServer: Exchange code for tokens + AuthServer->>AuthServer: Auto-provision user if enabled + AuthServer-->>Browser: Redirect to success URL diff --git a/docs/diagrams/architecture/authorization-service-core-6.mmd b/docs/diagrams/architecture/authorization-service-core-6.mmd new file mode 100644 index 000000000..591437283 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-6.mmd @@ -0,0 +1,7 @@ +flowchart TD + InviteLink["Invitation Link"] --> Start["Start SSO Accept"] + Start --> Cookie["Set SSO Invite Cookie"] + Cookie --> IdP["Redirect to IdP"] + IdP --> Callback["SSO Success"] + Callback --> Register["Register User By Invitation"] + Register --> Redirect["Redirect To Target Tenant"] diff --git a/docs/diagrams/architecture/authorization-service-core-7.mmd b/docs/diagrams/architecture/authorization-service-core-7.mmd new file mode 100644 index 000000000..ed52da464 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-7.mmd @@ -0,0 +1,6 @@ +flowchart TD + StartReg["Start Tenant SSO Registration"] --> CookieReg["Set SSO Reg Cookie"] + CookieReg --> IdPReg["Redirect to IdP"] + IdPReg --> CallbackReg["SSO Success"] + CallbackReg --> CreateTenant["Register Tenant + Owner"] + CreateTenant --> RedirectTenant["Redirect to New Tenant Context"] diff --git a/docs/diagrams/architecture/authorization-service-core-8.mmd b/docs/diagrams/architecture/authorization-service-core-8.mmd new file mode 100644 index 000000000..457821009 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-8.mmd @@ -0,0 +1,4 @@ +flowchart LR + OAuthAuth["OAuth2Authorization"] --> Mapper["MongoAuthorizationMapper"] + Mapper --> MongoEntity["MongoOAuth2Authorization"] + MongoEntity --> Mongo[("MongoDB")] diff --git a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-2.mmd b/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-2.mmd deleted file mode 100644 index 61f25309b..000000000 --- a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-2.mmd +++ /dev/null @@ -1,11 +0,0 @@ -sequenceDiagram - participant Browser - participant Controller as InvitationRegistrationController - participant SSOService as SsoInvitationService - participant IdP as External IdP - - Browser->>Controller: GET /invitations/accept/sso - Controller->>SSOService: startAccept(request) - SSOService-->>Controller: SsoAuthorizeData - Controller->>Browser: Set-Cookie + 303 Redirect - Browser->>IdP: OAuth2 Authorization Request diff --git a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-3.mmd b/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-3.mmd deleted file mode 100644 index b6038e501..000000000 --- a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-3.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Start["User Clicks Register with SSO"] --> Clear["Clear Auth State"] - Clear --> StartReg["SsoTenantRegistrationService.startRegistration()"] - StartReg --> Cookie["Set COOKIE_SSO_REG"] - Cookie --> Redirect["Redirect to IdP"] diff --git a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-4.mmd b/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-4.mmd deleted file mode 100644 index 5e5fca6db..000000000 --- a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - User["User enters email"] --> Controller["TenantDiscoveryController"] - Controller --> Service["TenantDiscoveryService"] - Service --> Response["TenantDiscoveryResponse"] diff --git a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-5.mmd b/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-5.mmd deleted file mode 100644 index 2b315124b..000000000 --- a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Controllers["Auth Controllers"] --> TenantLayer["Tenant Context Layer"] - Controllers --> SSOFlow["SSO Flow & Handlers"] - Controllers --> AuthServer["OAuth2 Authorization Server"] - AuthServer --> Persistence["Mongo Authorization Service"] diff --git a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos.mmd b/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos.mmd deleted file mode 100644 index 9ab0a2ede..000000000 --- a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - Client["Browser / Client App"] --> Controllers["Auth Controllers"] - Controllers --> Services["Domain Services"] - Services --> Persistence["Authorization & Tenant Persistence"] - Services --> Security["Spring Security / OAuth2"] - Security --> Client diff --git a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-2.mmd b/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-2.mmd deleted file mode 100644 index f417b8ed2..000000000 --- a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-2.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Start["Generate Key Pair"] --> InitRandom["Initialize SecureRandom"] - InitRandom --> InitGenerator["Initialize KeyPairGenerator"] - InitGenerator --> Generate["Generate RSA KeyPair"] - Generate --> ConvertPem["Convert to PEM"] - ConvertPem --> CreateKid["Create kid UUID"] - CreateKid --> Result["Return AuthenticationKeyPair"] diff --git a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-3.mmd b/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-3.mmd deleted file mode 100644 index e99c37703..000000000 --- a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-3.mmd +++ /dev/null @@ -1,11 +0,0 @@ -flowchart TD - Request["Request RSAKey for Tenant"] --> CheckActive["Check Active Key Count"] - CheckActive --> Found{"Active Key Exists?"} - Found -->|"Yes"| LoadDoc["Load TenantKey Document"] - Found -->|"No"| Create["Generate New Key"] - Create --> Encrypt["Encrypt Private PEM"] - Encrypt --> Save["Save TenantKey"] - Save --> LoadDoc - LoadDoc --> Parse["Parse PEM to RSA Keys"] - Parse --> BuildJwk["Build Nimbus RSAKey"] - BuildJwk --> Return["Return RSAKey"] diff --git a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-4.mmd b/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-4.mmd deleted file mode 100644 index 485dad1ca..000000000 --- a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - SaveCall["Save Authorization"] --> ExtractPkce["Extract PKCE Parameters"] - ExtractPkce --> MapEntity["Map to Mongo Entity"] - MapEntity --> Persist["Save to Mongo"] - Persist --> Verify["Debug Verify PKCE"] diff --git a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-5.mmd b/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-5.mmd deleted file mode 100644 index a88a11966..000000000 --- a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-5.mmd +++ /dev/null @@ -1,11 +0,0 @@ -flowchart TD - Lookup["Find By Token"] --> TypeCheck{"Token Type?"} - TypeCheck -->|"Access"| AccessQuery["Find By Access Token"] - TypeCheck -->|"Refresh"| RefreshQuery["Find By Refresh Token"] - TypeCheck -->|"Code"| CodeQuery["Find By Authorization Code"] - TypeCheck -->|"Null"| MultiQuery["Try All Token Fields"] - AccessQuery --> MapBack["Map to Domain"] - RefreshQuery --> MapBack - CodeQuery --> MapBack - MultiQuery --> MapBack - MapBack --> ReturnAuth["Return OAuth2Authorization"] diff --git a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-6.mmd b/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-6.mmd deleted file mode 100644 index 5d14b5009..000000000 --- a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-6.mmd +++ /dev/null @@ -1,18 +0,0 @@ -sequenceDiagram - participant User - participant AuthServer - participant KeyService - participant AuthService - participant MongoDB - - User->>AuthServer: Authorization Request with PKCE - AuthServer->>AuthService: Save OAuth2Authorization - AuthService->>MongoDB: Persist Authorization Code - AuthServer->>KeyService: Get Tenant Signing Key - KeyService->>MongoDB: Load or Create TenantKey - KeyService->>AuthServer: Return RSAKey - AuthServer->>User: Issue JWT Access Token - User->>AuthServer: Token Exchange - AuthServer->>AuthService: Find Authorization by Code - AuthService->>MongoDB: Retrieve Authorization - AuthServer->>User: Return Access and Refresh Tokens diff --git a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence.mmd b/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence.mmd deleted file mode 100644 index cdd77c54f..000000000 --- a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence.mmd +++ /dev/null @@ -1,23 +0,0 @@ -flowchart TD - ClientApp["OAuth2 Client Application"] --> AuthServer["Authorization Server"] - - subgraph key_layer["Tenant Key Layer"] - TenantKeyService["TenantKeyService"] --> KeyGenerator["RsaAuthenticationKeyPairGenerator"] - TenantKeyService --> PemUtil["PemUtil"] - TenantKeyService --> TenantKeyRepo["TenantKeyRepository"] - TenantKeyService --> EncryptionService["EncryptionService"] - end - - subgraph client_layer["Client Registration Layer"] - RegisteredRepo["MongoRegisteredClientRepository"] --> MongoClientRepo["RegisteredClientMongoRepository"] - end - - subgraph auth_layer["Authorization Persistence Layer"] - AuthService["MongoAuthorizationService"] --> AuthMapper["MongoAuthorizationMapper"] - AuthService --> MongoAuthRepo["MongoOAuth2AuthorizationRepository"] - AuthMapper --> MongoEntity["MongoOAuth2Authorization"] - end - - AuthServer --> TenantKeyService - AuthServer --> RegisteredRepo - AuthServer --> AuthService diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-2.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-2.mmd deleted file mode 100644 index eabc1c302..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-2.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Request["Incoming HTTP Request"] --> Filter["TenantContextFilter"] - Filter --> ThreadLocal["TenantContext ThreadLocal"] - ThreadLocal --> Services["UserService and Key Services"] diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-3.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-3.mmd deleted file mode 100644 index 668f63b02..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-3.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Request["OAuth2 Endpoint Request"] --> Match["Authorization Endpoints Matcher"] - Match --> Authenticated["Require Authentication"] - Authenticated --> JWTValidation["JWT Resource Server Enabled"] - JWTValidation --> Response["OAuth2 Response"] diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-4.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-4.mmd deleted file mode 100644 index 0fc1b3d3e..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - JWTRequest["JWKS or Token Signing"] --> TenantId["TenantContext.getTenantId()"] - TenantId --> KeyService["TenantKeyService.getOrCreateActiveKey"] - KeyService --> RSAKey["Tenant RSAKey"] - RSAKey --> JWKSet["JWKSet Returned"] diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-5.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-5.mmd deleted file mode 100644 index 01f61db5e..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Auth["Authenticated Principal"] --> Lookup["UserService.findActiveByEmailAndTenant"] - Lookup --> Claims["Add Claims to JWT"] - Claims --> Token["Signed Access Token"] diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-6.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-6.mmd deleted file mode 100644 index ae1c09e93..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-6.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - User["User Clicks SSO"] --> OAuthLogin["oauth2Login()"] - OAuthLogin --> Resolver["SsoAuthorizationRequestResolver"] - Resolver --> Provider["External IdP"] - Provider --> Callback["OIDC Callback"] - Callback --> OidcService["Custom OidcUserService"] - OidcService --> AutoProvision["Auto Provision If Needed"] - AutoProvision --> Success["AuthSuccessHandler"] diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-7.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-7.mmd deleted file mode 100644 index cf74b4686..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-7.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - OAuthFlow["OAuth2 Login Request"] --> Repo["DynamicClientRegistrationRepository"] - Repo --> TenantCtx["TenantContext"] - TenantCtx --> DynamicSvc["DynamicClientRegistrationService"] - DynamicSvc --> ClientReg["ClientRegistration"] diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-8.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-8.mmd deleted file mode 100644 index 9532e7d9d..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-8.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - Step1["Incoming Request"] --> Step2["TenantContextFilter Resolves Tenant"] - Step2 --> Step3{{"OAuth2 Endpoint?"}} - Step3 -->|"Yes"| Step4["AuthorizationServerConfig"] - Step3 -->|"No"| Step5["SecurityConfig"] - Step4 --> Step6["JWT Issued with Tenant Claims"] - Step5 --> Step7["Authenticated Session or SSO"] - Step6 --> EndNode["Response"] - Step7 --> EndNode diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant.mmd deleted file mode 100644 index 93afcb216..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant.mmd +++ /dev/null @@ -1,20 +0,0 @@ -flowchart TD - Client["Browser or API Client"] --> Gateway["Gateway Service"] - Gateway --> Authz["Authorization Service Core Server And Tenant"] - - subgraph authz_core["Authorization Layer"] - TenantFilter["TenantContextFilter"] - AuthServerConfig["AuthorizationServerConfig"] - SecurityConfigBean["SecurityConfig"] - DynamicClientRepo["DynamicClientRegistrationRepository"] - TenantKeySvc["TenantKeyService"] - end - - Authz --> TenantFilter - TenantFilter --> AuthServerConfig - TenantFilter --> SecurityConfigBean - AuthServerConfig --> TenantKeySvc - SecurityConfigBean --> DynamicClientRepo - - AuthServerConfig --> JWKS["Per-Tenant JWKS"] - AuthServerConfig --> JWT["Signed JWT Tokens"] diff --git a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-2.mmd b/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-2.mmd deleted file mode 100644 index c1938fa0b..000000000 --- a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-2.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - OidcUser["OIDC User Claims"] --> Email["email"] - OidcUser --> Preferred["preferred_username"] - OidcUser --> Upn["upn"] - OidcUser --> Unique["unique_name"] - - Email --> Resolved["Resolved Email"] - Preferred --> Resolved - Upn --> Resolved - Unique --> Resolved diff --git a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-3.mmd b/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-3.mmd deleted file mode 100644 index 0151dca81..000000000 --- a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-3.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Cookie["Invite Cookie"] --> Decode["Decode Invite Payload"] - Oidc["OIDC User"] --> Extract["Extract Names and Email"] - Decode --> BuildReq["Build InvitationRegistrationRequest"] - Extract --> BuildReq - BuildReq --> Register["InvitationRegistrationService.registerByInvitation"] - Register --> Redirect["Clear Cookie and Redirect"] diff --git a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-4.mmd b/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-4.mmd deleted file mode 100644 index 579d6dffa..000000000 --- a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-4.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - Cookie["Registration Cookie"] --> Decode["Decode Tenant Payload"] - Oidc["OIDC User"] --> Email["Resolve Email"] - Decode --> Validate["Validate Tenant Name and Domain"] - Email --> BuildReq["Build TenantRegistrationRequest"] - Validate --> BuildReq - BuildReq --> Register["TenantRegistrationService.registerTenant"] - Register --> Redirect["Clear Cookie and Redirect"] diff --git a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-5.mmd b/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-5.mmd deleted file mode 100644 index cc926f590..000000000 --- a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-5.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - SSOConfigService["SSO Config Service"] --> GoogleStrategy["Google Client Registration Strategy"] - SSOConfigService --> MicrosoftStrategy["Microsoft Client Registration Strategy"] - - GoogleStrategy --> GoogleProps["Google SSO Properties"] - MicrosoftStrategy --> MicrosoftProps["Microsoft SSO Properties"] diff --git a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-6.mmd b/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-6.mmd deleted file mode 100644 index 12322027f..000000000 --- a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-6.mmd +++ /dev/null @@ -1,14 +0,0 @@ -sequenceDiagram - participant Browser - participant AuthServer as "Authorization Server" - participant Success as "Auth Success Handler" - participant Flow as "SSO Flow Handler" - participant Service as "Registration Service" - - Browser->>AuthServer: OAuth2 Login (Google or Microsoft) - AuthServer->>Success: onAuthenticationSuccess - Success->>Success: Update lastLogin - Success->>Flow: Delegate to flow handler - Flow->>Service: Register user or tenant - Service->>Flow: Return tenant context - Flow->>Browser: Redirect to target tenant diff --git a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils.mmd b/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils.mmd deleted file mode 100644 index ce65039e2..000000000 --- a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils.mmd +++ /dev/null @@ -1,15 +0,0 @@ -flowchart TD - Browser["Browser"] -->|"OAuth2 Login"| AuthServer["Authorization Server"] - AuthServer -->|"Authentication Success"| AuthSuccessHandler["Auth Success Handler"] - - AuthSuccessHandler -->|"Delegate"| SsoFlowHandlers["SSO Flow Handlers"] - AuthSuccessHandler -->|"Update User"| UserService["User Service"] - - SsoFlowHandlers --> InviteHandler["Invite SSO Handler"] - SsoFlowHandlers --> TenantRegHandler["Tenant Registration SSO Handler"] - - InviteHandler --> InvitationService["Invitation Registration Service"] - TenantRegHandler --> TenantRegService["Tenant Registration Service"] - - AuthSuccessHandler --> RedirectsUtil["Redirect Utilities"] - AuthSuccessHandler --> AuthStateUtils["Auth State Utilities"] diff --git a/docs/diagrams/architecture/authorization-service-core.mmd b/docs/diagrams/architecture/authorization-service-core.mmd new file mode 100644 index 000000000..823394c48 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core.mmd @@ -0,0 +1,7 @@ +flowchart LR + User["User Browser"] -->|"Login / OAuth2"| AuthServer["Authorization Service Core"] + AuthServer -->|"OIDC Redirect"| Google["Google OIDC"] + AuthServer -->|"OIDC Redirect"| Microsoft["Microsoft OIDC"] + AuthServer -->|"JWT Access Token"| Gateway["Gateway Service Core"] + Gateway -->|"Forward Request"| ApiService["API Service Core"] + AuthServer -->|"Persist Authorization"| Mongo[("MongoDB")] diff --git a/docs/diagrams/architecture/client-core-agent-ingress-2.mmd b/docs/diagrams/architecture/client-core-agent-ingress-2.mmd new file mode 100644 index 000000000..c3b1fd485 --- /dev/null +++ b/docs/diagrams/architecture/client-core-agent-ingress-2.mmd @@ -0,0 +1,2 @@ +flowchart LR + Task["Tool Install Task"] --> Executor["Virtual Thread Executor"] --> Worker["Virtual Thread"] diff --git a/docs/diagrams/architecture/client-core-agent-ingress-3.mmd b/docs/diagrams/architecture/client-core-agent-ingress-3.mmd new file mode 100644 index 000000000..d8472a905 --- /dev/null +++ b/docs/diagrams/architecture/client-core-agent-ingress-3.mmd @@ -0,0 +1,9 @@ +sequenceDiagram + participant Agent + participant AuthController as "Agent Auth Controller" + participant AuthService as "Agent Auth Service" + + Agent->>AuthController: POST /oauth/token + AuthController->>AuthService: issueClientToken() + AuthService-->>AuthController: AgentTokenResponse + AuthController-->>Agent: 200 OK diff --git a/docs/diagrams/architecture/client-core-agent-ingress-4.mmd b/docs/diagrams/architecture/client-core-agent-ingress-4.mmd new file mode 100644 index 000000000..74b184c51 --- /dev/null +++ b/docs/diagrams/architecture/client-core-agent-ingress-4.mmd @@ -0,0 +1,5 @@ +flowchart TD + Request["AgentRegistrationRequest"] --> Validate["Validation"] + Validate --> CreateMachine["Create or Update Machine"] + CreateMachine --> AssignTags["Assign or Create Tags"] + AssignTags --> PostProcess["Post Registration Processor"] diff --git a/docs/diagrams/architecture/client-core-agent-ingress-5.mmd b/docs/diagrams/architecture/client-core-agent-ingress-5.mmd new file mode 100644 index 000000000..304d652a0 --- /dev/null +++ b/docs/diagrams/architecture/client-core-agent-ingress-5.mmd @@ -0,0 +1,8 @@ +flowchart TD + NATS["NATS / JetStream"] --> Heartbeat["Machine Heartbeat Listener"] + NATS --> Installed["Installed Agent Listener"] + NATS --> ToolConn["Tool Connection Listener"] + + Heartbeat --> MachineStatus["Machine Status Service"] + Installed --> InstalledService["Installed Agent Service"] + ToolConn --> ToolService["Tool Connection Service"] diff --git a/docs/diagrams/architecture/client-core-agent-ingress-6.mmd b/docs/diagrams/architecture/client-core-agent-ingress-6.mmd new file mode 100644 index 000000000..6132940f4 --- /dev/null +++ b/docs/diagrams/architecture/client-core-agent-ingress-6.mmd @@ -0,0 +1,5 @@ +flowchart TD + Message["InstalledAgentMessage"] --> Parse["Deserialize JSON"] + Parse --> Extract["Extract machineId"] + Extract --> Process["InstalledAgentService.addInstalledAgent"] + Process --> Ack["message.ack()"] diff --git a/docs/diagrams/architecture/client-core-agent-ingress.mmd b/docs/diagrams/architecture/client-core-agent-ingress.mmd new file mode 100644 index 000000000..5b215da89 --- /dev/null +++ b/docs/diagrams/architecture/client-core-agent-ingress.mmd @@ -0,0 +1,16 @@ +flowchart TD + Agent["Endpoint Agent"] -->|"POST /api/agents/register"| AgentController["Agent Controller"] + Agent -->|"POST /oauth/token"| AuthController["Agent Auth Controller"] + + AgentController --> RegistrationService["Agent Registration Service"] + RegistrationService --> RegistrationProcessor["Agent Registration Processor"] + + Agent -->|"Heartbeat (NATS)"| HeartbeatListener["Machine Heartbeat Listener"] + Agent -->|"Installed Agent (JetStream)"| InstalledListener["Installed Agent Listener"] + Agent -->|"Tool Connection (JetStream)"| ToolListener["Tool Connection Listener"] + + HeartbeatListener --> MachineStatusService["Machine Status Service"] + InstalledListener --> InstalledAgentService["Installed Agent Service"] + ToolListener --> ToolConnectionService["Tool Connection Service"] + + ToolListener --> Transformer["Tool Agent ID Transformers"] diff --git a/docs/diagrams/architecture/data-access-mongo-sync-2.mmd b/docs/diagrams/architecture/data-access-mongo-sync-2.mmd new file mode 100644 index 000000000..a8cd65033 --- /dev/null +++ b/docs/diagrams/architecture/data-access-mongo-sync-2.mmd @@ -0,0 +1,8 @@ +flowchart LR + Config["Configuration Layer"] --> Repo["Custom Repositories"] + Repo --> Domain["Domain-Specific Data Access"] + Repo --> Ticketing["Ticketing & Notifications"] + Repo --> Identity["User & Tenant"] + Repo --> Tools["Scripts & Integrated Tools"] + Retry["Retry & Resilience"] --> Repo + Seed["Seed Catalogs"] --> Repo diff --git a/docs/diagrams/architecture/data-access-mongo-sync-3.mmd b/docs/diagrams/architecture/data-access-mongo-sync-3.mmd new file mode 100644 index 000000000..901f6d266 --- /dev/null +++ b/docs/diagrams/architecture/data-access-mongo-sync-3.mmd @@ -0,0 +1,4 @@ +flowchart TD + A["spring.data.mongodb.enabled"] --> B{{"Tenant isolation?"}} + B -->|No| C["MongoSyncConfig"] + B -->|Yes| D["TenantAwareSyncConfig"] diff --git a/docs/diagrams/architecture/data-access-mongo-sync-4.mmd b/docs/diagrams/architecture/data-access-mongo-sync-4.mmd new file mode 100644 index 000000000..bd1b618c2 --- /dev/null +++ b/docs/diagrams/architecture/data-access-mongo-sync-4.mmd @@ -0,0 +1,5 @@ +flowchart TD + Descriptors["NotificationContextDescriptor"] --> Jackson["Subtype Registration"] + Jackson --> Converters["Read / Write Converters"] + Converters --> MongoConverter["MappingMongoConverter"] + MongoConverter --> Mongo[("MongoDB")] diff --git a/docs/diagrams/architecture/data-access-mongo-sync-5.mmd b/docs/diagrams/architecture/data-access-mongo-sync-5.mmd new file mode 100644 index 000000000..5fd9e5f94 --- /dev/null +++ b/docs/diagrams/architecture/data-access-mongo-sync-5.mmd @@ -0,0 +1,9 @@ +flowchart TD + Client["Client Request"] --> Query["Build Mongo Query"] + Query --> Cursor{{"Cursor Provided?"}} + Cursor -->|Yes| Criteria["Add lt/gt _id Criteria"] + Cursor -->|No| NoCursor["First Page"] + Criteria --> Sort["Apply Sort + _id Tie Breaker"] + NoCursor --> Sort + Sort --> Limit["Apply Limit"] + Limit --> Result["Return Page"] diff --git a/docs/diagrams/architecture/data-access-mongo-sync-6.mmd b/docs/diagrams/architecture/data-access-mongo-sync-6.mmd new file mode 100644 index 000000000..3e920020a --- /dev/null +++ b/docs/diagrams/architecture/data-access-mongo-sync-6.mmd @@ -0,0 +1,11 @@ +sequenceDiagram + participant Client + participant Repo + participant Mongo + + Client->>Repo: findPageForRecipient() + Repo->>Mongo: Query NotificationReadState + Mongo-->>Repo: ReadState rows + Repo->>Mongo: Query Notifications by _id + Mongo-->>Repo: Notification docs + Repo-->>Client: NotificationPage diff --git a/docs/diagrams/architecture/data-access-mongo-sync-7.mmd b/docs/diagrams/architecture/data-access-mongo-sync-7.mmd new file mode 100644 index 000000000..63c45aab0 --- /dev/null +++ b/docs/diagrams/architecture/data-access-mongo-sync-7.mmd @@ -0,0 +1,6 @@ +flowchart TD + Filter["TicketQueryFilter"] --> Build["Build Query"] + Build --> Cursor["Apply Cursor"] + Cursor --> Sort["Sort + _id"] + Sort --> Find["MongoTemplate.find()"] + Build --> Agg["Aggregation Pipelines"] diff --git a/docs/diagrams/architecture/data-access-mongo-sync-8.mmd b/docs/diagrams/architecture/data-access-mongo-sync-8.mmd new file mode 100644 index 000000000..d50fb67c8 --- /dev/null +++ b/docs/diagrams/architecture/data-access-mongo-sync-8.mmd @@ -0,0 +1,6 @@ +flowchart TD + Operation["Optimistic Update"] --> Conflict{{"Version Conflict?"}} + Conflict -->|Yes| Retry["Retry"] + Retry --> Success{{"Success?"}} + Success -->|Yes| Done["Complete"] + Success -->|No| Exhausted["Error"] diff --git a/docs/diagrams/architecture/data-access-mongo-sync.mmd b/docs/diagrams/architecture/data-access-mongo-sync.mmd new file mode 100644 index 000000000..2652a4212 --- /dev/null +++ b/docs/diagrams/architecture/data-access-mongo-sync.mmd @@ -0,0 +1,7 @@ +flowchart TD + Controllers["REST / GraphQL Controllers"] --> Services["Business Services"] + Services --> Repos["Data Access Mongo Sync Repositories"] + Repos --> Mongo[("MongoDB")] + + Repos --> Docs["Mongo Documents"] + Docs --> Mongo diff --git a/docs/diagrams/architecture/data-kafka-configuration-and-retry-2.mmd b/docs/diagrams/architecture/data-kafka-configuration-and-retry-2.mmd deleted file mode 100644 index 9ba2cc60f..000000000 --- a/docs/diagrams/architecture/data-kafka-configuration-and-retry-2.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - SpringBoot[Spring Boot] --> DefaultKafkaAutoConfig[KafkaAutoConfiguration] - DefaultKafkaAutoConfig -->|Excluded| OssKafkaConfig - OssKafkaConfig --> OssTenantKafkaAutoConfiguration - OssTenantKafkaAutoConfiguration --> ProducerFactory - OssTenantKafkaAutoConfiguration --> ConsumerFactory - OssTenantKafkaAutoConfiguration --> KafkaTemplate - OssTenantKafkaAutoConfiguration --> KafkaAdmin diff --git a/docs/diagrams/architecture/data-kafka-configuration-and-retry-3.mmd b/docs/diagrams/architecture/data-kafka-configuration-and-retry-3.mmd deleted file mode 100644 index 28fb95c63..000000000 --- a/docs/diagrams/architecture/data-kafka-configuration-and-retry-3.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - ConfigProps[KafkaTopicProperties] --> AdminBean[KafkaAdmin] - AdminBean --> TopicBuilder - TopicBuilder --> NewTopic - NewTopic --> KafkaCluster diff --git a/docs/diagrams/architecture/data-kafka-configuration-and-retry-4.mmd b/docs/diagrams/architecture/data-kafka-configuration-and-retry-4.mmd deleted file mode 100644 index 24014625a..000000000 --- a/docs/diagrams/architecture/data-kafka-configuration-and-retry-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - KafkaCluster --> ListenerContainer - ListenerContainer --> ConsumerFactory - ConsumerFactory --> JsonDeserializer - ListenerContainer --> AckMode[RECORD Ack Mode] diff --git a/docs/diagrams/architecture/data-kafka-configuration-and-retry-5.mmd b/docs/diagrams/architecture/data-kafka-configuration-and-retry-5.mmd deleted file mode 100644 index 7647e6808..000000000 --- a/docs/diagrams/architecture/data-kafka-configuration-and-retry-5.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - DebeziumMessage --> Payload - Payload --> Before[Before State] - Payload --> After[After State] - Payload --> Source - Payload --> Operation - Payload --> Timestamp - Source --> Database - Source --> Table - Source --> Connector diff --git a/docs/diagrams/architecture/data-kafka-configuration-and-retry-6.mmd b/docs/diagrams/architecture/data-kafka-configuration-and-retry-6.mmd deleted file mode 100644 index f36f73356..000000000 --- a/docs/diagrams/architecture/data-kafka-configuration-and-retry-6.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Producer --> KafkaCluster - KafkaCluster -->|Failure| RecoveryHandler - RecoveryHandler --> StructuredLog diff --git a/docs/diagrams/architecture/data-kafka-configuration-and-retry.mmd b/docs/diagrams/architecture/data-kafka-configuration-and-retry.mmd deleted file mode 100644 index b54be1cd0..000000000 --- a/docs/diagrams/architecture/data-kafka-configuration-and-retry.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - AppService[Application Service] --> KafkaInfra[Data Kafka Configuration And Retry] - KafkaInfra --> KafkaCluster[Kafka Cluster] - KafkaCluster --> StreamService[Stream Service Core] - KafkaCluster --> OtherConsumers[Other Consumers] diff --git a/docs/diagrams/architecture/data-model-and-repositories-mongo-2.mmd b/docs/diagrams/architecture/data-model-and-repositories-mongo-2.mmd new file mode 100644 index 000000000..086037fbf --- /dev/null +++ b/docs/diagrams/architecture/data-model-and-repositories-mongo-2.mmd @@ -0,0 +1,5 @@ +flowchart LR + Request["Incoming Request"] --> Security["Security Layer"] + Security --> TenantContext["Tenant Context"] + TenantContext --> TenantProvider["TenantIdProvider"] + TenantProvider --> Repository["Mongo Repository"] diff --git a/docs/diagrams/architecture/data-model-and-repositories-mongo-3.mmd b/docs/diagrams/architecture/data-model-and-repositories-mongo-3.mmd new file mode 100644 index 000000000..71d4391eb --- /dev/null +++ b/docs/diagrams/architecture/data-model-and-repositories-mongo-3.mmd @@ -0,0 +1,4 @@ +flowchart TD + Ticket["Ticket"] --> Attachment["TicketAttachment"] + Ticket --> Note["TicketNote"] + Ticket --> StatusDef["TicketStatusDefinition"] diff --git a/docs/diagrams/architecture/data-model-and-repositories-mongo-4.mmd b/docs/diagrams/architecture/data-model-and-repositories-mongo-4.mmd new file mode 100644 index 000000000..c26215eaf --- /dev/null +++ b/docs/diagrams/architecture/data-model-and-repositories-mongo-4.mmd @@ -0,0 +1,2 @@ +flowchart LR + Notification --> ReadState["NotificationReadState"] diff --git a/docs/diagrams/architecture/data-model-and-repositories-mongo-5.mmd b/docs/diagrams/architecture/data-model-and-repositories-mongo-5.mmd new file mode 100644 index 000000000..35f292dc8 --- /dev/null +++ b/docs/diagrams/architecture/data-model-and-repositories-mongo-5.mmd @@ -0,0 +1,4 @@ +flowchart LR + ApiInput["API Filter Input"] --> Mapper["Service Mapper"] + Mapper --> QueryFilter["*QueryFilter"] + QueryFilter --> Repository["Custom Repository Impl"] diff --git a/docs/diagrams/architecture/data-model-and-repositories-mongo.mmd b/docs/diagrams/architecture/data-model-and-repositories-mongo.mmd new file mode 100644 index 000000000..d345bd2ed --- /dev/null +++ b/docs/diagrams/architecture/data-model-and-repositories-mongo.mmd @@ -0,0 +1,7 @@ +flowchart TD + Controllers["REST Controllers / GraphQL"] --> Services["Business Services"] + Services --> Repositories["Mongo Repositories"] + Repositories --> Documents["Mongo Documents"] + Documents --> MongoDB[("MongoDB")] + + TenantProvider["TenantIdProvider"] --> Repositories diff --git a/docs/diagrams/architecture/data-mongo-base-repositories-2.mmd b/docs/diagrams/architecture/data-mongo-base-repositories-2.mmd deleted file mode 100644 index aece0a12e..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories-2.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - ApiKey["API Key Document"] --> ApiKeyRepo["BaseApiKeyRepository"] - ApiKeyRepo --> UserScope["User-Scoped Queries"] - ApiKeyRepo --> Expiry["Expiration Queries"] diff --git a/docs/diagrams/architecture/data-mongo-base-repositories-3.mmd b/docs/diagrams/architecture/data-mongo-base-repositories-3.mmd deleted file mode 100644 index 42408866f..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Tenant["Tenant Document"] --> TenantRepo["BaseTenantRepository"] - TenantRepo --> DomainLookup["Domain-Based Resolution"] - DomainLookup --> AuthLayer["Authorization & SSO"] diff --git a/docs/diagrams/architecture/data-mongo-base-repositories-4.mmd b/docs/diagrams/architecture/data-mongo-base-repositories-4.mmd deleted file mode 100644 index 85b9e185f..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Tool["Integrated Tool Document"] --> ToolRepo["BaseIntegratedToolRepository"] - ToolRepo --> TypeLookup["Type-Based Resolution"] - TypeLookup --> Gateway["Gateway & Integration Layer"] diff --git a/docs/diagrams/architecture/data-mongo-base-repositories-5.mmd b/docs/diagrams/architecture/data-mongo-base-repositories-5.mmd deleted file mode 100644 index de3fabd01..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - UserDoc["User Document"] --> UserRepo["BaseUserRepository"] - UserRepo --> EmailLookup["Email-Based Lookup"] - UserRepo --> StatusCheck["Status Validation"] - StatusCheck --> Security["Authentication & Access Control"] diff --git a/docs/diagrams/architecture/data-mongo-base-repositories-6.mmd b/docs/diagrams/architecture/data-mongo-base-repositories-6.mmd deleted file mode 100644 index 19d21ead5..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories-6.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Request["Incoming Request"] --> DomainExtract["Extract Domain"] - DomainExtract --> TenantRepo["BaseTenantRepository"] - TenantRepo --> TenantResolved["Tenant Context Established"] - TenantResolved --> UserRepo["BaseUserRepository"] - UserRepo --> AuthResult["User Validated"] diff --git a/docs/diagrams/architecture/data-mongo-base-repositories-7.mmd b/docs/diagrams/architecture/data-mongo-base-repositories-7.mmd deleted file mode 100644 index 8d3041b86..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories-7.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - Base["Data Mongo Base Repositories"] --> SyncImpl["Mongo Sync Implementations"] - Base --> ReactiveImpl["Mongo Reactive Implementations"] - SyncImpl --> ServiceLayer["Service Layer"] - ReactiveImpl --> ServiceLayer diff --git a/docs/diagrams/architecture/data-mongo-base-repositories.mmd b/docs/diagrams/architecture/data-mongo-base-repositories.mmd deleted file mode 100644 index da1da35d2..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Domain["Mongo Domain Model"] --> BaseRepo["Data Mongo Base Repositories"] - BaseRepo --> Blocking["Blocking Repositories"] - BaseRepo --> Reactive["Reactive Repositories"] - - Blocking --> Services["Application Services"] - Reactive --> Services diff --git a/docs/diagrams/architecture/data-mongo-domain-model-2.mmd b/docs/diagrams/architecture/data-mongo-domain-model-2.mmd deleted file mode 100644 index a298cd876..000000000 --- a/docs/diagrams/architecture/data-mongo-domain-model-2.mmd +++ /dev/null @@ -1,22 +0,0 @@ -classDiagram - class User { - id - email - firstName - lastName - roles - status - createdAt - updatedAt - } - - class AuthUser { - tenantId - passwordHash - loginProvider - externalUserId - lastLogin - imageUrl - } - - User <|-- AuthUser diff --git a/docs/diagrams/architecture/data-mongo-domain-model-3.mmd b/docs/diagrams/architecture/data-mongo-domain-model-3.mmd deleted file mode 100644 index 1f61e011e..000000000 --- a/docs/diagrams/architecture/data-mongo-domain-model-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -stateDiagram-v2 - [*] --> ACTIVE - ACTIVE --> OFFLINE - OFFLINE --> ACTIVE - ACTIVE --> MAINTENANCE - MAINTENANCE --> ACTIVE diff --git a/docs/diagrams/architecture/data-mongo-domain-model-4.mmd b/docs/diagrams/architecture/data-mongo-domain-model-4.mmd deleted file mode 100644 index 9fcd8cbc2..000000000 --- a/docs/diagrams/architecture/data-mongo-domain-model-4.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Ticket["Ticket"] --> Note["TicketNote"] - Ticket --> Attachment["TicketAttachment"] - Ticket --> User["User (Assigned)"] - Ticket --> Device["Device"] - Ticket --> Org["Organization"] diff --git a/docs/diagrams/architecture/data-mongo-domain-model-5.mmd b/docs/diagrams/architecture/data-mongo-domain-model-5.mmd deleted file mode 100644 index dfbf696fa..000000000 --- a/docs/diagrams/architecture/data-mongo-domain-model-5.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart TD - Notification["Notification"] --> ReadState["NotificationReadState"] - ReadState --> Recipient["User or Organization"] diff --git a/docs/diagrams/architecture/data-mongo-domain-model-6.mmd b/docs/diagrams/architecture/data-mongo-domain-model-6.mmd deleted file mode 100644 index 86de08803..000000000 --- a/docs/diagrams/architecture/data-mongo-domain-model-6.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - TagAssignment["TagAssignment"] --> Device - TagAssignment --> Ticket - TagAssignment --> Organization diff --git a/docs/diagrams/architecture/data-mongo-domain-model.mmd b/docs/diagrams/architecture/data-mongo-domain-model.mmd deleted file mode 100644 index 6171a289c..000000000 --- a/docs/diagrams/architecture/data-mongo-domain-model.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - Gateway["Gateway Service"] --> ApiService["API Service Core"] - ApiService --> DomainModel["Data Mongo Domain Model"] - AuthService["Authorization Service"] --> DomainModel - StreamService["Stream Service"] --> DomainModel - ManagementService["Management Service"] --> DomainModel - ExternalApi["External API Service"] --> DomainModel - - DomainModel --> MongoDB[("MongoDB")] diff --git a/docs/diagrams/architecture/data-mongo-query-filters-2.mmd b/docs/diagrams/architecture/data-mongo-query-filters-2.mmd deleted file mode 100644 index 631c49dee..000000000 --- a/docs/diagrams/architecture/data-mongo-query-filters-2.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Filter["EventQueryFilter"] --> ByUser["userIds"] - Filter --> ByType["eventTypes"] - Filter --> ByStart["startDate"] - Filter --> ByEnd["endDate"] diff --git a/docs/diagrams/architecture/data-mongo-query-filters-3.mmd b/docs/diagrams/architecture/data-mongo-query-filters-3.mmd deleted file mode 100644 index af95d5ac1..000000000 --- a/docs/diagrams/architecture/data-mongo-query-filters-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - OrgFilter["OrganizationQueryFilter"] --> SizeRange["Employee Range"] - OrgFilter --> Contract["Active Contract"] - OrgFilter --> Status["Organization Status"] diff --git a/docs/diagrams/architecture/data-mongo-query-filters-4.mmd b/docs/diagrams/architecture/data-mongo-query-filters-4.mmd deleted file mode 100644 index 2e3aebcb4..000000000 --- a/docs/diagrams/architecture/data-mongo-query-filters-4.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - TicketFilter["TicketQueryFilter"] --> Statuses["TicketStatus List"] - TicketFilter --> OrgScope["Organization IDs"] - TicketFilter --> Assignees["Assignee IDs"] - TicketFilter --> Labels["Label IDs"] - TicketFilter --> Devices["Device IDs"] - TicketFilter --> Source["Creation Sources"] - TicketFilter --> TimeFrom["createdAtFrom"] - TicketFilter --> TimeTo["createdAtTo"] diff --git a/docs/diagrams/architecture/data-mongo-query-filters-5.mmd b/docs/diagrams/architecture/data-mongo-query-filters-5.mmd deleted file mode 100644 index 3e677a756..000000000 --- a/docs/diagrams/architecture/data-mongo-query-filters-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - UserFilter["UserQueryFilter"] --> Email["emailRegex"] - UserFilter --> Name["nameRegex"] - UserFilter --> Status["UserStatus"] diff --git a/docs/diagrams/architecture/data-mongo-query-filters-6.mmd b/docs/diagrams/architecture/data-mongo-query-filters-6.mmd deleted file mode 100644 index 17d98509a..000000000 --- a/docs/diagrams/architecture/data-mongo-query-filters-6.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - ServiceLayer["Service Layer"] --> QueryFilter["Query Filter"] - QueryFilter --> CustomRepository["Custom Repository Impl"] - CustomRepository --> CriteriaBuilder["Mongo Criteria Builder"] - CriteriaBuilder --> Mongo[("MongoDB Collection")] diff --git a/docs/diagrams/architecture/data-mongo-query-filters.mmd b/docs/diagrams/architecture/data-mongo-query-filters.mmd deleted file mode 100644 index 8dc0be82a..000000000 --- a/docs/diagrams/architecture/data-mongo-query-filters.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart LR - ApiLayer["API Layer\nREST + GraphQL"] --> FilterDTOs["Filter Input DTOs"] - FilterDTOs --> QueryFilters["Data Mongo Query Filters"] - QueryFilters --> CustomRepos["Custom Mongo Repositories"] - CustomRepos --> MongoDB[("MongoDB")] - - DomainModel["Domain Documents"] --> CustomRepos diff --git a/docs/diagrams/architecture/data-mongo-reactive-repositories-2.mmd b/docs/diagrams/architecture/data-mongo-reactive-repositories-2.mmd deleted file mode 100644 index 14dd06f78..000000000 --- a/docs/diagrams/architecture/data-mongo-reactive-repositories-2.mmd +++ /dev/null @@ -1,9 +0,0 @@ -sequenceDiagram - participant Service as "Auth Service" - participant Repo as "ReactiveOAuthClientRepository" - participant DB as "MongoDB" - - Service->>Repo: findByClientId(clientId) - Repo->>DB: Reactive query - DB-->>Repo: OAuthClient document - Repo-->>Service: Mono diff --git a/docs/diagrams/architecture/data-mongo-reactive-repositories-3.mmd b/docs/diagrams/architecture/data-mongo-reactive-repositories-3.mmd deleted file mode 100644 index 6c8b5f7c2..000000000 --- a/docs/diagrams/architecture/data-mongo-reactive-repositories-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - BaseRepo["BaseUserRepository"] --> ReactiveRepo["ReactiveUserRepository"] - ReactiveMongo["ReactiveMongoRepository"] --> ReactiveRepo - ReactiveRepo --> UserDoc["User Document"] diff --git a/docs/diagrams/architecture/data-mongo-reactive-repositories.mmd b/docs/diagrams/architecture/data-mongo-reactive-repositories.mmd deleted file mode 100644 index 86ebbcd15..000000000 --- a/docs/diagrams/architecture/data-mongo-reactive-repositories.mmd +++ /dev/null @@ -1,12 +0,0 @@ -flowchart TD - App["Application Context"] --> Config["MongoReactiveConfig"] - Config -->|"@EnableReactiveMongoRepositories"| Scan["Reactive Repository Scan"] - - Scan --> OAuthRepo["ReactiveOAuthClientRepository"] - Scan --> UserRepo["ReactiveUserRepository"] - - OAuthRepo --> OAuthClient["OAuthClient Document"] - UserRepo --> UserDoc["User Document"] - - OAuthClient --> MongoDB[("MongoDB")] - UserDoc --> MongoDB diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-2.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-2.mmd deleted file mode 100644 index 82524c587..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-2.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Factory["MongoDatabaseFactory"] --> Resolver["DefaultDbRefResolver"] - Resolver --> Converter["MappingMongoConverter"] - Conversions["MongoCustomConversions"] --> Converter diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-3.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-3.mmd deleted file mode 100644 index 8150f2900..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Startup["Application Startup"] --> EnsureIndexes["Ensure Required Indexes"] - EnsureIndexes --> DropLegacy["Drop Stale Indexes"] - DropLegacy --> LogResult["Log Success Or Skip"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-4.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-4.mmd deleted file mode 100644 index d94e29831..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-4.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - ClientRequest["Client Request With Cursor"] --> ParseCursor["Parse ObjectId"] - ParseCursor --> LoadCursorDoc["Load Cursor Document"] - LoadCursorDoc --> CompareSortField["Compare Sort Field"] - CompareSortField --> AddCriteria["Add $or Criteria"] - AddCriteria --> ApplySort["Sort By Field And _id"] - ApplySort --> Limit["Apply Limit"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-5.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-5.mmd deleted file mode 100644 index c303aee69..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Match["Match itemId"] --> Group["Group By targetType"] - Group --> Count["Count"] - Count --> MapResult["Map To EnumMap"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-6.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-6.mmd deleted file mode 100644 index 093df5ca8..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-6.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - SearchCriteria["Search OR Criteria"] --> Combine - CursorCriteria["Cursor OR Criteria"] --> Combine - Combine["Combine Using $and"] --> FinalQuery["Final Query"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-7.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-7.mmd deleted file mode 100644 index a321e8c2d..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-7.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - FetchReadStates["Fetch NotificationReadState"] --> ExtractIds - ExtractIds["Extract Notification IDs"] --> FetchNotifications - FetchNotifications["Fetch Notification Documents"] --> Merge["Merge With Status"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-8.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-8.mmd deleted file mode 100644 index fb0f8a57d..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-8.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart LR - MatchResolved["Match Resolved Tickets"] --> ProjectDiff["Compute resolvedAt - createdAt"] - ProjectDiff --> GroupAvg["Average Resolution Time"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-9.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-9.mmd deleted file mode 100644 index 306ed7466..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-9.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - RetryStart["Retry Attempt"] --> OnError["onError()"] - OnError --> LogWarn["Log Warning"] - RetryEnd["Retry Complete"] --> Close - Close --> SuccessOrFail["Log Success Or Error"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories.mmd deleted file mode 100644 index b4114df5b..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - ApiLayer["API Service Layer"] --> RepositoryLayer["Custom Repositories"] - RepositoryLayer --> MongoTemplate["MongoTemplate"] - MongoTemplate --> MongoDB[("MongoDB")] - - DomainModel["Domain Documents"] --> RepositoryLayer - QueryFilters["Query Filter DTOs"] --> RepositoryLayer - - Config["Mongo Sync Config"] --> RepositoryLayer - IndexConfig["Mongo Index Config"] --> MongoDB diff --git a/docs/diagrams/architecture/data-nats-notifications-2.mmd b/docs/diagrams/architecture/data-nats-notifications-2.mmd deleted file mode 100644 index 1f9f77542..000000000 --- a/docs/diagrams/architecture/data-nats-notifications-2.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - Start["Broadcast Request"] --> Validate["Validate NotificationCommand"] - Validate --> Persist["Persist Notification"] - Persist --> CreateReadStates["Create Read States"] - CreateReadStates --> Publish["Publish to NATS"] - Publish --> End["Complete"] - - CreateReadStates -->|"failure"| Rollback["Delete Notification"] diff --git a/docs/diagrams/architecture/data-nats-notifications-3.mmd b/docs/diagrams/architecture/data-nats-notifications-3.mmd deleted file mode 100644 index bd2f7d461..000000000 --- a/docs/diagrams/architecture/data-nats-notifications-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - Cmd["NotificationCommand"] --> Build["Build Notification"] - Build --> Save["Save to Repository"] - Save --> Category["Resolve Category"] - Category --> RS["Create Read States"] - RS --> Publish["Publish Per Recipient"] diff --git a/docs/diagrams/architecture/data-nats-notifications-4.mmd b/docs/diagrams/architecture/data-nats-notifications-4.mmd deleted file mode 100644 index 07c44667a..000000000 --- a/docs/diagrams/architecture/data-nats-notifications-4.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - Doc["Notification Document"] --> Msg["NotificationMessage"] - - Msg --> Id["id"] - Msg --> Severity["severity"] - Msg --> Title["title"] - Msg --> Desc["description"] - Msg --> Created["createdAt"] - Msg --> Context["context"] diff --git a/docs/diagrams/architecture/data-nats-notifications-5.mmd b/docs/diagrams/architecture/data-nats-notifications-5.mmd deleted file mode 100644 index 4dbe3a60f..000000000 --- a/docs/diagrams/architecture/data-nats-notifications-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Persist["Persist Notification"] --> Durable["Durable Storage"] - Durable --> RealTime["Real-Time Publish"] - - RealTime -->|"failure"| CatchUp["Client Catch-Up via API"] diff --git a/docs/diagrams/architecture/data-nats-notifications-6.mmd b/docs/diagrams/architecture/data-nats-notifications-6.mmd deleted file mode 100644 index c34ee6d92..000000000 --- a/docs/diagrams/architecture/data-nats-notifications-6.mmd +++ /dev/null @@ -1,13 +0,0 @@ -sequenceDiagram - participant Service - participant Broadcaster - participant Mongo - participant NATS - participant Client - - Service->>Broadcaster: broadcast(command) - Broadcaster->>Mongo: save(notification) - Broadcaster->>Mongo: createReadStates() - Broadcaster->>NATS: publish(subject, message) - NATS->>Client: deliver real-time event - Client->>Mongo: query for reconciliation diff --git a/docs/diagrams/architecture/data-nats-notifications.mmd b/docs/diagrams/architecture/data-nats-notifications.mmd deleted file mode 100644 index 15eddc886..000000000 --- a/docs/diagrams/architecture/data-nats-notifications.mmd +++ /dev/null @@ -1,13 +0,0 @@ -flowchart TD - Caller["Application Service"] --> Command["NotificationCommand"] - Command --> Broadcaster["NotificationBroadcaster"] - - Broadcaster --> Repo["NotificationRepository"] - Broadcaster --> ReadState["NotificationReadStateService"] - Broadcaster --> Registry["NotificationContextDescriptorRegistry"] - - Broadcaster -->|"optional"| Publisher["NotificationNatsPublisher"] - Publisher --> Nats["NATS Server"] - - Repo --> Mongo[("MongoDB")] - ReadState --> Mongo diff --git a/docs/diagrams/architecture/data-pinot-repositories-2.mmd b/docs/diagrams/architecture/data-pinot-repositories-2.mmd deleted file mode 100644 index d1b088f2f..000000000 --- a/docs/diagrams/architecture/data-pinot-repositories-2.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - AppConfig["Spring Context"] --> PinotConfig["PinotConfig"] - PinotConfig --> BrokerConn["pinotBrokerConnection()"] - PinotConfig --> ControllerConn["pinotControllerConnection()"] - BrokerConn --> PinotCluster["Pinot Broker"] - ControllerConn --> PinotController["Pinot Controller"] diff --git a/docs/diagrams/architecture/data-pinot-repositories-3.mmd b/docs/diagrams/architecture/data-pinot-repositories-3.mmd deleted file mode 100644 index f1a0d62fd..000000000 --- a/docs/diagrams/architecture/data-pinot-repositories-3.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - Start["Facet Request"] --> BuildQuery["PinotQueryBuilder"] - BuildQuery --> ApplyFilters["applyDeviceFilters()"] - ApplyFilters --> ExcludeField{"Exclude Facet Field?"} - ExcludeField -->|"Yes"| SkipFilter["Skip That Filter"] - ExcludeField -->|"No"| ApplyFilter["Apply Filter"] - SkipFilter --> GroupBy["GROUP BY facetField"] - ApplyFilter --> GroupBy - GroupBy --> Execute["executeKeyCountQuery()"] diff --git a/docs/diagrams/architecture/data-pinot-repositories-4.mmd b/docs/diagrams/architecture/data-pinot-repositories-4.mmd deleted file mode 100644 index 45bd4e272..000000000 --- a/docs/diagrams/architecture/data-pinot-repositories-4.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart LR - Request["Log Query Request"] --> Builder["PinotQueryBuilder"] - Builder --> DateRange["whereDateRange()"] - DateRange --> Filters["whereIn() / whereEquals()"] - Filters --> Search["whereRelevanceLogSearch()"] - Search --> Cursor["whereCursor()"] - Cursor --> Sorting["orderBySortInput()"] - Sorting --> Limit["limit()"] - Limit --> Execute["executeLogQuery()"] diff --git a/docs/diagrams/architecture/data-pinot-repositories-5.mmd b/docs/diagrams/architecture/data-pinot-repositories-5.mmd deleted file mode 100644 index 2868297f8..000000000 --- a/docs/diagrams/architecture/data-pinot-repositories-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Query["Incoming Query"] --> TenantFilter["Apply tenantId"] - TenantFilter --> PinotExec["Execute on Shared Table"] - PinotExec --> TenantScoped["Tenant Scoped Result"] diff --git a/docs/diagrams/architecture/data-pinot-repositories-6.mmd b/docs/diagrams/architecture/data-pinot-repositories-6.mmd deleted file mode 100644 index 263641427..000000000 --- a/docs/diagrams/architecture/data-pinot-repositories-6.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart LR - Client["UI / API Request"] --> ApiService["API Service"] - ApiService --> LogRepo["PinotClientLogRepository"] - LogRepo --> PinotBroker["Pinot Broker"] - PinotBroker --> PinotServer["Pinot Servers"] - PinotServer --> PinotBroker - PinotBroker --> LogRepo - LogRepo --> ApiService - ApiService --> Client diff --git a/docs/diagrams/architecture/data-pinot-repositories.mmd b/docs/diagrams/architecture/data-pinot-repositories.mmd deleted file mode 100644 index 5bedb0d88..000000000 --- a/docs/diagrams/architecture/data-pinot-repositories.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - StreamService["Stream Service Core Kafka And Handlers"] -->|"Ingest Events"| PinotCluster["Apache Pinot Cluster"] - MongoDB["Data Mongo Domain Model"] -->|"Transactional Data"| Services["API & Management Services"] - PinotCluster -->|"Analytical Queries"| PinotRepos["Data Pinot Repositories"] - PinotRepos -->|"Facet Counts & Logs"| Services diff --git a/docs/diagrams/architecture/data-redis-cache-2.mmd b/docs/diagrams/architecture/data-redis-cache-2.mmd deleted file mode 100644 index 03a831121..000000000 --- a/docs/diagrams/architecture/data-redis-cache-2.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Start["Application Startup"] --> RedisEnabled{"Redis Enabled?"} - RedisEnabled -->|Yes| CreateManager["Create RedisCacheManager"] - RedisEnabled -->|No| Skip["Skip Redis Cache Config"] - CreateManager --> DefaultConfig["Apply Default TTL 6h"] - DefaultConfig --> FleetOverride["Override Fleet TTL 1h"] - FleetOverride --> Ready["CacheManager Ready"] diff --git a/docs/diagrams/architecture/data-redis-cache-3.mmd b/docs/diagrams/architecture/data-redis-cache-3.mmd deleted file mode 100644 index 4ce667c3d..000000000 --- a/docs/diagrams/architecture/data-redis-cache-3.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart LR - TenantA["Tenant A"] --> KeyBuilder - TenantB["Tenant B"] --> KeyBuilder - KeyBuilder --> RedisKeyA["tenantA:deviceCache::123"] - KeyBuilder --> RedisKeyB["tenantB:deviceCache::123"] - RedisKeyA --> RedisServer[("Redis")] - RedisKeyB --> RedisServer diff --git a/docs/diagrams/architecture/data-redis-cache.mmd b/docs/diagrams/architecture/data-redis-cache.mmd deleted file mode 100644 index e90265500..000000000 --- a/docs/diagrams/architecture/data-redis-cache.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - AppServices["Application Services"] --> CacheAbstraction["Spring Cache Abstraction"] - CacheAbstraction --> CacheManager["RedisCacheManager"] - CacheManager --> RedisConnection["RedisConnectionFactory"] - CacheManager --> KeyBuilder["OpenframeRedisKeyBuilder"] - RedisConnection --> RedisServer[("Redis Server")] - - ReactiveServices["Reactive Services"] --> ReactiveTemplate["ReactiveRedisTemplate"] - ReactiveTemplate --> ReactiveConnection["ReactiveRedisConnectionFactory"] - ReactiveConnection --> RedisServer diff --git a/docs/diagrams/architecture/eventing-and-messaging-kafka-nats-2.mmd b/docs/diagrams/architecture/eventing-and-messaging-kafka-nats-2.mmd new file mode 100644 index 000000000..f5cbf2705 --- /dev/null +++ b/docs/diagrams/architecture/eventing-and-messaging-kafka-nats-2.mmd @@ -0,0 +1,8 @@ +flowchart TD + Properties["OssTenantKafkaProperties"] --> ProducerFactory["ProducerFactory"] + Properties --> ConsumerFactory["ConsumerFactory"] + Properties --> KafkaAdmin["KafkaAdmin"] + + ProducerFactory --> KafkaTemplate["KafkaTemplate"] + ConsumerFactory --> ListenerFactory["ListenerContainerFactory"] + KafkaTemplate --> Producer["OssTenantKafkaProducer"] diff --git a/docs/diagrams/architecture/eventing-and-messaging-kafka-nats-3.mmd b/docs/diagrams/architecture/eventing-and-messaging-kafka-nats-3.mmd new file mode 100644 index 000000000..6588bac72 --- /dev/null +++ b/docs/diagrams/architecture/eventing-and-messaging-kafka-nats-3.mmd @@ -0,0 +1,6 @@ +flowchart TD + Root["DebeziumMessage"] --> Payload["Payload"] + Payload --> Before["before"] + Payload --> After["after"] + Payload --> Source["source"] + Payload --> Operation["operation"] diff --git a/docs/diagrams/architecture/eventing-and-messaging-kafka-nats-4.mmd b/docs/diagrams/architecture/eventing-and-messaging-kafka-nats-4.mmd new file mode 100644 index 000000000..6bb7fb76d --- /dev/null +++ b/docs/diagrams/architecture/eventing-and-messaging-kafka-nats-4.mmd @@ -0,0 +1,6 @@ +flowchart LR + Service["Business Service"] --> Broadcaster["NotificationBroadcaster"] + Broadcaster --> Repository["NotificationRepository"] + Broadcaster --> NatsPublisher["NotificationNatsPublisher"] + NatsPublisher --> UserSubject["user.{id}.notification"] + NatsPublisher --> MachineSubject["machine.{id}.notification"] diff --git a/docs/diagrams/architecture/eventing-and-messaging-kafka-nats.mmd b/docs/diagrams/architecture/eventing-and-messaging-kafka-nats.mmd new file mode 100644 index 000000000..97c1bd2ce --- /dev/null +++ b/docs/diagrams/architecture/eventing-and-messaging-kafka-nats.mmd @@ -0,0 +1,10 @@ +flowchart LR + ApiServices["API Services"] -->|"Publish Events"| KafkaLayer["Kafka Cluster"] + KafkaLayer -->|"Stream Processing"| StreamCore["Stream Processing Core"] + StreamCore -->|"Enriched Events"| Analytics["Analytics Pinot"] + + ApiServices -->|"Commands / Notifications"| NatsLayer["NATS Cluster"] + NatsLayer --> Agents["Connected Agents"] + NatsLayer --> Clients["User Clients"] + + Management["Management Services"] -->|"Topic Init"| KafkaLayer diff --git a/docs/diagrams/architecture/external-api-service-core-2.mmd b/docs/diagrams/architecture/external-api-service-core-2.mmd index f12ed4f87..49669bd12 100644 --- a/docs/diagrams/architecture/external-api-service-core-2.mmd +++ b/docs/diagrams/architecture/external-api-service-core-2.mmd @@ -1,7 +1,12 @@ -flowchart TD - Request["GET /api/v1/devices"] --> Criteria["DeviceFilterCriteria"] - Criteria --> Pagination["CursorPaginationCriteria"] - Pagination --> Query["DeviceService.queryDevices()"] - Query --> Result["Query Result"] - Result --> Mapper["DeviceMapper"] - Mapper --> Response["DevicesResponse"] +sequenceDiagram + participant Client + participant Gateway + participant ExternalApi as ExternalApiService + participant Services + + Client->>Gateway: Request + X-API-Key + Gateway->>ExternalApi: Forward validated request + ExternalApi->>Services: Execute business logic + Services-->>ExternalApi: Domain result + ExternalApi-->>Gateway: REST response + Gateway-->>Client: HTTP response diff --git a/docs/diagrams/architecture/external-api-service-core-3.mmd b/docs/diagrams/architecture/external-api-service-core-3.mmd index ab70d3f39..a6fe60753 100644 --- a/docs/diagrams/architecture/external-api-service-core-3.mmd +++ b/docs/diagrams/architecture/external-api-service-core-3.mmd @@ -1,5 +1,6 @@ -flowchart TD - EventRequest["GET /api/v1/events"] --> Filter["EventFilterCriteria"] - Filter --> Service["EventService.queryEvents()"] - Service --> Mapper["EventMapper"] - Mapper --> EventsResponse["EventsResponse"] +flowchart LR + Request["GET /api/v1/devices"] --> Filter["DeviceFilterCriteria"] + Filter --> Service["DeviceService.queryDevices()"] + Service --> Result["Paged Machine Result"] + Result --> Mapper["DeviceMapper"] + Mapper --> Response["DevicesResponse"] diff --git a/docs/diagrams/architecture/external-api-service-core-4.mmd b/docs/diagrams/architecture/external-api-service-core-4.mmd index 67917cb93..35ecbb220 100644 --- a/docs/diagrams/architecture/external-api-service-core-4.mmd +++ b/docs/diagrams/architecture/external-api-service-core-4.mmd @@ -1,4 +1,9 @@ -flowchart LR - OrgRequest["Organization Request"] --> QueryService["OrganizationQueryService"] - OrgRequest --> CommandService["OrganizationCommandService"] - CommandService --> Validation["Archive Rules"] +flowchart TD + Incoming["/tools/{toolId}/..."] --> Lookup["Find IntegratedTool"] + Lookup --> Enabled{"Enabled?"} + Enabled -->|No| Reject["400 / 404"] + Enabled -->|Yes| ResolveUrl["Resolve ToolUrl (API)"] + ResolveUrl --> BuildHeaders["Attach APIKey / Bearer"] + BuildHeaders --> Rewrite["ProxyUrlResolver"] + Rewrite --> HttpCall["Execute HTTP Request"] + HttpCall --> Return["Return ResponseEntity"] diff --git a/docs/diagrams/architecture/external-api-service-core-5.mmd b/docs/diagrams/architecture/external-api-service-core-5.mmd index 821c7120d..0232231be 100644 --- a/docs/diagrams/architecture/external-api-service-core-5.mmd +++ b/docs/diagrams/architecture/external-api-service-core-5.mmd @@ -1,6 +1,5 @@ -flowchart TD - Client["External Client"] --> ProxyController["IntegrationController"] - ProxyController --> RestProxyService["RestProxyService"] - RestProxyService --> ToolRepo["IntegratedToolRepository"] - RestProxyService --> Resolver["ProxyUrlResolver"] - Resolver --> TargetTool["Integrated Tool API"] +flowchart LR + Client["cursor + limit"] --> CursorCriteria["CursorPaginationCriteria.fromRest()"] + CursorCriteria --> Service + Service --> PageInfo + PageInfo --> Response diff --git a/docs/diagrams/architecture/external-api-service-core.mmd b/docs/diagrams/architecture/external-api-service-core.mmd index 1e702571b..0bea28d89 100644 --- a/docs/diagrams/architecture/external-api-service-core.mmd +++ b/docs/diagrams/architecture/external-api-service-core.mmd @@ -1,16 +1,22 @@ -flowchart LR - Client["External Client"] -->|"X-API-Key"| ExternalAPI["External Api Service Core"] +flowchart TD + Client["External Client / Integration"] -->|"X-API-Key"| Gateway["Gateway Service Core"] + Gateway --> ExternalApi["External Api Service Core"] - ExternalAPI --> DeviceService["Device Service"] - ExternalAPI --> EventService["Event Service"] - ExternalAPI --> LogService["Log Service"] - ExternalAPI --> OrganizationService["Organization Services"] - ExternalAPI --> ToolService["Tool Service"] + subgraph rest_layer["REST Controllers"] + DeviceCtrl["DeviceController"] + EventCtrl["EventController"] + LogCtrl["LogController"] + OrgCtrl["OrganizationController"] + ToolCtrl["ToolController"] + IntegrationCtrl["IntegrationController"] + end - ExternalAPI --> ProxyService["Rest Proxy Service"] - ProxyService --> IntegratedTool["Integrated Tool"] + ExternalApi --> rest_layer - DeviceService --> MongoDB[("MongoDB")] - EventService --> MongoDB - LogService --> Pinot[("Apache Pinot")] - ToolService --> MongoDB + rest_layer --> Services["Domain Services Layer"] + Services --> Mongo["Mongo Data Access"] + Services --> Pinot["Analytics (Pinot)"] + Services --> Kafka["Stream & Eventing"] + + IntegrationCtrl --> RestProxy["RestProxyService"] + RestProxy --> IntegratedTools["Integrated Tool APIs"] diff --git a/docs/diagrams/architecture/gateway-service-core-2.mmd b/docs/diagrams/architecture/gateway-service-core-2.mmd new file mode 100644 index 000000000..c688e5e57 --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core-2.mmd @@ -0,0 +1,9 @@ +flowchart TD + Netty["Netty & WebClient Config"] --> Security + Security["Security & JWT Layer"] --> Filters + Filters["Global & Web Filters"] --> Controllers + Controllers["REST Controllers"] --> Routing + Routing["WebSocket & Route Locator"] --> Upstream + Upstream["Tool Upstream Resolvers"] --> ExternalSystems + + ExternalSystems["Tools / NATS / Internal Services"] diff --git a/docs/diagrams/architecture/gateway-service-core-3.mmd b/docs/diagrams/architecture/gateway-service-core-3.mmd new file mode 100644 index 000000000..71b3b9b1b --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core-3.mmd @@ -0,0 +1,6 @@ +flowchart TD + Request["Incoming Request"] --> ExtractIssuer["Extract iss claim"] + ExtractIssuer --> Cache["Caffeine Cache"] + Cache --> Manager["ReactiveAuthenticationManager"] + Manager --> Decoder["NimbusReactiveJwtDecoder"] + Decoder --> Validate["Strict Issuer Validation"] diff --git a/docs/diagrams/architecture/gateway-service-core-4.mmd b/docs/diagrams/architecture/gateway-service-core-4.mmd new file mode 100644 index 000000000..91d07adb4 --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core-4.mmd @@ -0,0 +1,9 @@ +flowchart TD + Req["External API Request"] --> HasKey{"API Key Present?"} + HasKey -->|No| Reject["401 UNAUTHORIZED"] + HasKey -->|Yes| Validate["Validate API Key"] + Validate -->|Invalid| Reject + Validate -->|Valid| Rate{"Rate Limit OK?"} + Rate -->|No| Limit["429 RATE_LIMIT_EXCEEDED"] + Rate -->|Yes| AddHeaders["Add Context + Rate Headers"] + AddHeaders --> Continue["Forward to External API"] diff --git a/docs/diagrams/architecture/gateway-service-core-5.mmd b/docs/diagrams/architecture/gateway-service-core-5.mmd new file mode 100644 index 000000000..c41803054 --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core-5.mmd @@ -0,0 +1,9 @@ +flowchart LR + Client["WebSocket Client"] --> GatewayWS["WebSocket Gateway"] + + GatewayWS --> ToolApiFilter + GatewayWS --> ToolAgentFilter + GatewayWS --> NatsRoute + + ToolApiFilter --> UpstreamResolver + ToolAgentFilter --> UpstreamResolver diff --git a/docs/diagrams/architecture/gateway-service-core-6.mmd b/docs/diagrams/architecture/gateway-service-core-6.mmd new file mode 100644 index 000000000..f094d8119 --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core-6.mmd @@ -0,0 +1,9 @@ +flowchart TD + Request --> Registry["ToolUpstreamResolverRegistry"] + Registry -->|MeshCentral| MeshResolver + Registry -->|Tactical RMM| TacticalResolver + Registry -->|Other| DefaultResolver + + MeshResolver --> ProxyUrlResolver + TacticalResolver --> ProxyUrlResolver + DefaultResolver --> ToolUrlService diff --git a/docs/diagrams/architecture/gateway-service-core-7.mmd b/docs/diagrams/architecture/gateway-service-core-7.mmd new file mode 100644 index 000000000..92ec4eb40 --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core-7.mmd @@ -0,0 +1,9 @@ +flowchart TD + Client --> OriginFilter + OriginFilter --> AddAuthHeader + AddAuthHeader --> JwtAuth + JwtAuth --> RoleCheck + RoleCheck --> Controller + Controller --> UpstreamResolver + UpstreamResolver --> Proxy + Proxy --> ToolService diff --git a/docs/diagrams/architecture/gateway-service-core-8.mmd b/docs/diagrams/architecture/gateway-service-core-8.mmd new file mode 100644 index 000000000..8d669e60b --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core-8.mmd @@ -0,0 +1,5 @@ +flowchart TD + Client --> ApiKeyFilter + ApiKeyFilter --> RateLimit + RateLimit --> ContextHeaders + ContextHeaders --> ExternalApiService diff --git a/docs/diagrams/architecture/gateway-service-core-9.mmd b/docs/diagrams/architecture/gateway-service-core-9.mmd new file mode 100644 index 000000000..59747ae95 --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core-9.mmd @@ -0,0 +1,6 @@ +flowchart TD + Client --> SecurityDecorator + SecurityDecorator --> WsRoute + WsRoute --> UrlFilter + UrlFilter --> UpstreamResolver + UpstreamResolver --> WebSocketProxy diff --git a/docs/diagrams/architecture/gateway-service-core-security-and-routing-2.mmd b/docs/diagrams/architecture/gateway-service-core-security-and-routing-2.mmd deleted file mode 100644 index 1fb4bf101..000000000 --- a/docs/diagrams/architecture/gateway-service-core-security-and-routing-2.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - Request["Incoming Request"] --> Origin["OriginSanitizerFilter"] - Origin --> AuthHeader["AddAuthorizationHeaderFilter"] - AuthHeader --> JwtValidation["JWT Authentication"] - JwtValidation --> RoleCheck["Role Authorization Rules"] - RoleCheck --> ApiKeyCheck{"External API Path?"} - ApiKeyCheck -->|Yes| ApiKeyFilter["ApiKeyAuthenticationFilter"] - ApiKeyCheck -->|No| Continue["Continue Routing"] - ApiKeyFilter --> Continue - Continue --> Controller["Controller / Route"] diff --git a/docs/diagrams/architecture/gateway-service-core-security-and-routing-3.mmd b/docs/diagrams/architecture/gateway-service-core-security-and-routing-3.mmd deleted file mode 100644 index 8a4cf108f..000000000 --- a/docs/diagrams/architecture/gateway-service-core-security-and-routing-3.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - Req["/external-api/**"] --> CheckKey["Check X-API-Key Header"] - CheckKey --> Valid{"Valid Key?"} - Valid -->|No| Reject401["Return 401"] - Valid -->|Yes| RateCheck["RateLimitService.isAllowed"] - RateCheck --> Allowed{"Allowed?"} - Allowed -->|No| Reject429["Return 429 + Retry-After"] - Allowed -->|Yes| AddHeaders["Add RateLimit Headers"] - AddHeaders --> AddContext["Inject X-User-Id + X-Api-Key-Id"] - AddContext --> Forward["Forward to External API"] diff --git a/docs/diagrams/architecture/gateway-service-core-security-and-routing-4.mmd b/docs/diagrams/architecture/gateway-service-core-security-and-routing-4.mmd deleted file mode 100644 index dda93d4dd..000000000 --- a/docs/diagrams/architecture/gateway-service-core-security-and-routing-4.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart LR - Request["Tool Request"] --> Registry["ToolUpstreamResolverRegistry"] - Registry -->|Specific Tool| Mesh["MeshCentralUpstreamResolver"] - Registry -->|Specific Tool| Tactical["TacticalRmmUpstreamResolver"] - Registry -->|Fallback| Default["DefaultToolUpstreamResolver"] - Mesh --> Proxy - Tactical --> Proxy - Default --> Proxy diff --git a/docs/diagrams/architecture/gateway-service-core-security-and-routing-5.mmd b/docs/diagrams/architecture/gateway-service-core-security-and-routing-5.mmd deleted file mode 100644 index 44f992bcd..000000000 --- a/docs/diagrams/architecture/gateway-service-core-security-and-routing-5.mmd +++ /dev/null @@ -1,12 +0,0 @@ -flowchart TD - Client --> Gateway - Gateway --> OriginFilter - OriginFilter --> AuthHeader - AuthHeader --> JwtResolver - JwtResolver --> RoleAuth - RoleAuth --> RouteDecision - RouteDecision -->|REST| IntegrationController - RouteDecision -->|WebSocket| WebSocketGatewayConfig - IntegrationController --> UpstreamResolver - WebSocketGatewayConfig --> UpstreamResolver - UpstreamResolver --> IntegratedTool diff --git a/docs/diagrams/architecture/gateway-service-core-security-and-routing.mmd b/docs/diagrams/architecture/gateway-service-core-security-and-routing.mmd deleted file mode 100644 index 7b597a83e..000000000 --- a/docs/diagrams/architecture/gateway-service-core-security-and-routing.mmd +++ /dev/null @@ -1,20 +0,0 @@ -flowchart TD - Client["Client / Agent / UI"] --> Gateway["Gateway Service Core Security And Routing"] - - subgraph security_layer["Security Layer"] - AuthHeader["AddAuthorizationHeaderFilter"] - JwtConfig["JwtAuthConfig"] - SecurityConfig["GatewaySecurityConfig"] - ApiKeyFilter["ApiKeyAuthenticationFilter"] - OriginFilter["OriginSanitizerFilter"] - end - - subgraph routing_layer["Routing & Proxy Layer"] - IntegrationCtrl["IntegrationController"] - WsConfig["WebSocketGatewayConfig"] - UpstreamResolvers["ToolUpstreamResolvers"] - end - - Gateway --> security_layer - security_layer --> routing_layer - routing_layer --> Upstream["Integrated Tools / NATS / Services"] diff --git a/docs/diagrams/architecture/gateway-service-core.mmd b/docs/diagrams/architecture/gateway-service-core.mmd new file mode 100644 index 000000000..d3c1fca39 --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core.mmd @@ -0,0 +1,19 @@ +flowchart LR + Client["Browser / Agent / External API Client"] --> Gateway["Gateway Service Core"] + + Gateway --> ApiService["API Service Core"] + Gateway --> ExternalApi["External API Service Core"] + Gateway --> Authz["Authorization Service Core"] + Gateway --> Mesh["MeshCentral"] + Gateway --> Tactical["Tactical RMM"] + Gateway --> Nats["NATS WebSocket"] + + subgraph security["Security Layer"] + Jwt["JWT Validation"] + ApiKey["API Key Auth + Rate Limit"] + Roles["Role Based Access"] + end + + Gateway --> Jwt + Gateway --> ApiKey + Gateway --> Roles diff --git a/docs/diagrams/architecture/integrations-sdks-2.mmd b/docs/diagrams/architecture/integrations-sdks-2.mmd new file mode 100644 index 000000000..137f2e546 --- /dev/null +++ b/docs/diagrams/architecture/integrations-sdks-2.mmd @@ -0,0 +1,3 @@ +flowchart TD + AutoConfig["TacticalRmmConfig"] --> Bean["TacticalRmmClient Bean"] + Bean --> Services["Management or Gateway Services"] diff --git a/docs/diagrams/architecture/integrations-sdks-3.mmd b/docs/diagrams/architecture/integrations-sdks-3.mmd new file mode 100644 index 000000000..e411cf879 --- /dev/null +++ b/docs/diagrams/architecture/integrations-sdks-3.mmd @@ -0,0 +1,6 @@ +sequenceDiagram + participant Service + participant TacticalApi as "Tactical RMM API" + + Service->>TacticalApi: RunScriptRequest + TacticalApi-->>Service: Script execution result diff --git a/docs/diagrams/architecture/integrations-sdks-4.mmd b/docs/diagrams/architecture/integrations-sdks-4.mmd new file mode 100644 index 000000000..a19e9f100 --- /dev/null +++ b/docs/diagrams/architecture/integrations-sdks-4.mmd @@ -0,0 +1,3 @@ +flowchart TD + Command["Install Command String"] --> Parser["RegistrationSecretParser"] + Parser --> Secret["Extracted Auth Secret"] diff --git a/docs/diagrams/architecture/integrations-sdks-5.mmd b/docs/diagrams/architecture/integrations-sdks-5.mmd new file mode 100644 index 000000000..34db61da1 --- /dev/null +++ b/docs/diagrams/architecture/integrations-sdks-5.mmd @@ -0,0 +1,4 @@ +flowchart TD + Service["OpenFrame Service"] --> FleetApi["Fleet MDM API"] + FleetApi --> Response["HostSearchResponse"] + Response --> Hosts["List of Host"] diff --git a/docs/diagrams/architecture/integrations-sdks-6.mmd b/docs/diagrams/architecture/integrations-sdks-6.mmd new file mode 100644 index 000000000..8e6e111f1 --- /dev/null +++ b/docs/diagrams/architecture/integrations-sdks-6.mmd @@ -0,0 +1,7 @@ +sequenceDiagram + participant Service + participant FleetApi as "Fleet MDM API" + + Service->>FleetApi: RunLiveQueryRequest + FleetApi-->>Service: LiveQueryCampaign + Note over FleetApi: Results streamed asynchronously diff --git a/docs/diagrams/architecture/integrations-sdks-7.mmd b/docs/diagrams/architecture/integrations-sdks-7.mmd new file mode 100644 index 000000000..19c0aa57e --- /dev/null +++ b/docs/diagrams/architecture/integrations-sdks-7.mmd @@ -0,0 +1,6 @@ +flowchart TD + ManagementService["Management Service"] --> TacticalClient["TacticalRmmClient"] + TacticalClient --> CreateSchedule["CreateScriptScheduleRequest"] + CreateSchedule --> TacticalApi["Tactical RMM API"] + TacticalApi --> ScheduledScript["TacticalScheduledScript"] + ScheduledScript --> ManagementService diff --git a/docs/diagrams/architecture/integrations-sdks-8.mmd b/docs/diagrams/architecture/integrations-sdks-8.mmd new file mode 100644 index 000000000..0c4eaffaa --- /dev/null +++ b/docs/diagrams/architecture/integrations-sdks-8.mmd @@ -0,0 +1,6 @@ +flowchart TD + SyncService["Compliance Sync Service"] --> FleetClient["Fleet SDK"] + FleetClient --> PolicyRequest["CreatePolicyRequest"] + PolicyRequest --> FleetApi["Fleet MDM API"] + FleetApi --> Policy["Policy"] + Policy --> SyncService diff --git a/docs/diagrams/architecture/integrations-sdks.mmd b/docs/diagrams/architecture/integrations-sdks.mmd new file mode 100644 index 000000000..3635a5118 --- /dev/null +++ b/docs/diagrams/architecture/integrations-sdks.mmd @@ -0,0 +1,24 @@ +flowchart LR + subgraph OpenFramePlatform["OpenFrame Platform Services"] + Management["Management Services"] + Gateway["Gateway Services"] + Stream["Stream Processing"] + end + + subgraph IntegrationsSdks["Integrations Sdks Module"] + TacticalSdk["Tactical RMM SDK"] + FleetSdk["Fleet MDM SDK"] + end + + subgraph ExternalSystems["External Systems"] + TacticalApi["Tactical RMM API"] + FleetApi["Fleet MDM API"] + end + + Management --> TacticalSdk + Management --> FleetSdk + Gateway --> TacticalSdk + Stream --> FleetSdk + + TacticalSdk --> TacticalApi + FleetSdk --> FleetApi diff --git a/docs/diagrams/architecture/management-service-core-2.mmd b/docs/diagrams/architecture/management-service-core-2.mmd new file mode 100644 index 000000000..46974bfc7 --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-2.mmd @@ -0,0 +1,4 @@ +flowchart LR + API["Resync Endpoint"] --> Repo["MachineRepository"] + Repo --> Service["MachineTagEventService"] + Service --> Pinot["Pinot Analytics"] diff --git a/docs/diagrams/architecture/management-service-core-3.mmd b/docs/diagrams/architecture/management-service-core-3.mmd new file mode 100644 index 000000000..36bdcefcf --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-3.mmd @@ -0,0 +1,5 @@ +flowchart TD + Request["SaveToolRequest"] --> ToolService["IntegratedToolService"] + ToolService --> Mongo["MongoDB"] + ToolService --> Debezium["DebeziumService"] + ToolService --> Hooks["IntegratedToolPostSaveHook[]"] diff --git a/docs/diagrams/architecture/management-service-core-4.mmd b/docs/diagrams/architecture/management-service-core-4.mmd new file mode 100644 index 000000000..d598a8841 --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-4.mmd @@ -0,0 +1,3 @@ +flowchart TD + Init["NATS Stream Initializer"] --> Mgmt["NatsStreamManagementService"] + Mgmt --> NATS["NATS JetStream"] diff --git a/docs/diagrams/architecture/management-service-core-5.mmd b/docs/diagrams/architecture/management-service-core-5.mmd new file mode 100644 index 000000000..8c239e275 --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-5.mmd @@ -0,0 +1,5 @@ +flowchart TD + Start["Startup"] --> Tool["IntegratedToolService"] + Tool --> Tactical["TacticalRmmClient"] + Tactical --> Create["Create Script"] + Tactical --> Update["Update Script"] diff --git a/docs/diagrams/architecture/management-service-core-6.mmd b/docs/diagrams/architecture/management-service-core-6.mmd new file mode 100644 index 000000000..2b9c752d3 --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-6.mmd @@ -0,0 +1,4 @@ +flowchart TD + Query["Tickets Without Order"] --> Sort["Sort by CreatedAt DESC"] + Sort --> Rank["Generate LexoRank"] + Rank --> Update["Update Ticket Order Field"] diff --git a/docs/diagrams/architecture/management-service-core-7.mmd b/docs/diagrams/architecture/management-service-core-7.mmd new file mode 100644 index 000000000..4befee71c --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-7.mmd @@ -0,0 +1,4 @@ +flowchart TD + Tick["Scheduled Tick"] --> Check["Check PublishState"] + Check --> PublishClient["Publish Client Update"] + Check --> PublishTool["Publish Tool Agent Update"] diff --git a/docs/diagrams/architecture/management-service-core-8.mmd b/docs/diagrams/architecture/management-service-core-8.mmd new file mode 100644 index 000000000..a96218d0d --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-8.mmd @@ -0,0 +1,4 @@ +flowchart LR + Mongo["MongoDB"] --> Debezium["Kafka Connect"] + Services --> NATS["NATS Publisher"] + Redis["Redis"] --> Schedulers diff --git a/docs/diagrams/architecture/management-service-core-9.mmd b/docs/diagrams/architecture/management-service-core-9.mmd new file mode 100644 index 000000000..53f4568ce --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-9.mmd @@ -0,0 +1,9 @@ +flowchart TD + Boot["Application Boot"] --> Config["Configuration Beans"] + Config --> Initializers["ApplicationRunner Initializers"] + Initializers --> Streams["NATS Streams Created"] + Initializers --> Secrets["Secrets Created"] + Initializers --> ToolConfig["Tool Config Applied"] + + Boot --> Migrations["Mongock Change Units"] + Boot --> Schedulers["Background Jobs Activated"] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-2.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-2.mmd deleted file mode 100644 index 1c674ad9f..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-2.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Scheduler["Scheduled Job"] --> LockProvider["RedisLockProvider"] - LockProvider --> Redis[("Redis")] - Redis --> LockKey["Tenant Scoped Lock Key"] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-3.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-3.mmd deleted file mode 100644 index 2762a6a25..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-3.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Start["Application Startup"] --> SecretInit["AgentRegistrationSecretInitializer"] - SecretInit --> Service["AgentRegistrationSecretManagementService"] - Service --> Create["createInitialSecret()"] - Create --> Processor["DefaultAgentRegistrationSecretManagementProcessor"] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-4.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-4.mmd deleted file mode 100644 index b31c2e02b..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Init["NatsStreamConfigurationInitializer"] --> DefaultStreams["Default Stream Configurations"] - Init --> Additional["AdditionalStreamConfigurationProvider"] - DefaultStreams --> NatsService["NatsStreamManagementService"] - Additional --> NatsService diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-5.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-5.mmd deleted file mode 100644 index beb2eac54..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-5.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Startup["Application Startup"] --> LoadTool["Load Tactical RMM Tool"] - LoadTool --> FetchScripts["Fetch Existing Scripts"] - FetchScripts --> Compare["Compare by Name"] - Compare -->|"Missing"| Create["Create Script"] - Compare -->|"Exists"| Update["Update Script"] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-6.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-6.mmd deleted file mode 100644 index 6b701a8a0..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-6.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Tick["Scheduled Tick"] --> ClientCheck["Check Client PublishState"] - ClientCheck --> RetryClient["Publish If Needed"] - Tick --> AgentCheck["Check Tool Agents"] - AgentCheck --> RetryAgent["Publish If Needed"] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-7.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-7.mmd deleted file mode 100644 index d13f80b57..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-7.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart LR - Scheduler["Heartbeat Scheduler"] --> Service["DeviceHeartbeatOfflineDetectionService"] - Service --> Mongo[("MongoDB Device Records")] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-8.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-8.mmd deleted file mode 100644 index 434e76c29..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-8.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Client["API Client"] --> Controller["IntegratedToolController"] - Controller --> Save["IntegratedToolService.saveTool()"] - Save --> TenantCheck["TenantIdProvider"] - TenantCheck -->|"Registered"| Debezium["DebeziumService"] - Save --> Hooks["IntegratedToolPostSaveHook"] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-9.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-9.mmd deleted file mode 100644 index f545d4e47..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-9.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Boot["Application Boot"] --> Initializers - Initializers --> Messaging["NATS Streams Ready"] - Messaging --> Schedulers - Schedulers --> External["External Systems"] - Controllers --> External diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers.mmd deleted file mode 100644 index 7da107187..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers.mmd +++ /dev/null @@ -1,33 +0,0 @@ -flowchart TD - subgraph ConfigLayer["Configuration Layer"] - MgmtConfig["ManagementConfiguration"] - RetryConfig["RetryConfiguration"] - ShedLock["ShedLockConfig"] - end - - subgraph Initializers["Startup Initializers"] - SecretInit["AgentRegistrationSecretInitializer"] - AgentInit["IntegratedToolAgentInitializer"] - NatsInit["NatsStreamConfigurationInitializer"] - ClientInit["OpenFrameClientConfigurationInitializer"] - TacticalInit["TacticalRmmScriptsInitializer"] - end - - subgraph Schedulers["Background Schedulers"] - VersionFallback["AgentVersionUpdatePublishFallbackScheduler"] - ApiKeySync["ApiKeyStatsSyncScheduler"] - Heartbeat["DeviceHeartbeatOfflineDetectionScheduler"] - FleetMdm["FleetMdmSetupScheduler"] - end - - subgraph Controllers["Operational Controllers"] - ToolCtrl["IntegratedToolController"] - PinotCtrl["DevicePinotResyncController"] - ReleaseCtrl["ReleaseVersionController"] - end - - MgmtConfig --> Initializers - ShedLock --> Schedulers - RetryConfig --> Schedulers - Controllers --> Schedulers - Initializers --> Schedulers diff --git a/docs/diagrams/architecture/management-service-core.mmd b/docs/diagrams/architecture/management-service-core.mmd new file mode 100644 index 000000000..352b15128 --- /dev/null +++ b/docs/diagrams/architecture/management-service-core.mmd @@ -0,0 +1,11 @@ +flowchart TD + Controllers["REST Controllers"] --> Services["Management Services"] + Services --> DataLayer["MongoDB / Redis Repositories"] + Services --> Messaging["NATS / Kafka Publishers"] + Services --> ExternalTools["Integrated Tools (RMM / MDM)"] + + Initializers["Application Initializers"] --> Services + Schedulers["Distributed Schedulers"] --> Services + Migrations["Mongock Change Units"] --> DataLayer + + Locking["ShedLock (Redis)"] --> Schedulers diff --git a/docs/diagrams/architecture/processors-2.mmd b/docs/diagrams/architecture/processors-2.mmd deleted file mode 100644 index f5990ef98..000000000 --- a/docs/diagrams/architecture/processors-2.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Generator["Secret Generation Logic"] --> Persist["Persist AgentRegistrationSecret"] - Persist --> ProcessorCall["postProcessSecretGenerated()"] - ProcessorCall --> DefaultProcessor["DefaultAgentRegistrationSecretProcessor"] diff --git a/docs/diagrams/architecture/processors-3.mmd b/docs/diagrams/architecture/processors-3.mmd deleted file mode 100644 index 6a9ee343e..000000000 --- a/docs/diagrams/architecture/processors-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - InvitationService["Invitation Service"] --> SaveInvitation["Persist Invitation"] - SaveInvitation --> ProcessorHook["postProcessInvitationCreated()"] - ProcessorHook --> DefaultInvitationProcessor["DefaultInvitationProcessor"] diff --git a/docs/diagrams/architecture/processors-4.mmd b/docs/diagrams/architecture/processors-4.mmd deleted file mode 100644 index 695fd9f01..000000000 --- a/docs/diagrams/architecture/processors-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - SSOService["SSOConfigService"] --> SaveConfig["Persist SSOConfig"] - SaveConfig --> ProcessorHook["postProcessConfigSaved()"] - ProcessorHook --> DefaultSSOProcessor["DefaultSSOConfigProcessor"] diff --git a/docs/diagrams/architecture/processors-5.mmd b/docs/diagrams/architecture/processors-5.mmd deleted file mode 100644 index e24006f05..000000000 --- a/docs/diagrams/architecture/processors-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - UserController["UserController"] --> UserService["UserService"] - UserService --> PersistUser["Persist User"] - PersistUser --> ProcessorHook["postProcessUserUpdated()"] - ProcessorHook --> DefaultUserProcessor["DefaultUserProcessor"] diff --git a/docs/diagrams/architecture/processors-6.mmd b/docs/diagrams/architecture/processors-6.mmd deleted file mode 100644 index e738e3e2e..000000000 --- a/docs/diagrams/architecture/processors-6.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - SpringContext["Spring Application Context"] --> CheckCustom["Custom Bean Present?"] - CheckCustom -->|"Yes"| UseCustom["Use Custom Implementation"] - CheckCustom -->|"No"| UseDefault["Use Default Implementation"] diff --git a/docs/diagrams/architecture/processors-7.mmd b/docs/diagrams/architecture/processors-7.mmd deleted file mode 100644 index 13c39d0d0..000000000 --- a/docs/diagrams/architecture/processors-7.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Services["Services Module"] -->|"Business Logic"| Domain["Domain Model"] - Services -->|"Calls"| Processors["Processors Module"] - Processors -->|"Side Effects"| ExternalSystems["External Systems"] diff --git a/docs/diagrams/architecture/processors-8.mmd b/docs/diagrams/architecture/processors-8.mmd deleted file mode 100644 index 6394e195e..000000000 --- a/docs/diagrams/architecture/processors-8.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - API["API Layer"] --> ServiceLayer["Service Layer"] - ServiceLayer --> RepositoryLayer["Repository Layer"] - ServiceLayer --> ProcessorLayer["Processors"] - ProcessorLayer --> Integration["Integrations / Events / Audit"] diff --git a/docs/diagrams/architecture/processors.mmd b/docs/diagrams/architecture/processors.mmd deleted file mode 100644 index 8b6c815e4..000000000 --- a/docs/diagrams/architecture/processors.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - Controller["REST or GraphQL Controller"] --> Service["Core Service"] - Service --> DomainModel["Domain Model"] - Service --> Processor["Processor Interface"] - Processor --> DefaultImpl["Default Implementation"] - Processor --> CustomImpl["Custom Implementation (Optional)"] diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-2.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-2.mmd deleted file mode 100644 index 7ee915858..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-2.mmd +++ /dev/null @@ -1,11 +0,0 @@ -flowchart TD - JwtConfig["JwtConfig"] -->|"loadPublicKey()"| PublicKey["RSAPublicKey"] - JwtConfig -->|"loadPrivateKey()"| PrivateKey["RSAPrivateKey"] - - PublicKey --> RsaKey["RSAKey Builder"] - PrivateKey --> RsaKey - - RsaKey --> JwkSet["JWKSet"] - JwkSet --> JwtEncoder["NimbusJwtEncoder"] - - PublicKey --> JwtDecoder["NimbusJwtDecoder"] diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-3.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-3.mmd deleted file mode 100644 index 682c32589..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - PemString["PEM Private Key"] --> StripHeaders["Remove BEGIN/END markers"] - StripHeaders --> Base64Decode["Base64 Decode"] - Base64Decode --> KeySpec["PKCS8EncodedKeySpec"] - KeySpec --> KeyFactory["KeyFactory RSA"] - KeyFactory --> RsaPrivate["RSAPrivateKey"] diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-4.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-4.mmd deleted file mode 100644 index 57de5ec02..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-4.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Verifier["Code Verifier (random 32 bytes)"] --> Hash["SHA-256"] - Hash --> Challenge["Base64URL Encode"] - - Challenge -->|"Sent to Authorization Server"| AuthServer["Authorization Server"] - Verifier -->|"Stored by BFF"| BffStore["State JWT / Cookie"] diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-5.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-5.mmd deleted file mode 100644 index 6508a734f..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-5.mmd +++ /dev/null @@ -1,12 +0,0 @@ -sequenceDiagram - participant Browser - participant BFF as OAuth BFF Controller - participant Auth as Authorization Server - - Browser->>BFF: GET /oauth/login - BFF->>Auth: Redirect with state and PKCE challenge - Auth->>Browser: Login UI - Auth->>BFF: Callback with code and state - BFF->>Auth: Exchange code for tokens - Auth->>BFF: Access and Refresh tokens - BFF->>Browser: Set HttpOnly cookies and redirect diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-6.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-6.mmd deleted file mode 100644 index 32886c510..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-6.mmd +++ /dev/null @@ -1,9 +0,0 @@ -sequenceDiagram - participant Browser - participant BFF - participant Auth as Authorization Server - - Browser->>BFF: POST /oauth/refresh - BFF->>Auth: Refresh token request - Auth->>BFF: New tokens - BFF->>Browser: Set new cookies diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-7.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-7.mmd deleted file mode 100644 index 76d03d47e..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-7.mmd +++ /dev/null @@ -1,8 +0,0 @@ -sequenceDiagram - participant Browser - participant BFF - participant Auth as Authorization Server - - Browser->>BFF: GET /oauth/logout - BFF->>Auth: Revoke refresh token - BFF->>Browser: Clear auth cookies diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-8.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-8.mmd deleted file mode 100644 index add9b4272..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-8.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Requested["Requested redirectTo"] -->|"Has value"| UseRequested["Use requested value"] - Requested -->|"Empty"| RefererCheck["Check Referer header"] - RefererCheck -->|"Present"| UseReferer["Use Referer"] - RefererCheck -->|"Missing"| DefaultRoot["Use /"] diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff.mmd deleted file mode 100644 index 6b9d48043..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart LR - Browser["Browser Client"] -->|"GET /oauth/login"| Bff["OAuth BFF Controller"] - Bff -->|"Redirect to authorize"| AuthServer["Authorization Server"] - AuthServer -->|"Callback with code"| Bff - Bff -->|"Exchange code for tokens"| AuthServer - Bff -->|"Set HttpOnly Cookies"| Browser - - Browser -->|"API calls with cookies"| Gateway["Gateway Service"] - Gateway -->|"Validate JWT"| JwtDecoder["JwtDecoder Bean"] diff --git a/docs/diagrams/architecture/security-oauth-and-jwt-2.mmd b/docs/diagrams/architecture/security-oauth-and-jwt-2.mmd new file mode 100644 index 000000000..92c6a985c --- /dev/null +++ b/docs/diagrams/architecture/security-oauth-and-jwt-2.mmd @@ -0,0 +1,6 @@ +flowchart TD + JwtConfig["JwtConfig"] -->|"loadPublicKey()"| JwtSecurityConfig + JwtConfig -->|"loadPrivateKey()"| JwtSecurityConfig + + JwtSecurityConfig --> JwtEncoder["NimbusJwtEncoder"] + JwtSecurityConfig --> JwtDecoder["NimbusJwtDecoder"] diff --git a/docs/diagrams/architecture/security-oauth-and-jwt-3.mmd b/docs/diagrams/architecture/security-oauth-and-jwt-3.mmd new file mode 100644 index 000000000..a01487689 --- /dev/null +++ b/docs/diagrams/architecture/security-oauth-and-jwt-3.mmd @@ -0,0 +1,5 @@ +flowchart TD + AppConfig["application.yml jwt.*"] --> JwtConfig + JwtConfig -->|"Base64 Decode"| KeyFactory + KeyFactory --> RSAPublic["RSAPublicKey"] + KeyFactory --> RSAPrivate["RSAPrivateKey"] diff --git a/docs/diagrams/architecture/security-oauth-and-jwt-4.mmd b/docs/diagrams/architecture/security-oauth-and-jwt-4.mmd new file mode 100644 index 000000000..4ab22bfb5 --- /dev/null +++ b/docs/diagrams/architecture/security-oauth-and-jwt-4.mmd @@ -0,0 +1,6 @@ +flowchart TD + GenerateVerifier["generateCodeVerifier()"] --> Hash["SHA-256"] + Hash --> Encode["Base64URL"] + Encode --> Challenge["code_challenge"] + + GenerateState["generateState()"] --> Encode diff --git a/docs/diagrams/architecture/security-oauth-and-jwt-5.mmd b/docs/diagrams/architecture/security-oauth-and-jwt-5.mmd new file mode 100644 index 000000000..7e0c8c5e1 --- /dev/null +++ b/docs/diagrams/architecture/security-oauth-and-jwt-5.mmd @@ -0,0 +1,13 @@ +sequenceDiagram + participant Browser + participant BFF as OAuthBffController + participant Auth as AuthorizationServer + + Browser->>BFF: GET /oauth/login + BFF->>BFF: Generate state and PKCE + BFF->>Auth: Redirect to authorize endpoint + Auth->>Browser: Login page + Browser->>Auth: Submit credentials + Auth->>BFF: Redirect with code and state + BFF->>Auth: Exchange code for tokens + BFF->>Browser: Set auth cookies and redirect diff --git a/docs/diagrams/architecture/security-oauth-and-jwt-6.mmd b/docs/diagrams/architecture/security-oauth-and-jwt-6.mmd new file mode 100644 index 000000000..579e8ba89 --- /dev/null +++ b/docs/diagrams/architecture/security-oauth-and-jwt-6.mmd @@ -0,0 +1,5 @@ +flowchart TD + Tokens["TokenResponse"] --> CreateTicket + CreateTicket --> Store["ConcurrentHashMap"] + DevClient["Dev Client"] -->|"ticket"| Consume + Consume --> Store diff --git a/docs/diagrams/architecture/security-oauth-and-jwt-7.mmd b/docs/diagrams/architecture/security-oauth-and-jwt-7.mmd new file mode 100644 index 000000000..6ac04b6e9 --- /dev/null +++ b/docs/diagrams/architecture/security-oauth-and-jwt-7.mmd @@ -0,0 +1,9 @@ +flowchart TD + Login["Login"] --> AccessToken["Access Token"] + Login --> RefreshToken["Refresh Token"] + + AccessToken --> ApiCall["API Call"] + RefreshToken --> Refresh["/oauth/refresh"] + + Refresh --> AccessToken + Logout["Logout"] --> Revoke["Revoke Refresh Token"] diff --git a/docs/diagrams/architecture/security-oauth-and-jwt.mmd b/docs/diagrams/architecture/security-oauth-and-jwt.mmd new file mode 100644 index 000000000..e7bc651dc --- /dev/null +++ b/docs/diagrams/architecture/security-oauth-and-jwt.mmd @@ -0,0 +1,9 @@ +flowchart LR + Browser["Browser / Frontend"] -->|"/oauth/login"| BffController["OAuthBffController"] + BffController -->|"Redirect to IdP"| AuthServer["Authorization Server"] + AuthServer -->|"Authorization Code"| BffController + BffController -->|"Token Exchange"| AuthServer + BffController -->|"Set Auth Cookies"| Browser + + ApiService["API Services"] -->|"Validate JWT"| JwtDecoder["JwtDecoder"] + JwtEncoder["JwtEncoder"] -->|"Signs Tokens"| JwtDecoder diff --git a/docs/diagrams/architecture/services-2.mmd b/docs/diagrams/architecture/services-2.mmd deleted file mode 100644 index cf3d02dd0..000000000 --- a/docs/diagrams/architecture/services-2.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - Request["User Update Request"] --> Load["Load User from Repository"] - Load --> Check{"User Exists?"} - Check -->|"No"| Error["Throw IllegalArgumentException"] - Check -->|"Yes"| Modify["Apply Field Updates"] - Modify --> Save["Save via UserRepository"] - Save --> PostProcess["UserProcessor.postProcessUserUpdated()"] - PostProcess --> Response["Return UserResponse"] diff --git a/docs/diagrams/architecture/services-3.mmd b/docs/diagrams/architecture/services-3.mmd deleted file mode 100644 index 0dcbea462..000000000 --- a/docs/diagrams/architecture/services-3.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - DeleteRequest["Soft Delete Request"] --> LoadUser["Load User"] - LoadUser --> SelfCheck{"Requester == Target?"} - SelfCheck -->|"Yes"| SelfError["Throw Self Delete Exception"] - SelfCheck -->|"No"| OwnerCheck{"Has OWNER Role?"} - OwnerCheck -->|"Yes"| OwnerError["Throw OperationNotAllowedException"] - OwnerCheck -->|"No"| MarkDeleted["Set Status = DELETED"] - MarkDeleted --> SaveDeleted["Save User"] - SaveDeleted --> PostDelete["UserProcessor.postProcessUserDeleted()"] diff --git a/docs/diagrams/architecture/services-4.mmd b/docs/diagrams/architecture/services-4.mmd deleted file mode 100644 index 38db55be0..000000000 --- a/docs/diagrams/architecture/services-4.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - AdminRequest["Upsert SSOConfigRequest"] --> Validate["validateAutoProvision()"] - Validate --> Normalize["Normalize Domains"] - Normalize --> DomainValidation["DomainValidationService.validate*"] - DomainValidation --> Encrypt["Encrypt Client Secret"] - Encrypt --> SaveConfig["Save via SSOConfigRepository"] - SaveConfig --> PostProcess["SSOConfigProcessor.postProcessConfigSaved()"] - PostProcess --> ReturnResponse["Return SSOConfigResponse"] diff --git a/docs/diagrams/architecture/services-5.mmd b/docs/diagrams/architecture/services-5.mmd deleted file mode 100644 index d5f4c90fe..000000000 --- a/docs/diagrams/architecture/services-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - DomainService["DomainValidationService"] --> Interface["DomainExistenceValidator"] - Interface --> DefaultImpl["DefaultDomainExistenceValidator"] - Interface --> CustomImpl["SaaS Override Implementation"] diff --git a/docs/diagrams/architecture/services-6.mmd b/docs/diagrams/architecture/services-6.mmd deleted file mode 100644 index 5627a721f..000000000 --- a/docs/diagrams/architecture/services-6.mmd +++ /dev/null @@ -1,11 +0,0 @@ -sequenceDiagram - participant Controller - participant Service - participant Repository - participant Processor - - Controller->>Service: Update Request - Service->>Repository: Save Entity - Repository-->>Service: Saved Entity - Service->>Processor: postProcess(...) - Service-->>Controller: Response DTO diff --git a/docs/diagrams/architecture/services-7.mmd b/docs/diagrams/architecture/services-7.mmd deleted file mode 100644 index 9ffceb5d9..000000000 --- a/docs/diagrams/architecture/services-7.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Client["Client Application"] --> Gateway["Gateway Service"] - Gateway --> ApiCore["API Service Core"] - ApiCore --> ServicesNode["Services Module"] - ServicesNode --> Mongo[("MongoDB")] - ServicesNode --> Authz["Authorization Service"] - ServicesNode --> Crypto["Encryption Service"] diff --git a/docs/diagrams/architecture/services.mmd b/docs/diagrams/architecture/services.mmd deleted file mode 100644 index b3c9c9b9e..000000000 --- a/docs/diagrams/architecture/services.mmd +++ /dev/null @@ -1,26 +0,0 @@ -flowchart LR - subgraph Controllers["API Layer"] - RestControllers["REST Controllers"] - GraphQL["GraphQL DataFetchers"] - end - - subgraph ServicesLayer["Services Module"] - UserServiceNode["UserService"] - SSOServiceNode["SSOConfigService"] - DomainValidator["DefaultDomainExistenceValidator"] - end - - subgraph Processors["Processors"] - UserProcessorNode["UserProcessor"] - SSOProcessorNode["SSOConfigProcessor"] - end - - subgraph DataLayer["Data Layer"] - UserRepo["UserRepository"] - SSORepo["SSOConfigRepository"] - MongoDocs["Mongo Domain Models"] - end - - Controllers --> ServicesLayer - ServicesLayer --> Processors - ServicesLayer --> DataLayer diff --git a/docs/diagrams/architecture/stream-processing-core-2.mmd b/docs/diagrams/architecture/stream-processing-core-2.mmd new file mode 100644 index 000000000..f1d73ed8a --- /dev/null +++ b/docs/diagrams/architecture/stream-processing-core-2.mmd @@ -0,0 +1,13 @@ +sequenceDiagram + participant Kafka + participant Listener as JsonKafkaListener + participant Processor as GenericJsonMessageProcessor + participant Deserializer + participant Enrichment + participant Handler + + Kafka->>Listener: CommonDebeziumMessage + MessageType + Listener->>Processor: process(message, type) + Processor->>Deserializer: Deserialize by type + Deserializer->>Enrichment: Enrich data + Enrichment->>Handler: Handle operation diff --git a/docs/diagrams/architecture/stream-processing-core-3.mmd b/docs/diagrams/architecture/stream-processing-core-3.mmd new file mode 100644 index 000000000..bf654560a --- /dev/null +++ b/docs/diagrams/architecture/stream-processing-core-3.mmd @@ -0,0 +1,3 @@ +flowchart LR + SourceEvent["toolType + sourceEventType"] --> Mapper["EventTypeMapper"] + Mapper --> Unified["UnifiedEventType"] diff --git a/docs/diagrams/architecture/stream-processing-core-4.mmd b/docs/diagrams/architecture/stream-processing-core-4.mmd new file mode 100644 index 000000000..527fa8955 --- /dev/null +++ b/docs/diagrams/architecture/stream-processing-core-4.mmd @@ -0,0 +1,4 @@ +flowchart TD + Message --> Transform["transform()"] + Transform --> Operation["getOperationType()"] + Operation --> Dispatch["handleCreate / handleUpdate / handleDelete"] diff --git a/docs/diagrams/architecture/stream-processing-core-5.mmd b/docs/diagrams/architecture/stream-processing-core-5.mmd new file mode 100644 index 000000000..94bc9399a --- /dev/null +++ b/docs/diagrams/architecture/stream-processing-core-5.mmd @@ -0,0 +1,7 @@ +flowchart TD + ActivitiesTopic["fleet activities"] --> ReKey["Re-key by activityId"] + HostActivitiesTopic["fleet host activities"] --> ReKeyHost["Re-key by activityId"] + ReKey --> Join["Left Join (5s window)"] + ReKeyHost --> Join + Join --> HeaderAdd["Add MessageType Header"] + HeaderAdd --> OutputTopic["enriched fleet events"] diff --git a/docs/diagrams/architecture/stream-processing-core.mmd b/docs/diagrams/architecture/stream-processing-core.mmd new file mode 100644 index 000000000..579aaf65b --- /dev/null +++ b/docs/diagrams/architecture/stream-processing-core.mmd @@ -0,0 +1,15 @@ +flowchart TD + KafkaTopics["Kafka Topics\n(Integrated Tool Events)"] --> JsonListener["JsonKafkaListener"] + JsonListener --> Processor["GenericJsonMessageProcessor"] + Processor --> Deserializers["Tool Event Deserializers"] + Deserializers --> Enrichment["IntegratedToolDataEnrichmentService"] + Enrichment --> Mapping["EventTypeMapper"] + Mapping --> Handlers["DebeziumMessageHandler"] + Handlers --> CassandraHandler["DebeziumCassandraMessageHandler"] + Handlers --> TenantKafkaHandler["TenantDebeziumKafkaMessageHandler"] + + subgraph streams_layer["Kafka Streams Enrichment"] + Activities["Fleet Activities Topic"] --> Joiner["ActivityEnrichmentService"] + HostActivities["Fleet Host Activities Topic"] --> Joiner + Joiner --> EnrichedTopic["Enriched Fleet Events Topic"] + end diff --git a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-2.mmd b/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-2.mmd deleted file mode 100644 index 90b3c0db9..000000000 --- a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-2.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - A["Kafka Debezium Event"] --> B["JsonKafkaListener"] - B --> C["GenericJsonMessageProcessor"] - C --> D["Tool-Specific Deserializer"] - D --> E["Unified DeserializedDebeziumMessage"] - E --> F["IntegratedToolDataEnrichmentService"] - F --> G["EventTypeMapper"] - G --> H["DebeziumMessageHandler"] - H --> I["Destination Handler"] - I --> J["Cassandra or Kafka"] diff --git a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-3.mmd b/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-3.mmd deleted file mode 100644 index e3f6fdecd..000000000 --- a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - ActivityTopic["Fleet Activities"] --> Join - HostActivityTopic["Fleet Host Activities"] --> Join - Join["Left Join (5s Window)"] --> Enriched["Enriched ActivityMessage"] - Enriched --> HeaderAdder["Add MESSAGE_TYPE_HEADER"] - HeaderAdder --> OutputTopic["Fleet MDM Events Topic"] diff --git a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-4.mmd b/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-4.mmd deleted file mode 100644 index 021b220e7..000000000 --- a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - ToolType["IntegratedToolType"] --> Key - SourceType["Source Event Type"] --> Key - Key["tool:sourceType"] --> Mapper["EventTypeMapper"] - Mapper --> Unified["UnifiedEventType"] diff --git a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-5.mmd b/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-5.mmd deleted file mode 100644 index 260c90f84..000000000 --- a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Event["DeserializedDebeziumMessage"] --> MachineLookup["MachineIdCacheService"] - MachineLookup --> OrgLookup["Organization Cache"] - OrgLookup --> TenantResolution["TenantIdProvider or ClusterTenantIdResolver"] - TenantResolution --> Enriched["IntegratedToolEnrichedData"] diff --git a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers.mmd b/docs/diagrams/architecture/stream-service-core-kafka-and-handlers.mmd deleted file mode 100644 index 27fcc8807..000000000 --- a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers.mmd +++ /dev/null @@ -1,43 +0,0 @@ -flowchart LR - subgraph Tools["Integrated Tools"] - MeshCentral["MeshCentral"] - Tactical["Tactical RMM"] - Fleet["Fleet MDM"] - end - - subgraph KafkaIn["Inbound Kafka Topics"] - InTopics["Debezium Topics"] - end - - subgraph StreamCore["Stream Service Core Kafka And Handlers"] - Listener["JsonKafkaListener"] - Processor["GenericJsonMessageProcessor"] - Deserializer["Tool Event Deserializers"] - Enrichment["IntegratedToolDataEnrichmentService"] - Mapper["EventTypeMapper"] - Handler["DebeziumMessageHandler"] - CassandraHandler["DebeziumCassandraMessageHandler"] - TenantKafkaHandler["TenantDebeziumKafkaMessageHandler"] - end - - subgraph Storage["Storage & Downstream"] - Cassandra["Cassandra UnifiedLogEvent"] - KafkaOut["Outbound Kafka Topics"] - end - - MeshCentral --> InTopics - Tactical --> InTopics - Fleet --> InTopics - - InTopics --> Listener - Listener --> Processor - Processor --> Deserializer - Deserializer --> Enrichment - Enrichment --> Mapper - Mapper --> Handler - - Handler --> CassandraHandler - Handler --> TenantKafkaHandler - - CassandraHandler --> Cassandra - TenantKafkaHandler --> KafkaOut diff --git a/docs/getting-started/first-steps.md b/docs/getting-started/first-steps.md index a9bcf6f73..88815f10c 100644 --- a/docs/getting-started/first-steps.md +++ b/docs/getting-started/first-steps.md @@ -1,178 +1,176 @@ # First Steps -After completing the [Quick Start](quick-start.md), here are the first five things to explore in **openframe-oss-lib** to become productive quickly. +After successfully building `openframe-oss-lib`, here are the five most important things to do to explore the library and start integrating it into your services. ---- - -## 1. Understand the Module Structure +[![OpenFrame v0.3.7 - Enhanced Developer Experience](https://img.youtube.com/vi/O8hbBO5Mym8/maxresdefault.jpg)](https://www.youtube.com/watch?v=O8hbBO5Mym8) -The repository is a Maven multi-module project. Each module is independently deployable and follows a consistent pattern: +--- -```text -openframe-/ -β”œβ”€β”€ src/ -β”‚ β”œβ”€β”€ main/java/com/openframe/... # Production code -β”‚ └── test/java/com/openframe/... # Unit and integration tests -└── pom.xml # Module POM (inherits parent) -``` +## 1. Explore the Module Structure -Start by reviewing the parent POM at the repository root to understand the full module list and shared dependency versions: +The repository is a multi-module Maven project. Understanding which module does what is the best starting point. ```bash -cat pom.xml +# List all modules in the project +mvn help:evaluate -Dexpression=project.modules -q -DforceStdout ``` -Key properties to note: +Or simply open the root `pom.xml` and review the `` section. The modules follow a layered dependency structure: + +```mermaid +graph TD + A["openframe-exception"] --> B["openframe-core"] + B --> C["openframe-core-crypto"] + C --> D["openframe-data-mongo-common"] + D --> E["openframe-data-mongo-sync"] + D --> F["openframe-data-mongo-reactive"] + E --> G["openframe-api-lib"] + G --> H["openframe-api-service-core"] + H --> I["openframe-authorization-service-core"] + H --> J["openframe-gateway-service-core"] + H --> K["openframe-client-core"] + H --> L["openframe-management-service-core"] +``` -| Property | Value | -|----------|-------| -| `revision` | Current unified version (`5.79.3`) | -| `java.version` | `21` | -| `spring-boot-starter-parent` | `3.3.0` | -| `spring-cloud.version` | `2023.0.3` | +Start with the lower-level modules (`openframe-core`, `openframe-exception`) before moving to higher-level ones. --- -## 2. Explore the Core Domain Model +## 2. Understand the Multi-Tenancy Model -The best entry point for understanding the data model is `openframe-data-mongo-common`. This module defines all MongoDB documents: +Every entity in OpenFrame is **tenant-scoped**. Before writing any business logic, understand how tenancy flows through the system. -```bash -ls openframe-data-mongo-common/src/main/java/com/openframe/data/document/ -``` +**The key class is `TenantIdProvider`:** -Key domain areas: +```java +// Default implementation reads from TENANT_ID environment variable +// OSS deployments default to "oss" +public class DefaultTenantIdProvider implements TenantIdProvider { + @Override + public String getTenantId() { + return System.getenv().getOrDefault("TENANT_ID", "oss"); + } +} +``` -| Package | Domain | -|---------|--------| -| `device/` | `Device`, `Machine`, `DeviceHealth` | -| `organization/` | `Organization`, `ContactInformation` | -| `user/` | `User`, `AuthUser`, `Invitation` | -| `ticket/` | `Ticket`, `TicketNote`, `TicketAttachment` | -| `tool/` | `IntegratedTool`, `ToolConnection`, `ToolCredentials` | -| `notification/` | `Notification`, `NotificationContext`, `ReadStatus` | -| `tenant/` | `Tenant`, `TenantKey`, `SSOPerTenantConfig` | -| `oauth/` | `MongoRegisteredClient`, `OAuthToken` | +**Key points:** -Read the domain model reference documentation for a full data-flow diagram and entity relationships: -[./reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md](./reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md) +- Every MongoDB collection uses `tenantId` as a filter field +- JWT tokens carry `tenant_id` as a claim +- The `TENANT_ID` environment variable controls the OSS single-tenant context +- In SaaS mode, the tenant is resolved from the JWT issuer and request path --- -## 3. Try the Security Modules +## 3. Set Up a Local Infrastructure Stack -The security stack is a critical foundation for any OpenFrame service. Explore: +To develop and test any service locally, you need the infrastructure running. Use Docker to start MongoDB: -### `openframe-security-core` - -Provides JWT signing and verification: - -```java -// Inject the JwtService to sign or validate tokens -@Autowired -private JwtService jwtService; +```bash +# Start MongoDB for integration tests +cd openframe-data-mongo-sync/src/test/docker +docker compose up -d ``` -Properties to configure (in `application.yml`): +For a full local development stack, you'll also need NATS, Redis, and Kafka. These are typically configured in your deployment environment. Refer to your deployment documentation or ask in the OpenMSP Slack community for environment-specific setup. -```yaml -jwt: - public-key: classpath:keys/public.pem - private-key: classpath:keys/private.pem - issuer: https://your-tenant.openframe.ai - audience: openframe-api -``` +**Minimum required for basic module development:** -### `openframe-security-oauth` +| Service | Docker Image | Default Port | +|---------|-------------|-------------| +| MongoDB | `mongo:7` | `27017` | +| Redis | `redis:7` | `6379` | +| NATS | `nats:2-alpine` | `4222` | -The OAuth BFF module provides ready-made endpoints for browser-based OAuth flows. To enable: +--- -```yaml -openframe: - gateway: - oauth: - enable: true -``` +## 4. Explore the Reference Documentation -Exposed endpoints automatically: +The `docs/reference/architecture/` directory contains detailed documentation for every major module. Start with the architecture overview: -- `GET /oauth/login` -- `GET /oauth/callback` -- `POST /oauth/refresh` -- `GET /oauth/logout` +| Document | What You'll Learn | +|----------|-----------------| +| `api-service-core-config-and-security` | Security config, JWT multi-issuer, GraphQL scalars | +| `authorization-service-core` | OAuth2/OIDC server, SSO flows, tenant key management | +| `gateway-service-core` | Reactive gateway, WebSocket proxy, rate limiting | +| `data-model-and-repositories-mongo` | Domain documents, multi-tenancy, query filters | +| `management-service-core` | Bootstrapping, schedulers, data migrations | +| `eventing-and-messaging-kafka-nats` | Kafka producers, NATS publishers, Debezium CDC | +| `client-core-agent-ingress` | Agent registration, heartbeat, NATS listeners | +| `integrations-sdks` | Tactical RMM and Fleet MDM SDK usage | --- -## 4. Run Your First Integration Test +## 5. Write Your First Extension -The `openframe-data-mongo-sync` module has a comprehensive integration test suite using Testcontainers. Run it to verify your local Docker setup: +OpenFrame modules are designed to be extended through **processor interfaces** and **hooks**. A common first integration is implementing a custom `AgentRegistrationProcessor`: -```bash -# Start Docker first, then run integration tests -mvn verify -pl openframe-data-mongo-sync -Pfailsafe +```java +@Component +public class MyCustomAgentRegistrationProcessor + implements AgentRegistrationProcessor { + + @Override + public void processAfterRegistration( + AgentRegistrationRequest request, + AgentRegistrationResponse response) { + // Your custom logic after an agent registers + // e.g., send a notification, update a CRM, etc. + } +} ``` -Testcontainers will automatically: -1. Pull the MongoDB Docker image -2. Start a containerized MongoDB instance -3. Run all `*IT.java` tests against it -4. Tear down the container on completion +Other common extension points: -Integration test classes follow the `*IT.java` naming convention (configured in the parent `maven-surefire-plugin`). +| Interface | Purpose | +|-----------|---------| +| `AgentRegistrationProcessor` | Hook into agent registration lifecycle | +| `InvitationProcessor` | Custom logic during user invitation flow | +| `UserProcessor` | Custom user creation and update handling | +| `SSOConfigProcessor` | Extend SSO configuration behavior | +| `RegistrationProcessor` | Hook into tenant registration flow | +| `IntegratedToolPostSaveHook` | React to tool configuration saves | --- -## 5. Explore the Gateway Module +## Key Configuration Properties -The gateway is the entry point for all service traffic. Review: +When building a Spring Boot service on top of `openframe-oss-lib`, these properties are commonly configured: -```bash -ls openframe-gateway-service-core/src/main/java/com/openframe/gateway/ -``` - -Key files to read: +```yaml +# application.yml example skeleton +spring: + data: + mongodb: + uri: mongodb://localhost:27017/openframe -| File | Purpose | -|------|---------| -| `security/GatewaySecurityConfig.java` | Main reactive security filter chain | -| `security/filter/ApiKeyAuthenticationFilter.java` | API key authentication + rate limiting | -| `config/ws/WebSocketGatewayConfig.java` | WebSocket routing configuration | -| `upstream/DefaultToolUpstreamResolver.java` | Tool proxy URL resolution | +openframe: + security: + jwt: + cache: + expire-after: 3600 + refresh-after: 1800 + maximum-size: 100 -The gateway supports these route patterns: + gateway: + disable-cors: false -```text -/api/** β†’ ADMIN role required -/tools/agent/** β†’ AGENT role required -/ws/tools/** β†’ WebSocket proxy to integrated tools -/external-api/** β†’ API key authentication +jwt: + issuer: https://auth.yourdomain.com + # public-key and private-key loaded from secrets manager ``` ---- - -## Where to Get Help - -| Resource | Link | -|----------|------| -| OpenMSP Community (Slack) | [https://www.openmsp.ai/](https://www.openmsp.ai/) | -| OpenFrame Platform | [https://openframe.ai](https://openframe.ai) | -| Flamingo | [https://flamingo.run](https://flamingo.run) | -| Reference Architecture | [./reference/architecture/README.md](./reference/architecture/README.md) | - -> **Note:** We use Slack for all community support and discussions. Please join the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) instead of creating GitHub Issues. +> Always refer to each module's `@ConfigurationProperties` classes for the full list of available properties. --- -## What to Explore Next - -Once comfortable with the basics: +## Get Help from the Community -- Review the **Authorization Service** to understand multi-tenant JWT issuance -- Explore **Stream Service Core** for Kafka / Debezium event processing -- Look at **Management Service Core** for startup initializers and schedulers -- Check the **External API Service** for integration patterns +> The **OpenMSP Slack** is the primary support channel for all OpenFrame questions. +> +> πŸ’¬ [Join the community β†’](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) -All reference documentation is available under: -[./reference/architecture/](./reference/architecture/) - -[![OpenFrame v0.3.7 - Enhanced Developer Experience](https://img.youtube.com/vi/O8hbBO5Mym8/maxresdefault.jpg)](https://www.youtube.com/watch?v=O8hbBO5Mym8) +You can also: +- Browse the full architecture reference in `docs/reference/architecture/` +- Review the inline code documentation alongside each source file +- Check the `openframe-test-service-core` module for real usage examples and test patterns diff --git a/docs/getting-started/introduction.md b/docs/getting-started/introduction.md index 1bcf37af4..be1950eaa 100644 --- a/docs/getting-started/introduction.md +++ b/docs/getting-started/introduction.md @@ -1,136 +1,175 @@ -# OpenFrame OSS Lib β€” Introduction +# Introduction to OpenFrame OSS Lib -**openframe-oss-lib** is the foundational backend library powering the [OpenFrame](https://openframe.ai) platform β€” the AI-driven, open-source MSP infrastructure stack built by [Flamingo](https://flamingo.run). +**`openframe-oss-lib`** is the modular backend foundation of the [OpenFrame platform](https://openframe.ai) β€” the unified AI-driven MSP platform by [Flamingo](https://flamingo.run). It provides a production-ready, multi-tenant, event-driven, analytics-enabled backend library that powers IT support operations at scale. -This repository provides the **shared Spring Boot libraries** that every OpenFrame service is built upon: authentication, API layers, gateway routing, streaming, persistence, and more. - -[![OpenFrame Product Walkthrough (Beta Access)](https://img.youtube.com/vi/awc-yAnkhIo/maxresdefault.jpg)](https://www.youtube.com/watch?v=awc-yAnkhIo) +[![Getting Started with OpenFrame - Organization Setup Basics](https://img.youtube.com/vi/-_56_qYvMWk/maxresdefault.jpg)](https://www.youtube.com/watch?v=-_56_qYvMWk) --- ## What Is OpenFrame OSS Lib? -OpenFrame OSS Lib is a multi-module Maven project (Java 21, Spring Boot 3.3) that delivers the **core backend infrastructure** for the OpenFrame MSP platform. Rather than a standalone application, it is a set of reusable modules consumed by every service deployed inside an OpenFrame installation. - -The library covers: +`openframe-oss-lib` is a **Spring Boot 3 multi-module Maven project** written in Java 21. It ships as a set of reusable library modules that are composed together to build a fully functional MSP backend platform including: -- **Multi-tenant authentication and authorization** β€” OAuth2 Authorization Server, per-tenant RSA key pairs, JWT issuance, SSO integration -- **API service layers** β€” REST controllers and Relay-compliant GraphQL execution -- **Gateway routing and security** β€” reactive edge gateway, API key rate limiting, WebSocket proxying -- **Event ingestion and streaming** β€” Kafka / Debezium CDC processing, Kafka Streams enrichment -- **Real-time messaging** β€” NATS publish/subscribe for device and tool notifications -- **Polyglot persistence** β€” MongoDB (sync + reactive), Redis caching, Apache Pinot analytics, Cassandra log storage -- **Management initializers and schedulers** β€” distributed ShedLock-based cluster-safe schedulers, startup bootstrapping -- **External REST API** β€” API key–authenticated integration surface for third parties -- **SDKs** β€” Fleet MDM and Tactical RMM Java clients +- **REST + GraphQL API surface** β€” Device, ticket, organization, and script management +- **OAuth2 / OIDC Authorization Server** β€” Multi-tenant identity with Google and Microsoft SSO support +- **JWT-based Gateway Security** β€” Reactive API gateway with API key authentication and rate limiting +- **Event-driven Messaging** β€” Hybrid Kafka (durable) and NATS (real-time) messaging +- **MongoDB Persistence** β€” Tenant-aware document model for all core aggregates +- **Apache Pinot Analytics** β€” High-performance time-series analytics for logs and device metrics +- **Tool Integrations** β€” SDK adapters for Tactical RMM and Fleet MDM +- **Agent Ingress** β€” Endpoint agent registration, heartbeat, and command dispatch --- -## Key Features & Benefits +## Key Features | Feature | Description | |---------|-------------| -| Multi-tenant by default | Every module is designed with tenant isolation: scoped keys, scoped caches, scoped scheduler locks | -| Spring Boot 3.3 / Java 21 | Latest LTS Java with virtual threads ready, modern Spring Security | -| Modular Maven structure | 30+ independent modules β€” include only what you need | -| Open source, GitHub Packages | Published to GitHub Maven Packages with unified versioning | -| Tool integrations | First-class support for Tactical RMM, Fleet MDM, MeshCentral | -| AI-ready event model | Unified event type model enables Mingo AI and Fae agent consumption | -| GraphQL + REST | Relay-compliant GraphQL for the internal API and versioned REST for external integrations | -| Reactive gateway | Spring Cloud Gateway + WebFlux + Netty for high-concurrency proxying | +| **Multi-tenancy** | Every domain entity is tenant-scoped; JWT tokens carry tenant identity | +| **OAuth2 + OIDC** | Full Authorization Server with dynamic SSO provider registration | +| **Reactive Gateway** | Spring Cloud Gateway with WebSocket proxying and upstream tool routing | +| **GraphQL with DataLoaders** | Relay-compliant GraphQL API with N+1-safe batching | +| **Hybrid Messaging** | Kafka for durable event streaming, NATS for real-time agent communication | +| **Analytics-Ready** | Apache Pinot integration for sub-second log and device queries | +| **Agent Protocol** | NATS JetStream-based heartbeat, installation, and command dispatch | +| **Extensible by Design** | Processor interfaces and hooks allow tenant-specific customization | --- ## Target Audience -This library is intended for: +This library is aimed at: -- **Platform engineers** building or extending OpenFrame microservices -- **MSP developers** integrating their tooling with the OpenFrame event model -- **Open-source contributors** improving the Flamingo/OpenFrame ecosystem - -> **Community:** Questions, discussions, and support happen on the [OpenMSP Slack Community](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA). We don't use GitHub Issues. +- **MSP Platform Engineers** building or extending OpenFrame-based services +- **Backend developers** working on multi-tenant SaaS applications +- **DevOps engineers** deploying and operating the OpenFrame stack +- **Contributors** adding new features, modules, or tool integrations --- -## High-Level Architecture +## Architecture Overview + +The following diagram shows how all modules interact end-to-end: ```mermaid flowchart TD - Client["Client / Browser / Agent"] --> Gateway["Gateway Service Core"] - Gateway --> ExternalAPI["External API Service"] - Gateway --> ApiCore["API Service Core (GraphQL + REST)"] - - ApiCore --> Authz["Authorization Service Core"] - ApiCore --> Stream["Stream Service Core (Kafka)"] - ApiCore --> Management["Management Service Core"] - - Authz --> Mongo["MongoDB"] - ApiCore --> Mongo - Stream --> Kafka["Kafka / Debezium"] - Stream --> Pinot["Apache Pinot"] - ApiCore --> Redis["Redis"] - Management --> NATS["NATS"] -``` + subgraph Edge["Edge"] + Client["Browser / Agent"] + end ---- + subgraph GatewayLayer["Gateway Service Core"] + Gateway["Reactive API Gateway"] + end -## Module Overview + subgraph SecurityLayer["Authorization & Security"] + AuthServer["Authorization Service Core"] + SecurityCore["Security OAuth & JWT"] + end -The library is organized into functional groups: + subgraph ApiLayer["API Layer"] + Rest["REST Controllers"] + GraphQL["GraphQL Layer"] + DataLoaders["GraphQL DataLoaders"] + end -```mermaid -graph LR - subgraph apiFoundation["API Foundation"] - A1["openframe-api-lib"] - A2["openframe-api-service-core"] + subgraph DomainLayer["Business & Mapping"] + Services["Business Services"] + Mapping["Mapping & Domain Services"] end - subgraph auth["Authorization"] - B1["openframe-authorization-service-core"] - B2["openframe-security-core"] - B3["openframe-security-oauth"] + + subgraph PersistenceLayer["Persistence"] + MongoDocs["Mongo Documents"] + MongoSync["Mongo Sync Repositories"] end - subgraph gateway["Gateway"] - C1["openframe-gateway-service-core"] + + subgraph MessagingLayer["Messaging"] + Kafka["Kafka"] + Nats["NATS"] end - subgraph data["Data Layer"] - D1["openframe-data-mongo-*"] - D2["openframe-data-redis"] - D3["openframe-data-kafka"] - D4["openframe-data-nats"] - D5["openframe-data-cassandra"] - D6["openframe-data-pinot"] + + subgraph StreamLayer["Stream Processing"] + StreamCore["Stream Processing Core"] end - subgraph tools["Tool SDKs"] - E1["sdk/fleetmdm"] - E2["sdk/tacticalrmm"] + + subgraph AnalyticsLayer["Analytics"] + Pinot["Apache Pinot"] end - subgraph management["Management"] - F1["openframe-management-service-core"] - F2["openframe-stream-service-core"] + + subgraph AgentIngress["Client Core Agent Ingress"] + AgentAPI["Agent Registration & Auth"] + AgentListeners["NATS & JetStream Listeners"] end + + Client --> Gateway + Gateway --> Rest + Gateway --> GraphQL + Gateway --> AuthServer + + AuthServer --> SecurityCore + + Rest --> Services + GraphQL --> DataLoaders + DataLoaders --> Services + + Services --> Mapping + Mapping --> MongoSync + MongoSync --> MongoDocs + + Services --> Kafka + Services --> Nats + + Kafka --> StreamCore + StreamCore --> Pinot + + Nats --> AgentListeners + AgentListeners --> Services ``` --- -## Current Version - -The current library version is **5.79.3** β€” published to [GitHub Packages](https://github.com/flamingo-stack/openframe-oss-lib). +## Repository Module Summary + +| Module | Purpose | +|--------|---------| +| `openframe-api-service-core` | REST + GraphQL API with security config | +| `openframe-api-lib` | Shared DTO contracts and domain services | +| `openframe-authorization-service-core` | OAuth2/OIDC Authorization Server | +| `openframe-security-core` | JWT infrastructure (encoder/decoder) | +| `openframe-security-oauth` | BFF OAuth controller and cookie strategy | +| `openframe-gateway-service-core` | Reactive edge gateway with routing and rate limiting | +| `openframe-client-core` | Agent ingress: registration, auth, NATS listeners | +| `openframe-management-service-core` | Bootstrapping, schedulers, migrations | +| `openframe-data-mongo-common` | Domain documents and base repository contracts | +| `openframe-data-mongo-sync` | MongoTemplate repository implementations | +| `openframe-data-mongo-reactive` | Reactive MongoDB repositories | +| `openframe-data-kafka` | Kafka producer, consumer, and config | +| `openframe-data-nats` | NATS publisher and real-time messaging | +| `openframe-data-pinot` | Apache Pinot query client | +| `openframe-data-redis` | Redis cache and rate limiting | +| `openframe-stream-service-core` | Kafka Streams and Debezium CDC processing | +| `openframe-external-api-service-core` | External REST API for third-party consumers | +| `openframe-debezium-initializer` | Debezium connector lifecycle management | +| `openframe-pinot-initializer` | Pinot schema and table initialization | +| `sdk/tacticalrmm` | Tactical RMM SDK (agents, scripts, schedules) | +| `sdk/fleetmdm` | Fleet MDM SDK (hosts, policies, queries) | +| `openframe-test-service-core` | E2E test framework and utilities | --- -## Links & Resources +## Community & Support + +> We use the **OpenMSP Slack community** for all discussions and support β€” we don't use GitHub Issues or GitHub Discussions. -- **OpenFrame Platform:** [https://openframe.ai](https://openframe.ai) -- **Flamingo:** [https://flamingo.run](https://flamingo.run) -- **GitHub Repository:** [https://github.com/flamingo-stack/openframe-oss-lib](https://github.com/flamingo-stack/openframe-oss-lib) -- **OpenMSP Community (Slack):** [https://www.openmsp.ai/](https://www.openmsp.ai/) -- **Reference Architecture Docs:** [./reference/architecture/README.md](./reference/architecture/README.md) +- πŸ’¬ **Join Slack**: [https://www.openmsp.ai/](https://www.openmsp.ai/) +- 🌐 **Flamingo Platform**: [https://flamingo.run](https://flamingo.run) +- πŸ”§ **OpenFrame**: [https://openframe.ai](https://openframe.ai) --- ## Next Steps -- Review the [Prerequisites Guide](prerequisites.md) for environment requirements -- Follow the [Quick Start Guide](quick-start.md) to set up the library -- Explore [First Steps](first-steps.md) to understand the key modules +Ready to dive in? Continue with: + +- **[Prerequisites](prerequisites.md)** β€” What you need before getting started +- **[Quick Start](quick-start.md)** β€” Build and run in 5 minutes +- **[First Steps](first-steps.md)** β€” Explore key features after setup diff --git a/docs/getting-started/prerequisites.md b/docs/getting-started/prerequisites.md index 8a194c1e6..09a375129 100644 --- a/docs/getting-started/prerequisites.md +++ b/docs/getting-started/prerequisites.md @@ -1,56 +1,49 @@ # Prerequisites -Before working with **openframe-oss-lib**, ensure your development environment meets all requirements below. +Before working with `openframe-oss-lib`, ensure your development environment meets the following requirements. --- ## Required Software -| Tool | Minimum Version | Purpose | -|------|----------------|---------| -| Java (JDK) | 21 | Required by all modules (Spring Boot 3.3 baseline) | -| Apache Maven | 3.9+ | Build and dependency management | -| Git | 2.x | Source code management | -| Docker | 24.x | Running integration test containers | -| Node.js | 20+ | Required for the documentation tooling (`package.json` present) | +| Tool | Minimum Version | Notes | +|------|----------------|-------| +| **Java (JDK)** | 21 | The project requires Java 21 (LTS). Use OpenJDK or Eclipse Temurin. | +| **Apache Maven** | 3.9+ | Used for building all modules. Maven wrapper (`mvnw`) is preferred. | +| **Git** | 2.x | For cloning and version control. | +| **Node.js** | 18+ | Required for documentation tooling and the `openframe-frontend-core` module. | +| **Docker** | 24+ | Required to run integration test infrastructure (MongoDB, NATS, Redis, Kafka). | +| **Docker Compose** | 2.x | Used for spinning up integration test environments. | --- -## Java Version +## Infrastructure Dependencies -This library targets **Java 21** (LTS). Ensure your `JAVA_HOME` points to a Java 21 JDK: +The OpenFrame platform integrates with several external services. Depending on the modules you work with, you may need access to: -```bash -java -version -# Should output: openjdk 21.x.x ... -``` - -Recommended distributions: +| Service | Purpose | Required For | +|---------|---------|-------------| +| **MongoDB** | Primary operational database | All modules | +| **Redis** | Caching, rate limiting, distributed locking | Gateway, Management, Data-Redis modules | +| **Apache Kafka** | Durable event streaming | Stream service, Data-Kafka modules | +| **NATS** | Real-time agent messaging | Client core, Data-NATS modules | +| **Apache Pinot** | Time-series analytics | Stream, Pinot, Data-Pinot modules | +| **Kafka Connect / Debezium** | Change Data Capture from MongoDB | Debezium initializer module | -- [Eclipse Temurin 21](https://adoptium.net/) -- [Amazon Corretto 21](https://aws.amazon.com/corretto/) -- [GraalVM 21](https://www.graalvm.org/) +> For integration tests, a Docker Compose configuration is provided in `openframe-data-mongo-sync/src/test/docker/` to start a local MongoDB instance. --- -## Maven - -Maven 3.9 or higher is required: - -```bash -mvn -version -# Apache Maven 3.9.x ... -``` - -The project uses the `flatten-maven-plugin` for CI-friendly versioning (`${revision}`), so Maven 3.9+ is required for proper resolution. +## GitHub Access ---- +### GitHub Packages (Maven Registry) -## GitHub Packages Access +The modules are published to the GitHub Packages Maven registry. You will need: -The library is published to **GitHub Maven Packages**. To consume it as a dependency in downstream services you need a GitHub Personal Access Token (PAT) with `read:packages` permission. +1. A GitHub account with access to the `flamingo-stack` organization +2. A **GitHub Personal Access Token (PAT)** with `read:packages` scope -Add to `~/.m2/settings.xml`: +Configure your `~/.m2/settings.xml` to authenticate: ```xml @@ -64,100 +57,96 @@ Add to `~/.m2/settings.xml`: ``` -> Your `GITHUB_PAT` must have the `read:packages` scope to resolve dependencies, and `write:packages` to publish new versions. - --- -## Infrastructure Services (for Integration Tests) - -Some modules run integration tests against live services via Testcontainers. Ensure Docker is running and the following images can be pulled: - -| Service | Used By | -|---------|---------| -| MongoDB | `openframe-data-mongo-sync`, `openframe-api-service-core` integration tests | -| NATS | `openframe-data-nats` integration tests | - -Testcontainers will automatically spin up and tear down containers during integration test execution. The only requirement is a running Docker daemon. +## Java Version Verification ```bash -docker info -# Should not return an error +java -version +# Expected output: +# openjdk version "21.x.x" ... + +mvn -version +# Expected output: +# Apache Maven 3.x.x ... ``` --- ## Environment Variables -The following environment variables may be needed depending on which modules you are developing: +Some modules require environment variables to be configured. The following are commonly used: -| Variable | Required For | Description | -|----------|-------------|-------------| -| `GITHUB_ACTOR` | Publishing | GitHub username for package publishing | -| `GITHUB_TOKEN` | Publishing | GitHub PAT for package publishing | +| Variable | Description | Example | +|----------|-------------|---------| +| `TENANT_ID` | Default tenant identifier for OSS deployments | `oss` | +| `SPRING_DATA_MONGODB_URI` | MongoDB connection string | `mongodb://localhost:27017/openframe` | +| `SPRING_REDIS_HOST` | Redis host | `localhost` | +| `SPRING_KAFKA_BOOTSTRAP_SERVERS` | Kafka broker address | `localhost:9092` | +| `NATS_SERVER` | NATS server URL | `nats://localhost:4222` | +| `jwt.public-key` | RSA public key (PEM, base64 encoded) | Refer to your environment configuration | +| `jwt.private-key` | RSA private key (PEM, base64 encoded) | Refer to your environment configuration | +| `oauth.client.default.id` | Default OAuth client ID | Refer to your environment configuration | +| `oauth.client.default.secret` | Default OAuth client secret | Refer to your environment configuration | -For local development without publishing, only local Maven settings are needed. +> **Security note**: Never commit secrets or private keys to version control. Use environment-specific configuration management or a secrets manager. --- -## Recommended IDE +## IDE Recommendations | IDE | Notes | |-----|-------| -| IntelliJ IDEA (Ultimate or Community) | Best support for Spring Boot, Maven multi-module, Lombok | -| VS Code + Java Extension Pack | Alternative for lighter setup | +| **IntelliJ IDEA** | Recommended. Excellent Lombok and Spring Boot support. Use the Community or Ultimate edition. | +| **VS Code** | Works well with the Java Extension Pack and Spring Boot Extension Pack. | +| **Eclipse** | Supported but requires manual Lombok and annotation processor configuration. | + +### IntelliJ IDEA Setup + +1. Open the root `pom.xml` as a Maven project +2. Enable annotation processing: `Settings β†’ Build β†’ Compiler β†’ Annotation Processors β†’ Enable` +3. Install the Lombok plugin if not already present +4. Set the project SDK to Java 21 + +--- + +## Lombok Configuration -### IntelliJ Setup Checklist +This project uses [Project Lombok](https://projectlombok.org/) for boilerplate reduction. Ensure: -1. Import as **Maven project** (not Gradle) -2. Set **Project SDK** to Java 21 -3. Enable **Annotation Processors** for Lombok support -4. Set Maven delegate: `Settings β†’ Build Tools β†’ Maven β†’ Runner β†’ Delegate IDE build/run actions to Maven` +- The Lombok JAR is available in your Maven local repository (it's declared as a dependency in the parent POM) +- Your IDE has the Lombok plugin or annotation processing enabled --- -## Verification Commands +## Checking Readiness -Run these checks before beginning development: +Run the following commands to confirm your environment is ready: ```bash -# Verify Java version +# 1. Verify Java 21 java -version -# Verify Maven version +# 2. Verify Maven mvn -version -# Verify Docker is running +# 3. Verify Docker is running docker info -# Verify Git configuration -git config user.name -git config user.email +# 4. Clone the repository +git clone https://github.com/flamingo-stack/openframe-oss-lib.git +cd openframe-oss-lib -# Verify GitHub Packages access (requires settings.xml configured) -mvn dependency:resolve -pl openframe-core -q +# 5. Attempt to compile (skipping tests initially) +mvn compile -DskipTests ``` ---- - -## System Requirements - -| Resource | Minimum | Recommended | -|----------|---------|-------------| -| RAM | 8 GB | 16 GB | -| Disk | 5 GB free | 20 GB free | -| CPU | 4 cores | 8+ cores | -| OS | macOS, Linux, Windows (WSL2) | macOS or Linux | - -> Building the entire repository (`mvn install`) compiles 30+ modules. Sufficient RAM (especially heap) prevents OOM during compilation. - -To increase Maven heap: - -```bash -export MAVEN_OPTS="-Xmx4g -XX:MaxMetaspaceSize=512m" -``` +A successful compile without errors indicates your environment is configured correctly. --- -## Community & Support +## Need Help? -If you run into setup issues, reach out on the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA). +> Join the **OpenMSP Slack community** to ask questions and get support from the community and maintainers. +> +> πŸ‘‰ [https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) diff --git a/docs/getting-started/quick-start.md b/docs/getting-started/quick-start.md index 8237d6d25..01a7a7ca7 100644 --- a/docs/getting-started/quick-start.md +++ b/docs/getting-started/quick-start.md @@ -1,39 +1,50 @@ # Quick Start -Get up and running with **openframe-oss-lib** in 5 minutes. +Get `openframe-oss-lib` cloned and compiled in under 5 minutes. + +[![OpenFrame Product Walkthrough (Beta Access)](https://img.youtube.com/vi/awc-yAnkhIo/maxresdefault.jpg)](https://www.youtube.com/watch?v=awc-yAnkhIo) --- ## TL;DR ```bash -# 1. Clone the repository +# Clone the repository git clone https://github.com/flamingo-stack/openframe-oss-lib.git cd openframe-oss-lib -# 2. Build the full library (skip tests for speed) +# Build all modules (skip tests for speed) mvn install -DskipTests - -# 3. Verify the build -mvn verify -pl openframe-core -DskipTests ``` +That's it. The library modules are now available in your local Maven repository. + --- -## Step 1 β€” Clone the Repository +## Step-by-Step Setup + +### 1. Clone the Repository ```bash git clone https://github.com/flamingo-stack/openframe-oss-lib.git cd openframe-oss-lib ``` -The repository contains 30+ Maven modules under a single parent POM at the root. +### 2. Verify Java 21 ---- +```bash +java -version +``` -## Step 2 β€” Configure GitHub Packages (Required for Dependency Resolution) +Expected output: + +```text +openjdk version "21.x.x" ... +``` -Add your GitHub credentials to `~/.m2/settings.xml`: +### 3. Configure GitHub Packages Access + +This repository publishes to GitHub Packages. Add your credentials to `~/.m2/settings.xml`: ```xml @@ -47,154 +58,133 @@ Add your GitHub credentials to `~/.m2/settings.xml`: ``` -> A GitHub Personal Access Token (PAT) with `read:packages` scope is required. See the [Prerequisites Guide](prerequisites.md) for details. - ---- - -## Step 3 β€” Build the Library - -Build all modules without running tests for the fastest initial setup: +### 4. Build All Modules ```bash mvn install -DskipTests ``` -Expected output: +This will build all 30+ modules in dependency order and install them into your local Maven repository. + +**Expected output:** ```text -[INFO] Reactor Summary for OpenFrame OSS Libraries 5.79.3: -[INFO] -[INFO] openframe-exception .................... SUCCESS [ 3.5 s] -[INFO] openframe-core ........................ SUCCESS [ 2.1 s] -[INFO] openframe-core-crypto ................. SUCCESS [ 1.8 s] -[INFO] openframe-data-mongo-common ........... SUCCESS [ 4.2 s] +[INFO] Reactor Summary for OpenFrame OSS Libraries 6.0.10: +[INFO] OpenFrame OSS Libraries ........................... SUCCESS +[INFO] openframe-exception ............................... SUCCESS +[INFO] openframe-core .................................... SUCCESS +[INFO] openframe-core-crypto ............................. SUCCESS +[INFO] openframe-data-mongo-common ....................... SUCCESS +[INFO] openframe-data-mongo-sync ......................... SUCCESS ... [INFO] BUILD SUCCESS ``` ---- - -## Step 4 β€” Add a Module as a Dependency - -To use a specific module in your own Spring Boot service, add the parent BOM and the desired dependency to your `pom.xml`: - -```xml - - - - - com.openframe.oss - openframe-oss-lib - 5.79.3 - pom - import - - - - - - - - com.openframe.oss - openframe-core - +### 5. Build a Specific Module - - - com.openframe.oss - openframe-data-mongo-sync - +If you only need to work on one module: - - - com.openframe.oss - openframe-api-service-core - - +```bash +# Example: build only the API service core module +mvn install -pl openframe-api-service-core -am -DskipTests ``` +The `-am` flag builds all modules that `openframe-api-service-core` depends on. + --- -## Step 5 β€” Run Tests for a Specific Module +## Using the Library in Your Project -To run tests for a single module: +Once built, add individual modules as Maven dependencies: -```bash -# Run unit tests only for a specific module -mvn test -pl openframe-core +```xml + + + com.openframe.oss + openframe-api-service-core + 6.0.10 + -# Run integration tests (requires Docker) -mvn verify -pl openframe-data-mongo-sync + + + + + com.openframe.oss + openframe-oss-lib + 6.0.10 + pom + import + + + ``` -> Integration tests use Testcontainers and require a running Docker daemon. - --- -## Available Modules β€” Quick Reference - -| Module | GroupId | Description | -|--------|---------|-------------| -| `openframe-core` | `com.openframe.oss` | Core utilities, pagination, validation | -| `openframe-exception` | `com.openframe.oss` | Standard exception hierarchy | -| `openframe-core-crypto` | `com.openframe.oss` | Encryption utilities | -| `openframe-security-core` | `com.openframe.oss` | JWT, PKCE, cookie service | -| `openframe-security-oauth` | `com.openframe.oss` | OAuth2 BFF layer | -| `openframe-authorization-service-core` | `com.openframe.oss` | Multi-tenant OAuth2 auth server | -| `openframe-api-lib` | `com.openframe.oss` | API contracts, filter DTOs | -| `openframe-api-service-core` | `com.openframe.oss` | REST + GraphQL API service layer | -| `openframe-gateway-service-core` | `com.openframe.oss` | Reactive gateway, routing, security | -| `openframe-data-mongo-common` | `com.openframe.oss` | MongoDB domain documents | -| `openframe-data-mongo-sync` | `com.openframe.oss` | Synchronous MongoDB repositories | -| `openframe-data-mongo-reactive` | `com.openframe.oss` | Reactive MongoDB repositories | -| `openframe-data-redis` | `com.openframe.oss` | Redis cache configuration | -| `openframe-data-kafka` | `com.openframe.oss` | Kafka multi-tenant configuration | -| `openframe-data-nats` | `com.openframe.oss` | NATS real-time messaging | -| `openframe-data-cassandra` | `com.openframe.oss` | Cassandra log storage | -| `openframe-data-pinot` | `com.openframe.oss` | Apache Pinot analytics | -| `openframe-management-service-core` | `com.openframe.oss` | Schedulers, initializers | -| `openframe-stream-service-core` | `com.openframe.oss` | Kafka streams, event enrichment | -| `openframe-external-api-service-core` | `com.openframe.oss` | External REST API | -| `sdk/fleetmdm` | `com.openframe.oss` | Fleet MDM Java SDK | -| `sdk/tacticalrmm` | `com.openframe.oss` | Tactical RMM Java SDK | +## Available Modules (Quick Reference) + +| Module Artifact ID | Description | +|-------------------|-------------| +| `openframe-core` | Core utilities (pagination, constants, slug) | +| `openframe-exception` | Shared exception hierarchy and error codes | +| `openframe-security-core` | JWT encoder/decoder, PKCE utilities | +| `openframe-security-oauth` | OAuth BFF controller and token cookies | +| `openframe-api-lib` | Shared DTO contracts and domain services | +| `openframe-api-service-core` | REST + GraphQL API service core | +| `openframe-authorization-service-core` | OAuth2/OIDC Authorization Server | +| `openframe-gateway-service-core` | Reactive API Gateway | +| `openframe-client-core` | Agent registration and ingress | +| `openframe-management-service-core` | Management services and schedulers | +| `openframe-data-mongo-common` | MongoDB domain documents | +| `openframe-data-mongo-sync` | MongoDB sync repositories | +| `openframe-data-mongo-reactive` | Reactive MongoDB repositories | +| `openframe-data-redis` | Redis cache and rate-limit support | +| `openframe-data-kafka` | Kafka producers and configuration | +| `openframe-data-nats` | NATS messaging publishers | +| `openframe-data-pinot` | Apache Pinot query repositories | +| `openframe-stream-service-core` | Kafka Streams and CDC processing | +| `openframe-external-api-service-core` | External REST API for consumers | +| `openframe-debezium-initializer` | Debezium connector lifecycle | +| `openframe-pinot-initializer` | Pinot schema initialization | +| `fleetmdm` | Fleet MDM SDK | +| `tacticalrmm` | Tactical RMM SDK | +| `openframe-test-service-core` | E2E test utilities | --- -## Expected Results +## Running Unit Tests -After a successful `mvn install -DskipTests` you should see: +```bash +# Run all unit tests +mvn test -```text -[INFO] BUILD SUCCESS -[INFO] Total time: 2-4 minutes (depending on hardware) -[INFO] Finished at: ... +# Run tests for a single module +mvn test -pl openframe-api-service-core ``` -All modules are installed into your local Maven repository (`~/.m2/repository/com/openframe/oss/`). +## Running Integration Tests ---- - -## Video Walkthrough +Integration tests require Docker to be running (for MongoDB, Redis, NATS, etc.): -[![Getting Started with OpenFrame - Organization Setup Basics](https://img.youtube.com/vi/-_56_qYvMWk/maxresdefault.jpg)](https://www.youtube.com/watch?v=-_56_qYvMWk) +```bash +# Run integration tests (requires Docker) +mvn verify -pl openframe-data-mongo-sync +``` --- -## Troubleshooting +## Expected Results + +After a successful build: -| Problem | Solution | -|---------|---------| -| `Could not resolve dependencies` | Check `~/.m2/settings.xml` for GitHub credentials | -| `OutOfMemoryError` during build | Set `export MAVEN_OPTS="-Xmx4g"` | -| Integration tests fail | Ensure Docker daemon is running | -| `flatten-maven-plugin` errors | Upgrade Maven to 3.9+ | +- All module JARs are installed in your local Maven repository under `~/.m2/repository/com/openframe/oss/` +- You can import and extend any module in your downstream Spring Boot services +- The complete OpenFrame library stack is available as a dependency BOM --- ## Next Steps -After building successfully, explore: - -- [First Steps Guide](first-steps.md) for key module walkthroughs -- [Development Environment Setup](../development/setup/environment.md) for IDE configuration -- [Architecture Overview](../development/architecture/README.md) for system design patterns +- Follow the **[First Steps Guide](first-steps.md)** to explore the key modules +- Review the **[Prerequisites](prerequisites.md)** if you encounter build issues +- Explore the architecture docs in `./reference/architecture/` for detailed module design diff --git a/docs/reference/architecture/README.md b/docs/reference/architecture/README.md index 3d2a87c5d..d078bec1b 100644 --- a/docs/reference/architecture/README.md +++ b/docs/reference/architecture/README.md @@ -1,447 +1,442 @@ # OpenFrame OSS Lib – Repository Overview -The **openframe-oss-lib** repository contains the foundational backend libraries powering the OpenFrame platform. It provides: +The **`openframe-oss-lib`** repository is the modular backend foundation of the OpenFrame platform. +It provides a complete multi-tenant, event-driven, analytics-enabled architecture for: -- Multi-tenant authentication and authorization -- API service layers (REST + GraphQL) -- Gateway routing and security -- Event ingestion and streaming (Kafka) -- Real-time messaging (NATS) -- Polyglot persistence (MongoDB, Redis, Pinot) -- Management initializers and distributed schedulers +- βœ… API services (REST + GraphQL) +- βœ… OAuth2 / OIDC authorization +- βœ… JWT-based security +- βœ… Agent ingress & command execution +- βœ… Kafka & NATS messaging +- βœ… MongoDB operational persistence +- βœ… Apache Pinot analytics +- βœ… Tool integrations (Tactical RMM, Fleet MDM, etc.) -This repository represents the **core backend infrastructure stack** of OpenFrame and is designed to be modular, extensible, and multi-tenant by default. +This repository is structured as a set of independent but cohesive modules forming a layered, scalable backend system. --- -# High-Level Architecture +# 🎯 Purpose of the Repository -OpenFrame follows a layered, service-oriented architecture with strong separation between: +`openframe-oss-lib` delivers: -- Identity & Authorization -- API Surface (Internal + External) -- Gateway & Edge Security -- Stream Processing -- Data Access Layer -- Caching & Messaging -- Management & Operations +1. **Multi-tenant API infrastructure** +2. **OAuth2 Authorization Server** +3. **JWT security & BFF orchestration** +4. **Reactive Gateway layer** +5. **Device & ticket lifecycle management** +6. **Real-time event ingestion & enrichment** +7. **High-performance analytics via Pinot** +8. **Tool SDK abstraction layer** +9. **Agent ingress & command execution** +10. **Operational management & migrations** -```mermaid -flowchart TD - Client["Client / Browser / Agent"] --> Gateway["Gateway Service Core"] - Gateway --> ExternalAPI["External API Service"] - Gateway --> ApiCore["API Service Core"] - - ApiCore --> Authz["Authorization Service Core"] - ApiCore --> Stream["Stream Service Core"] - ApiCore --> Management["Management Service Core"] - - Authz --> Mongo["MongoDB"] - ApiCore --> Mongo - Stream --> Kafka["Kafka"] - Stream --> Pinot["Pinot"] - ApiCore --> Redis["Redis"] - Management --> NATS["NATS"] -``` +It is designed to support both: +- 🟒 OSS single-tenant deployments +- πŸ”΅ SaaS multi-tenant environments --- -# Core Modules +# πŸ— End-to-End Architecture -Below is a structured overview of the major modules contained in this repository. +Below is the complete system architecture across all modules: ---- +```mermaid +flowchart TD -## 1. API Foundation + subgraph Edge + Client["Browser / Agent"] + end + + subgraph GatewayLayer["Gateway Service Core"] + Gateway["Reactive API Gateway"] + end + + subgraph SecurityLayer["Authorization & Security"] + AuthServer["Authorization Service Core"] + SecurityCore["Security OAuth & JWT"] + end + + subgraph ApiLayer["API Layer"] + Rest["REST Controllers"] + GraphQL["GraphQL Layer"] + DataLoaders["GraphQL DataLoaders"] + ApiDtos["API DTOs"] + end + + subgraph DomainLayer["Business & Mapping"] + Services["Business Services"] + Mapping["Mapping & Domain Services"] + end + + subgraph PersistenceLayer["Persistence"] + MongoDocs["Mongo Documents"] + MongoSync["Mongo Sync Repositories"] + end + + subgraph MessagingLayer["Messaging"] + Kafka["Kafka"] + Nats["NATS"] + end + + subgraph StreamLayer["Stream Processing"] + StreamCore["Stream Processing Core"] + end + + subgraph AnalyticsLayer["Analytics"] + Pinot["Analytics Pinot"] + end + + subgraph AgentIngress["Client Core Agent Ingress"] + AgentAPI["Agent Registration & Auth"] + AgentListeners["NATS & JetStream Listeners"] + end -### πŸ”Ή `api-contracts-and-pagination` -Defines reusable API contracts: + Client --> Gateway + Gateway --> Rest + Gateway --> GraphQL + Gateway --> AuthServer -- Relay-style pagination (`ConnectionArgs`, cursors) -- Count-aware query results -- Opaque cursor encoding (`CursorCodec`) -- Standard mutation inputs + AuthServer --> SecurityCore -πŸ“„ Core Docs: -- `CountedGenericQueryResult` -- `ConnectionArgs` -- `CursorCodec` -- `MutationDeleteInput` + Rest --> Services + GraphQL --> DataLoaders + DataLoaders --> Services ---- + Services --> Mapping + Mapping --> MongoSync + MongoSync --> MongoDocs -### πŸ”Ή `api-domain-filters-dtos` -Strongly-typed filter DTOs for: + Services --> Kafka + Services --> Nats -- Devices -- Events -- Logs -- Knowledge Base -- Organizations -- Tools + Kafka --> StreamCore + StreamCore --> Pinot -Bridges API input β†’ Mongo query filters. + Nats --> AgentListeners + AgentListeners --> Services +``` --- -### πŸ”Ή `api-lib-core-services` -Reusable domain services: +# πŸ“¦ Repository Module Structure -- Installed agent resolution -- Tool connections -- Ticket queries -- Knowledge base lifecycle hooks -- Device status processors - -Optimized for DataLoader and GraphQL batching. +The repository is composed of the following major modules: --- -## 2. API Service Core +## 1️⃣ API Contract & DTO Layer -### πŸ”Ή REST Controllers -Internal operational endpoints: +### `api-lib-dto-contracts` +**Purpose:** Defines stable transport contracts shared between REST, GraphQL, and services. -- Organizations -- Devices -- Users -- Invitations -- SSO Config -- API Keys -- Release versions -- Agent lifecycle control +Includes: +- Device filters +- Organization responses +- Script commands +- Tool filters +- Relay pagination primitives +- Command dispatch contracts -πŸ“„ Module: `api-service-core-rest-controllers` +πŸ‘‰ Reference: **Api Lib Dto Contracts documentation** --- -### πŸ”Ή GraphQL Layer -Relay-compliant GraphQL execution layer: - -- Data fetchers -- DataLoaders -- Relay Node resolution -- Cursor-based pagination -- Polymorphic type resolution +### `api-service-core-dtos` +Defines REST & GraphQL DTOs for: +- SSO +- OAuth +- Invitations +- Knowledge base +- Notifications +- Force tool operations +- Relay connections -πŸ“„ Modules: -- `api-service-core-graphql-datafetchers` -- `api-service-core-dataloaders` -- `api-service-core-graphql-dtos` -- `api-service-core-relay-type-resolution` +πŸ‘‰ Reference: **Api Service Core Dtos documentation** --- -### πŸ”Ή Security & Config -Provides: - -- JWT resource server config -- Multi-issuer support -- Custom GraphQL scalars -- OAuth client initialization -- Argument resolvers +## 2️⃣ API Service Core -πŸ“„ Module: `api-service-core-config-and-security` +### `api-service-core-rest-controllers` +Thin HTTP controllers for: +- Organizations +- Devices +- API Keys +- Invitations +- SSO +- User management +- Force tool operations --- -## 3. Authorization Service Core - -Multi-tenant OAuth2 Authorization Server. - -### Capabilities +### `api-service-core-graphql-layer` +Relay-compliant GraphQL API: +- Node resolution +- Cursor pagination +- Mutations +- Type resolvers +- DataLoader integration -- Per-tenant RSA key pairs -- JWT issuance with `tenant_id` -- Dynamic client registration -- SSO flows (Google, Microsoft) -- PKCE support -- Invitation-based onboarding -- Tenant discovery & registration - -```mermaid -flowchart TD - Browser --> AuthController["Auth Controllers"] - AuthController --> TenantContext["TenantContextFilter"] - TenantContext --> AuthServer["AuthorizationServerConfig"] - AuthServer --> TenantKeyService["TenantKeyService"] - TenantKeyService --> Mongo - AuthServer --> JWT["Signed JWT"] -``` +--- -πŸ“„ Modules: -- `authorization-service-core-server-and-tenant` -- `authorization-service-core-auth-controllers-and-dtos` -- `authorization-service-core-keys-and-authorization-persistence` -- `authorization-service-core-sso-flow-and-utils` +### `api-service-core-graphql-dataloaders` +Prevents N+1 queries with batched loading for: +- Machines +- Organizations +- Tags +- Knowledge Base +- Tickets +- Tool connections --- -## 4. Gateway Service Core - -Reactive edge gateway built on Spring Cloud Gateway + WebFlux. +### `api-service-core-business-services` +Core domain orchestration: +- User lifecycle +- SSO configuration +- Domain validation +- Extension processors -### Responsibilities +--- -- JWT validation (multi-issuer) -- Role-based authorization -- API key authentication + rate limiting -- WebSocket proxying -- Tool upstream resolution -- Origin sanitization -- CORS enforcement +### `api-lib-mapping-and-domain-services` +Mapping layer between: +- DTOs +- Domain documents +- Repository layer -```mermaid -flowchart TD - Request --> OriginSanitizer - OriginSanitizer --> JwtAuth - JwtAuth --> RoleCheck - RoleCheck --> RouteDecision - RouteDecision --> ToolResolver - ToolResolver --> UpstreamTool -``` +--- -πŸ“„ Module: `gateway-service-core-security-and-routing` +### `api-service-core-config-and-security` +Infrastructure configuration: +- JWT resource server +- OAuth integration +- Custom GraphQL scalars +- OAuth client initialization --- -## 5. Stream Service Core (Kafka) +## 3️⃣ Authorization & Security -Event ingestion and normalization backbone. +### `authorization-service-core` +Full OAuth2 + OIDC Authorization Server: -### Responsibilities - -- Consume Debezium CDC events -- Tool-specific deserialization -- Event enrichment (tenant, device, org) -- Unified event type mapping -- Cassandra persistence -- Kafka republishing -- Kafka Streams enrichment +- Multi-tenant JWT issuers +- RSA signing keys per tenant +- PKCE enforcement +- Dynamic SSO providers +- Invitation onboarding +- Tenant registration ```mermaid -flowchart TD - DebeziumEvent --> Deserializer - Deserializer --> Enrichment - Enrichment --> EventTypeMapper - EventTypeMapper --> Cassandra - EventTypeMapper --> KafkaOut +flowchart LR + User["User"] --> Auth["Authorization Server"] + Auth --> IdP["Google / Microsoft"] + Auth --> JWT["Tenant Scoped JWT"] + JWT --> Gateway ``` -πŸ“„ Module: `stream-service-core-kafka-and-handlers` - --- -## 6. Data Layer - -### MongoDB - -- Domain documents (`User`, `Organization`, `Device`, `Ticket`, etc.) -- Base repositories (sync + reactive) -- Custom query repositories -- Cursor pagination logic -- Index configuration - -πŸ“„ Modules: -- `data-mongo-domain-model` -- `data-mongo-query-filters` -- `data-mongo-base-repositories` -- `data-mongo-sync-config-and-custom-repositories` -- `data-mongo-reactive-repositories` +### `security-oauth-and-jwt` +Provides: +- JWT encoder/decoder +- PKCE utilities +- OAuth BFF controller +- Secure cookie strategy --- -### Redis +## 4️⃣ Gateway Layer -- Tenant-aware cache key prefixing -- Spring Cache integration -- Reactive + blocking templates +### `gateway-service-core` -πŸ“„ Module: `data-redis-cache` +Reactive Spring Cloud Gateway: ---- +- JWT validation +- API key authentication +- Role-based routing +- WebSocket proxying +- Tool upstream resolution +- Rate limiting -### Kafka +--- -- Tenant-scoped Kafka configuration -- Topic auto-provisioning -- Debezium message modeling -- Retry & recovery handling +## 5️⃣ Persistence Layer -πŸ“„ Module: `data-kafka-configuration-and-retry` +### `data-model-and-repositories-mongo` +Defines: +- MongoDB documents +- Query filters +- Base repositories +- Tenant ID provider --- -### NATS (Real-Time Notifications) - -- Persist-first notification strategy -- Read-state tracking -- Per-recipient NATS publish -- Graceful degradation if NATS disabled - -πŸ“„ Module: `data-nats-notifications` +### `data-access-mongo-sync` +MongoTemplate-based advanced repository implementations: +- Cursor pagination +- Aggregations +- Ticket lifecycle +- Device filtering +- Notification joins +- Optimistic locking retries --- -### Pinot (Analytics) +## 6️⃣ Messaging & Eventing -- Log exploration -- Device faceted filtering -- High-performance aggregation queries +### `eventing-and-messaging-kafka-nats` -```mermaid -flowchart LR - StreamService --> PinotCluster - PinotCluster --> PinotRepositories - PinotRepositories --> ApiService -``` +Hybrid messaging model: -πŸ“„ Module: `data-pinot-repositories` +- Kafka for durable streaming +- NATS for real-time messaging +- Debezium CDC integration +- Notification broadcasting +- Agent command execution --- -## 7. Management Service Core +## 7️⃣ Stream Processing -Operational backbone of the platform. +### `stream-processing-core` -### Startup Initializers +Real-time event enrichment engine: -- Agent registration secret bootstrap -- NATS stream provisioning -- Client configuration loading -- Tactical RMM script initialization - -### Distributed Schedulers (ShedLock + Redis) - -- Offline device detection -- API key stats sync -- Fleet MDM setup -- Version publish fallback +- Tool event deserialization +- Tenant resolution +- Unified event taxonomy +- Kafka Streams joins +- Cassandra integration +- Pinot ingestion support ```mermaid flowchart TD - Startup --> Initializers - Initializers --> StreamsReady - StreamsReady --> Schedulers - Schedulers --> ExternalSystems + Kafka --> Deserializer + Deserializer --> Enrichment + Enrichment --> UnifiedEvent + UnifiedEvent --> Cassandra + UnifiedEvent --> Pinot ``` -πŸ“„ Module: `management-service-core-initializers-and-schedulers` - --- -## 8. External API Service Core +## 8️⃣ Analytics -Public REST interface for third-party integrations. +### `analytics-pinot` -- API key–based authentication -- Cursor-based pagination -- Device / Event / Log / Organization APIs -- Tool proxy endpoints -- OpenAPI documentation +High-performance read layer: -πŸ“„ Module: `external-api-service-core` +- Device faceted filtering +- Log search +- Cursor pagination +- Distinct filter options +- Tenant isolation +- Broker warm-up --- -## 9. Security Core & OAuth BFF - -Frontend-safe OAuth2 BFF layer. +## 9️⃣ Agent Ingress -- PKCE utilities -- JWT encoder/decoder -- OAuth login flow -- Token refresh & revocation -- HttpOnly cookie management +### `client-core-agent-ingress` -```mermaid -sequenceDiagram - participant Browser - participant BFF - participant AuthServer - - Browser->>BFF: GET /oauth/login - BFF->>AuthServer: Redirect with PKCE - AuthServer->>BFF: Callback with code - BFF->>AuthServer: Exchange code - BFF->>Browser: Set cookies -``` +Handles: -πŸ“„ Module: `security-core-and-oauth-bff` +- Agent registration +- OAuth token issuance +- Tool ID transformation +- Machine heartbeat tracking +- JetStream event consumption --- -# End-to-End System View +## πŸ”Ÿ Management & Operations -```mermaid -flowchart TD - Client --> Gateway - Gateway --> ExternalAPI - Gateway --> ApiCore +### `management-service-core` - ApiCore --> Authz - ApiCore --> Mongo - ApiCore --> Redis - ApiCore --> Pinot +Operational control plane: - Authz --> Mongo - Authz --> JWT +- NATS stream initialization +- Tool lifecycle management +- Version publishing +- Distributed schedulers +- Mongo migrations (Mongock) +- Ticket status seeding - Stream --> Kafka - Stream --> Cassandra - Stream --> Pinot +--- - Management --> Redis - Management --> NATS -``` +## 1️⃣1️⃣ Integrations SDKs ---- +### `integrations-sdks` -# Design Principles +Typed SDK abstractions for: + +- Tactical RMM +- Fleet MDM + +Provides: +- Strongly typed request/response models +- Auto-configuration +- Script scheduling support +- Policy management -The repository follows consistent architectural principles: +--- -1. **Multi-Tenancy First** - - Tenant ID embedded in JWT - - Tenant-scoped keys and locks - - Tenant-aware caching +# πŸ” Cross-Cutting Architecture Patterns -2. **Separation of Concerns** - - Domain model isolated from transport - - Query filters separate from API DTOs - - Gateway isolated from business logic +- βœ… Multi-tenant isolation at all layers +- βœ… Relay-compliant GraphQL +- βœ… Cursor-based pagination everywhere +- βœ… Distributed scheduling via ShedLock +- βœ… Event-driven architecture (Kafka + NATS) +- βœ… Asymmetric JWT signing +- βœ… PKCE enforcement +- βœ… Broker warm-up for analytics +- βœ… Soft-delete instead of hard deletes +- βœ… Extension-point processor pattern -3. **Polyglot Persistence** - - MongoDB β†’ transactional - - Pinot β†’ analytical - - Redis β†’ caching - - Kafka β†’ streaming - - NATS β†’ real-time +--- -4. **Event-Driven Architecture** - - Debezium ingestion - - Kafka normalization - - Unified event model +# 🧠 End-to-End Request Lifecycle Example -5. **Extensibility** - - Conditional beans - - Processor hooks - - Strategy patterns - - Pluggable resolvers +```mermaid +sequenceDiagram + participant Browser + participant Gateway + participant Auth + participant API + participant Mongo + participant Kafka + participant Pinot + + Browser->>Gateway: GraphQL Query + Gateway->>Auth: Validate JWT + Gateway->>API: Forward request + API->>Mongo: Fetch domain data + API-->>Browser: Response + + API->>Kafka: Publish event + Kafka->>Pinot: Analytics ingestion +``` --- -# Conclusion +# βœ… Summary -The **openframe-oss-lib** repository is the foundational backend library stack of OpenFrame. It provides: +The **`openframe-oss-lib`** repository is a complete backend platform composed of: -- A multi-tenant OAuth2 authorization server -- Internal and external API layers -- A reactive gateway -- Event streaming and normalization -- Analytics querying -- Distributed management workflows -- Secure OAuth BFF flows -- Polyglot data infrastructure +- Secure multi-tenant authorization +- Reactive gateway routing +- REST & GraphQL APIs +- Domain-driven services +- Mongo operational storage +- Kafka + NATS messaging +- Stream enrichment pipelines +- Pinot analytics +- Agent ingress services +- Tool integration SDKs +- Operational management control plane -Together, these modules form a production-grade, scalable, multi-tenant backend architecture that powers the OpenFrame platform end-to-end. \ No newline at end of file +It provides a modular, scalable, and extensible foundation for building a unified, AI-enhanced MSP backend platform. \ No newline at end of file diff --git a/docs/reference/architecture/analytics-pinot/analytics-pinot.md b/docs/reference/architecture/analytics-pinot/analytics-pinot.md new file mode 100644 index 000000000..15dffc321 --- /dev/null +++ b/docs/reference/architecture/analytics-pinot/analytics-pinot.md @@ -0,0 +1,348 @@ +# Analytics Pinot + +The **Analytics Pinot** module provides high-performance, real-time analytics capabilities for the OpenFrame platform using Apache Pinot. It is responsible for querying large volumes of time-series and event data (devices, logs, and related metadata) with low latency, enabling dashboards, filtering, search, and reporting use cases. + +This module acts as the analytical read layer of the system, complementing MongoDB (operational storage) and Kafka/NATS (event ingestion and streaming). + +--- + +## 1. Purpose and Responsibilities + +Analytics Pinot is designed to: + +- Provide fast, faceted filtering for devices +- Enable time-range log search with cursor-based pagination +- Support distinct filter option queries (e.g., severity, event type, tool type) +- Offer organization-based analytics projections +- Abstract Apache Pinot query construction and execution behind repository interfaces +- Warm up Pinot brokers at application startup to reduce first-query latency + +It is optimized for read-heavy workloads and analytical queries across large datasets. + +--- + +## 2. High-Level Architecture + +Analytics Pinot sits between application services (API layer, GraphQL, external APIs) and the Pinot cluster. + +```mermaid +flowchart LR + Client["API Layer / GraphQL / External API"] --> DeviceRepo["PinotClientDeviceRepository"] + Client --> LogRepo["PinotClientLogRepository"] + + DeviceRepo --> QueryBuilder["PinotQueryBuilder"] + LogRepo --> QueryBuilder + + QueryBuilder --> BrokerConn["Pinot Broker Connection"] + BrokerConn --> PinotCluster[("Apache Pinot Cluster")] + + ControllerConn["Pinot Controller Connection"] --> PinotCluster + + Warmup["PinotBrokerWarmup"] --> BrokerConn +``` + +### Key Architectural Concepts + +- **Repository Pattern**: Encapsulates Pinot query logic. +- **Query Builder Abstraction**: Dynamically builds SQL queries with filters and sorting. +- **Broker vs Controller Connections**: + - Broker: Executes analytical queries. + - Controller: Used for cluster-level operations if needed. +- **Startup Warm-Up**: Ensures brokers are ready before first user query. + +--- + +## 3. Configuration Layer + +### 3.1 PinotConfig + +`PinotConfig` defines Spring beans for connecting to Apache Pinot: + +- `pinotBrokerConnection()` +- `pinotControllerConnection()` + +Configuration properties: + +```text +pinot.broker.url +pinot.controller.url +pinot.tables.devices.name +pinot.tables.logs.name +``` + +These connections are injected into repositories via `@Qualifier("pinotBrokerConnection")`. + +### 3.2 PinotBrokerWarmup + +`PinotBrokerWarmup` is conditionally enabled via: + +```text +pinot.broker.warmup.enabled=true +``` + +On `ApplicationReadyEvent`, it executes: + +```sql +SELECT COUNT(*) FROM "devices" LIMIT 1 +``` + +Purpose: + +- Pre-load broker routing tables +- Reduce cold-start latency +- Fail gracefully (non-blocking) if warm-up fails + +--- + +## 4. Data Models + +### 4.1 OrganizationOption + +A lightweight projection model: + +```text +id -> organization identifier +name -> display name +``` + +Used when returning distinct organization filter options from logs. + +### 4.2 PinotEventEntity + +Currently acts as a placeholder entity for Pinot-backed event projections. It can be extended to represent denormalized event documents optimized for analytical reads. + +--- + +## 5. Device Analytics Repository + +### PinotClientDeviceRepository + +Responsible for device-level analytics and filter aggregation. + +#### Core Responsibilities + +- Faceted filter queries (status, device type, OS type, organization, tags) +- Filtered device count queries +- Multi-dimensional filtering +- Exclusion of logically deleted devices + +### Faceted Query Pattern + +Each filter option query follows this structure: + +```sql +SELECT {facetField}, COUNT(*) as count +FROM devices +WHERE +GROUP BY {facetField} +ORDER BY count DESC +``` + +Implemented through: + +- `executeFacetQuery(...)` +- `applyDeviceFilters(...)` + +### Device Filter Logic + +All queries: + +- Exclude `DELETED` status +- Apply OR-based filtering for: + - status + - deviceType + - osType + - organizationId + - tags + - tagKeyValues + +```mermaid +flowchart TD + Start["Start Device Query"] --> ExcludeDeleted["Exclude DELETED Status"] + ExcludeDeleted --> ApplyStatus["Apply Status Filters"] + ApplyStatus --> ApplyType["Apply Device Type Filters"] + ApplyType --> ApplyOs["Apply OS Type Filters"] + ApplyOs --> ApplyOrg["Apply Organization Filters"] + ApplyOrg --> ApplyTags["Apply Tag Filters"] + ApplyTags --> Execute["Execute Pinot Query"] +``` + +### Counting Devices + +`getFilteredDeviceCount(...)` builds a `SELECT COUNT(*)` query using the same filtering logic to ensure consistency between facet results and total counts. + +--- + +## 6. Log Analytics Repository + +### PinotClientLogRepository + +Handles time-series log queries with advanced filtering and sorting. + +#### Core Capabilities + +- Time-range filtering +- Full-text relevance search +- Cursor-based pagination +- Distinct filter option queries +- Organization option projections +- Controlled sorting via allowlist + +### 6.1 Log Query Flow + +```mermaid +flowchart TD + Input["Log Query Request"] --> Build["Build PinotQueryBuilder"] + Build --> DateRange["Apply Date Range"] + DateRange --> Filters["Apply Tool / Event / Severity / Org Filters"] + Filters --> Search["Apply Relevance Search (optional)"] + Search --> Cursor["Apply Cursor"] + Cursor --> Sort["Validate & Apply Sort"] + Sort --> Limit["Apply Limit"] + Limit --> Execute["Execute Pinot Query"] +``` + +### 6.2 Sorting Safety + +Only fields in `SORTABLE_COLUMNS` are allowed: + +```text +eventTimestamp +severity +eventType +toolType +organizationId +deviceId +ingestDay +``` + +If the requested field is invalid: + +- Sorting falls back to `eventTimestamp` +- A primary key (`toolEventId`) ensures deterministic pagination + +### 6.3 Search vs Standard Query + +- `findLogs(...)` β†’ Structured filtering only +- `searchLogs(...)` β†’ Includes `whereRelevanceLogSearch(searchTerm)` + +This allows Pinot's indexing and relevance scoring to power free-text search. + +### 6.4 Distinct Option Queries + +Used to dynamically populate filter dropdowns: + +- `getEventTypeOptions(...)` +- `getSeverityOptions(...)` +- `getToolTypeOptions(...)` +- `getAvailableDateRanges(...)` +- `getOrganizationOptions(...)` + +These use `SELECT DISTINCT` patterns optimized for analytics. + +--- + +## 7. Query Execution Abstraction + +Both repositories extend `AbstractPinotRepository`. + +Responsibilities abstracted away: + +- Query execution against Pinot +- Result set mapping +- Column index resolution +- Count query execution +- Key-count mapping + +This ensures: + +- Centralized Pinot integration logic +- Reduced duplication +- Easier instrumentation and logging + +--- + +## 8. Multi-Tenancy Model + +All queries require a `tenantId`. + +The `PinotQueryBuilder` enforces tenant scoping at query construction time. + +```mermaid +flowchart LR + Request["Tenant Scoped Request"] --> Builder["PinotQueryBuilder(tenantId)"] + Builder --> Query["SELECT ... WHERE tenantId = X"] + Query --> Pinot[("Pinot Cluster")] +``` + +This guarantees: + +- Tenant isolation at the analytics layer +- Safe multi-tenant data access + +--- + +## 9. Integration with the Platform + +Analytics Pinot interacts with: + +- Stream processing modules (for ingestion into Pinot tables) +- Device and log-producing services +- API and GraphQL layers consuming analytics projections + +Typical data flow: + +```mermaid +flowchart LR + Producers["Devices / Tools / Stream Processors"] --> Kafka["Kafka / Streaming"] + Kafka --> PinotIngest["Pinot Real-Time Tables"] + PinotIngest --> Analytics["Analytics Pinot Module"] + Analytics --> ApiLayer["API / GraphQL"] + ApiLayer --> UI["Dashboard / Client"] +``` + +--- + +## 10. Performance Characteristics + +Analytics Pinot is optimized for: + +- Large-scale time-series data +- High-cardinality filter dimensions +- Fast aggregation queries +- Real-time dashboard refresh + +Key optimizations include: + +- Faceted aggregation queries +- Date partition filtering (`ingestDay`) +- Controlled sortable fields +- Broker warm-up at startup + +--- + +## 11. Design Principles + +- βœ… Separation of analytical and operational storage +- βœ… Repository abstraction over raw Pinot client +- βœ… Explicit multi-tenant scoping +- βœ… Deterministic cursor-based pagination +- βœ… Defensive sorting validation +- βœ… Non-blocking warm-up behavior + +--- + +## 12. Summary + +The **Analytics Pinot** module provides the analytical backbone of OpenFrame, enabling real-time insights into devices and logs at scale. + +It: + +- Connects securely to Apache Pinot +- Exposes optimized repositories for devices and logs +- Supports faceted filtering and full-text search +- Enforces tenant isolation +- Ensures predictable and safe sorting behavior +- Improves startup performance with broker warm-up + +Together, these capabilities allow the platform to deliver responsive dashboards, advanced filtering, and scalable analytics over large operational datasets. diff --git a/docs/reference/architecture/api-contracts-and-pagination/api-contracts-and-pagination.md b/docs/reference/architecture/api-contracts-and-pagination/api-contracts-and-pagination.md deleted file mode 100644 index 8072b0290..000000000 --- a/docs/reference/architecture/api-contracts-and-pagination/api-contracts-and-pagination.md +++ /dev/null @@ -1,329 +0,0 @@ -# Api Contracts And Pagination - -## Overview - -The **Api Contracts And Pagination** module defines the foundational API data contracts used across OpenFrame services, with a strong focus on: - -- Relay-style cursor-based pagination -- Count-aware query results -- Opaque cursor encoding and decoding -- Standardized mutation input structures - -This module sits at the boundary between transport layers (REST and GraphQL) and domain/query services. It ensures consistent, predictable API behavior across the platform while hiding internal persistence details (such as MongoDB identifiers or composite keys) from API consumers. - ---- - -## Core Responsibilities - -The Api Contracts And Pagination module provides: - -1. **Generic query result wrappers** with total/filtered counts -2. **Relay-compliant pagination arguments** -3. **Opaque cursor encoding/decoding utilities** -4. **Standard mutation input contracts (e.g., delete by ID)** - -These contracts are reused by: - -- GraphQL data fetchers -- REST controllers -- Domain query services -- MongoDB custom repositories - ---- - -## Architectural Positioning - -The module acts as a shared contract layer between API entry points and data/query layers. - -```mermaid -flowchart TD - Client["Client Application"] --> API["REST Controllers / GraphQL DataFetchers"] - API --> Contracts["Api Contracts And Pagination"] - Contracts --> Services["Domain Query Services"] - Services --> Repositories["Mongo Repositories"] - Repositories --> Database[("MongoDB")] -``` - -### Key Characteristics - -- **Transport-agnostic**: Works with both REST and GraphQL. -- **Persistence-agnostic**: Does not expose internal database identifiers. -- **Validation-aware**: Uses Jakarta validation annotations for input constraints. -- **Relay-compliant**: Follows the GraphQL Relay Connection specification patterns. - ---- - -# Core Components - -## 1. CountedGenericQueryResult - -**Class:** `CountedGenericQueryResult` -**Extends:** `GenericQueryResult` - -### Purpose - -Adds a `filteredCount` field to a generic query result, allowing APIs to return: - -- The current page of results -- Total elements (from base class) -- Filtered result count (after applying search/filter criteria) - -### Structure - -```text -CountedGenericQueryResult - β”œβ”€ List results - β”œβ”€ int totalCount - └─ int filteredCount -``` - -### Why It Matters - -This structure enables: - -- Efficient UI pagination -- Accurate result counts after filtering -- Consistent response shape across domains (devices, events, users, etc.) - -### Example Usage Flow - -```mermaid -flowchart LR - Request["Query Request with Filters"] --> Service["Query Service"] - Service --> Repo["Custom Repository"] - Repo --> Result["CountedGenericQueryResult"] - Result --> API["Serialized API Response"] -``` - -The `filteredCount` is especially important when: - -- Full dataset size is large -- Filters significantly reduce results -- UI must display: "Showing 10 of 245 filtered results" - ---- - -## 2. ConnectionArgs - -**Class:** `ConnectionArgs` - -### Purpose - -Defines Relay-style pagination arguments used primarily in GraphQL connections. - -Supports: - -- Forward pagination: `first` + `after` -- Backward pagination: `last` + `before` - -### Field Definitions - -```text -ConnectionArgs - β”œβ”€ Integer first (min 1, max 100) - β”œβ”€ String after - β”œβ”€ Integer last (min 1, max 100) - └─ String before -``` - -### Validation Rules - -- `first` and `last` must be between 1 and 100. -- Validation is enforced via Jakarta annotations. -- Prevents excessive data loading and abuse. - -### Pagination Model - -```mermaid -flowchart TD - Client["Client"] --> Args["ConnectionArgs"] - Args --> Decode["CursorCodec.decode()"] - Decode --> Query["Repository Query with Limit"] - Query --> Encode["CursorCodec.encode()"] - Encode --> Response["Connection Response with Edges"] -``` - -### Design Principles - -- Hard limit (100) protects backend resources. -- Supports bi-directional pagination. -- Compatible with Relay Connection pattern. - ---- - -## 3. CursorCodec - -**Class:** `CursorCodec` - -### Purpose - -Encodes and decodes opaque pagination cursors using Base64. - -This ensures that internal cursor values such as: - -- MongoDB ObjectIds -- Composite keys (e.g., "timestamp_eventId") -- Internal database offsets - -are not exposed directly to API consumers. - -### Encoding Flow - -```mermaid -flowchart LR - Raw["Raw Internal Cursor"] --> Encode["Base64 Encode"] - Encode --> Opaque["Opaque Cursor String"] -``` - -### Decoding Flow - -```mermaid -flowchart LR - Opaque["Opaque Cursor"] --> Decode["Base64 Decode"] - Decode --> Raw["Raw Internal Cursor"] -``` - -### Key Behaviors - -- Returns `null` for null or empty input. -- Gracefully handles invalid Base64 input. -- Hides implementation details from clients. - -### Example - -```text -Raw Cursor: 665fae2b3a91c87b12345678 -Encoded: NjY1ZmFlMmIzYTkxYzg3YjEyMzQ1Njc4 -``` - -This pattern guarantees that clients cannot infer database structure or ordering logic. - ---- - -## 4. MutationDeleteInput - -**Class:** `MutationDeleteInput` - -### Purpose - -Defines a standard mutation input structure for delete operations. - -### Structure - -```text -MutationDeleteInput - └─ String id (must not be blank) -``` - -### Design Rationale - -- Enforces explicit ID-based deletion. -- Aligns with GraphQL mutation input patterns. -- Supports validation via `@NotBlank`. - -### Mutation Flow - -```mermaid -flowchart TD - Client["Client Mutation"] --> Input["MutationDeleteInput"] - Input --> Validate["Validation Layer"] - Validate --> Service["Domain Service"] - Service --> Repo["Repository Delete by ID"] - Repo --> Response["Mutation Result"] -``` - ---- - -# Pagination Strategy in the Platform - -The Api Contracts And Pagination module supports two complementary patterns: - -## 1. Cursor-Based Pagination (GraphQL) - -Used for connection-based queries: - -- Forward and backward navigation -- Opaque cursors -- Relay compliance - -Benefits: - -- Stable pagination even with concurrent inserts -- No reliance on numeric offsets -- Efficient indexed queries - -## 2. Counted Query Results (REST or Hybrid) - -Used where: - -- UI requires total/filtered counts -- Offset-based or filtered queries are needed -- Simpler REST-style pagination is sufficient - ---- - -# End-to-End Data Flow Example - -Below is a complete pagination lifecycle example. - -```mermaid -flowchart TD - Step1["Client Requests first 20 after Cursor"] --> Step2["ConnectionArgs Validated"] - Step2 --> Step3["CursorCodec.decode()"] - Step3 --> Step4["Repository Query with Limit 21"] - Step4 --> Step5["Determine hasNextPage"] - Step5 --> Step6["CursorCodec.encode() for Edges"] - Step6 --> Step7["Return Connection Response"] -``` - -### Why Fetch 21 for a Limit of 20? - -To determine if: - -- There is a next page -- `hasNextPage` should be true - -This pattern ensures efficient and correct pagination semantics. - ---- - -# Design Principles - -The Api Contracts And Pagination module follows these core principles: - -1. **Consistency Across Domains** - Devices, events, users, organizations, and tools all use uniform contracts. - -2. **Opaque by Default** - Clients never see internal IDs or composite keys. - -3. **Validation at the Edge** - Input constraints are enforced before hitting service layers. - -4. **Resource Safety** - Pagination limits prevent unbounded queries. - -5. **Transport Layer Neutrality** - Usable by both REST controllers and GraphQL data fetchers. - ---- - -# Summary - -The **Api Contracts And Pagination** module is a foundational contract layer for OpenFrame APIs. - -It standardizes: - -- How lists are queried -- How pagination works -- How cursors are encoded -- How deletions are requested - -By centralizing these patterns, the platform achieves: - -- Predictable API behavior -- Secure data exposure boundaries -- Efficient pagination at scale -- Strong validation guarantees - -This module underpins consistent API design across the entire OpenFrame ecosystem. \ No newline at end of file diff --git a/docs/reference/architecture/api-domain-filters-dtos/api-domain-filters-dtos.md b/docs/reference/architecture/api-domain-filters-dtos/api-domain-filters-dtos.md deleted file mode 100644 index 9cead9cef..000000000 --- a/docs/reference/architecture/api-domain-filters-dtos/api-domain-filters-dtos.md +++ /dev/null @@ -1,366 +0,0 @@ -# Api Domain Filters Dtos - -The **Api Domain Filters Dtos** module defines the domain-level Data Transfer Objects (DTOs) used to: - -- Express filtering criteria for core entities (Devices, Events, Logs, Knowledge Base, Organizations, Tools) -- Provide structured filter metadata for UI dropdowns and faceted search -- Standardize list and response payloads shared across REST and GraphQL layers - -This module acts as a **contract layer** between: - -- GraphQL DataFetchers and REST Controllers -- Core services and query layers -- Mongo query filter builders and repositories - -It does not contain business logic. Instead, it provides strongly typed filter contracts that drive query construction in downstream modules. - ---- - -## Architectural Role - -At a high level, Api Domain Filters Dtos sits between API input DTOs and persistence query filters. - -```mermaid -flowchart LR - Client["Client (REST or GraphQL)"] --> ApiInput["GraphQL Input / REST Request"] - ApiInput --> DomainFilter["Api Domain Filters Dtos"] - DomainFilter --> QueryLayer["Mongo Query Filters"] - QueryLayer --> Repository["Mongo Repositories"] - Repository --> Database[("MongoDB")] -``` - -### Responsibilities - -1. **Normalize filter structures** across APIs -2. **Encapsulate filtering intent** in strongly typed criteria classes -3. **Support UI filter metadata responses** (counts, labels, selectable options) -4. **Bridge domain enums and database query filters** - ---- - -# Core Filter Patterns - -Across entities, this module follows consistent patterns: - -## 1. *FilterCriteria Classes* - -These represent the filtering rules submitted by the client. - -Examples: - -- `LogFilterCriteria` -- `DeviceFilterCriteria` -- `EventFilterCriteria` -- `KnowledgeBaseFilterCriteria` -- `ToolFilterCriteria` - -These classes typically include: - -- Date ranges (`startDate`, `endDate`) -- Entity type filters -- Status filters -- Organization scoping -- Tag-based filtering - -They are consumed by service/query layers to build Mongo queries. - ---- - -## 2. *Filters Classes (Faceted Responses)* - -These classes provide filter metadata for UI dropdowns and dashboards. - -Examples: - -- `LogFilters` -- `DeviceFilters` -- `EventFilters` -- `ToolFilters` - -They usually contain: - -- Available filter values -- Display labels -- Optional counts per filter value - -These enable dynamic filtering experiences (faceted search). - ---- - -## 3. *List & Response DTOs* - -Used to standardize list responses and entity projections: - -- `OrganizationList` -- `ToolList` -- `OrganizationResponse` - -These are shared between: - -- GraphQL DTO layer -- REST external API layer - ---- - -# Entity-Specific Filter Structures - -## Log Filtering - -### LogFilterCriteria - -Encapsulates audit log filtering rules: - -- `startDate` / `endDate` -- `eventTypes` -- `toolTypes` -- `severities` -- `organizationIds` -- `deviceId` - -```mermaid -flowchart TD - LogFilterCriteria["LogFilterCriteria"] --> DateRange["Date Range"] - LogFilterCriteria --> EventTypes["Event Types"] - LogFilterCriteria --> ToolTypes["Tool Types"] - LogFilterCriteria --> Severities["Severities"] - LogFilterCriteria --> Organizations["Organization IDs"] - LogFilterCriteria --> Device["Device ID"] -``` - -### LogFilters - -Provides available values for: - -- Tool types -- Event types -- Severities -- Organizations (via `OrganizationFilterOption`) - -### OrganizationFilterOption - -Represents a selectable organization option: - -- `id` -- `name` - -Used primarily for UI dropdown population. - ---- - -## Device Filtering - -### DeviceFilterCriteria - -Supports multi-dimensional filtering: - -- `statuses` (DeviceStatus enum) -- `deviceTypes` (DeviceType enum) -- `osTypes` -- `organizationIds` -- `tagKeys` -- `tagValues` - -```mermaid -flowchart TD - DeviceFilterCriteria["DeviceFilterCriteria"] --> Statuses["Device Statuses"] - DeviceFilterCriteria --> Types["Device Types"] - DeviceFilterCriteria --> OsTypes["OS Types"] - DeviceFilterCriteria --> Orgs["Organization IDs"] - DeviceFilterCriteria --> Tags["Tag Keys / Values"] -``` - -### DeviceFilters - -Faceted filter response containing: - -- `statuses` -- `deviceTypes` -- `osTypes` -- `organizationIds` -- `tagKeys` -- `filteredCount` - -Each option is represented by: - -### DeviceFilterOption - -- `value` -- `label` -- `count` - -### TagFilterOption - -- `key` -- `value` -- `count` - -This enables advanced tag-based device filtering. - ---- - -## Event Filtering - -### EventFilterCriteria - -Filters system/user events by: - -- `userIds` -- `eventTypes` -- `startDate` -- `endDate` - -### EventFilters - -Returns available: - -- User IDs -- Event types - -This supports activity feed filtering and auditing dashboards. - ---- - -## Knowledge Base Filtering - -### KnowledgeBaseFilterCriteria - -Used to query hierarchical content: - -- `parentId` -- `type` (KnowledgeBaseItemType enum) -- `tagIds` -- `statuses` (KnowledgeBaseArticleStatus enum) - -This enables: - -- Folder-level filtering -- Tag-based content queries -- Status-based moderation filtering - ---- - -## Organization Filtering & Responses - -### OrganizationFilterOptions - -Internal filtering structure for: - -- `category` -- `minEmployees` -- `maxEmployees` -- `hasActiveContract` -- `status` - -### OrganizationList - -Wrapper for returning multiple Organization documents. - -### OrganizationResponse - -Shared response DTO across GraphQL and REST. - -Contains: - -- Identity fields (`id`, `organizationId`) -- Business metadata -- Revenue and contract information -- Lifecycle timestamps -- Status fields - -```mermaid -flowchart LR - Organization["Organization (Mongo Document)"] --> OrganizationResponse["OrganizationResponse DTO"] - OrganizationResponse --> RestAPI["REST API"] - OrganizationResponse --> GraphQL["GraphQL API"] -``` - ---- - -## Tool Filtering - -### ToolFilterCriteria - -Filters integrated tools by: - -- `enabled` -- `type` -- `category` -- `platformCategory` - -### ToolFilters - -Provides available: - -- Tool types -- Categories -- Platform categories - -### ToolList - -Wrapper for returning a list of `IntegratedTool` domain documents. - ---- - -# Interaction with Other Layers - -## With GraphQL Layer - -GraphQL input DTOs are transformed into FilterCriteria objects, which are then passed to service/query layers. - -```mermaid -flowchart LR - GraphQLInput["GraphQL Filter Input"] --> Criteria["FilterCriteria DTO"] - Criteria --> Service["Query Service"] - Service --> MongoFilter["Mongo Query Filter"] -``` - -## With Mongo Query Filters - -Each FilterCriteria maps conceptually to a corresponding Mongo query filter class in the query layer. - -Example conceptual mapping: - -```mermaid -flowchart TD - DeviceFilterCriteria --> DeviceQueryFilter["DeviceQueryFilter"] - EventFilterCriteria --> EventQueryFilter["EventQueryFilter"] - ToolFilterCriteria --> ToolQueryFilter["ToolQueryFilter"] -``` - -The DTO layer remains persistence-agnostic, while query modules handle database-specific logic. - ---- - -# Design Principles - -The Api Domain Filters Dtos module follows these architectural principles: - -1. **Separation of Concerns** - DTOs describe intent; services implement logic. - -2. **Strong Typing** - Uses enums and structured lists instead of raw maps. - -3. **UI-Driven Design** - Includes filter option and count DTOs for faceted filtering. - -4. **Cross-API Reusability** - Shared response DTOs prevent duplication across REST and GraphQL. - -5. **Extensibility** - New filter dimensions can be added without impacting existing consumers. - ---- - -# Summary - -The **Api Domain Filters Dtos** module provides the structured filtering and response contracts that power: - -- Device management -- Audit logs -- Event tracking -- Knowledge base content -- Organization management -- Tool integrations - -It forms the backbone of query expressiveness across the OpenFrame API stack, enabling consistent filtering semantics across services, APIs, and persistence layers. \ No newline at end of file diff --git a/docs/reference/architecture/api-lib-core-services/api-lib-core-services.md b/docs/reference/architecture/api-lib-core-services/api-lib-core-services.md deleted file mode 100644 index f06349115..000000000 --- a/docs/reference/architecture/api-lib-core-services/api-lib-core-services.md +++ /dev/null @@ -1,298 +0,0 @@ -# Api Lib Core Services - -## Overview - -The **Api Lib Core Services** module provides the core business service layer for the OpenFrame API library. It sits between the API-facing layers (REST controllers and GraphQL data fetchers) and the persistence layer (Mongo repositories and domain documents). - -This module encapsulates reusable domain logic related to: - -- Installed agents per machine -- Tool connections and integrations -- Ticket querying and search -- Knowledge base publication lifecycle -- Device status post-processing - -It is designed to be: - -- βœ… Framework-aligned (Spring Boot, Spring Data) -- βœ… Multi-tenant ready (delegating to repository-level filters) -- βœ… GraphQL DataLoader-friendly (batch-oriented methods) -- βœ… Extensible via conditional beans and processors - ---- - -## Architectural Position - -The Api Lib Core Services module acts as a **pure service layer** within the larger OpenFrame architecture. - -```mermaid -flowchart TD - Controllers["REST Controllers"] --> Services["Api Lib Core Services"] - DataFetchers["GraphQL Data Fetchers"] --> Services - Services --> Repositories["Mongo Repositories"] - Repositories --> MongoDB[("MongoDB")] - - Services --> Processors["Domain Processors"] -``` - -### Upstream Consumers - -- REST Controllers (API Service Core) -- GraphQL Data Fetchers -- DataLoaders (batch loading layer) - -### Downstream Dependencies - -- Mongo domain documents (Machine, Ticket, ToolConnection, InstalledAgent) -- Spring Data repositories -- Mongo event lifecycle hooks - -The module intentionally contains **no HTTP or GraphQL annotations**, ensuring reusability across different API surfaces. - ---- - -## Core Components - -### 1. InstalledAgentService - -**Purpose:** -Provides read access and batch-loading logic for installed agents across machines. - -**Key Capabilities:** - -- Fetch installed agents for a single machine -- Fetch installed agents for multiple machines (DataLoader optimized) -- Lookup by ID -- Lookup by machine and agent type - -```mermaid -flowchart LR - DataLoader["InstalledAgentDataLoader"] --> Service["InstalledAgentService"] - Service --> Repo["InstalledAgentRepository"] - Repo --> Mongo[("MongoDB")] -``` - -#### Batch-Oriented Design - -The method: - -```java -getInstalledAgentsForMachines(List machineIds) -``` - -- Fetches all agents using `findByMachineIdIn` -- Groups them in-memory by machine ID -- Returns a list aligned with the original input order - -This structure avoids the **N+1 query problem** in GraphQL environments. - ---- - -### 2. ToolConnectionService - -**Purpose:** -Manages retrieval of tool connection data associated with machines. - -**Responsibilities:** - -- Fetch tool connections by ID -- Fetch tool connections per machine -- Batch-fetch tool connections for multiple machines - -```mermaid -flowchart LR - ToolDataLoader["ToolConnectionDataLoader"] --> ToolService["ToolConnectionService"] - ToolService --> ToolRepo["ToolConnectionRepository"] - ToolRepo --> Mongo[("MongoDB")] -``` - -Similar to InstalledAgentService, this service is optimized for batch resolution in GraphQL environments. - ---- - -### 3. TicketQueryService - -**Purpose:** -Encapsulates ticket search and retrieval logic with filtering and cursor-based pagination support. - -**Key Methods:** - -- `findById(String ticketId)` -- `searchTickets(TicketQueryFilter filter, String search, int limit)` - -```mermaid -flowchart TD - API["API Layer"] --> TicketService["TicketQueryService"] - TicketService --> QueryFilter["TicketQueryFilter"] - TicketService --> Repo["TicketRepository"] - Repo --> Mongo[("MongoDB")] -``` - -#### Query Construction Flow - -1. Build a `Query` object using: - - Structured filter (`TicketQueryFilter`) - - Free-text search -2. Delegate to repository method: - - `findTicketsWithCursor(...)` -3. Apply default sorting: - - Field: `createdAt` - - Direction: `DESC` - -This service centralizes ticket query semantics and ensures consistent sorting behavior. - ---- - -### 4. KnowledgeBasePublishLifecycleListener - -**Purpose:** -Automatically stamps `publishedAt` when a knowledge base article transitions into the `PUBLISHED` state for the first time. - -This is implemented as a: - -- Spring `@Component` -- Extends `AbstractMongoEventListener` -- Hooks into `onBeforeConvert` - -```mermaid -flowchart TD - Save["Save KnowledgeBaseItem"] --> MongoEvent["BeforeConvertEvent"] - MongoEvent --> Listener["KnowledgeBasePublishLifecycleListener"] - Listener --> Stamp["Set publishedAt if null"] - Stamp --> Persist[("MongoDB")] -``` - -### Semantic Guarantees - -- βœ… `publishedAt` is set only once -- βœ… Unpublish/republish cycles do not overwrite the original timestamp -- βœ… Aligns with Schema.org `datePublished` and Atom `published` - -This ensures historical integrity and stable audit semantics. - ---- - -### 5. DefaultDeviceStatusProcessor - -**Purpose:** -Provides a default implementation for device status post-processing logic. - -It implements `DeviceStatusProcessor` and is registered with: - -- `@Component` -- `@ConditionalOnMissingBean` - -```mermaid -flowchart LR - DeviceStatusUpdate["Machine Status Updated"] --> Processor["DeviceStatusProcessor"] - Processor --> DefaultImpl["DefaultDeviceStatusProcessor"] -``` - -### Extensibility Model - -Because it is conditionally registered, applications can: - -- Provide a custom `DeviceStatusProcessor` -- Override default behavior without modifying the core module - -The default implementation only logs status changes, making it safe and side-effect free. - ---- - -## Design Patterns Used - -### 1. Service Layer Pattern -Encapsulates domain logic and shields API layers from persistence details. - -### 2. DataLoader-Friendly Batch APIs -Both InstalledAgentService and ToolConnectionService provide: - -- `List>` aligned responses -- Deterministic ordering -- Single query per batch - -### 3. Repository Delegation -All query logic ultimately delegates to repositories responsible for: - -- Query construction -- Pagination mechanics -- Sorting logic - -### 4. Event-Driven Lifecycle Hooks -KnowledgeBasePublishLifecycleListener demonstrates: - -- Mongo lifecycle event integration -- Declarative business rules -- Immutable publication timestamp semantics - -### 5. Conditional Bean Extension -DefaultDeviceStatusProcessor enables pluggable domain behavior using Spring Boot auto-configuration conventions. - ---- - -## Data Flow Example: GraphQL Machine Query - -When querying machines with nested installed agents and tool connections: - -```mermaid -flowchart TD - Client["GraphQL Client"] --> DataFetcher["MachineDataFetcher"] - DataFetcher --> AgentLoader["InstalledAgentDataLoader"] - DataFetcher --> ToolLoader["ToolConnectionDataLoader"] - - AgentLoader --> AgentService["InstalledAgentService"] - ToolLoader --> ToolService["ToolConnectionService"] - - AgentService --> AgentRepo["InstalledAgentRepository"] - ToolService --> ToolRepo["ToolConnectionRepository"] - - AgentRepo --> Mongo[("MongoDB")] - ToolRepo --> Mongo -``` - -This ensures: - -- βœ… No N+1 queries -- βœ… Deterministic batch resolution -- βœ… Clean separation between data resolution and domain logic - ---- - -## Transaction and Consistency Model - -- Read-only services are annotated with `@Transactional(readOnly = true)` -- Lifecycle listeners operate before persistence conversion -- Repository layer enforces query-level consistency - -The module intentionally avoids complex transaction orchestration and instead relies on: - -- Repository atomic operations -- Mongo consistency guarantees - ---- - -## Extension Guidelines - -When extending Api Lib Core Services: - -1. βœ… Keep services stateless -2. βœ… Delegate persistence to repositories -3. βœ… Provide batch-friendly APIs when used by GraphQL -4. βœ… Use conditional beans for override points -5. βœ… Keep lifecycle logic idempotent - ---- - -## Summary - -The **Api Lib Core Services** module is the domain-centric service backbone of the OpenFrame API stack. - -It provides: - -- Clean separation of concerns -- Batch-optimized data access patterns -- Event-driven lifecycle management -- Extensible processing hooks -- Consistent query semantics - -By centralizing domain service logic in this module, the broader OpenFrame platform maintains a modular, scalable, and extensible API architecture. \ No newline at end of file diff --git a/docs/reference/architecture/api-lib-dto-contracts/api-lib-dto-contracts.md b/docs/reference/architecture/api-lib-dto-contracts/api-lib-dto-contracts.md new file mode 100644 index 000000000..f16fbfc86 --- /dev/null +++ b/docs/reference/architecture/api-lib-dto-contracts/api-lib-dto-contracts.md @@ -0,0 +1,407 @@ +# Api Lib Dto Contracts + +## Overview + +The **Api Lib Dto Contracts** module defines the shared Data Transfer Objects (DTOs) that form the contract layer between: + +- GraphQL resolvers in `api-service-core-graphql-layer` +- REST controllers in `api-service-core-rest-controllers` +- External REST APIs in `external-api-service-core` +- Business services in `api-lib-mapping-and-domain-services` +- Domain models and repositories in `data-model-and-repositories-mongo` + +This module is intentionally free of transport logic, persistence logic, and business rules. It provides **stable, reusable contract models** that: + +- Define input payloads for commands and mutations +- Define filter criteria for search operations +- Provide response shapes for REST and GraphQL +- Implement Relay-compatible pagination primitives +- Standardize command dispatch and execution contracts + +It acts as the **boundary language** of the OpenFrame API ecosystem. + +--- + +## Architectural Role + +The module sits between the API layer and the domain layer, serving as a stable contract boundary. + +```mermaid +flowchart LR + Client["Client Applications"] --> GraphQL["GraphQL Layer"] + Client --> Rest["REST Controllers"] + + GraphQL --> Dto["Api Lib Dto Contracts"] + Rest --> Dto + + Dto --> Services["Business Services"] + Services --> Domain["Domain Models"] + Domain --> Mongo["Mongo Repositories"] +``` + +### Key Responsibilities + +- βœ… Define **input contracts** (commands, filters, mutation inputs) +- βœ… Define **output contracts** (responses, lists, filter options) +- βœ… Encapsulate **pagination rules** (Relay connection spec) +- βœ… Standardize **cross-module payloads** (Organization, Script, Tool, Device) +- βœ… Avoid leaking internal domain objects unnecessarily + +--- + +## Design Principles + +### 1. Clear Separation of Concerns + +DTOs do not: + +- Access repositories +- Contain business logic +- Perform validation beyond structural constraints + +They may use: + +- Jakarta validation annotations (`@NotBlank`, `@NotNull`, `@Positive`) +- Enum references from domain models +- Lombok for immutability and builders + +--- + +### 2. Shared Between GraphQL and REST + +Several DTOs (e.g., `OrganizationResponse`) are explicitly designed for reuse across both internal GraphQL and external REST APIs. + +This ensures: + +- Identical serialization +- Consistent field naming +- No duplication of contract logic + +--- + +### 3. Tenant Awareness by Context, Not Payload + +Inputs such as `CreateScriptInput` explicitly avoid including `tenantId`. Tenant attribution is derived from authentication context in upper layers. + +This prevents: + +- Cross-tenant spoofing +- Contract-level security leaks +- Client-controlled multi-tenant switching + +--- + +# Module Breakdown + +The Api Lib Dto Contracts module can be grouped into the following logical areas: + +```mermaid +flowchart TD + Root["Api Lib Dto Contracts"] --> Audit["Audit & Logs"] + Root --> Device["Device"] + Root --> Event["Event"] + Root --> Knowledge["Knowledge Base"] + Root --> Organization["Organization"] + Root --> Script["Script"] + Root --> Tool["Tool"] + Root --> Command["Command Dispatch"] + Root --> Shared["Shared & Pagination"] +``` + +--- + +# Audit & Log DTOs + +### Core Classes + +- `LogFilterCriteria` +- `LogFilters` +- `OrganizationFilterOption` + +### Responsibilities + +- Define filter inputs for log queries +- Provide dropdown option models for UI +- Support time-bound and multi-field filtering + +### Example Fields + +- Date ranges (`startDate`, `endDate`) +- Event types +- Tool types +- Severity levels +- Organization filters +- Device scoping + +These DTOs are typically consumed by: + +- GraphQL `LogDataFetcher` +- REST log endpoints in `external-api-service-core` + +--- + +# Device DTOs + +### Core Classes + +- `DeviceFilterCriteria` +- `DeviceFilterOption` +- `DeviceFilters` +- `TagFilterOption` + +### Responsibilities + +- Express device filtering constraints +- Return aggregation-style filter options +- Support faceted search patterns + +### Pattern + +```mermaid +flowchart LR + Criteria["DeviceFilterCriteria"] --> Query["Repository Query"] + Query --> Result["DeviceFilters"] + Result --> Options["DeviceFilterOption"] +``` + +Key features: + +- Filter by status, type, OS, organization +- Tag-based filtering (key/value) +- Include `filteredCount` for UI summary statistics + +--- + +# Event DTOs + +### Core Classes + +- `EventFilterCriteria` +- `EventFilters` + +Used to: + +- Filter audit and system events +- Constrain by user, type, and time window + +These integrate with: + +- Event repositories +- Analytics pipelines +- Stream processing components + +--- + +# Knowledge Base DTOs + +### Core Classes + +- `CreateArticleCommand` +- `UpdateArticleCommand` +- `KnowledgeBaseFilterCriteria` +- `KnowledgeBaseAttachmentUpload` + +### Responsibilities + +- Create and update hierarchical knowledge base items +- Filter by parent, type, tags, and status +- Manage temporary attachment uploads + +The commands allow association with: + +- Organizations +- Devices +- Tickets +- Other knowledge articles + +This enables rich contextual linking. + +--- + +# Organization DTOs + +### Core Classes + +- `OrganizationFilterOptions` +- `OrganizationList` +- `OrganizationResponse` + +`OrganizationResponse` is a shared cross-layer contract used by: + +- GraphQL resolvers +- External REST APIs + +It includes: + +- Financial data (monthly revenue) +- Contract lifecycle dates +- Contact information +- Status tracking metadata + +This DTO is a canonical representation of organization state at the API boundary. + +--- + +# Script DTOs + +### Core Classes + +- `CreateScriptInput` +- `UpdateScriptInput` +- `ScriptFilterInput` +- `ScriptResponse` +- `ScriptEnvVarInput` + +### Command Lifecycle + +```mermaid +flowchart TD + CreateInput["CreateScriptInput"] --> Service["Script Service"] + Service --> Stored["Script Document"] + Stored --> Response["ScriptResponse"] +``` + +### Notable Characteristics + +- Explicit enum usage (`ScriptShell`, `ScriptPlatform`, `ScriptStatus`) +- Validation constraints on required fields +- Full replacement semantics for update +- Secret flag on environment variables +- Tenant context inferred from authentication + +This set of DTOs standardizes script management across integrations. + +--- + +# Command Dispatch DTOs + +### Core Classes + +- `RunCommandInput` +- `CancelExecutionInput` +- `CommandDispatchResponse` +- `CancelDispatchResponse` + +### Flow + +```mermaid +sequenceDiagram + participant Client + participant Api + participant Agent + + Client->>Api: RunCommandInput + Api->>Agent: Dispatch Command + Agent-->>Api: ExecutionId + Api-->>Client: CommandDispatchResponse + + Client->>Api: CancelExecutionInput + Api->>Agent: Cancel Command + Api-->>Client: CancelDispatchResponse +``` + +These DTOs standardize remote command execution across: + +- Web clients +- Agent services +- Message buses (Kafka / NATS) + +They ensure consistent execution tracking via `executionId`. + +--- + +# Tool DTOs + +### Core Classes + +- `ToolFilterCriteria` +- `ToolFilters` +- `ToolList` + +Responsibilities: + +- Filter integrated tools +- Return tool catalog lists +- Provide platform and category grouping + +`ToolList` wraps domain `IntegratedTool` objects for API consumption. + +--- + +# Shared & Pagination DTOs + +### Core Classes + +- `ConnectionArgs` +- `CursorCodec` +- `MutationDeleteInput` +- `CountedGenericQueryResult` + +## Relay Pagination + +`ConnectionArgs` implements forward and backward pagination semantics: + +- `first` + `after` +- `last` + `before` + +`CursorCodec` encodes opaque base64 cursors to prevent leaking database identifiers. + +```mermaid +flowchart LR + Raw["Raw Cursor"] --> Encode["Base64 Encode"] + Encode --> Opaque["Opaque Cursor"] + Opaque --> Decode["Base64 Decode"] + Decode --> Raw +``` + +## Counted Query Results + +`CountedGenericQueryResult` extends a generic query result with: + +- `filteredCount` + +This supports UI patterns where: + +- Total available +- Total filtered +- Page slice + +must all be visible simultaneously. + +--- + +# Integration with Other Modules + +The Api Lib Dto Contracts module integrates tightly with: + +- `api-lib-mapping-and-domain-services` for mapping between domain models and DTOs +- `api-service-core-graphql-layer` for GraphQL input/output contracts +- `api-service-core-rest-controllers` for REST serialization +- `external-api-service-core` for public API responses +- `data-model-and-repositories-mongo` for domain entity references +- `security-oauth-and-jwt` for authentication-derived context + +However, it does not depend on those modules' runtime logic. + +--- + +# Summary + +The **Api Lib Dto Contracts** module is the **contract foundation** of the OpenFrame API architecture. + +It provides: + +- A consistent and reusable API language +- Strongly-typed filter and mutation inputs +- Stable response models +- Relay-compliant pagination +- Standardized command execution contracts + +By isolating these definitions in a dedicated module, the system ensures: + +- Clean layering +- Reduced duplication +- Easier refactoring +- Stable public API boundaries + +This module is critical for maintaining long-term API consistency across GraphQL, REST, streaming integrations, and external SDK consumers. \ No newline at end of file diff --git a/docs/reference/architecture/api-lib-mapping-and-domain-services/api-lib-mapping-and-domain-services.md b/docs/reference/architecture/api-lib-mapping-and-domain-services/api-lib-mapping-and-domain-services.md new file mode 100644 index 000000000..7a8163e4b --- /dev/null +++ b/docs/reference/architecture/api-lib-mapping-and-domain-services/api-lib-mapping-and-domain-services.md @@ -0,0 +1,335 @@ +# Api Lib Mapping And Domain Services + +## Overview + +The **Api Lib Mapping And Domain Services** module acts as a shared application layer inside the OpenFrame API library. It bridges: + +- External API contracts (DTOs from Api Lib Dto Contracts) +- Persistent domain entities (from Data Model And Repositories Mongo) +- Higher-level API layers (REST controllers and GraphQL data fetchers) + +This module centralizes: + +- Entity ↔ DTO mapping logic +- Query orchestration and filtering logic +- Batch-fetch support for GraphQL DataLoaders +- Domain lifecycle hooks +- Extension points for domain-specific processors + +By keeping this logic in a shared module, both REST and GraphQL layers reuse consistent behavior without duplicating mapping or query logic. + +--- + +## Architectural Context + +The module sits between the API layer and the persistence layer. + +```mermaid +flowchart TD + RestControllers["REST Controllers"] --> ApiServices["Api Lib Mapping And Domain Services"] + GraphQL["GraphQL Data Fetchers"] --> ApiServices + ApiServices --> Repositories["Mongo Repositories"] + ApiServices --> DomainEntities["Domain Documents"] + ApiServices --> DtoContracts["DTO Contracts"] + + subgraph data_layer["Data Layer"] + Repositories --> MongoDB[("MongoDB")] + end +``` + +### Upstream Consumers + +- REST controllers from Api Service Core Rest Controllers +- GraphQL data fetchers from Api Service Core Graphql Layer +- GraphQL DataLoaders from Api Service Core Graphql Dataloaders + +### Downstream Dependencies + +- Domain documents and repositories from Data Model And Repositories Mongo +- Mongo query builders from Data Access Mongo Sync + +--- + +## Core Responsibilities + +The module provides: + +1. **Mapping layer** – Converting DTOs to entities and back. +2. **Query services** – Encapsulating complex filtering and search logic. +3. **Batch aggregation services** – Optimized for GraphQL DataLoader patterns. +4. **Lifecycle listeners** – Domain-specific persistence hooks. +5. **Extension processors** – Default implementations for customizable domain logic. + +--- + +## Components + +### 1. Organization Mapper + +**Component:** `OrganizationMapper` + +The Organization Mapper is responsible for converting between: + +- Create and Update request DTOs +- Organization domain entities +- Organization response DTOs + +#### Responsibilities + +- Generate immutable `organizationId` values (UUID-based) +- Support partial updates (only non-null fields applied) +- Enforce immutability of `organizationId` +- Map nested objects: + - ContactInformation + - ContactPerson + - Address +- Convert entity status enums to response-friendly string values + +#### Mapping Flow + +```mermaid +flowchart LR + CreateReq["CreateOrganizationRequest"] --> Mapper["OrganizationMapper"] + Mapper --> Entity["Organization Document"] + Entity --> Mapper + Mapper --> Response["OrganizationResponse"] +``` + +#### Key Design Decisions + +- `organizationId` is generated once and cannot be updated. +- Partial updates prevent accidental overwrites. +- Mailing address can automatically copy physical address. +- Defensive null handling avoids NPEs. + +This ensures consistent behavior across both REST and GraphQL APIs. + +--- + +### 2. Installed Agent Service + +**Component:** `InstalledAgentService` + +This service provides read operations for InstalledAgent documents. + +#### Responsibilities + +- Fetch installed agents by machine ID +- Fetch installed agents by machine ID and agent type +- Fetch all installed agents +- Support batch loading for GraphQL DataLoader + +#### Batch Loading Pattern + +The `getInstalledAgentsForMachines` method is optimized for GraphQL DataLoader usage: + +```mermaid +flowchart TD + DataLoader["InstalledAgentDataLoader"] --> Service["InstalledAgentService"] + Service --> Repo["InstalledAgentRepository"] + Repo --> Mongo[("MongoDB")] + Service --> Grouping["Group by machineId"] + Grouping --> OrderedResult["List>"] +``` + +Key behavior: + +- Single query for multiple machine IDs +- In-memory grouping by machineId +- Result order matches input order + +This eliminates the N+1 query problem in GraphQL. + +--- + +### 3. Tool Connection Service + +**Component:** `ToolConnectionService` + +Provides read-only access to ToolConnection documents. + +#### Responsibilities + +- Fetch tool connections by ID +- Fetch tool connections for a single machine +- Batch fetch tool connections for multiple machines + +Its batch API mirrors the Installed Agent Service pattern, making it suitable for GraphQL DataLoader integration. + +```mermaid +flowchart TD + ToolDataLoader["ToolConnectionDataLoader"] --> ToolService["ToolConnectionService"] + ToolService --> ToolRepo["ToolConnectionRepository"] + ToolRepo --> Mongo[("MongoDB")] +``` + +--- + +### 4. Ticket Query Service + +**Component:** `TicketQueryService` + +Encapsulates ticket search logic using repository-level query builders. + +#### Responsibilities + +- Find ticket by ID +- Search tickets using: + - TicketQueryFilter + - Free-text search + - Result limit +- Apply default sorting (`createdAt DESC`) + +#### Query Flow + +```mermaid +flowchart TD + Controller["REST or GraphQL"] --> TicketService["TicketQueryService"] + TicketService --> BuildQuery["buildTicketQuery()"] + BuildQuery --> CursorQuery["findTicketsWithCursor()"] + CursorQuery --> Mongo[("MongoDB")] +``` + +The service delegates query construction to the repository while enforcing: + +- Default sort field +- Descending sort direction +- Read-only transactional boundary + +This ensures consistent search semantics across API entry points. + +--- + +### 5. Knowledge Base Publish Lifecycle Listener + +**Component:** `KnowledgeBasePublishLifecycleListener` + +A Mongo lifecycle listener that enforces "first published at" semantics. + +#### Behavior + +On `BeforeConvert`: + +- If status == PUBLISHED +- And `publishedAt` is null +- Then set `publishedAt = Instant.now()` + +```mermaid +flowchart TD + Save["Save KnowledgeBaseItem"] --> BeforeConvert["BeforeConvertEvent"] + BeforeConvert --> Check["Status == PUBLISHED?"] + Check -->|"Yes and publishedAt null"| Stamp["Set publishedAt"] + Check -->|"Otherwise"| NoChange["No change"] +``` + +#### Design Guarantees + +- Timestamp is set only once. +- Unpublish/republish does not overwrite the original value. +- Aligns with Schema.org `datePublished` semantics. + +This ensures canonical publishing behavior without requiring API-layer logic. + +--- + +### 6. Default Device Status Processor + +**Component:** `DefaultDeviceStatusProcessor` + +Provides a default implementation of the `DeviceStatusProcessor` extension point. + +#### Responsibilities + +- React to device status updates +- Log status changes + +```mermaid +flowchart LR + Machine["Machine"] --> Processor["DeviceStatusProcessor"] + Processor --> Log["Debug log"] +``` + +#### Extensibility + +- Marked with `@ConditionalOnMissingBean` +- Automatically replaced if a custom implementation is defined + +This allows: + +- Product-specific customization +- Plugin-style overrides +- Future event-driven expansions + +--- + +## Cross-Cutting Design Patterns + +### 1. Read-Only Service Pattern + +Most services: + +- Are annotated with `@Transactional(readOnly = true)` +- Delegate persistence to repositories +- Avoid embedding business mutations + +This keeps the module focused on orchestration and mapping rather than domain mutation. + +--- + +### 2. Batch-Oriented Design for GraphQL + +Both InstalledAgentService and ToolConnectionService: + +- Accept collections of IDs +- Perform single repository queries +- Group results in-memory +- Return ordered lists + +This aligns directly with GraphQL DataLoader batching semantics. + +--- + +### 3. Strict Mapping Separation + +Mapping logic is isolated in mappers rather than: + +- Controllers +- Repositories +- Domain entities + +Benefits: + +- Clear boundary between API contracts and domain models +- Easier DTO evolution +- Reduced accidental entity exposure + +--- + +## How This Module Fits Into the Platform + +Within the larger OpenFrame architecture: + +- Api Service Core Rest Controllers call services in this module. +- Api Service Core Graphql Layer uses these services for queries. +- Api Service Core Graphql Dataloaders depend on the batch methods here. +- Data Model And Repositories Mongo provides entities and repositories. + +High-level interaction: + +```mermaid +flowchart TD + Client["Client"] --> Gateway["Gateway Service Core"] + Gateway --> ApiCore["Api Service Core"] + ApiCore --> ApiLib["Api Lib Mapping And Domain Services"] + ApiLib --> MongoLayer["Data Model And Repositories Mongo"] + MongoLayer --> Mongo[("MongoDB")] +``` + +The Api Lib Mapping And Domain Services module therefore serves as: + +- A shared mapping layer +- A query orchestration layer +- A lifecycle hook provider +- A GraphQL optimization layer + +It ensures consistent, reusable domain access patterns across all API entry points while keeping controllers and data fetchers thin and focused. diff --git a/docs/reference/architecture/api-organization-mapping/api-organization-mapping.md b/docs/reference/architecture/api-organization-mapping/api-organization-mapping.md deleted file mode 100644 index 86fb3e3d8..000000000 --- a/docs/reference/architecture/api-organization-mapping/api-organization-mapping.md +++ /dev/null @@ -1,322 +0,0 @@ -# Api Organization Mapping - -The **Api Organization Mapping** module is responsible for translating between external API data transfer objects (DTOs) and the internal persistence model for organizations. It acts as the boundary layer between: - -- REST and GraphQL API contracts -- Domain entities stored in MongoDB -- Service-layer business logic - -At its core is the `OrganizationMapper`, a Spring component that ensures consistent, safe, and predictable conversion between request objects, domain entities, and response DTOs. - ---- - -## 1. Purpose and Responsibilities - -The Api Organization Mapping module provides: - -- βœ… Conversion from API request DTOs to `Organization` entities -- βœ… Partial update (patch-style) entity updates -- βœ… Conversion from `Organization` entities to response DTOs -- βœ… Nested object mapping (contact information, addresses, contacts) -- βœ… Controlled immutability for system-critical fields (e.g., `organizationId`) - -It centralizes mapping logic to avoid duplication across: - -- REST controllers -- GraphQL data fetchers -- Service layer components - ---- - -## 2. Core Component - -### OrganizationMapper - -Location: - -```text -com.openframe.api.mapper.OrganizationMapper -``` - -Type: - -- Spring `@Component` -- Stateless mapper -- Shared across REST and GraphQL APIs - -### Primary Methods - -| Method | Purpose | -|--------|----------| -| `toEntity(CreateOrganizationRequest)` | Create new `Organization` entity | -| `updateEntity(Organization, UpdateOrganizationRequest)` | Partial update of entity | -| `toResponse(Organization)` | Convert entity to API response | - ---- - -## 3. High-Level Architecture - -The mapper sits between API contracts and the domain model. - -```mermaid -flowchart LR - subgraph ApiLayer["API Layer"] - RestController["OrganizationController"] - GraphQLFetcher["OrganizationDataFetcher"] - end - - Mapper["OrganizationMapper"] - - subgraph DomainLayer["Domain & Persistence"] - OrgEntity["Organization Entity"] - MongoRepo["Mongo Repository"] - end - - RestController -->|"Create/Update Request"| Mapper - GraphQLFetcher -->|"Mutation Input"| Mapper - - Mapper -->|"Entity"| OrgEntity - OrgEntity --> MongoRepo - - MongoRepo --> OrgEntity - OrgEntity -->|"Response DTO"| Mapper - Mapper -->|"OrganizationResponse"| RestController - Mapper -->|"GraphQL DTO"| GraphQLFetcher -``` - -### Related Modules - -- API contracts and DTO definitions: [Api Domain Filters Dtos](api-domain-filters-dtos.md) -- REST layer: [Api Service Core Rest Controllers](api-service-core-rest-controllers.md) -- GraphQL layer: [Api Service Core Graphql Datafetchers](api-service-core-graphql-datafetchers.md) -- Persistence model: [Data Mongo Domain Model](data-mongo-domain-model.md) - ---- - -## 4. Create Flow Mapping - -### 4.1 Entity Creation - -`toEntity(CreateOrganizationRequest request)`: - -- Generates a new `organizationId` using UUID -- Copies request fields into the entity -- Sets `isDefault` to `false` -- Converts nested contact structures - -```mermaid -flowchart TD - Request["CreateOrganizationRequest"] --> MapperCreate["toEntity()"] - MapperCreate --> GenId["Generate UUID"] - MapperCreate --> MapFields["Map Scalar Fields"] - MapperCreate --> MapContact["Map Contact Information"] - MapContact --> MapAddress["Map Address"] - MapContact --> MapContacts["Map Contact Persons"] - GenId --> OrgEntity["Organization Entity"] - MapFields --> OrgEntity - MapContact --> OrgEntity -``` - -### 4.2 UUID Generation - -The organization ID is: - -- Generated internally -- Immutable after creation -- Independent from MongoDB primary key (`id`) - -This ensures: - -- External-safe identifier exposure -- Decoupling from database implementation -- Stable public references - ---- - -## 5. Partial Update Strategy - -### 5.1 Patch-Style Updates - -`updateEntity(existing, request)`: - -- Only updates fields that are **non-null** in the request -- Leaves unspecified fields unchanged -- Does **not** allow updating `organizationId` - -```mermaid -flowchart TD - UpdateRequest["UpdateOrganizationRequest"] --> CheckField["Field != null?"] - CheckField -->|"Yes"| ApplyUpdate["Set Field on Entity"] - CheckField -->|"No"| Skip["Keep Existing Value"] - ApplyUpdate --> OrgEntity["Updated Organization"] - Skip --> OrgEntity -``` - -### 5.2 Immutability Rules - -The following field is intentionally immutable: - -- `organizationId` - -Reason: - -- Prevents identity mutation -- Protects referential integrity across services -- Ensures stable tenant-level references - ---- - -## 6. Response Mapping - -`toResponse(Organization organization)` converts the domain entity into `OrganizationResponse`. - -### 6.1 Field Transformations - -- `status` β†’ converted to string via `status.name()` -- `createdAt` and `updatedAt` preserved -- Nested contact information mapped to DTO equivalents - -```mermaid -flowchart LR - OrgEntity["Organization Entity"] --> MapperResponse["toResponse()"] - MapperResponse --> MapScalars["Map Scalar Fields"] - MapperResponse --> MapStatus["Enum to String"] - MapperResponse --> MapNested["Map Contact Information DTO"] - MapScalars --> Response["OrganizationResponse"] - MapStatus --> Response - MapNested --> Response -``` - ---- - -## 7. Nested Contact Mapping - -The mapper handles complex nested structures: - -### 7.1 ContactInformation - -Structure includes: - -- Contacts (list of `ContactPerson`) -- Physical address -- Mailing address -- `mailingAddressSameAsPhysical` flag - -### 7.2 Mailing Address Copy Logic - -If `mailingAddressSameAsPhysical == true`: - -- Mailing address is a **copy** of physical address -- Avoids shared object references -- Prevents unintended mutation side effects - -```mermaid -flowchart TD - Dto["ContactInformationDto"] --> CheckFlag{{"Same As Physical?"}} - CheckFlag -->|"Yes"| CopyAddr["Copy Physical Address"] - CheckFlag -->|"No"| MapMail["Map Mailing Address"] - CopyAddr --> Entity["ContactInformation Entity"] - MapMail --> Entity -``` - -This defensive copying ensures data integrity when updates occur later. - ---- - -## 8. Interaction with Other Modules - -### 8.1 REST Controllers - -Controllers such as `OrganizationController`: - -- Accept request DTOs -- Call services -- Use `OrganizationMapper` to convert entities to responses - -See: [Api Service Core Rest Controllers](api-service-core-rest-controllers.md) - -### 8.2 GraphQL Data Fetchers - -`OrganizationDataFetcher` uses the same mapper to: - -- Convert mutation inputs -- Map domain entities to GraphQL DTOs - -See: [Api Service Core Graphql Datafetchers](api-service-core-graphql-datafetchers.md) - -### 8.3 Domain Model - -Entity involved: - -- `com.openframe.data.document.organization.Organization` - -Defined in: [Data Mongo Domain Model](data-mongo-domain-model.md) - ---- - -## 9. End-to-End Organization Lifecycle (Simplified) - -```mermaid -flowchart LR - Client["Client App"] --> RestAPI["REST or GraphQL API"] - RestAPI --> MapperCreate["OrganizationMapper"] - MapperCreate --> Service["Organization Service"] - Service --> Repo["Mongo Repository"] - Repo --> DB["MongoDB"] - DB --> Repo - Repo --> Service - Service --> MapperResponse["OrganizationMapper"] - MapperResponse --> RestAPI - RestAPI --> Client -``` - -The Api Organization Mapping module ensures that: - -- API contracts remain stable -- Domain entities remain clean and persistence-focused -- Mapping rules are centralized and consistent - ---- - -## 10. Design Principles - -### 10.1 Single Responsibility - -The mapper: - -- Does not contain business logic -- Does not access repositories -- Does not perform validation - -It strictly transforms data structures. - -### 10.2 Stateless & Thread-Safe - -- No shared mutable state -- No injected dependencies -- Safe for concurrent use - -### 10.3 Explicit Mapping Over Reflection - -All fields are mapped explicitly rather than using reflection-based mapping frameworks. - -Benefits: - -- Better compile-time safety -- Clear documentation of field transformations -- Predictable behavior -- Easier debugging - ---- - -## 11. Key Takeaways - -The **Api Organization Mapping** module: - -- Acts as the transformation boundary between API and domain -- Protects immutable identity fields -- Implements safe partial update semantics -- Handles complex nested structures with defensive copying -- Is shared across REST and GraphQL layers - -By centralizing organization mapping logic, the platform ensures consistent behavior across all API surfaces while keeping the domain model isolated from transport-layer concerns. \ No newline at end of file diff --git a/docs/reference/architecture/api-service-core-business-services/api-service-core-business-services.md b/docs/reference/architecture/api-service-core-business-services/api-service-core-business-services.md new file mode 100644 index 000000000..985aa06ea --- /dev/null +++ b/docs/reference/architecture/api-service-core-business-services/api-service-core-business-services.md @@ -0,0 +1,320 @@ +# Api Service Core Business Services + +## Overview + +The **Api Service Core Business Services** module contains the core domain-level business logic for the API layer. It sits between REST/GraphQL controllers and the data access layer, orchestrating: + +- User lifecycle management +- Single Sign-On (SSO) configuration +- Agent registration secret processing +- Invitation processing +- Domain validation policies +- Post-processing hooks for extensibility + +This module is intentionally designed with extension points (via processor interfaces and `@ConditionalOnMissingBean`) so that SaaS or enterprise deployments can override default behaviors without modifying the core OSS implementation. + +--- + +## Architectural Context + +Within the broader OpenFrame platform, Api Service Core Business Services operates as a domain orchestration layer. + +```mermaid +flowchart TD + Controllers["REST & GraphQL Controllers"] --> Services["Api Service Core Business Services"] + Services --> Repositories["Mongo Repositories"] + Services --> Mappers["DTO Mappers"] + Services --> Processors["Domain Processors (Extension Points)"] + Services --> Encryption["Encryption Service"] + Services --> DomainValidation["Domain Validation Service"] + + Repositories --> Mongo["MongoDB"] +``` + +### Key Responsibilities + +- Enforce domain invariants +- Validate input beyond DTO constraints +- Orchestrate repository operations +- Trigger post-processing workflows +- Integrate with encryption and domain validation services + +--- + +## Core Components + +The module includes the following primary components: + +- `DefaultDomainExistenceValidator` +- `SSOConfigService` +- `DefaultAgentRegistrationSecretProcessor` +- `DefaultInvitationProcessor` +- `DefaultSSOConfigProcessor` +- `DefaultUserProcessor` +- `UserService` + +They can be grouped into three logical domains: + +1. **User Domain Services** +2. **SSO Configuration Services** +3. **Extensibility Processors & Validators** + +--- + +# User Domain Services + +## UserService + +The `UserService` is the central orchestrator for user lifecycle management. + +### Responsibilities + +- Retrieve users by ID or email +- Paginated listing of users +- Update mutable fields (first name, last name) +- Soft delete users +- Enforce deletion constraints +- Trigger post-processing hooks + +### Dependencies + +- `UserRepository` +- `UserMapper` +- `UserProcessor` + +### User Lifecycle Flow + +```mermaid +flowchart TD + Request["Controller Request"] --> GetUser["Load User from Repository"] + GetUser --> Validate["Apply Business Rules"] + Validate --> Persist["Save User"] + Persist --> PostProcess["UserProcessor Hook"] + PostProcess --> Response["Return UserResponse"] +``` + +### Business Rules Enforced + +- A user cannot delete themselves (`UserSelfDeleteNotAllowedException`). +- Owner accounts cannot be deleted (`OperationNotAllowedException`). +- Deletion is soft (status set to `DELETED`). +- Post-processing occurs after: + - Fetch + - Update + - Delete + +This ensures consistent domain behavior independent of controller implementation. + +--- + +# SSO Configuration Services + +## SSOConfigService + +The `SSOConfigService` manages Single Sign-On provider configurations per tenant. + +### Responsibilities + +- List enabled providers (for login UI) +- List available providers (from properties) +- Retrieve full configuration for admin editing +- Create/update (upsert) provider configuration +- Delete configuration +- Toggle enabled status +- Validate auto-provisioning rules + +### Dependencies + +- `SSOConfigRepository` +- `EncryptionService` +- `SSOProperties` +- `SSOConfigProcessor` +- `SSOConfigMapper` +- `DomainValidationService` + +### Configuration Upsert Flow + +```mermaid +flowchart TD + Request["SSOConfigRequest"] --> Normalize["Normalize Domains"] + Normalize --> ValidatePublic["Validate Generic Public Domains"] + ValidatePublic --> ValidateExist["Validate Domain Exists"] + ValidateExist --> ValidateAuto["Validate Auto-Provision Rules"] + ValidateAuto --> MapEntity["Map to SSOConfig Entity"] + MapEntity --> Encrypt["Encrypt Client Secret"] + Encrypt --> Save["Save via Repository"] + Save --> PostProcess["SSOConfigProcessor Hook"] + PostProcess --> Response["SSOConfigResponse"] +``` + +### Domain Validation Rules + +Before saving configuration: + +1. Domains are: + - Trimmed + - Lowercased + - Deduplicated +2. Generic public domains are validated. +3. Existence validation is executed. +4. If `autoProvisionUsers` is true: + - At least one allowed domain must exist. + - For Microsoft provider, `msTenantId` is mandatory. + +### Encryption Handling + +- Client secrets are encrypted at save time. +- Decrypted only when returning full admin configuration. + +This ensures secrets are never stored in plain text. + +--- + +# Extensibility Processors & Validators + +This module provides default implementations for several extension interfaces using Spring’s `@ConditionalOnMissingBean`. + +These defaults are no-op implementations and are intended to be overridden in more advanced deployments. + +## Processor Pattern + +```mermaid +flowchart LR + Service["Domain Service"] --> ProcessorInterface["Processor Interface"] + ProcessorInterface --> DefaultImpl["Default Implementation"] + ProcessorInterface --> CustomImpl["Custom Deployment Implementation"] +``` + +If a custom bean is provided, the default implementation is automatically disabled. + +--- + +## DefaultUserProcessor + +Hooks triggered by `UserService`: + +- `postProcessUserDeleted` +- `postProcessUserGet` +- `postProcessUserUpdated` + +Default behavior: debug-level logging only. + +Custom implementations could: + +- Publish domain events +- Trigger audit logging +- Notify external systems + +--- + +## DefaultSSOConfigProcessor + +Hooks triggered by `SSOConfigService`: + +- After config saved +- After config deleted +- After config toggled + +Default behavior: debug logging. + +--- + +## DefaultInvitationProcessor + +Hooks triggered when invitations are: + +- Created +- Revoked + +Default behavior: debug logging. + +--- + +## DefaultAgentRegistrationSecretProcessor + +Hooks triggered when: + +- Agent registration secret is generated +- Agent registration secret is deactivated + +Default behavior: debug logging. + +In SaaS deployments, this could integrate with: + +- Audit pipelines +- Event streaming systems +- Secret rotation workflows + +--- + +## DefaultDomainExistenceValidator + +The `DefaultDomainExistenceValidator` provides a permissive default implementation: + +```mermaid +flowchart TD + Domains["List of Domains"] --> Validator["DomainExistenceValidator"] + Validator --> Result["Return false (no blocking)"] +``` + +### Behavior + +- Always returns `false`. +- Does not block domain validation in OSS mode. +- Intended to be overridden in SaaS multi-tenant environments. + +This design allows: + +- Strict domain control in enterprise deployments. +- Relaxed behavior in OSS mode. + +--- + +# Cross-Module Relationships + +Although this documentation focuses on Api Service Core Business Services, it integrates closely with: + +- Data repositories (Mongo persistence layer) +- Encryption services +- Domain validation infrastructure +- REST controllers +- GraphQL data fetchers + +The business services layer is the authoritative source of domain rules, ensuring that both REST and GraphQL interfaces behave consistently. + +--- + +# Design Principles + +## 1. Clear Domain Boundaries + +Services encapsulate domain logic. Controllers delegate orchestration to this module. + +## 2. Extensibility by Default + +Every major domain service exposes processor hooks. + +## 3. Secure-by-Design + +- Client secrets encrypted at rest. +- Domain validation enforced before auto-provision. +- Owner deletion protection. + +## 4. Multi-Tenant Awareness + +Although tenant resolution occurs in other modules, domain validation and SSO rules support per-tenant isolation. + +--- + +# Summary + +The **Api Service Core Business Services** module is the domain orchestration engine of the API layer. + +It: + +- Enforces core business rules +- Coordinates persistence and mapping +- Protects critical invariants (users, SSO, secrets) +- Provides extension hooks for advanced deployments + +By centralizing domain logic in this module, the platform ensures consistent behavior across REST, GraphQL, and future interfaces while maintaining strong extensibility and security guarantees. diff --git a/docs/reference/architecture/api-service-core-config-and-security/api-service-core-config-and-security.md b/docs/reference/architecture/api-service-core-config-and-security/api-service-core-config-and-security.md index 0c8137f06..625944520 100644 --- a/docs/reference/architecture/api-service-core-config-and-security/api-service-core-config-and-security.md +++ b/docs/reference/architecture/api-service-core-config-and-security/api-service-core-config-and-security.md @@ -1,67 +1,76 @@ # Api Service Core Config And Security -The **Api Service Core Config And Security** module defines the foundational runtime configuration for the OpenFrame API service. It centralizes: +## Overview -- Core Spring Boot bean configuration -- Authentication and argument resolution +The **Api Service Core Config And Security** module provides foundational configuration and security infrastructure for the OpenFrame API service. It is responsible for: + +- Spring Boot application-level bean configuration +- Security filter chain and OAuth2 resource server integration +- JWT issuer-based authentication resolution +- GraphQL custom scalar configuration - OAuth client bootstrapping -- GraphQL custom scalar definitions -- Outbound HTTP client configuration -- JWT resource server integration with issuer-based caching +- REST client provisioning -This module is intentionally lightweight in business logic. Its responsibility is to configure and secure the API runtime so that higher-level modules such as REST controllers, GraphQL data fetchers, and service layers can operate consistently and securely. +This module does **not** implement business logic or endpoint definitions. Instead, it supplies the cross-cutting infrastructure required by: ---- +- REST Controllers (API layer) +- GraphQL Data Fetchers +- Business Services +- Data Access Layers +- External integrations -## 1. Architectural Role in the System +It acts as the **security and configuration backbone** of the API service. -The Api Service Core Config And Security module sits at the heart of the API runtime. It wires together: +--- -- Spring Boot auto-configuration -- Spring Security (OAuth2 Resource Server) -- GraphQL DGS custom scalars -- OAuth client initialization -- Shared infrastructure beans (PasswordEncoder, RestTemplate) +## Architectural Position -### High-Level Placement +The module sits between the API surface (REST + GraphQL) and the underlying authorization and data layers. ```mermaid -flowchart LR - Gateway["Gateway Service"] -->|"JWT Forwarded"| ApiService["API Service Runtime"] - ApiService -->|"Uses"| ConfigModule["Api Service Core Config And Security"] - ApiService --> Controllers["REST Controllers"] - ApiService --> GraphQL["GraphQL Data Fetchers"] - ApiService --> Services["Domain Services"] - Services --> DataLayer["Mongo / Kafka / Redis"] +flowchart TD + Client["Client Application"] --> Gateway["Gateway Service Core"] + Gateway --> ApiService["API Service"] + + subgraph ApiCore["Api Service Core"] + Config["Api Service Core Config And Security"] + RestLayer["REST Controllers"] + GraphQlLayer["GraphQL Layer"] + Business["Business Services"] + end + + ApiService --> Config + ApiService --> RestLayer + ApiService --> GraphQlLayer + RestLayer --> Business + GraphQlLayer --> Business + + Config --> AuthService["Authorization Service Core"] + Business --> DataLayer["Mongo + Repositories"] ``` -### Security Responsibility Split - -The API service does **not** perform full authentication enforcement. Instead: +### Key Architectural Responsibilities -- The **Gateway Service**: - - Validates JWT tokens - - Handles public vs protected routes - - Injects `Authorization` headers from cookies -- The **Api Service Core Config And Security** module: - - Enables OAuth2 Resource Server support - - Resolves JWT issuers dynamically - - Exposes `@AuthenticationPrincipal` support - -This layered approach keeps the API service stateless and focused on business logic. +| Responsibility | Description | +|---------------|------------| +| Authentication Support | Enables OAuth2 resource server behavior for JWT validation | +| Principal Resolution | Injects authenticated principals into controller methods | +| OAuth Client Initialization | Seeds default OAuth client at startup | +| GraphQL Scalar Support | Registers Date, Instant, and Long custom scalars | +| REST Integration | Provides `RestTemplate` bean for external calls | +| Password Encoding | Configures BCrypt encoder for secure hashing | --- -## 2. Core Configuration Components +# Core Components -### 2.1 ApiApplicationConfig +## 1. ApiApplicationConfig -**Component:** -- `ApiApplicationConfig` +**Class:** `ApiApplicationConfig` -Provides shared infrastructure beans. +Provides foundational Spring beans required across the application. -#### PasswordEncoder Bean +### Password Encoder ```java @Bean @@ -70,281 +79,287 @@ public PasswordEncoder passwordEncoder() { } ``` -- Uses BCrypt for secure password hashing. -- Shared across services handling credentials. -- Ensures consistent hashing strategy across modules. +### Purpose + +- Ensures consistent password hashing using BCrypt +- Used by user services and authentication-related workflows +- Provides secure password storage compatibility with Authorization Service Core --- -### 2.2 AuthenticationConfig +## 2. AuthenticationConfig -**Component:** -- `AuthenticationConfig` +**Class:** `AuthenticationConfig` -Registers a custom Spring MVC argument resolver: +Registers a custom `HandlerMethodArgumentResolver`: - `AuthPrincipalArgumentResolver` -This enables controller methods like: +### Responsibility + +Enables controller methods to use: ```java -public ResponseEntity getProfile(@AuthenticationPrincipal AuthPrincipal principal) +@AuthenticationPrincipal AuthPrincipal principal ``` -#### Flow +This allows: + +- Injection of authenticated user context +- Simplified controller method signatures +- Decoupling from raw Spring Security APIs + +### Flow ```mermaid -flowchart TD - Request["HTTP Request"] --> Security["OAuth2 Resource Server"] - Security --> Controller["Controller Method"] - Controller --> Resolver["AuthPrincipalArgumentResolver"] - Resolver --> Principal["AuthPrincipal Injected"] +flowchart LR + JwtToken["JWT Token"] --> SecurityLayer["Security Filter Chain"] + SecurityLayer --> AuthPrincipal["AuthPrincipal"] + AuthPrincipal --> Resolver["AuthPrincipalArgumentResolver"] + Resolver --> Controller["REST Controller Method"] ``` -This ensures: -- Clean controller signatures -- Strong typing for authenticated users -- No manual extraction of JWT claims - --- -### 2.3 SecurityConfig +## 3. SecurityConfig -**Component:** -- `SecurityConfig` +**Class:** `SecurityConfig` -This is the central Spring Security configuration. +This is the core security configuration for the API service. -#### Key Characteristics +### Design Philosophy -- CSRF disabled (stateless API) -- All routes `permitAll()` at API layer -- OAuth2 Resource Server enabled -- Multi-issuer JWT support -- Caffeine-based JWT provider cache +The Gateway Service Core handles: + +- JWT validation +- PermitAll path filtering +- Header normalization +- Cookie-to-header authorization mapping + +The API service acts as an **OAuth2 Resource Server** primarily to support: + +- `@AuthenticationPrincipal` +- Method-level security context ### JWT Issuer-Based Authentication -The module uses: +The configuration supports **multi-issuer JWT validation** using: - `JwtIssuerAuthenticationManagerResolver` -- A `LoadingCache` +- Caffeine-backed cache of `JwtAuthenticationProvider` -This allows dynamic resolution of authentication managers based on the JWT issuer. +### JWT Provider Cache ```mermaid flowchart TD - Incoming["Incoming Request with JWT"] --> ExtractIssuer["Extract iss Claim"] - ExtractIssuer --> CacheCheck["Check JwtAuthenticationProvider Cache"] - CacheCheck -->|"Miss"| CreateDecoder["Create JwtDecoder from Issuer"] - CreateDecoder --> StoreCache["Store in Caffeine Cache"] - CacheCheck -->|"Hit"| UseProvider["Reuse Cached Provider"] - StoreCache --> Authenticate["Authenticate JWT"] - UseProvider --> Authenticate - Authenticate --> Principal["SecurityContext Updated"] + Request["Incoming Request"] --> ExtractIssuer["Extract JWT Issuer"] + ExtractIssuer --> CacheCheck["Caffeine Cache Lookup"] + + CacheCheck -->|"Hit"| Provider["JwtAuthenticationProvider"] + CacheCheck -->|"Miss"| CreateDecoder["JwtDecoders.fromIssuerLocation()"] + + CreateDecoder --> Provider + Provider --> Authenticate["Authenticate JWT"] ``` -#### Cache Configuration +### Cache Properties -The cache is controlled by properties: +Configured via application properties: - `openframe.security.jwt.cache.expire-after` - `openframe.security.jwt.cache.refresh-after` - `openframe.security.jwt.cache.maximum-size` -This ensures: -- Efficient decoder reuse -- Controlled memory usage -- Automatic refresh of issuer metadata +### Security Filter Chain + +Key characteristics: + +- CSRF disabled (stateless API) +- `anyRequest().permitAll()` +- OAuth2 Resource Server enabled + +This may look permissive, but authentication enforcement is delegated to the Gateway layer. --- -### 2.4 DataInitializer +## 4. DataInitializer + +**Class:** `DataInitializer` -**Component:** -- `DataInitializer` +Implements `CommandLineRunner` to initialize a default OAuth client at startup. -Implements a `CommandLineRunner` to initialize OAuth clients at application startup. +### Behavior -#### Responsibilities +1. Reads environment properties: + - `oauth.client.default.id` + - `oauth.client.default.secret` +2. Checks repository for existing client +3. Updates secret if changed +4. Creates client if missing -- Reads properties: - - `oauth.client.default.id` - - `oauth.client.default.secret` -- Checks if client exists -- Updates secret if changed -- Creates client if missing +### Initialization Flow ```mermaid flowchart TD - Startup["Application Startup"] --> LoadProps["Read OAuth Properties"] - LoadProps --> QueryRepo["Find Client by ID"] - QueryRepo -->|"Exists"| CompareSecret["Compare Secrets"] - CompareSecret -->|"Different"| UpdateSecret["Update and Save"] - CompareSecret -->|"Same"| Skip["No Action"] - QueryRepo -->|"Missing"| CreateClient["Create OAuthClient"] + Startup["Application Startup"] --> ReadEnv["Read OAuth Properties"] + ReadEnv --> Lookup["Find OAuth Client"] + + Lookup -->|"Exists"| CompareSecret["Compare Client Secret"] + CompareSecret -->|"Different"| UpdateSecret["Update Secret"] + CompareSecret -->|"Same"| Skip["No Change"] + + Lookup -->|"Missing"| CreateClient["Create OAuth Client"] ``` -#### Design Rationale +### Why This Matters -- Ensures environment-driven configuration. -- Avoids manual database seeding. -- Keeps OAuth client configuration aligned with deployment configuration. +- Ensures consistent OAuth configuration +- Prevents manual database bootstrapping +- Supports containerized and ephemeral deployments --- -### 2.5 GraphQL Custom Scalars +## 5. GraphQL Scalar Configurations -The module defines three custom DGS scalars. +The module defines three custom GraphQL scalars using Netflix DGS. -#### DateScalarConfig +### DateScalarConfig -- Scalar name: `Date` -- Backed by `LocalDate` +- GraphQL Type: `Date` +- Java Type: `LocalDate` - Format: `yyyy-MM-dd` +- Validates input and provides descriptive errors -Validation ensures: -- Strict format compliance -- Clear error messages for invalid input - ---- - -#### InstantScalarConfig - -- Scalar name: `Instant` -- Backed by `Instant` -- ISO-8601 format (e.g. `2026-01-01T10:15:30Z`) +### InstantScalarConfig -Provides: -- Consistent time serialization -- Accurate timezone handling +- GraphQL Type: `Instant` +- Java Type: `Instant` +- ISO-8601 compliant +- Supports serialization and literal parsing ---- - -#### LongScalarConfig +### LongScalarConfig -- Scalar name: `Long` -- Supports 64-bit integers -- Required for values exceeding GraphQL `Int` limits +- GraphQL Type: `Long` +- Java Type: `Long` +- Required because GraphQL `Int` is limited to 32-bit +- Handles: + - Numeric values + - String values + - Literal values -Handles: -- Numeric literals -- String-based numeric input -- Safe coercion with validation +### GraphQL Type Integration ```mermaid flowchart LR - GraphQLInput["GraphQL Query Input"] --> Scalar["Custom Scalar Coercing"] - Scalar --> Validation["Format Validation"] - Validation --> Parsed["Java Type (LocalDate / Instant / Long)"] - Parsed --> DataFetcher["Data Fetcher Execution"] + GraphQlSchema["GraphQL Schema"] --> DateScalar["Date Scalar"] + GraphQlSchema --> InstantScalar["Instant Scalar"] + GraphQlSchema --> LongScalar["Long Scalar"] + + DateScalar --> LocalDate["LocalDate"] + InstantScalar --> InstantType["Instant"] + LongScalar --> LongType["Long"] ``` -These scalars standardize type handling across all GraphQL data fetchers. +These scalars ensure: + +- Strong typing +- Input validation +- Consistent serialization across clients --- -### 2.6 RestTemplateConfig +## 6. RestTemplateConfig -**Component:** -- `RestTemplateConfig` +**Class:** `RestTemplateConfig` -Defines a singleton `RestTemplate` bean. +Provides a reusable `RestTemplate` bean. -```java -@Bean -public RestTemplate restTemplate() { - return new RestTemplate(); -} -``` +### Purpose + +- Used by services calling external APIs +- Supports integration with: + - Authorization service + - External connectors + - Third-party integrations -Used for: -- Outbound HTTP calls -- Internal service-to-service communication -- OAuth metadata resolution (indirectly via Spring Security) +While simple, centralizing it enables: -Centralizing the bean allows: - Future interceptors +- Tracing - Timeout configuration -- Observability instrumentation +- Load-balancing extensions --- -## 3. End-to-End Security Flow +# End-to-End Request Lifecycle -Below is a simplified request lifecycle involving this module. +The following diagram illustrates how a request flows through the system with this module in place. ```mermaid sequenceDiagram participant Client participant Gateway - participant Api as "API Service" + participant Api participant Security as "SecurityConfig" participant Controller - Client->>Gateway: Request with Cookie or Token - Gateway->>Gateway: Validate JWT + Client->>Gateway: HTTP Request with JWT Gateway->>Api: Forward request with Authorization header - Api->>Security: Resolve issuer and authenticate - Security->>Api: Populate SecurityContext - Api->>Controller: Invoke handler - Controller->>Controller: Inject AuthPrincipal + Api->>Security: Resolve issuer and validate JWT + Security->>Controller: Inject AuthPrincipal + Controller->>Api: Return response + Api->>Gateway: HTTP response + Gateway->>Client: Final response ``` -### Key Observations - -- Authentication enforcement happens upstream. -- API service trusts Gateway filtering. -- Multi-tenant issuer resolution is supported dynamically. -- Controllers remain clean and declarative. - --- -## 4. Design Principles +# Interaction with Other Modules -The Api Service Core Config And Security module follows these principles: +Although this module contains no business logic, it is critical to: -### 4.1 Separation of Concerns +- REST Controllers (authentication context injection) +- GraphQL Layer (custom scalar support) +- Business Services (password encoding and security context) +- Authorization Service Core (JWT issuer validation) +- Data repositories (OAuth client persistence) -- Gateway: authentication + edge security -- API: resource server support + identity propagation -- Services: business logic +It provides the secure and consistent runtime environment required for the broader OpenFrame platform. -### 4.2 Statelessness +--- -- No server-side sessions -- JWT-based identity -- Cache only for issuer metadata +# Design Principles -### 4.3 Extensibility +### 1. Gateway-First Security +Authentication enforcement lives at the Gateway. The API layer remains lightweight and issuer-aware. -- Centralized bean definitions -- Pluggable scalar definitions -- Configurable cache properties +### 2. Multi-Tenant JWT Support +Issuer-based resolution allows dynamic support for multiple identity providers. -### 4.4 Environment-Driven Configuration +### 3. Declarative Configuration +Spring annotations (`@Configuration`, `@Bean`) ensure predictable lifecycle behavior. -- OAuth clients initialized via properties -- Cache behavior driven by configuration -- No hard-coded credentials +### 4. Fail-Fast Validation +GraphQL scalars enforce strict input formats. + +### 5. Idempotent Bootstrapping +OAuth client initialization is safe across restarts and deployments. --- -## 5. Summary +# Summary -The **Api Service Core Config And Security** module is the foundation of the OpenFrame API runtime. It: +The **Api Service Core Config And Security** module is the infrastructural foundation of the OpenFrame API service. -- Enables secure JWT-based authentication -- Integrates with multi-issuer OAuth providers -- Standardizes GraphQL scalar behavior -- Provides core infrastructure beans -- Bootstraps OAuth client configuration +It provides: -While minimal in business logic, this module is critical for ensuring: +- JWT-based resource server support +- Multi-issuer authentication resolution +- Password encoding +- OAuth client bootstrapping +- GraphQL scalar definitions +- REST client configuration -- Secure request handling -- Consistent authentication context propagation -- Reliable GraphQL type handling -- Clean separation between infrastructure and domain layers +Without this module, higher-level modules (controllers, GraphQL, business services) would lack authentication context, scalar type safety, and initialization guarantees. -It acts as the runtime spine of the API service, ensuring that all higher-level modules operate in a secure, predictable, and standardized environment. +It ensures the API service remains secure, extensible, and production-ready. \ No newline at end of file diff --git a/docs/reference/architecture/api-service-core-dataloaders/api-service-core-dataloaders.md b/docs/reference/architecture/api-service-core-dataloaders/api-service-core-dataloaders.md deleted file mode 100644 index 7caecbce5..000000000 --- a/docs/reference/architecture/api-service-core-dataloaders/api-service-core-dataloaders.md +++ /dev/null @@ -1,343 +0,0 @@ -# Api Service Core Dataloaders - -## Overview - -The **Api Service Core Dataloaders** module provides batched, asynchronous data loading capabilities for the GraphQL layer of the OpenFrame API Service Core. - -Built on top of **Netflix DGS** and the standard `org.dataloader` pattern, this module eliminates N+1 query problems by batching and caching entity lookups during a single GraphQL request lifecycle. - -It acts as a bridge between: - -- GraphQL Data Fetchers (resolvers) -- Domain Services -- Mongo repositories -- Underlying domain documents - -This module is a critical performance layer in the API stack. - ---- - -## Architectural Context - -The Api Service Core Dataloaders module sits between the GraphQL Data Fetchers and the service/repository layer. - -```mermaid -flowchart TD - Client["GraphQL Client"] --> GQL["GraphQL Data Fetchers"] - GQL --> DL["Api Service Core Dataloaders"] - DL --> Service["Domain Services"] - DL --> Repo["Mongo Repositories"] - Service --> Mongo[("MongoDB")] - Repo --> Mongo -``` - -### Responsibilities - -- Batch load related entities by IDs -- Preserve request ordering -- Avoid redundant queries within a request -- Delegate to appropriate service or repository -- Return results asynchronously via `CompletableFuture` - ---- - -## Why DataLoaders Are Required - -Without DataLoaders, a query such as: - -```graphql -query { - devices { - id - organization { - name - } - } -} -``` - -would result in: - -- 1 query to load devices -- N queries to load organizations - -With DataLoaders: - -- 1 query to load devices -- 1 batched query to load all required organizations - -This dramatically improves performance and scalability. - ---- - -## DataLoader Execution Model - -All loaders implement: - -```java -BatchLoader -``` - -and are annotated with: - -```java -@DgsDataLoader(name = "...") -``` - -Each loader: - -1. Receives a list of keys -2. Removes nulls -3. Batch queries the database or service -4. Maps results by ID -5. Returns results in the same order as input -6. Executes asynchronously via `CompletableFuture.supplyAsync` - -```mermaid -sequenceDiagram - participant Resolver as "GraphQL Resolver" - participant Loader as "DataLoader" - participant Service as "Service/Repository" - - Resolver->>Loader: load([id1, id2, id3]) - Loader->>Service: Batch fetch entities - Service-->>Loader: List of entities - Loader-->>Resolver: Ordered results -``` - ---- - -# Core DataLoaders - -Below are the loaders provided by this module. - ---- - -## InstalledAgentDataLoader - -**Purpose:** Batch loads InstalledAgent objects grouped by machine ID. - -**Delegates to:** `InstalledAgentService` - -```mermaid -flowchart LR - Resolver --> InstalledAgentDataLoader - InstalledAgentDataLoader --> InstalledAgentService - InstalledAgentService --> Mongo -``` - -- Key: `machineId` -- Value: `List` -- Use case: Resolving installed agents for multiple machines in one request - ---- - -## KnowledgeBaseAttachmentDataLoader - -**Purpose:** Batch loads attachments for knowledge base items. - -**Delegates to:** `KnowledgeBaseAttachmentService` - -- Key: `knowledgeBaseItemId` -- Value: `List` -- Includes debug logging for batch size visibility - ---- - -## KnowledgeBaseItemDataLoader - -**Purpose:** Batch loads KnowledgeBaseItem by ID. - -**Used for:** Polymorphic AssignableTarget resolution for KNOWLEDGE_ARTICLE target type. - -**Delegates to:** `KnowledgeBaseItemRepository` - -### Special Behavior - -- Filters null IDs -- De-duplicates IDs -- Returns `null` for missing items -- Preserves request order - -```mermaid -flowchart TD - Resolver --> KnowledgeBaseItemDataLoader - KnowledgeBaseItemDataLoader --> KnowledgeBaseItemRepository - KnowledgeBaseItemRepository --> Mongo -``` - ---- - -## KnowledgeBaseTagDataLoader - -**Purpose:** Batch loads Tags associated with Knowledge Base items. - -**Delegates to:** `KnowledgeBaseTagService` - -- Key: `knowledgeBaseItemId` -- Value: `List` - ---- - -## MachineDataLoader - -**Purpose:** Batch loads Machine entities by machine ID. - -**Used for:** AssignableTarget resolution for DEVICE target type. - -**Delegates to:** `MachineRepository` - -### Behavior - -- Filters null IDs -- Performs `findByMachineIdIn` -- Maps results by machineId -- Preserves input ordering - ---- - -## OrganizationDataLoader - -**Purpose:** Batch loads Organization entities by organization ID. - -**Key Optimization:** Prevents N+1 queries when resolving organization for many machines. - -**Delegates to:** `OrganizationRepository` - -### Additional Logic - -- Filters out soft-deleted organizations -- Only returns organizations with `ACTIVE` status - -```mermaid -flowchart TD - Resolver --> OrganizationDataLoader - OrganizationDataLoader --> OrganizationRepository - OrganizationRepository --> Mongo -``` - ---- - -## TagDataLoader - -**Purpose:** Batch loads Tags for machines. - -**Delegates to:** `TagService` - -- Key: `machineId` -- Value: `List` - ---- - -## TicketDataLoader - -**Purpose:** Batch loads Ticket entities by ID. - -**Used for:** AssignableTarget resolution for TICKET target type. - -**Delegates to:** `TicketRepository` - -### Behavior - -- Filters null IDs -- Uses `findAllById` -- Preserves input order - ---- - -## ToolConnectionDataLoader - -**Purpose:** Batch loads ToolConnection entities for machines. - -**Delegates to:** `ToolConnectionService` - -- Key: `machineId` -- Value: `List` - ---- - -## UserDataLoader - -**Purpose:** Batch loads UserResponse DTOs by user ID. - -**Delegates to:** `UserService` - -### Important Detail - -The loader goes through `UserService` rather than directly querying a repository. This ensures: - -- Registered UserProcessor implementations enrich responses -- SaaS-specific user augmentation is applied -- Profile image and additional computed fields are included - -**Used by:** KnowledgeBaseItem author resolver - ---- - -# Cross-Module Relationships - -The Api Service Core Dataloaders module works closely with: - -- [Api Service Core GraphQL Datafetchers](../api-service-core-graphql-datafetchers/api-service-core-graphql-datafetchers.md) -- Domain services in the API layer -- Mongo repositories from the data layer - -```mermaid -flowchart TD - subgraph graphql_layer["GraphQL Layer"] - DF["Data Fetchers"] - DL["DataLoaders"] - end - - subgraph service_layer["Service Layer"] - SVC["Domain Services"] - end - - subgraph data_layer["Data Layer"] - REPO["Mongo Repositories"] - DB[("MongoDB")] - end - - DF --> DL - DL --> SVC - DL --> REPO - SVC --> DB - REPO --> DB -``` - ---- - -# Performance Characteristics - -βœ… Eliminates N+1 query issues -βœ… Reduces database round trips -βœ… Maintains deterministic ordering -βœ… Enables async non-blocking resolution -βœ… Scales efficiently with large GraphQL queries - ---- - -# Design Patterns Used - -- **Batch Loading Pattern** -- **Repository Pattern** -- **Service Layer Abstraction** -- **Asynchronous Execution via CompletableFuture** -- **GraphQL DataLoader Pattern** - ---- - -# Summary - -The **Api Service Core Dataloaders** module is a performance-critical layer in the OpenFrame GraphQL architecture. - -It ensures that complex nested queries: - -- Execute efficiently -- Avoid excessive database calls -- Preserve domain encapsulation -- Maintain clean separation between GraphQL and persistence layers - -Without this module, GraphQL queries across devices, organizations, tickets, knowledge base items, and users would not scale effectively. - -This module enables high-performance, production-grade GraphQL execution within the Api Service Core. \ No newline at end of file diff --git a/docs/reference/architecture/api-service-core-dtos/api-service-core-dtos.md b/docs/reference/architecture/api-service-core-dtos/api-service-core-dtos.md new file mode 100644 index 000000000..aa950f1ec --- /dev/null +++ b/docs/reference/architecture/api-service-core-dtos/api-service-core-dtos.md @@ -0,0 +1,381 @@ +# Api Service Core Dtos + +## Overview + +The **Api Service Core Dtos** module defines the primary Data Transfer Objects (DTOs) used by the Api Service Core layer. These DTOs represent: + +- REST request and response payloads +- GraphQL input and output types +- Pagination and Relay-style connection models +- OAuth, OIDC, and SSO interaction models +- Force update and remote action commands +- Knowledge base, invitation, notification, and user management payloads + +This module acts as the **API contract layer** between: + +- [Api Service Core Rest Controllers](../api-service-core-rest-controllers/api-service-core-rest-controllers.md) +- [Api Service Core GraphQL Layer](../api-service-core-graphql-layer/api-service-core-graphql-layer.md) +- [Api Service Core Business Services](../api-service-core-business-services/api-service-core-business-services.md) +- Data model and repository modules +- External authentication and authorization services + +It ensures a strict separation between internal domain models (Mongo documents, services, stream models) and externally exposed API contracts. + +--- + +## Architectural Role + +The Api Service Core Dtos module sits at the boundary of the API layer. + +```mermaid +flowchart LR + Client["Client Applications"] --> Rest["REST Controllers"] + Client --> GraphQL["GraphQL Layer"] + + Rest --> DTOs["Api Service Core Dtos"] + GraphQL --> DTOs + + DTOs --> Services["Business Services"] + Services --> Repos["Mongo Repositories"] + Services --> Auth["Authorization Service Core"] +``` + +### Responsibilities + +1. Define stable API-facing models +2. Encapsulate validation constraints (Jakarta Validation) +3. Support GraphQL Relay pagination patterns +4. Decouple REST/GraphQL from persistence documents +5. Model OAuth, OIDC, and SSO communication payloads + +--- + +## DTO Categories + +The module can be logically grouped into the following domains. + +--- + +## 1. Core API and Pagination Models + +### GenericEdge + +`GenericEdge` models a Relay-style edge containing: + +- `node` – the actual data object +- `cursor` – opaque pagination cursor + +### CountedGenericConnection + +Extends a generic connection model and adds: + +- `filteredCount` – total count after filters are applied + +This supports GraphQL cursor-based pagination. + +```mermaid +flowchart TD + Connection["CountedGenericConnection"] --> Edge["GenericEdge"] + Edge --> Node["Domain Node"] + Connection --> PageInfo["PageInfo"] + Connection --> FilteredCount["filteredCount"] +``` + +These types are heavily used in the GraphQL data fetchers. + +--- + +## 2. Authentication, OAuth, and SSO DTOs + +This group models interactions with OAuth providers, OpenID Connect, and SSO configuration. + +### SSO Configuration + +- `SSOConfigRequest` +- `SSOConfigResponse` +- `SSOConfigStatusResponse` +- `SSOProviderInfo` + +These DTOs support tenant-level SSO configuration and provider discovery. + +```mermaid +flowchart TD + Admin["Admin User"] --> Req["SSOConfigRequest"] + Req --> Service["SSOConfigService"] + Service --> Resp["SSOConfigResponse"] + Service --> Status["SSOConfigStatusResponse"] +``` + +### OAuth Flow DTOs + +- `AuthorizationResponse` +- `GoogleTokenRequest` +- `SocialAuthRequest` +- `TokenResponse` + +These model the authorization code and token exchange steps. + +### OIDC Discovery and UserInfo + +- `OpenIDConfiguration` +- `UserInfo` +- `UserInfoRequest` + +They represent: + +- OIDC discovery metadata +- Claims returned by identity providers +- Internal user mapping requests + +This layer integrates with the Authorization Service Core module. + +--- + +## 3. User and Invitation DTOs + +### User Management + +- `UserResponse` +- `UpdateUserRequest` + +`UserResponse` encapsulates: + +- Identity attributes +- Roles +- Status +- Profile image +- Audit timestamps + +### Invitation Management + +- `CreateInvitationRequest` +- `InvitationPageResponse` +- `UpdateInvitationStatusRequest` + +These support user onboarding flows integrated with the authorization system. + +```mermaid +flowchart TD + Admin["Admin"] --> CreateInvite["CreateInvitationRequest"] + CreateInvite --> Processor["Invitation Processor"] + Processor --> PageResp["InvitationPageResponse"] +``` + +--- + +## 4. Agent Registration and API Keys + +### AgentRegistrationSecretResponse + +Represents a registration secret for client agents: + +- `id` +- `key` +- `createdAt` +- `active` + +Used during agent onboarding and validated by client ingress services. + +### ApiKeyResponse + +Exposes metadata about API keys without revealing the secret: + +- Request counters +- Usage statistics +- Expiration metadata + +This ensures secure observability without secret leakage. + +--- + +## 5. Device, Event, Log, and Tool Filters + +GraphQL-specific input types designed for flexible querying. + +### DeviceFilterInput +- Statuses +- Device types +- Organization filters +- Tag-based filtering + +### EventFilterInput +- User-based filtering +- Date range filtering +- Event type filtering + +### LogFilterInput +- Date ranges +- Tool types +- Severity filters +- Organization scope + +### ToolFilterInput +- Enabled flag +- Type and category + +These inputs map to repository-level query filters while keeping API concerns separate from Mongo query objects. + +```mermaid +flowchart LR + FilterInput["FilterInput DTO"] --> DataFetcher["GraphQL DataFetcher"] + DataFetcher --> QueryService["Query Service"] + QueryService --> Repository["Mongo Repository"] +``` + +--- + +## 6. Knowledge Base DTOs + +Supports article creation, updates, folder management, and attachment workflows. + +### Article Management + +- `CreateArticleInput` +- `UpdateArticleInput` +- `DeleteFolderInput` +- `KnowledgeBaseFilterInput` + +### Attachment Flow + +- `CreateKnowledgeBaseAttachmentInput` +- `CreateKnowledgeBaseTempAttachmentInput` +- `LinkKnowledgeBaseTempAttachmentsInput` + +These DTOs enable multi-step upload flows: + +1. Temporary upload +2. Linking to article +3. Final persistence + +--- + +## 7. Notifications and Assignments + +### NotificationFilterInput + +Used to filter notifications by read/unread state. + +### AssignedItemCount + +Encapsulates assignment summary data: + +- `targetType` +- `count` + +Supports dashboards and aggregation endpoints. + +--- + +## 8. Force Update and Remote Action DTOs + +These DTOs trigger remote operations on agents or tool integrations. + +### Client Update + +- `ForceClientUpdateRequest` (force namespace) +- `ForceClientUpdateRequest` (update namespace) + +### Tool Operations + +- `ForceToolInstallationAllRequest` +- `ForceToolReinstallationRequest` +- `ForceToolUpdateRequest` +- `ForceToolAgentUpdateRequest` +- `ForceToolAgentUpdateAllRequest` +- `ForceToolAgentUpdateResponse` + +```mermaid +flowchart TD + Admin["Admin"] --> Request["ForceToolAgentUpdateRequest"] + Request --> Controller["ForceAgentController"] + Controller --> Service["Update Processor"] + Service --> Messaging["Kafka or NATS"] +``` + +These DTOs are typically translated into messages published to eventing systems. + +--- + +## 9. Client Configuration DTO + +### ClientConfigurationResponse + +Exposes minimal configuration metadata such as client version. + +This DTO is typically consumed by: + +- Desktop agents +- Web UI clients + +--- + +## Validation Strategy + +Many DTOs use Jakarta Validation annotations such as: + +- `@NotBlank` +- `@NotNull` +- `@Size` + +This ensures: + +- Early request validation at controller or GraphQL boundary +- Clear error reporting +- Reduced service-layer defensive logic + +--- + +## Design Principles + +### 1. Strict API Contract Isolation + +DTOs do not expose: + +- Mongo document internals +- Repository-specific fields +- Security-sensitive data + +### 2. GraphQL and REST Parity + +The module supports both paradigms: + +- REST-specific responses (e.g., ApiKeyResponse) +- GraphQL-specific inputs (e.g., DeviceFilterInput) +- Relay-compatible pagination models + +### 3. Multi-Tenant and Security Awareness + +DTOs related to: + +- SSO configuration +- OAuth token exchange +- Agent registration secrets + +are designed to align with tenant-aware security policies. + +--- + +## How This Module Fits in the System + +```mermaid +flowchart TD + Client["Frontend or Agent"] --> Controllers["REST Controllers"] + Client --> GraphQL["GraphQL Layer"] + + Controllers --> DTOs["Api Service Core Dtos"] + GraphQL --> DTOs + + DTOs --> Business["Business Services"] + Business --> Data["Mongo Data Layer"] + Business --> Auth["Authorization Service"] + Business --> Streams["Stream Processing"] +``` + +The **Api Service Core Dtos** module forms the contract backbone of the API layer. It ensures: + +- Stability of external interfaces +- Clean separation of concerns +- Reusable pagination and filtering patterns +- Secure handling of authentication-related data + +It is intentionally lightweight, declarative, and validation-focused, allowing controllers and services to evolve independently while preserving API compatibility. diff --git a/docs/reference/architecture/api-service-core-graphql-datafetchers/api-service-core-graphql-datafetchers.md b/docs/reference/architecture/api-service-core-graphql-datafetchers/api-service-core-graphql-datafetchers.md deleted file mode 100644 index 6757c6391..000000000 --- a/docs/reference/architecture/api-service-core-graphql-datafetchers/api-service-core-graphql-datafetchers.md +++ /dev/null @@ -1,439 +0,0 @@ -# Api Service Core Graphql Datafetchers - -## Overview - -The **Api Service Core Graphql Datafetchers** module is the GraphQL execution layer of the OpenFrame API Service Core. It exposes domain capabilities (devices, events, organizations, knowledge base, notifications, assignments, tools, logs, and tags) through Netflix DGS-based GraphQL data fetchers. - -This module acts as the orchestration layer between: - -- GraphQL schema and Relay node model -- Application services (DeviceService, EventService, KnowledgeBaseService, etc.) -- DataLoaders for batching and N+1 mitigation -- Security context (JWT-based principals) -- Cursor-based pagination and filter criteria - -It does **not** implement business logic directly. Instead, it: - -- Decodes Relay global IDs -- Maps GraphQL inputs into domain filter criteria -- Delegates to services -- Wraps results into Relay-compatible connection types -- Resolves nested fields using DataLoaders - ---- - -## High-Level Architecture - -```mermaid -flowchart LR - Client["GraphQL Client"] --> Schema["GraphQL Schema"] - Schema --> DataFetchers["DGS DataFetchers"] - DataFetchers --> Mappers["GraphQL Mappers"] - DataFetchers --> Services["Application Services"] - Services --> Repositories["Mongo / Pinot / Cassandra Repositories"] - - DataFetchers --> DataLoaders["DataLoaders (Batching)"] - DataLoaders --> Services - - DataFetchers --> Security["SecurityContext + JWT"] -``` - -### Responsibilities of This Module - -- Implements `@DgsQuery`, `@DgsMutation`, and `@DgsData` -- Converts GraphQL inputs into service-layer DTOs -- Encodes and decodes Relay global IDs -- Builds cursor-based pagination criteria -- Returns Relay-compatible `Connection` and `Edge` types -- Resolves polymorphic types (NotificationContext, Node interface) - ---- - -## Core Design Patterns - -### 1. Relay Global ID Pattern - -All node-based entities use Relay global IDs. - -```text -Global ID = Base64(TypeName:RawId) -Example: "Machine:abc123" -> encoded -``` - -Pattern used everywhere: - -- Decode input ID using `Relay.fromGlobalId()` -- Fetch entity using raw ID -- Encode output ID using `Relay.toGlobalId()` - -This enables: - -- Uniform `node(id: ID!)` resolution -- Type-safe global references -- Cross-entity linking - ---- - -### 2. Cursor-Based Pagination - -Pagination is built using: - -- `ConnectionArgs` -- `CursorPaginationCriteria` -- `CountedGenericQueryResult` or `GenericQueryResult` -- `GenericEdge` -- `CountedGenericConnection` or `GenericConnection` - -```mermaid -flowchart TD - Query["GraphQL Query"] --> Args["ConnectionArgs"] - Args --> Pagination["CursorPaginationCriteria"] - Pagination --> ServiceQuery["Service.query(...)"] - ServiceQuery --> Result["QueryResult"] - Result --> Mapper["GraphQL Mapper"] - Mapper --> Connection["Relay Connection"] -``` - -This ensures consistent pagination behavior across: - -- Devices -- Events -- Organizations -- Logs -- Knowledge base -- Assignments -- Notifications - ---- - -### 3. DataLoader-Based N+1 Prevention - -Nested fields are resolved asynchronously using DGS DataLoaders. - -Example relationships: - -- Machine β†’ Tags -- Machine β†’ InstalledAgents -- Machine β†’ ToolConnections -- Machine β†’ Organization -- KnowledgeBaseItem β†’ Tags -- KnowledgeBaseItem β†’ Attachments -- KnowledgeBaseItem β†’ Author -- Assignment β†’ Target entity - -```mermaid -flowchart LR - Query["devices query"] --> DeviceFetcher - DeviceFetcher --> DeviceService - DeviceFetcher --> MachineNode["Machine"] - MachineNode -->|"tags"| TagLoader - MachineNode -->|"installedAgents"| AgentLoader - TagLoader --> TagService - AgentLoader --> InstalledAgentService -``` - -All DataLoader calls return `CompletableFuture` to ensure batched execution. - ---- - -## Data Fetcher Components - -### AssignmentDataFetcher - -Handles: - -- Assignment queries -- Assign/unassign mutations -- Assignment counts by target type -- Target resolution using DataLoaders - -Key patterns: - -- Uses `AssignmentService` -- Maps results via `GraphQLAssignmentMapper` -- Resolves polymorphic assignment targets (Organization, Machine, Ticket, KnowledgeBaseItem) - ---- - -### DeviceDataFetcher - -Responsible for: - -- Device listing with filtering and pagination -- Device filters -- Device lookup by ID -- Resolving nested relations (tags, agents, organization) - -Key dependencies: - -- `DeviceService` -- `DeviceFilterService` -- `TagService` -- `GraphQLDeviceMapper` - -Implements full Relay connection pattern. - ---- - -### EventDataFetcher - -Provides: - -- Event listing -- Event lookup -- Event creation and update -- Event filter retrieval - -Features: - -- Converts `EventFilterInput` into `EventFilterCriteria` -- Uses `GraphQLEventMapper` -- Returns `GenericConnection` - ---- - -### KnowledgeBaseDataFetcher - -One of the most feature-rich fetchers. - -Supports: - -- Folder and article queries -- Publishing/unpublishing -- Archiving/unarchiving -- Folder tree retrieval -- Tag management -- Attachment uploads (temp + permanent) -- Attachment linking - -Security-aware behavior: - -- Extracts user ID from JWT -- Uses `AuthPrincipal` -- Applies mutation wrapper pattern for success/error payloads - -```mermaid -flowchart TD - Mutation["createArticle"] --> GetUser["Extract JWT User"] - GetUser --> Mapper - Mapper --> Service["KnowledgeBaseService"] - Service --> DB["Mongo Repository"] -``` - -Also resolves nested fields: - -- Tags -- Attachments -- Author -- Parent ID - ---- - -### LogDataFetcher - -Conditionally enabled via: - -```text -spring.data.cassandra.enabled=true -``` - -Responsibilities: - -- Log listing -- Log filters -- Log detail retrieval - -Uses: - -- `LogService` -- `GraphQLLogMapper` - -Provides cursor-based pagination for log events. - ---- - -### NodeDataFetcher - -Implements Relay `node(id: ID!)` and `nodes(ids: [ID!])`. - -```mermaid -flowchart TD - NodeQuery["node(id)"] --> Decode["Relay.fromGlobalId"] - Decode --> TypeSwitch["NodeType enum"] - TypeSwitch --> DeviceService - TypeSwitch --> OrganizationService - TypeSwitch --> EventService - TypeSwitch --> ToolService - TypeSwitch --> TagService -``` - -Supports types such as: - -- Machine -- Organization -- Event -- IntegratedTool -- Tag -- ToolConnection -- InstalledAgent -- Tenant - -This enables universal entity lookup via a single GraphQL entry point. - ---- - -### NotificationDataFetcher - -Handles authenticated notification access. - -Security rules: - -- Requires `ADMIN` or `AGENT` authority -- Determines recipient type (USER or MACHINE) -- Extracts identity from JWT - -Operations: - -- List notifications -- Mark as read -- Mark all as read -- Delete -- Delete all read -- Unread counts by category - -```mermaid -flowchart TD - Query["notifications"] --> CurrentRecipient - CurrentRecipient --> Principal["AuthPrincipal"] - Principal --> NotificationService - NotificationService --> ReadStateService -``` - -Also includes strict ID decoding validation for notification IDs. - ---- - -### OrganizationDataFetcher - -Provides: - -- Organization listing (paginated) -- Organization lookup -- Filter options - -Uses: - -- `OrganizationService` -- `OrganizationQueryService` -- `GraphQLOrganizationMapper` - ---- - -### TagDataFetcher - -Supports: - -- Tag listing -- Key/value suggestions -- Tag creation -- Tag updates -- Tag deletion - -Converts Relay ID for mutation operations. - ---- - -### ToolsDataFetcher - -Provides: - -- Integrated tool listing -- Tool filters - -Uses: - -- `ToolService` -- `GraphQLToolMapper` - ---- - -### NotificationContextGraphQlTypeResolver - -Resolves polymorphic GraphQL types for `NotificationContext`. - -```mermaid -flowchart TD - NotificationContext --> Resolver - Resolver -->|"type discriminator"| TypeMap - TypeMap --> GraphQLType -``` - -- Maps discriminator string β†’ GraphQL type name -- Falls back to `GenericContext` if not found -- Enables GraphQL union/interface resolution - ---- - -## Security Integration - -The module integrates tightly with JWT-based security. - -Key aspects: - -- Uses `SecurityContextHolder` -- Extracts `AuthPrincipal` from JWT -- Determines `ActorType` (USER or AGENT) -- Applies `@PreAuthorize` annotations - -Security is enforced at: - -- Query level (Notification access) -- Mutation level -- User-aware operations (Knowledge base, notifications) - ---- - -## Interaction with Other Layers - -```mermaid -flowchart LR - GraphQL["GraphQL DataFetchers"] --> Services - Services --> Domain["Domain Models"] - Services --> Repositories - Services --> Messaging["NATS / Kafka"] - Services --> Cache["Redis"] -``` - -The Api Service Core Graphql Datafetchers module: - -- Does not access repositories directly -- Does not contain domain logic -- Acts as translation + orchestration layer - ---- - -## Key Architectural Benefits - -1. Clear separation between API and business logic -2. Uniform Relay support across all entities -3. Consistent pagination model -4. Batch loading via DataLoaders -5. Centralized Node resolution -6. Strong JWT-based identity handling -7. Polymorphic type resolution for flexible schemas - ---- - -## Summary - -The **Api Service Core Graphql Datafetchers** module is the central GraphQL execution layer of the OpenFrame platform. - -It: - -- Bridges GraphQL schema to service layer -- Implements Relay global ID and connection patterns -- Uses DataLoaders to prevent N+1 issues -- Enforces JWT-based security -- Provides consistent pagination and filtering - -By keeping business logic in services and focusing purely on API orchestration, this module ensures a clean, scalable, and maintainable GraphQL architecture for the OpenFrame ecosystem. diff --git a/docs/reference/architecture/api-service-core-graphql-dataloaders/api-service-core-graphql-dataloaders.md b/docs/reference/architecture/api-service-core-graphql-dataloaders/api-service-core-graphql-dataloaders.md new file mode 100644 index 000000000..7182ee641 --- /dev/null +++ b/docs/reference/architecture/api-service-core-graphql-dataloaders/api-service-core-graphql-dataloaders.md @@ -0,0 +1,344 @@ +# Api Service Core Graphql Dataloaders + +## Overview + +The **Api Service Core Graphql Dataloaders** module provides batched and cached data access for the GraphQL layer using the Netflix DGS `@DgsDataLoader` abstraction and the `org.dataloader.BatchLoader` contract. + +Its primary responsibility is to eliminate the **N+1 query problem** in GraphQL field resolution by: + +- Batching multiple entity lookups into a single repository or service call +- Preserving request-level ordering +- Returning results aligned with input keys +- Delegating domain logic to services and repositories + +This module acts as a performance optimization layer between: + +- The **GraphQL DataFetchers** (api-service-core-graphql-layer) +- The **Business Services & Repositories** (api-service-core-business-services, data-access-mongo-sync) + +It does not contain business logic. Instead, it orchestrates efficient bulk loading. + +--- + +## Architectural Context + +Within the overall OpenFrame backend architecture, the Api Service Core Graphql Dataloaders module sits between GraphQL field resolvers and domain/data layers. + +```mermaid +flowchart TD + Client["GraphQL Client"] --> GraphQL["GraphQL DataFetchers"] + GraphQL --> DataLoaders["Api Service Core Graphql Dataloaders"] + DataLoaders --> Services["Business Services"] + DataLoaders --> Repositories["Mongo Repositories"] + Services --> Repositories + Repositories --> MongoDB[("MongoDB")] +``` + +### Key Characteristics + +- **Request-scoped batching** via DGS DataLoader registry +- **Asynchronous resolution** using `CompletableFuture` +- **Order preservation** to match GraphQL execution semantics +- **Soft-delete awareness** (e.g., Organization status filtering) +- **Multi-entity polymorphic support** (used by AssignableTarget resolution) + +--- + +## The N+1 Problem in GraphQL + +Without DataLoaders: + +```text +Query 100 machines + └─ For each machine β†’ load organization + └─ 100 additional database queries +``` + +With DataLoaders: + +```text +Query 100 machines + └─ Collect 100 organizationIds + └─ Single batched query: findByOrganizationIdIn(...) +``` + +This dramatically reduces database load and improves latency. + +--- + +## Core DataLoaders + +Each DataLoader implements: + +- `BatchLoader` +- `CompletionStage>` or `CompletionStage>>` +- Asynchronous batching via `CompletableFuture.supplyAsync` + +Below is a breakdown of each loader and its responsibility. + +--- + +### InstalledAgentDataLoader + +**Purpose:** Batch loads installed agents for multiple machines. + +- Key: `machineId` +- Value: `List` +- Delegates to: `InstalledAgentService` + +```mermaid +flowchart LR + GraphQL["Machine.installedAgents"] --> Loader["InstalledAgentDataLoader"] + Loader --> Service["InstalledAgentService"] + Service --> Repo["InstalledAgent Repository"] +``` + +--- + +### KnowledgeBaseItemDataLoader + +**Purpose:** Batch loads KnowledgeBaseItem by ID. + +- Key: `itemId` +- Value: `KnowledgeBaseItem` +- Delegates to: `KnowledgeBaseItemRepository` +- Used by: AssignableTarget resolution (KNOWLEDGE_ARTICLE) + +**Important Behavior:** + +- Filters null IDs +- De-duplicates IDs +- Preserves input order +- Returns null for missing entries + +--- + +### KnowledgeBaseAttachmentDataLoader + +**Purpose:** Batch loads attachments for Knowledge Base articles. + +- Key: `itemId` +- Value: `List` +- Delegates to: `KnowledgeBaseAttachmentService` + +Includes debug logging for batch size observability. + +--- + +### KnowledgeBaseTagDataLoader + +**Purpose:** Batch loads tags for Knowledge Base items. + +- Key: `itemId` +- Value: `List` +- Delegates to: `KnowledgeBaseTagService` + +--- + +### MachineDataLoader + +**Purpose:** Batch loads Machine entities by machineId. + +- Key: `machineId` +- Value: `Machine` +- Delegates to: `MachineRepository` +- Used by: AssignableTarget resolution (DEVICE) + +Implements: + +- Null filtering +- ID de-duplication +- Map-based reordering + +--- + +### OrganizationDataLoader + +**Purpose:** Batch loads organizations while excluding soft-deleted entries. + +- Key: `organizationId` +- Value: `Organization` +- Delegates to: `OrganizationRepository` + +**Additional Logic:** + +- Filters organizations where `OrganizationStatus != ACTIVE` +- Prevents N+1 lookups when resolving machines β†’ organizations + +```mermaid +flowchart TD + Machines["Machine List"] --> OrgField["Machine.organization"] + OrgField --> OrgLoader["OrganizationDataLoader"] + OrgLoader --> Repo["OrganizationRepository.findByOrganizationIdIn"] + Repo --> Filter["Filter ACTIVE Only"] +``` + +--- + +### TagDataLoader + +**Purpose:** Batch loads tags assigned to machines. + +- Key: `machineId` +- Value: `List` +- Delegates to: `TagService` + +--- + +### TicketDataLoader + +**Purpose:** Batch loads Ticket entities by ID. + +- Key: `ticketId` +- Value: `Ticket` +- Delegates to: `TicketRepository` +- Used by: AssignableTarget resolution (TICKET) + +Follows standard: + +- Null filtering +- De-duplication +- Order preservation + +--- + +### ToolConnectionDataLoader + +**Purpose:** Batch loads tool connections for machines. + +- Key: `machineId` +- Value: `List` +- Delegates to: `ToolConnectionService` + +Used for resolving machine β†’ tool integration relationships. + +--- + +### UserDataLoader + +**Purpose:** Batch loads enriched user responses by ID. + +- Key: `userId` +- Value: `UserResponse` +- Delegates to: `UserService` + +**Notable Behavior:** + +- Uses `UserService` instead of direct repository access +- Ensures `UserProcessor` logic runs (e.g., SaaS image enrichment) +- Used by `KnowledgeBaseItem.author` resolver + +```mermaid +flowchart LR + KBItem["KnowledgeBaseItem.author"] --> UserLoader["UserDataLoader"] + UserLoader --> UserService["UserService"] + UserService --> Processor["UserProcessor Enrichment"] + Processor --> Repository["User Repository"] +``` + +--- + +## Batch Loading Pattern + +All loaders follow a consistent pattern: + +```mermaid +flowchart TD + A["Collect Keys"] --> B["Filter Null Values"] + B --> C["Deduplicate IDs"] + C --> D["Bulk Repository/Service Call"] + D --> E["Map Results by ID"] + E --> F["Rebuild Output List in Input Order"] + F --> G["Return CompletableFuture"] +``` + +### Why Order Preservation Matters + +GraphQL requires that results align with input keys. Therefore: + +- Input list index `i` must match output index `i` +- Missing entities must return `null` at the correct index + +Failure to preserve order leads to incorrect field resolution. + +--- + +## Async Execution Model + +Each DataLoader uses: + +```java +CompletableFuture.supplyAsync(() -> serviceCall()) +``` + +This ensures: + +- Non-blocking GraphQL execution +- Parallel resolution of independent fields +- Efficient thread utilization + +The DGS framework handles: + +- Per-request DataLoader registry +- Caching within a request +- Dispatch timing during execution + +--- + +## Polymorphic Resolution Support + +Several DataLoaders are used by the AssignableTarget polymorphic resolver in the GraphQL layer: + +| Target Type | DataLoader | +|-------------|------------| +| DEVICE | MachineDataLoader | +| TICKET | TicketDataLoader | +| KNOWLEDGE_ARTICLE | KnowledgeBaseItemDataLoader | + +This enables dynamic type resolution without triggering N+1 repository calls. + +--- + +## Performance Considerations + +### Benefits + +- Reduced database round trips +- Improved GraphQL response time +- Request-scoped caching +- Better throughput under load + +### Potential Risks + +- Large batch sizes may increase memory pressure +- Async thread pool saturation if misconfigured +- Over-fetching if DataLoader usage is not scoped properly + +--- + +## Design Principles + +The Api Service Core Graphql Dataloaders module follows these principles: + +1. **No Business Logic** – Delegates to services/repositories +2. **Batch First** – Always prefer `findByIdIn` style queries +3. **Null Safe** – Defensive handling of null IDs +4. **Order Deterministic** – Output aligns with input +5. **Service-Aware** – Use service layer when enrichment is required + +--- + +## Summary + +The **Api Service Core Graphql Dataloaders** module is a critical performance optimization layer for the GraphQL API. + +It ensures that: + +- Complex nested GraphQL queries remain efficient +- Database access is minimized +- Polymorphic type resolution scales +- Business logic remains centralized in services + +Without this module, the GraphQL layer would suffer from severe N+1 query amplification under realistic workloads. + +It is a foundational component enabling scalable GraphQL operations in the OpenFrame backend ecosystem. \ No newline at end of file diff --git a/docs/reference/architecture/api-service-core-graphql-dtos/api-service-core-graphql-dtos.md b/docs/reference/architecture/api-service-core-graphql-dtos/api-service-core-graphql-dtos.md deleted file mode 100644 index f118770b9..000000000 --- a/docs/reference/architecture/api-service-core-graphql-dtos/api-service-core-graphql-dtos.md +++ /dev/null @@ -1,434 +0,0 @@ -# Api Service Core Graphql Dtos - -The **Api Service Core Graphql Dtos** module defines the GraphQL-facing Data Transfer Objects (DTOs) used by the Api Service Core. These DTOs represent: - -- GraphQL connection and edge wrappers -- Query filter input types -- Mutation input types -- Structured response models -- Lightweight aggregation models - -This module acts as the **schema contract layer** between GraphQL data fetchers and the underlying domain, service, and persistence layers. - -It does not contain business logic. Instead, it formalizes how data flows into and out of the GraphQL API. - ---- - -## Architectural Role - -Within the overall Api Service Core architecture, this module sits between: - -- GraphQL Data Fetchers (execution layer) -- Domain DTOs and persistence documents (data layer) -- Services and processors (business layer) - -### High-Level Placement - -```mermaid -flowchart TD - Client["GraphQL Client"] --> Schema["GraphQL Schema"] - Schema --> DataFetcher["GraphQL Data Fetchers"] - DataFetcher --> Dtos["Api Service Core Graphql Dtos"] - Dtos --> Services["Core Services"] - Services --> Repositories["Mongo Repositories"] - Repositories --> Database[("Database")] -``` - -The **Api Service Core Graphql Dtos** module defines the types used in: - -- Query arguments -- Mutation inputs -- Relay-style connections -- Filter inputs -- Structured API responses - ---- - -# Module Structure - -The module can be logically grouped into the following categories: - -1. Relay Connection Models -2. Filter Input DTOs -3. Mutation Input DTOs -4. Domain Response DTOs -5. Aggregation and Assignment DTOs - ---- - -# 1. Relay Connection Models - -GraphQL in Api Service Core follows the Relay-style pagination model. This module provides reusable generic wrappers. - -## GenericEdge - -```java -public class GenericEdge { - private T node; - private String cursor; -} -``` - -Responsibilities: - -- Wraps a single node -- Provides a cursor for pagination -- Forms part of a GraphQL Connection response - -## CountedGenericConnection - -```java -public class CountedGenericConnection - extends GenericConnection { - private int filteredCount; -} -``` - -Responsibilities: - -- Extends the base connection model -- Adds `filteredCount` for total result size -- Supports filtered pagination use cases - -### Relay Pagination Flow - -```mermaid -flowchart LR - Query["GraphQL Query"] --> Args["Connection Arguments"] - Args --> Fetcher["Data Fetcher"] - Fetcher --> Service["Service Layer"] - Service --> Edge["GenericEdge"] - Edge --> Connection["CountedGenericConnection"] - Connection --> Client["GraphQL Client"] -``` - -This ensures: - -- Stable cursor-based pagination -- Support for filtering -- Predictable total counts for UI rendering - ---- - -# 2. Filter Input DTOs - -These classes represent GraphQL input types for querying and filtering domain entities. - -They are optimized for client-side flexibility and map to domain-level filter criteria. - -## LogFilterInput - -Supports filtering by: - -- Date range (`startDate`, `endDate`) -- Event types -- Tool types -- Severities -- Organization IDs -- Device ID - -Used in log and audit queries. - ---- - -## DeviceFilterInput - -Supports filtering by: - -- Device status -- Device type -- Operating system types -- Organization IDs -- Tag keys and values - -Enables complex device search and inventory queries. - ---- - -## EventFilterInput - -Supports filtering by: - -- User IDs -- Event types -- Date range - -Used by event-related GraphQL queries. - ---- - -## KnowledgeBaseFilterInput - -Supports filtering by: - -- Parent folder -- Item type -- Tag IDs - -Used to navigate and filter knowledge base structures. - ---- - -## OrganizationFilterInput - -Supports filtering by: - -- Category -- Employee range -- Contract status -- Organization status - ---- - -## ToolFilterInput - -Supports filtering by: - -- Enabled state -- Tool type -- Category -- Platform category - ---- - -## NotificationFilterInput - -Supports filtering by: - -- Read/unread status - ---- - -### Filter Mapping Flow - -```mermaid -flowchart TD - GraphQLInput["GraphQL Filter Input"] --> DataFetcher["Data Fetcher"] - DataFetcher --> Criteria["Domain Filter Criteria"] - Criteria --> Repository["Custom Repository"] - Repository --> Results["Domain Documents"] - Results --> Connection["Connection DTO"] -``` - -Filter inputs are intentionally: - -- Client-oriented -- Flexible -- Decoupled from persistence implementation - ---- - -# 3. Mutation Input DTOs - -These DTOs define structured input contracts for GraphQL mutations. - -They frequently include validation annotations such as: - -- `@NotBlank` -- `@NotNull` -- `@Size` - -## Event Mutations - -### CreateEventInput - -Fields: - -- `userId` (required) -- `type` (required) -- `data` - -Used for creating audit or system events. - ---- - -## Knowledge Base Mutations - -### CreateArticleInput - -Supports: - -- Article metadata -- Status -- Tag assignments -- Cross-entity assignments - -### UpdateArticleInput - -Supports partial updates of: - -- Name -- Parent -- Content -- Summary - -### DeleteFolderInput - -Supports: - -- Folder deletion -- Child handling strategy -- Optional move target - -### CreateKnowledgeBaseAttachmentInput - -Defines: - -- Target article -- File metadata - -### CreateKnowledgeBaseTempAttachmentInput - -Defines temporary upload metadata before final linking. - -### LinkKnowledgeBaseTempAttachmentsInput - -Supports bulk linking of temporary attachments to an article. - ---- - -## User Mutations - -### UpdateUserRequest - -Supports updating: - -- First name -- Last name - -Includes size validation constraints. - ---- - -### Mutation Execution Flow - -```mermaid -flowchart TD - Client["GraphQL Mutation"] --> Input["Mutation Input DTO"] - Input --> Validation["Bean Validation"] - Validation --> Processor["Service or Processor"] - Processor --> Repository["Mongo Repository"] - Repository --> Response["Response DTO"] -``` - -The DTO layer guarantees: - -- Schema consistency -- Validation enforcement -- Clear mutation contracts - ---- - -# 4. Domain Response DTOs - -These classes represent structured GraphQL output types. - -## UserResponse - -Encapsulates: - -- Identity fields -- Roles -- Status -- Profile image -- Audit timestamps - -Designed to: - -- Hide internal domain representation -- Expose only client-relevant fields -- Maintain immutability via builder pattern - ---- - -# 5. Aggregation and Assignment DTOs - -## AssignedItemCount - -```java -public class AssignedItemCount { - private AssignmentTargetType targetType; - private int count; -} -``` - -Purpose: - -- Represents assignment counts grouped by target type -- Used in dashboards and summary queries -- Bridges domain enum types into GraphQL responses - ---- - -# Design Principles - -The **Api Service Core Graphql Dtos** module follows several architectural principles: - -## 1. Clear Separation of Concerns - -- DTOs contain no business logic -- Services implement logic -- Repositories handle persistence - -## 2. GraphQL-Optimized Models - -- Designed for GraphQL schema shape -- Optimized for client flexibility -- Uses input-specific classes instead of reusing domain objects - -## 3. Validation at the Boundary - -Validation annotations ensure: - -- Early error detection -- Strong input guarantees -- Reduced service-layer complexity - -## 4. Relay Compliance - -- Edge-based pagination -- Connection wrappers -- Cursor-based navigation - ---- - -# End-to-End Data Flow Example - -```mermaid -flowchart TD - UI["Frontend UI"] --> Query["GraphQL Query or Mutation"] - Query --> DTOIn["Input DTO"] - DTOIn --> Fetcher["Data Fetcher"] - Fetcher --> Service["Core Service"] - Service --> Repo["Mongo Repository"] - Repo --> Domain["Domain Document"] - Domain --> DTOOut["Edge or Response DTO"] - DTOOut --> Connection["CountedGenericConnection"] - Connection --> UI -``` - -This module ensures the GraphQL API remains: - -- Strongly typed -- Stable -- Decoupled from internal persistence structures -- Easy to evolve without breaking clients - ---- - -# Summary - -The **Api Service Core Graphql Dtos** module defines the GraphQL contract layer of the Api Service Core. It provides: - -- Relay-compatible connection models -- Rich filter input types -- Structured mutation inputs -- Clean response DTOs -- Aggregation models for summary data - -By isolating API-facing structures in a dedicated module, the system maintains: - -- Strong separation between API and domain -- High schema clarity -- Safer refactoring of internal models -- Predictable GraphQL behavior diff --git a/docs/reference/architecture/api-service-core-graphql-layer/api-service-core-graphql-layer.md b/docs/reference/architecture/api-service-core-graphql-layer/api-service-core-graphql-layer.md new file mode 100644 index 000000000..b3725b385 --- /dev/null +++ b/docs/reference/architecture/api-service-core-graphql-layer/api-service-core-graphql-layer.md @@ -0,0 +1,433 @@ +# Api Service Core Graphql Layer + +## Overview + +The **Api Service Core Graphql Layer** module is the primary GraphQL entry point for the OpenFrame platform. It exposes a unified, Relay-compliant API over core domain entities such as devices, organizations, events, knowledge base items, scripts, notifications, assignments, and integrated tools. + +Built on the Netflix DGS framework, this layer: + +- Translates GraphQL queries and mutations into domain service calls +- Implements Relay-style global node resolution and cursor-based pagination +- Coordinates DataLoader-based batching to prevent N+1 query problems +- Applies authentication and authorization at the GraphQL boundary +- Maps DTOs and domain models into GraphQL types + +It sits between frontend clients (e.g., OpenFrame UI) and the underlying business services, repositories, and messaging infrastructure. + +--- + +## Architectural Context + +The Api Service Core Graphql Layer interacts with multiple adjacent modules: + +- **API Service Core Business Services** – encapsulate domain logic +- **API Service Core GraphQL DataLoaders** – batch and cache related entity loading +- **API Service Core DTOs & DTO Contracts** – define API-facing data structures +- **Data Model & Repositories (Mongo)** – persistence layer +- **Security (OAuth & JWT)** – authentication and principal resolution +- **Eventing & Messaging (Kafka / NATS)** – async notifications and command dispatch + +### High-Level Architecture + +```mermaid +flowchart TD + Client["Frontend / API Client"] -->|"GraphQL HTTP"| DGS["DGS GraphQL Engine"] + + subgraph graphql_layer["Api Service Core Graphql Layer"] + DF["DataFetchers"] + TR["Type Resolvers"] + NodeQ["Node & Relay Support"] + end + + subgraph dataloaders["GraphQL DataLoaders"] + DL1["Entity Batch Loaders"] + end + + subgraph services["Business Services"] + S1["Domain Services"] + end + + subgraph persistence["Mongo Repositories"] + DB[("MongoDB")] + end + + Client --> DGS + DGS --> DF + DF --> S1 + DF --> DL1 + DL1 --> S1 + S1 --> DB + + DF --> TR + DF --> NodeQ +``` + +--- + +## Core Design Patterns + +### 1. Netflix DGS-Based Resolvers + +Each domain area is implemented as a `@DgsComponent` containing: + +- `@DgsQuery` methods – GraphQL queries +- `@DgsMutation` methods – GraphQL mutations +- `@DgsData` methods – field resolvers for nested types +- `@DgsTypeResolver` – runtime type resolution for interfaces/unions + +This enables strict separation between: + +- Schema definitions (GraphQL SDL) +- Resolver logic (DataFetchers) +- Domain services + +--- + +### 2. Relay Global ID Pattern + +The module uses `graphql.relay.Relay` for global node handling. + +Key characteristics: + +- All entities expose an `id` field encoded as a global ID +- Incoming IDs are decoded using `RELAY.fromGlobalId()` +- Outgoing IDs are encoded using `RELAY.toGlobalId()` +- A unified `node(id: ID!)` query supports polymorphic fetching + +### Relay Flow + +```mermaid +flowchart LR + GID["Global ID"] --> Decode["Relay.fromGlobalId"] + Decode --> Resolve["NodeDataFetcher.resolveNode"] + Resolve --> Service["Domain Service"] + Service --> Entity["Domain Entity"] + Entity --> Encode["Relay.toGlobalId"] +``` + +--- + +### 3. Cursor-Based Pagination + +Most list queries follow a consistent pattern: + +- Accept `first`, `after`, `last`, `before` +- Convert to `ConnectionArgs` +- Map to `CursorPaginationCriteria` +- Return `GenericConnection` or `CountedGenericConnection` + +This ensures: + +- Stable pagination +- Forward/backward navigation +- Total count availability (when needed) + +--- + +### 4. DataLoader for N+1 Prevention + +Nested entity resolution (e.g., machine β†’ tags, article β†’ author) is performed using named DataLoaders. + +Example usage pattern: + +- Fetch parent entities via service +- Resolve related entities via `dfe.getDataLoader("...")` +- Batch load related IDs in a single repository call + +This dramatically reduces database round-trips in complex queries. + +--- + +## Module Responsibilities by Domain + +Below is a breakdown of major DataFetcher components and their responsibilities. + +--- + +### AssignmentDataFetcher + +Handles assignment relationships between items and targets. + +Responsibilities: + +- Query assignment counts by target type +- Paginated retrieval of assigned items +- Assign / unassign items +- Resolve `AssignableTarget` polymorphically + +Supports assignment between: + +- Organizations +- Devices (Machines) +- Tickets +- Knowledge Base Articles + +--- + +### CommandDataFetcher + +Exposes RMM ad-hoc command dispatch operations. + +Responsibilities: + +- `runCommand` mutation +- `cancelExecution` mutation + +Delegates to `CommandDispatchService`, which integrates with messaging infrastructure. + +--- + +### DeviceDataFetcher + +Provides device-centric GraphQL queries and nested resolvers. + +Responsibilities: + +- Device search and filtering +- Cursor-based device listing +- Device lookup by ID +- Nested resolution of: + - Tags + - ToolConnections + - InstalledAgents + - Organization + +Relies heavily on DataLoaders for efficient nested resolution. + +--- + +### EventDataFetcher + +Manages event lifecycle and querying. + +Responsibilities: + +- Paginated event listing +- Event filtering +- Event creation and update +- Relay-compliant ID resolution + +Events are stored in Mongo and may be enriched via stream-processing pipelines. + +--- + +### KnowledgeBaseDataFetcher + +Handles knowledge base content management. + +Responsibilities: + +- Folder & article tree queries +- Article CRUD lifecycle (create, update, publish, archive) +- Tag assignment +- Attachment upload URL generation +- Temp attachment lifecycle management +- Author resolution via DataLoader + +Includes security-aware mutation execution using the authenticated principal. + +--- + +### LogDataFetcher + +Conditionally enabled audit log resolver. + +Responsibilities: + +- Log filtering +- Cursor-based log listing +- Detailed log lookup + +Activated only when Cassandra-backed logging is enabled. + +--- + +### NotificationDataFetcher + +Provides user and agent notification querying and mutation operations. + +Responsibilities: + +- List notifications +- Mark as read / delete +- Unread counts by category +- Principal-aware recipient resolution + +Supports multiple actor types: + +- USER +- MACHINE (agent) + +Applies `@PreAuthorize` guards at the GraphQL boundary. + +--- + +### OrganizationDataFetcher + +Exposes organization search and lookup operations. + +Responsibilities: + +- Paginated organization listing +- Organization filtering +- Lookup by global ID or raw organizationId + +--- + +### ScriptDataFetcher + +Provides RMM script CRUD operations. + +Responsibilities: + +- Script lookup +- Script search and filtering +- Create / update / delete + +Tenant scoping is handled inside `ScriptService`. + +--- + +### TagDataFetcher + +Manages tag CRUD and suggestions. + +Responsibilities: + +- Tag listing +- Autocomplete for tag keys and values +- Tag creation, update, deletion + +--- + +### ToolsDataFetcher + +Handles integrated tool discovery and filtering. + +Responsibilities: + +- Integrated tool listing +- Tool filtering options + +--- + +### NodeDataFetcher + +Implements the Relay `node(id: ID!)` and `nodes(ids: [ID!]!)` queries. + +Responsibilities: + +- Decode global ID +- Map type name to `NodeType` +- Delegate to correct domain service + +Supports polymorphic node resolution for: + +- Machine +- Organization +- Event +- IntegratedTool +- Tenant +- ItemAssignment +- Ticket +- KnowledgeBaseItem +- User + +--- + +## Type Resolvers + +### NodeTypeResolver + +Resolves GraphQL `Node` interface to concrete types at runtime. + +### AssignableTargetTypeResolver + +Resolves `AssignableTarget` union/interface for assignment targets. + +### NotificationContextGraphQlTypeResolver + +Maps `NotificationContext` discriminator values to concrete GraphQL types dynamically. + +--- + +## End-to-End Query Flow Example + +```mermaid +sequenceDiagram + participant Client + participant GraphQL + participant DataFetcher + participant Service + participant Repository + + Client->>GraphQL: Query devices(first: 20) + GraphQL->>DataFetcher: devices() + DataFetcher->>Service: queryDevices() + Service->>Repository: findByFilter() + Repository-->>Service: Device list + Service-->>DataFetcher: QueryResult + DataFetcher-->>GraphQL: Connection + GraphQL-->>Client: JSON response +``` + +--- + +## Security Model + +The Api Service Core Graphql Layer integrates with JWT-based authentication. + +Security characteristics: + +- Uses Spring Security context +- Extracts `AuthPrincipal` from JWT +- Applies `@PreAuthorize` on sensitive operations +- Supports USER and AGENT actor types +- Enforces type validation for global IDs + +Mutations that modify state often: + +- Resolve current user ID +- Validate actor type +- Delegate authorization checks to service layer + +--- + +## Error Handling Strategy + +The module uses: + +- Validation annotations (`@Valid`, `@NotBlank`) +- Explicit `IllegalArgumentException` for invalid IDs +- Optional returns mapped to `null` where appropriate +- Controlled mutation wrappers (e.g., `executeMutation`) for safe error payload construction + +--- + +## Why This Layer Matters + +The Api Service Core Graphql Layer provides: + +- A unified API surface across all OpenFrame domains +- Strict type safety and pagination consistency +- Relay compliance for modern frontend clients +- Efficient batching via DataLoader +- Centralized authentication enforcement +- Clear separation between API contract and domain logic + +It acts as the orchestration boundary between user-facing GraphQL clients and the platform’s rich, multi-tenant backend ecosystem. + +--- + +## Summary + +The **Api Service Core Graphql Layer** is the orchestration and translation layer of the OpenFrame platform. It: + +- Implements Relay-compliant GraphQL +- Encapsulates domain access through DataFetchers +- Prevents N+1 issues with DataLoaders +- Delegates business logic to services +- Enforces authentication and authorization +- Exposes a consistent, scalable API contract + +This module is foundational to delivering a unified, extensible, and high-performance GraphQL API across the entire OpenFrame stack. diff --git a/docs/reference/architecture/api-service-core-relay-type-resolution/api-service-core-relay-type-resolution.md b/docs/reference/architecture/api-service-core-relay-type-resolution/api-service-core-relay-type-resolution.md deleted file mode 100644 index 5e9693efc..000000000 --- a/docs/reference/architecture/api-service-core-relay-type-resolution/api-service-core-relay-type-resolution.md +++ /dev/null @@ -1,276 +0,0 @@ -# Api Service Core Relay Type Resolution - -## Overview - -The **Api Service Core Relay Type Resolution** module is responsible for resolving polymorphic GraphQL types in the Api Service Core layer. It bridges: - -- Mongo domain model objects (e.g., `Machine`, `Organization`, `Ticket`) -- GraphQL interfaces and unions (e.g., `Node`, `AssignableTarget`) -- The Netflix DGS runtime type resolution mechanism - -In a Relay-compliant GraphQL architecture, interfaces such as `Node` and domain-specific unions such as `AssignableTarget` require runtime type resolution. This module provides that resolution logic via DGS `@DgsTypeResolver` components. - ---- - -## Why This Module Exists - -GraphQL interfaces and unions require a mechanism to determine the concrete GraphQL type for a returned object at runtime. - -For example: - -- A `Node` query may return a `Machine`, `User`, `Ticket`, or `Organization`. -- An `AssignableTarget` may represent an `Organization`, `Machine`, `Ticket`, or `KnowledgeBaseItem`. - -The GraphQL engine must map Java domain objects to their corresponding GraphQL schema type names. - -The **Api Service Core Relay Type Resolution** module provides that mapping in a centralized and explicit way. - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - Client["GraphQL Client"] --> Query["GraphQL Query"] - Query --> DGS["DGS Runtime"] - DGS --> Resolver["Type Resolver"] - Resolver --> DomainObj["Domain Object"] - DomainObj --> GraphQLType["GraphQL Type Name"] - - subgraph relay_layer["Relay Type Resolution Layer"] - Resolver - end - - subgraph domain_layer["Mongo Domain Model"] - DomainObj - end -``` - -### Flow Summary - -1. A GraphQL query resolves a field returning an interface or union. -2. DGS invokes the registered `@DgsTypeResolver`. -3. The resolver inspects the Java object using `instanceof` checks. -4. It returns the GraphQL type name as a `String`. -5. The runtime binds the object to the correct schema type. - ---- - -## Core Components - -This module contains two DGS type resolvers: - -- `AssignableTargetTypeResolver` -- `NodeTypeResolver` - -Both are annotated with `@DgsComponent` and use `@DgsTypeResolver`. - ---- - -## AssignableTargetTypeResolver - -**Component:** -`openframe-oss-lib.openframe-api-service-core.src.main.java.com.openframe.api.relay.AssignableTargetTypeResolver.AssignableTargetTypeResolver` - -### Responsibility - -Resolves the GraphQL union/interface `AssignableTarget` into a concrete schema type. - -### Supported Domain Types - -| Java Type | GraphQL Type Returned | -|------------|-----------------------| -| `Organization` | `Organization` | -| `Machine` | `Machine` | -| `Ticket` | `Ticket` | -| `KnowledgeBaseItem` | `KnowledgeBaseItem` | - -If an unsupported object is passed, the resolver throws: - -```text -IllegalArgumentException: Unknown AssignableTarget type -``` - -### Resolution Logic - -```mermaid -flowchart TD - Input["AssignableTarget Object"] --> CheckOrg{"Organization?"} - CheckOrg -->|"Yes"| Org["Return Organization"] - CheckOrg -->|"No"| CheckMachine{"Machine?"} - CheckMachine -->|"Yes"| Machine["Return Machine"] - CheckMachine -->|"No"| CheckTicket{"Ticket?"} - CheckTicket -->|"Yes"| Ticket["Return Ticket"] - CheckTicket -->|"No"| CheckKB{"KnowledgeBaseItem?"} - CheckKB -->|"Yes"| KB["Return KnowledgeBaseItem"] - CheckKB -->|"No"| Error["Throw IllegalArgumentException"] -``` - -### Architectural Role - -This resolver enables: - -- Polymorphic assignment targets -- Unified assignment models across multiple entity types -- Strong alignment between domain model and GraphQL schema - ---- - -## NodeTypeResolver - -**Component:** -`openframe-oss-lib.openframe-api-service-core.src.main.java.com.openframe.api.relay.NodeTypeResolver.NodeTypeResolver` - -### Responsibility - -Resolves the Relay `Node` interface to a concrete GraphQL type. - -This is central to: - -- Global object identification -- Relay-style pagination -- Cross-entity querying - -### Supported Domain Types - -| Java Type | GraphQL Type Returned | -|------------|-----------------------| -| `Machine` | `Machine` | -| `Organization` | `Organization` | -| `Event` | `Event` | -| `IntegratedTool` | `IntegratedTool` | -| `Tenant` | `Tenant` | -| `ItemAssignment` | `ItemAssignment` | -| `Ticket` | `Ticket` | -| `KnowledgeBaseItem` | `KnowledgeBaseItem` | -| `User` | `User` | - -If an unsupported object is encountered, an exception is thrown: - -```text -IllegalArgumentException: Unknown Node type -``` - -### Resolution Flow - -```mermaid -flowchart TD - NodeInput["Node Object"] --> CheckMachine{"Machine?"} - CheckMachine -->|"Yes"| ReturnMachine["Return Machine"] - CheckMachine -->|"No"| CheckOrg{"Organization?"} - CheckOrg -->|"Yes"| ReturnOrg["Return Organization"] - CheckOrg -->|"No"| CheckEvent{"Event?"} - CheckEvent -->|"Yes"| ReturnEvent["Return Event"] - CheckEvent -->|"No"| Continue["Other instanceof checks"] - Continue --> ReturnType["Return Matching Type"] -``` - ---- - -## Integration with GraphQL Layer - -This module works closely with: - -- [Api Service Core GraphQL Datafetchers](../api-service-core-graphql-datafetchers/api-service-core-graphql-datafetchers.md) -- [Api Service Core GraphQL Dtos](../api-service-core-graphql-dtos/api-service-core-graphql-dtos.md) - -### Interaction Diagram - -```mermaid -flowchart LR - DataFetcher["DataFetcher"] --> Domain["Domain Object"] - Domain --> Resolver["Node or AssignableTarget Resolver"] - Resolver --> SchemaType["GraphQL Schema Type"] - SchemaType --> Response["GraphQL Response"] -``` - -1. A DataFetcher returns a domain entity. -2. The entity implements or represents a `Node` or `AssignableTarget`. -3. DGS invokes the appropriate resolver. -4. The resolver returns the GraphQL type name. -5. The object is serialized correctly into the GraphQL response. - ---- - -## Relationship to the Domain Model - -This module depends directly on the Mongo domain model defined in: - -- Device entities (`Machine`) -- Organization entities (`Organization`, `Tenant`) -- User entities (`User`) -- Event entities (`Event`) -- Ticket entities (`Ticket`) -- Knowledge base entities (`KnowledgeBaseItem`) -- Tool entities (`IntegratedTool`) -- Assignment entities (`ItemAssignment`) - -These classes originate from the data layer and are not GraphQL-specific. - -The resolvers form a clean boundary between: - -```mermaid -flowchart TB - DomainModel["Mongo Domain Model"] --> RelayResolver["Relay Type Resolver"] - RelayResolver --> GraphQLSchema["GraphQL Schema Types"] -``` - -This keeps domain objects independent from GraphQL schema annotations. - ---- - -## Design Characteristics - -### 1. Explicit Type Mapping - -- No reflection-based resolution -- No schema name inference -- Explicit `instanceof` checks - -This ensures: - -- Predictable behavior -- Easy debugging -- Compile-time safety for supported types - -### 2. Fail-Fast Strategy - -Unknown types result in immediate `IllegalArgumentException`. - -Benefits: - -- Prevents silent schema mismatches -- Forces developers to update resolvers when introducing new Node types - -### 3. Relay Compliance - -By resolving the `Node` interface properly, the module enables: - -- Global ID-based queries -- Cursor-based pagination -- Cross-type collections - ---- - -## When to Update This Module - -You must update **Api Service Core Relay Type Resolution** when: - -- A new domain type implements the `Node` interface in the GraphQL schema -- A new entity becomes assignable under `AssignableTarget` -- A new Mongo document class is exposed through GraphQL - -Failure to update this module will result in runtime exceptions. - ---- - -## Summary - -The **Api Service Core Relay Type Resolution** module is a critical but focused infrastructure component that: - -- Implements GraphQL Relay runtime type resolution -- Maps Mongo domain entities to GraphQL schema types -- Enables polymorphic querying -- Enforces strict type safety and fail-fast behavior - -Although small in size, it is foundational to the correctness of the GraphQL execution layer in Api Service Core. \ No newline at end of file diff --git a/docs/reference/architecture/api-service-core-rest-controllers/api-service-core-rest-controllers.md b/docs/reference/architecture/api-service-core-rest-controllers/api-service-core-rest-controllers.md index 19b8e7b66..dfeb40c72 100644 --- a/docs/reference/architecture/api-service-core-rest-controllers/api-service-core-rest-controllers.md +++ b/docs/reference/architecture/api-service-core-rest-controllers/api-service-core-rest-controllers.md @@ -1,425 +1,430 @@ # Api Service Core Rest Controllers -The **Api Service Core Rest Controllers** module exposes the primary internal REST endpoints of the OpenFrame API Service Core. It acts as the HTTP boundary layer between clients (UI, agents, internal services) and the underlying application services, command/query services, and domain logic. +## Overview + +The **Api Service Core Rest Controllers** module exposes the primary internal REST endpoints of the OpenFrame API Service. It acts as the HTTP entry layer for administrative, operational, and internal platform interactions, delegating business logic to application services and returning structured DTO responses. This module is responsible for: -- Exposing secure REST endpoints for tenant-scoped operations -- Delegating business logic to dedicated service layers -- Translating HTTP semantics into domain/service calls -- Enforcing authentication context via `AuthPrincipal` -- Returning DTO-based responses for consistency and API stability +- User and organization management (mutations) +- API key lifecycle management +- Agent registration secret management +- Device status updates +- Forced client and tool agent operations +- SSO configuration management +- Invitation workflows +- Release version and health endpoints +- Current user identity resolution +- OpenFrame client configuration retrieval -It complements the GraphQL data fetchers and external API controllers by providing internal and operational REST endpoints. +All controllers are implemented using Spring Web MVC and follow a thin-controller pattern: validation and HTTP concerns are handled at the edge, while business logic is delegated to services in the Api Service Core Business Services and related modules. --- -## Architectural Role in the Platform - -Within the overall OpenFrame architecture, the Api Service Core Rest Controllers module sits at the edge of the API Service Core and depends on: +## Architectural Context -- Application services (command/query services) -- Domain services and processors -- Security context (`AuthPrincipal`) -- Mongo-backed persistence modules -- Tenant-aware authorization infrastructure +The Api Service Core Rest Controllers module sits between external callers (UI, internal services, gateway) and the domain/service layer. ```mermaid flowchart TD - Client["Client / UI / Agent"] --> Gateway["Gateway Service"] - Gateway --> ApiCore["API Service Core"] - - subgraph rest_layer["REST Controller Layer"] - Controllers["Api Service Core Rest Controllers"] - end - - subgraph service_layer["Service Layer"] - CommandServices["Command Services"] - QueryServices["Query Services"] - DomainProcessors["Domain Processors"] - end - - subgraph data_layer["Data Layer"] - Mongo["Mongo Repositories"] - Redis["Redis Cache"] - Kafka["Kafka / Events"] - end - - ApiCore --> Controllers - Controllers --> CommandServices - Controllers --> QueryServices - Controllers --> DomainProcessors - - CommandServices --> Mongo - QueryServices --> Mongo - DomainProcessors --> Kafka - DomainProcessors --> Redis -``` + Client["Web UI / Internal Service"] --> Gateway["Gateway Service Core"] + Gateway --> RestControllers["Api Service Core Rest Controllers"] -The controllers themselves contain minimal business logic and primarily orchestrate calls to services. + RestControllers --> BusinessServices["Api Service Core Business Services"] + RestControllers --> MappingLayer["Api Lib Mapping and Domain Services"] + RestControllers --> Dtos["Api Service Core Dtos"] ---- + BusinessServices --> Repositories["Data Model and Repositories Mongo"] + BusinessServices --> Messaging["Eventing and Messaging Kafka NATS"] + BusinessServices --> Authz["Authorization Service Core"] -## Controller Overview + Repositories --> Mongo[("MongoDB")] + Messaging --> Kafka[("Kafka")] + Messaging --> Nats[("NATS")] +``` + +### Responsibilities at This Layer -The module contains the following REST controllers: +- HTTP routing via `@RestController` +- Request validation via `@Valid` +- Authentication context resolution via `@AuthenticationPrincipal` +- HTTP status mapping +- Translation of domain models into response DTOs +- Minimal exception-to-HTTP conversion -- AgentRegistrationSecretController -- ApiKeyController -- DeviceController -- ForceAgentController -- HealthController -- InvitationController -- MeController -- OpenFrameClientConfigurationController -- OrganizationController -- ReleaseVersionController -- SSOConfigController -- UserController +The controllers do **not**: -Each controller is scoped to a specific functional domain. +- Contain business rules +- Directly interact with repositories +- Perform cross-service orchestration --- -# Endpoint Domains +## Controller Groups and Responsibilities -## 1. Agent Registration Secret +### 1. Agent Registration Secret Management +**Controller:** `AgentRegistrationSecretController` **Base Path:** `/agent/registration-secret` -Controller: `AgentRegistrationSecretController` +Handles lifecycle of agent registration secrets used during client or agent onboarding. + +Endpoints: -Responsibilities: +- `GET /agent/registration-secret/active` β†’ Get active secret +- `GET /agent/registration-secret` β†’ List all secrets +- `POST /agent/registration-secret/generate` β†’ Generate new secret (201 Created) -- Retrieve active registration secret -- List all historical secrets -- Generate new registration secret +Delegates to `AgentRegistrationSecretService` and returns `AgentRegistrationSecretResponse` DTOs. + +Typical flow: ```mermaid -sequenceDiagram - participant Admin - participant Controller as AgentRegistrationSecretController - participant Service as AgentRegistrationSecretService - - Admin->>Controller: POST /agent/registration-secret/generate - Controller->>Service: generateNewSecret() - Service-->>Controller: AgentRegistrationSecretResponse - Controller-->>Admin: 201 Created +flowchart LR + Admin["Admin Request"] --> Controller["AgentRegistrationSecretController"] + Controller --> Service["AgentRegistrationSecretService"] + Service --> Processor["DefaultAgentRegistrationSecretProcessor"] + Processor --> Repo["BaseApiKeyRepository / Tenant Repository"] ``` -This endpoint is typically used during agent provisioning and secure enrollment flows. - --- -## 2. API Key Management +### 2. API Key Management +**Controller:** `ApiKeyController` **Base Path:** `/api-keys` -Controller: `ApiKeyController` +Manages user-scoped API keys. -Key features: +Authentication is resolved via `AuthPrincipal` using `@AuthenticationPrincipal`. -- List user API keys -- Create new API key -- Update metadata -- Delete key -- Regenerate secret +Endpoints: -Authentication is derived from `AuthPrincipal`, ensuring API keys are scoped to the authenticated user. - -```mermaid -flowchart LR - User["Authenticated User"] --> Controller["ApiKeyController"] - Controller --> Service["ApiKeyService"] - Service --> Repo["BaseApiKeyRepository"] -``` +- `GET /api-keys` β†’ List keys for authenticated user +- `POST /api-keys` β†’ Create key (201 Created) +- `GET /api-keys/{keyId}` β†’ Fetch key +- `PUT /api-keys/{keyId}` β†’ Update key metadata +- `DELETE /api-keys/{keyId}` β†’ Delete key (204 No Content) +- `POST /api-keys/{keyId}/regenerate` β†’ Regenerate secret -Security Characteristics: +Key aspects: -- User-scoped access -- Regeneration rotates secret while preserving key identity -- Creation returns secret only once +- All operations are scoped to `principal.getId()` +- Service enforces ownership constraints +- Responses use `ApiKeyResponse` and `CreateApiKeyResponse` --- -## 3. Device Status Updates +### 3. Device Status Management +**Controller:** `DeviceController` **Base Path:** `/devices` -Controller: `DeviceController` +Provides internal status update capabilities for devices. -Primary responsibility: +Endpoint: -- Update device status via `PATCH /devices/{machineId}` +- `PATCH /devices/{machineId}` β†’ Update device status -This is typically invoked internally by agents or system processes to reflect device health or connectivity state. +Delegates to `DeviceService.updateStatusByMachineId`. -```mermaid -flowchart TD - Agent["Agent"] --> Controller["DeviceController"] - Controller --> Service["DeviceService"] - Service --> DeviceDoc["Device Document"] -``` +This endpoint is typically used by internal processes, stream processors, or system integrations. --- -## 4. Force Agent Operations +### 4. Forced Client and Tool Agent Operations +**Controller:** `ForceAgentController` **Base Path:** `/force` -Controller: `ForceAgentController` +Supports administrative force operations for: -Supports operational commands such as: +- Tool installation +- Tool reinstallation +- Tool updates +- Client updates +- Bulk operations across machines -- Force tool installation -- Force tool reinstallation -- Force tool update -- Force client update -- Bulk operations ("all") +Representative endpoints: + +- `POST /force/tool-agent/install` +- `POST /force/tool-agent/update` +- `POST /force/client/update` +- `POST /force/tool-agent/install/all` +- `POST /force/tool-agent/reinstall` These endpoints delegate to: -- ForceToolInstallationService -- ForceClientUpdateService -- ForceToolAgentUpdateService +- `ForceToolInstallationService` +- `ForceClientUpdateService` +- `ForceToolAgentUpdateService` + +High-level interaction: ```mermaid flowchart TD - Admin["Admin Action"] --> Controller["ForceAgentController"] - Controller --> InstallSvc["ForceToolInstallationService"] - Controller --> UpdateSvc["ForceToolAgentUpdateService"] - Controller --> ClientSvc["ForceClientUpdateService"] - - InstallSvc --> Kafka["Kafka Event"] - UpdateSvc --> Kafka - ClientSvc --> Kafka + Admin["Admin UI"] --> ForceController["ForceAgentController"] + ForceController --> InstallService["ForceToolInstallationService"] + ForceController --> UpdateService["ForceToolAgentUpdateService"] + ForceController --> ClientService["ForceClientUpdateService"] + + InstallService --> Messaging["NATS Publisher"] + UpdateService --> Messaging + ClientService --> Messaging ``` -These operations are typically asynchronous and propagate through event pipelines. +These services often trigger asynchronous messaging via NATS to connected agents. --- -## 5. Health Check +### 5. Health Endpoint + +**Controller:** `HealthController` -**Path:** `/health` +Endpoint: -Controller: `HealthController` +- `GET /health` β†’ Returns `OK` -- Lightweight liveness endpoint -- Returns `200 OK` with body `OK` -- Used by orchestrators and load balancers +Used for: + +- Kubernetes liveness/readiness probes +- Load balancer health checks +- Operational monitoring --- -## 6. Invitations +### 6. Invitation Management +**Controller:** `InvitationController` **Base Path:** `/invitations` -Controller: `InvitationController` +Manages tenant-level user invitations. -Supports: +Endpoints: -- Create invitation -- Paginated listing -- Revoke invitation -- Resend invitation +- `POST /invitations` β†’ Create invitation +- `GET /invitations` β†’ Paginated list +- `DELETE /invitations/{id}` β†’ Revoke invitation +- `POST /invitations/{id}/resend` β†’ Resend invitation -```mermaid -sequenceDiagram - participant Admin - participant Controller as InvitationController - participant Service as InvitationService - - Admin->>Controller: POST /invitations - Controller->>Service: createInvitation(request) - Service-->>Controller: InvitationResponse - Controller-->>Admin: 201 Created -``` +Delegates to `InvitationService` and returns: -Invitation flows integrate with SSO and tenant onboarding subsystems. +- `InvitationResponse` +- `InvitationPageResponse` + +Invitation registration flows are completed in the Authorization Service Core module. --- -## 7. Current User Context +### 7. Current User Identity + +**Controller:** `MeController` + +Endpoint: -**Path:** `/me` +- `GET /me` -Controller: `MeController` +Returns authenticated user details from `AuthPrincipal`: -Purpose: +- `id` +- `email` +- `displayName` +- `roles` +- `tenantId` -- Exposes authenticated user context -- Returns identity, roles, tenant ID -- Returns 401 if no authenticated principal +If no principal is resolved, returns HTTP 401 with structured error payload. + +Sequence overview: ```mermaid -flowchart TD - Request["GET /me"] --> AuthCheck["AuthPrincipal Present?"] - AuthCheck -->|"Yes"| Response["Return User Info"] - AuthCheck -->|"No"| Unauthorized["401 Unauthorized"] +sequenceDiagram + participant Client + participant Controller as Api Service + participant Security as JwtSecurityConfig + + Client->>Controller: GET /me with Bearer token + Controller->>Security: Resolve AuthPrincipal + Security-->>Controller: AuthPrincipal + Controller-->>Client: JSON user payload ``` -This endpoint is commonly used by frontend applications to bootstrap user state. - --- -## 8. OpenFrame Client Configuration +### 8. OpenFrame Client Configuration +**Controller:** `OpenFrameClientConfigurationController` **Base Path:** `/openframe-client/configuration` -Controller: `OpenFrameClientConfigurationController` +Endpoint: -Provides configuration metadata used by the OpenFrame client application. +- `GET /openframe-client/configuration` -Delegates to: +Returns `ClientConfigurationResponse` using `OpenFrameClientConfigurationQueryService`. -- OpenFrameClientConfigurationQueryService +This is typically used by client installers or runtime agents to fetch configuration parameters. --- -## 9. Organization Mutations +### 9. Organization Mutations +**Controller:** `OrganizationController` **Base Path:** `/organizations` -Controller: `OrganizationController` +Handles organization **mutations only**. Read operations are exposed in a different module for public access. -Handles: +Endpoints: -- Create organization -- Update organization -- Update status (ACTIVE / ARCHIVED) -- Check if archivable +- `POST /organizations` β†’ Create organization +- `PUT /organizations/{id}` β†’ Update organization +- `GET /organizations/{id}/can-archive` β†’ Archive eligibility check +- `PATCH /organizations/{id}/status` β†’ Update status (ACTIVE / ARCHIVED) -```mermaid -flowchart TD - Admin["Admin"] --> Controller["OrganizationController"] - Controller --> CommandSvc["OrganizationCommandService"] - Controller --> DomainSvc["OrganizationService"] - CommandSvc --> Repo["Organization Repository"] -``` +Key components involved: -Archiving rules: +- `OrganizationCommandService` +- `OrganizationService` +- `OrganizationMapper` -- Cannot archive if active devices exist -- May return `409 Conflict` +Archiving behavior: -Read operations are intentionally separated into external-facing modules. +```mermaid +flowchart TD + Request["PATCH status=ARCHIVED"] --> Service["OrganizationCommandService"] + Service --> Check["canArchiveOrganization()"] + Check -->|"false"| Conflict["409 Conflict"] + Check -->|"true"| Update["Update status to ARCHIVED"] +``` --- -## 10. Release Version +### 10. Release Version +**Controller:** `ReleaseVersionController` **Base Path:** `/release-version` -Controller: `ReleaseVersionController` +Endpoint: -Responsibilities: +- `GET /release-version` -- Return current platform release metadata -- Respond with 404 if not present +Returns `ReleaseVersionResponse` if a version exists, otherwise `404 Not Found`. Used by: -- UI build metadata -- Agent compatibility checks +- UI display - Monitoring tools +- Deployment verification workflows --- -## 11. SSO Configuration +### 11. SSO Configuration Management +**Controller:** `SSOConfigController` **Base Path:** `/sso` -Controller: `SSOConfigController` +Manages per-tenant SSO configuration. -Supports: +Endpoints include: -- List enabled providers -- List available providers -- Retrieve configuration -- Create or update provider config -- Toggle enablement -- Delete configuration +- `GET /sso/providers` β†’ Enabled providers +- `GET /sso/providers/available` β†’ All available provider strategies +- `GET /sso/{provider}` β†’ Full config +- `POST /sso/{provider}` β†’ Create config +- `PUT /sso/{provider}` β†’ Update config +- `PATCH /sso/{provider}/toggle` β†’ Enable/disable +- `DELETE /sso/{provider}` β†’ Remove config -```mermaid -flowchart TD - Admin["Admin"] --> Controller["SSOConfigController"] - Controller --> Service["SSOConfigService"] - Service --> Strategy["Provider Strategy"] - Strategy --> Provider["Google / Microsoft"] -``` +Delegates to `SSOConfigService` and uses: + +- `SSOConfigRequest` +- `SSOConfigResponse` +- `SSOConfigStatusResponse` +- `SSOProviderInfo` -This integrates with the Authorization Service Core and OAuth infrastructure. +SSO integrates with the Authorization Service Core module for OAuth and OIDC flows. --- -## 12. User Management +### 12. User Management +**Controller:** `UserController` **Base Path:** `/users` -Controller: `UserController` +Manages tenant-scoped users. -Supports: +Endpoints: -- Paginated listing -- Get by ID -- Update user -- Soft delete user +- `GET /users` β†’ Paginated list +- `GET /users/{id}` β†’ Get user +- `PUT /users/{id}` β†’ Update user +- `DELETE /users/{id}` β†’ Soft delete (audited by principal) -```mermaid -flowchart LR - Admin["Admin"] --> Controller["UserController"] - Controller --> Service["UserService"] - Service --> Repo["User Repository"] -``` +Delegates to `UserService` and uses: + +- `UserResponse` +- `UserPageResponse` +- `UpdateUserRequest` -Soft deletion ensures audit integrity and traceability. +Exception handling: + +- `IllegalArgumentException` is mapped to `404 Not Found` +- Deletions use soft-delete semantics --- -# Security Model +## Cross-Cutting Concerns -All controllers (except `/health`) rely on Spring Security and JWT-based authentication. +### Authentication and Authorization -Authentication Flow: +- `AuthPrincipal` injected via `@AuthenticationPrincipal` +- JWT validation handled in security configuration layer +- Tenant isolation enforced in service and repository layers -```mermaid -flowchart TD - Request["Incoming HTTP Request"] --> Filter["JWT Filter"] - Filter --> Principal["AuthPrincipal"] - Principal --> Controller["REST Controller"] -``` +### Validation + +- `@Valid` ensures DTO-level validation before service invocation +- Constraint violations return HTTP 400 automatically + +### Logging -Key characteristics: +- Controllers use SLF4J logging +- Sensitive data (e.g., secrets) are not logged -- Tenant-aware security context -- Role-based authorization -- Principal injection via `@AuthenticationPrincipal` -- Clear separation between authentication and business logic +### Error Handling + +- Explicit `ResponseStatusException` for controlled error mapping +- Standard Spring exception handling for validation and conversion --- -# Design Principles +## Design Principles + +The Api Service Core Rest Controllers module follows: + +1. Thin Controller Pattern + Controllers only coordinate HTTP and delegate to services. -The Api Service Core Rest Controllers module follows these principles: +2. Clear Separation of Concerns + - REST: This module + - Business logic: Business Services module + - Data access: Mongo repositories + - Messaging: Kafka/NATS modules + - Authentication: Security and Authorization modules -1. Thin controllers (no heavy business logic) -2. Explicit HTTP semantics (correct status codes) -3. DTO-based contract isolation -4. Clear separation of command vs query concerns -5. Tenant-aware multi-organization architecture +3. Explicit HTTP Semantics + - 201 for create + - 204 for no-content mutations + - 404 for missing resources + - 409 for conflict scenarios --- -# Summary +## Summary -The **Api Service Core Rest Controllers** module is the internal REST faΓ§ade of the OpenFrame API Service Core. It orchestrates: +The **Api Service Core Rest Controllers** module is the central REST entry point of the internal API service. It: -- Identity-scoped user operations -- Organization and tenant management -- API key lifecycle -- SSO configuration -- Agent lifecycle control -- Operational management endpoints +- Exposes secure, structured HTTP endpoints +- Bridges authentication context into business services +- Enforces validation and HTTP semantics +- Delegates all domain logic to specialized services -It serves as a critical integration layer between authenticated clients and the underlying domain, persistence, and event-driven infrastructure, ensuring a clean, secure, and maintainable API boundary. \ No newline at end of file +It plays a critical role in maintaining a clean boundary between transport-layer concerns and core business logic across the OpenFrame platform. \ No newline at end of file diff --git a/docs/reference/architecture/api-service-core-user-sso-services-and-processors/api-service-core-user-sso-services-and-processors.md b/docs/reference/architecture/api-service-core-user-sso-services-and-processors/api-service-core-user-sso-services-and-processors.md deleted file mode 100644 index 4943e9a41..000000000 --- a/docs/reference/architecture/api-service-core-user-sso-services-and-processors/api-service-core-user-sso-services-and-processors.md +++ /dev/null @@ -1,183 +0,0 @@ -# Api Service Core User Sso Services And Processors - -## Overview - -The **Api Service Core User Sso Services And Processors** module encapsulates user management, Single Sign-On (SSO) configuration, domain validation, and extensible post-processing hooks for user- and identity-related workflows inside the API Service Core. - -It acts as the orchestration layer between: - -- REST and GraphQL controllers in Api Service Core -- Mongo persistence (User, Invitation, SSOConfig, AgentRegistrationSecret) -- Encryption and domain validation services -- External Authorization Service Core (SSO, tenant, OAuth flows) - -This module is designed for **extensibility**. All major lifecycle events (user updates, SSO config changes, invitation creation, agent secret generation) are delegated to pluggable processor interfaces with default no-op implementations. - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - Controller["REST / GraphQL Controllers"] --> UserService["UserService"] - Controller --> SSOService["SSOConfigService"] - - UserService --> UserRepo["UserRepository"] - UserService --> UserProcessor["UserProcessor"] - - SSOService --> SSORepo["SSOConfigRepository"] - SSOService --> Encryption["EncryptionService"] - SSOService --> DomainValidation["DomainValidationService"] - SSOService --> SSOProcessor["SSOConfigProcessor"] - - subgraph processors_layer["Post-Processing Layer"] - UserProcessor --> DefaultUserProcessor["DefaultUserProcessor"] - SSOProcessor --> DefaultSSOProcessor["DefaultSSOConfigProcessor"] - InvitationProcessor --> DefaultInvitationProcessor["DefaultInvitationProcessor"] - AgentSecretProcessor --> DefaultAgentProcessor["DefaultAgentRegistrationSecretProcessor"] - end -``` - -### Key Responsibilities - -- Manage users (query, update, soft delete) -- Manage SSO provider configuration (CRUD + enable/disable) -- Enforce domain validation rules for auto-provisioned SSO -- Provide extension points for SaaS or enterprise overrides - ---- - -## Sub-Modules - -To keep responsibilities clean and extensible, this module can be logically divided into two sub-modules: - -### 1. Services - -Core business services handling user and SSO logic. - -πŸ‘‰ See: [Services](api-service-core-user-sso-services-and-processors/services/services.md) - -### 2. Processors - -Lifecycle hooks triggered after core operations (update, delete, toggle, create). - -πŸ‘‰ See: [Processors](api-service-core-user-sso-services-and-processors/processors/processors.md) - ---- - -## Domain Validation Strategy - -Domain validation plays a critical role in SSO auto-provisioning. - -```mermaid -flowchart TD - Request["SSOConfigRequest"] --> Normalize["Normalize Domains"] - Normalize --> GenericCheck["validateGenericPublicDomain()"] - GenericCheck --> ExistsCheck["validateExists()"] - ExistsCheck --> Decision{"Auto Provision?"} - Decision -->|"No"| Save["Save Config"] - Decision -->|"Yes"| RequireDomain["Require At Least One Domain"] - RequireDomain --> MicrosoftCheck{"Microsoft?"} - MicrosoftCheck -->|"Yes"| RequireTenant["Require msTenantId"] - MicrosoftCheck -->|"No"| Save - RequireTenant --> Save -``` - -### DefaultDomainExistenceValidator - -The default implementation always returns `false`, meaning: - -- OSS deployments do not block SSO domain configuration -- SaaS deployments can override with a stricter validator - -This is implemented using `@ConditionalOnMissingBean`, ensuring easy replacement. - ---- - -## User Lifecycle - -```mermaid -flowchart TD - ListUsers["listUsers()"] --> FetchPage["UserRepository.findAll()"] - FetchPage --> MapResponse["UserMapper.toResponse()"] - MapResponse --> PostProcess["UserProcessor.postProcessUserGet()"] - - DeleteUser["softDeleteUser()"] --> ValidateSelf["Prevent Self Delete"] - ValidateSelf --> ValidateOwner["Prevent OWNER Delete"] - ValidateOwner --> MarkDeleted["Set Status DELETED"] - MarkDeleted --> SaveUser["UserRepository.save()"] - SaveUser --> PostDelete["UserProcessor.postProcessUserDeleted()"] -``` - -### Soft Delete Rules - -- Users cannot delete themselves -- OWNER role accounts cannot be deleted -- Deletion sets status to `DELETED` instead of removing the record - -This preserves referential integrity and audit consistency. - ---- - -## SSO Configuration Lifecycle - -```mermaid -flowchart TD - Upsert["upsertConfig()"] --> Exists{"Config Exists?"} - Exists -->|"Yes"| Update["Update Existing"] - Exists -->|"No"| Create["Create New"] - Update --> Encrypt["Encrypt Client Secret"] - Create --> Encrypt - Encrypt --> Save["SSOConfigRepository.save()"] - Save --> PostProcess["SSOConfigProcessor.postProcessConfigSaved()"] -``` - -### Important Behaviors - -- Client secrets are encrypted before persistence -- Decryption occurs only when returning editable admin configuration -- Enabling/disabling triggers processor hooks -- Deletion triggers processor hooks - ---- - -## Extensibility Model - -All processors use: - -- `@Component` -- `@ConditionalOnMissingBean` - -This allows: - -- OSS default no-op behavior -- SaaS override with custom implementations -- Multi-tenant side effects (auditing, events, provisioning, sync) - -This pattern ensures the module is both **framework-level stable** and **deployment-level customizable**. - ---- - -## Integration With Other Modules - -This module integrates closely with: - -- Authorization Service Core (SSO flows, tenant registration, OAuth) -- Data Mongo Domain Model (User, Invitation, SSOConfig documents) -- Api Service Core REST Controllers (UserController, SSOConfigController) - -It does not implement OAuth flows directly β€” it manages configuration and lifecycle hooks that support those flows. - ---- - -## Summary - -The **Api Service Core User Sso Services And Processors** module is the identity orchestration layer of the API service. It: - -- Centralizes user management -- Governs SSO provider configuration -- Enforces domain safety rules -- Provides structured extension hooks -- Enables SaaS override without modifying core logic - -Its clean separation between services and processors makes it highly maintainable and enterprise-ready. \ No newline at end of file diff --git a/docs/reference/architecture/api-service-core-user-sso-services-and-processors/processors.md b/docs/reference/architecture/api-service-core-user-sso-services-and-processors/processors.md deleted file mode 100644 index a6f72e41b..000000000 --- a/docs/reference/architecture/api-service-core-user-sso-services-and-processors/processors.md +++ /dev/null @@ -1,303 +0,0 @@ -# Processors - -The **Processors** module provides extension points for domain-specific post-processing logic within the API Service Core, specifically around user management, invitations, SSO configuration, and agent registration secrets. - -In OpenFrame’s modular architecture, Processors act as **hooks** that execute after core service operations complete. By default, they provide no-op (logging-only) implementations, but they are designed to be overridden in custom or enterprise deployments. - -This module lives under the *API Service Core – User, SSO Services and Processors* domain and works closely with the sibling Services module. - ---- - -## 1. Purpose and Design Philosophy - -The Processors module exists to: - -- Provide **post-operation hooks** for core domain actions -- Enable **custom business logic injection** without modifying core services -- Maintain **clean separation of concerns** between service logic and side effects -- Support OSS deployments with safe default behavior - -All default implementations are annotated with: - -- `@Component` -- `@ConditionalOnMissingBean` - -This ensures: - -- The default implementation is used in OSS environments -- Custom implementations can transparently override the default behavior - ---- - -## 2. High-Level Architecture - -The Processors module sits between: - -- Core domain services (UserService, SSOConfigService, etc.) -- Downstream side effects (notifications, audit, integrations, etc.) - -```mermaid -flowchart LR - Controller["REST or GraphQL Controller"] --> Service["Core Service"] - Service --> DomainModel["Domain Model"] - Service --> Processor["Processor Interface"] - Processor --> DefaultImpl["Default Implementation"] - Processor --> CustomImpl["Custom Implementation (Optional)"] -``` - -### Key Characteristics - -- Services remain pure and focused on business rules -- Processors encapsulate side effects -- Spring’s conditional bean loading determines the active implementation - ---- - -## 3. Core Components - -The Processors module includes four default implementations: - -- DefaultAgentRegistrationSecretProcessor -- DefaultInvitationProcessor -- DefaultSSOConfigProcessor -- DefaultUserProcessor - -Each implements its respective processor interface and logs events by default. - ---- - -## 4. Agent Registration Secret Processing - -### Class - -`DefaultAgentRegistrationSecretProcessor` - -### Responsibilities - -- Post-process generated agent registration secrets -- Post-process deactivated secrets -- Provide safe logging in OSS mode - -### Interaction Flow - -```mermaid -flowchart TD - Generator["Secret Generation Logic"] --> Persist["Persist AgentRegistrationSecret"] - Persist --> ProcessorCall["postProcessSecretGenerated()"] - ProcessorCall --> DefaultProcessor["DefaultAgentRegistrationSecretProcessor"] -``` - -### Extension Use Cases - -A custom implementation may: - -- Push secrets to an external vault -- Emit audit events -- Trigger management workflows -- Notify external systems - -By default, only debug logging is performed. - ---- - -## 5. Invitation Processing - -### Class - -`DefaultInvitationProcessor` - -### Responsibilities - -- Post-process invitation creation -- Post-process invitation revocation - -### Flow - -```mermaid -flowchart TD - InvitationService["Invitation Service"] --> SaveInvitation["Persist Invitation"] - SaveInvitation --> ProcessorHook["postProcessInvitationCreated()"] - ProcessorHook --> DefaultInvitationProcessor["DefaultInvitationProcessor"] -``` - -### Typical Customization Scenarios - -A custom processor may: - -- Send email notifications -- Publish events to Kafka or NATS -- Integrate with external identity providers -- Trigger onboarding workflows - -The default implementation logs metadata such as invitation ID and email. - ---- - -## 6. SSO Configuration Processing - -### Class - -`DefaultSSOConfigProcessor` - -### Responsibilities - -- Post-process SSO config save -- Post-process SSO config deletion -- Post-process SSO config toggle (enable/disable) - -### Flow - -```mermaid -flowchart TD - SSOService["SSOConfigService"] --> SaveConfig["Persist SSOConfig"] - SaveConfig --> ProcessorHook["postProcessConfigSaved()"] - ProcessorHook --> DefaultSSOProcessor["DefaultSSOConfigProcessor"] -``` - -### Integration Context - -SSO configuration impacts: - -- OAuth/OIDC flows -- Tenant authentication -- Authorization Server behavior - -A custom processor could: - -- Register dynamic OAuth clients -- Synchronize configuration with external IdPs -- Invalidate authentication caches - -The default implementation performs debug logging only. - ---- - -## 7. User Processing - -### Class - -`DefaultUserProcessor` - -### Responsibilities - -- Post-process user deletion -- Post-process user updates -- Post-process user fetch (single) -- Post-process user fetch (paged) - -### Flow Example: User Update - -```mermaid -flowchart TD - UserController["UserController"] --> UserService["UserService"] - UserService --> PersistUser["Persist User"] - PersistUser --> ProcessorHook["postProcessUserUpdated()"] - ProcessorHook --> DefaultUserProcessor["DefaultUserProcessor"] -``` - -### Data Types Involved - -- User (Mongo domain model) -- UserResponse (API DTO) -- UserPageResponse (paginated DTO) - -### Customization Scenarios - -A custom implementation might: - -- Emit audit logs -- Synchronize user state with external systems -- Trigger deprovisioning in downstream tools -- Enforce domain-level policies - -The default behavior is debug-level logging. - ---- - -## 8. Conditional Bean Strategy - -All default processors use: - -- `@ConditionalOnMissingBean(value = Interface.class, ignored = DefaultImpl.class)` - -This enables the following resolution model: - -```mermaid -flowchart TD - SpringContext["Spring Application Context"] --> CheckCustom["Custom Bean Present?"] - CheckCustom -->|"Yes"| UseCustom["Use Custom Implementation"] - CheckCustom -->|"No"| UseDefault["Use Default Implementation"] -``` - -This pattern ensures: - -- Clean override without modifying core modules -- Backward compatibility -- Pluggable architecture for enterprise extensions - ---- - -## 9. Relationship to Services Module - -The Processors module complements the Services module located at: - -- [Services](../services/services.md) - -### Responsibility Split - -```mermaid -flowchart LR - Services["Services Module"] -->|"Business Logic"| Domain["Domain Model"] - Services -->|"Calls"| Processors["Processors Module"] - Processors -->|"Side Effects"| ExternalSystems["External Systems"] -``` - -- Services handle validation, persistence, and domain rules -- Processors handle post-operation side effects - -This clear boundary keeps the architecture maintainable and extensible. - ---- - -## 10. How Processors Fit Into the Overall System - -Within the broader OpenFrame platform: - -- Controllers (REST / GraphQL) invoke Services -- Services mutate domain entities -- Processors execute post-commit logic -- External systems (notifications, auth server, integrations) react - -```mermaid -flowchart TD - API["API Layer"] --> ServiceLayer["Service Layer"] - ServiceLayer --> RepositoryLayer["Repository Layer"] - ServiceLayer --> ProcessorLayer["Processors"] - ProcessorLayer --> Integration["Integrations / Events / Audit"] -``` - -This modular layering: - -- Prevents service bloat -- Encourages testability -- Enables OSS and enterprise feature differentiation - ---- - -## 11. Summary - -The **Processors** module provides structured extension points for: - -- Agent registration secrets -- Invitations -- SSO configuration -- User lifecycle operations - -By default, implementations are safe and logging-only. In advanced deployments, these processors become powerful integration points that allow OpenFrame to plug into broader ecosystems without altering core service logic. - -The design promotes: - -- Clean architecture -- Pluggability -- Multi-tenant extensibility -- Enterprise customization without forking core modules diff --git a/docs/reference/architecture/api-service-core-user-sso-services-and-processors/services.md b/docs/reference/architecture/api-service-core-user-sso-services-and-processors/services.md deleted file mode 100644 index 456083ead..000000000 --- a/docs/reference/architecture/api-service-core-user-sso-services-and-processors/services.md +++ /dev/null @@ -1,302 +0,0 @@ -# Services - -The **Services** module contains the core application services responsible for user management, Single Sign-On (SSO) configuration, and domain validation within the OpenFrame API Service Core. - -These services sit between controllers (REST and GraphQL) and the data layer, encapsulating business logic, validation, orchestration, and post-processing hooks. They integrate with repositories, processors, mappers, encryption utilities, and tenant/domain validation infrastructure. - ---- - -## 1. Module Responsibilities - -The Services module provides: - -- βœ… User lifecycle management (read, update, soft delete) -- βœ… SSO configuration management (CRUD, toggle, validation) -- βœ… Domain validation extension point (SaaS overrides) -- βœ… Post-processing hooks via processors -- βœ… Secure handling of secrets (via encryption service) - -Core components: - -- `DefaultDomainExistenceValidator` -- `SSOConfigService` -- `UserService` - ---- - -## 2. Architectural Position - -The Services module operates inside the API Service Core and collaborates with multiple surrounding modules. - -```mermaid -flowchart LR - subgraph Controllers["API Layer"] - RestControllers["REST Controllers"] - GraphQL["GraphQL DataFetchers"] - end - - subgraph ServicesLayer["Services Module"] - UserServiceNode["UserService"] - SSOServiceNode["SSOConfigService"] - DomainValidator["DefaultDomainExistenceValidator"] - end - - subgraph Processors["Processors"] - UserProcessorNode["UserProcessor"] - SSOProcessorNode["SSOConfigProcessor"] - end - - subgraph DataLayer["Data Layer"] - UserRepo["UserRepository"] - SSORepo["SSOConfigRepository"] - MongoDocs["Mongo Domain Models"] - end - - Controllers --> ServicesLayer - ServicesLayer --> Processors - ServicesLayer --> DataLayer -``` - -### Key Relationships - -- Controllers invoke Services for business operations. -- Services delegate side-effects to Processors. -- Services persist and query through repositories. -- Domain validation is pluggable and environment-aware. - ---- - -## 3. UserService - -`UserService` encapsulates all user-related domain logic for the API layer. - -### 3.1 Responsibilities - -- Retrieve users by ID or email -- Paginated listing -- Bulk fetch by IDs -- Update profile fields -- Soft delete users with safety checks -- Trigger lifecycle processors - -### 3.2 Core Dependencies - -- `UserRepository` -- `UserMapper` -- `UserProcessor` -- Mongo `User` document - -### 3.3 User Lifecycle Flow - -```mermaid -flowchart TD - Request["User Update Request"] --> Load["Load User from Repository"] - Load --> Check{"User Exists?"} - Check -->|"No"| Error["Throw IllegalArgumentException"] - Check -->|"Yes"| Modify["Apply Field Updates"] - Modify --> Save["Save via UserRepository"] - Save --> PostProcess["UserProcessor.postProcessUserUpdated()"] - PostProcess --> Response["Return UserResponse"] -``` - -### 3.4 Soft Delete Safety Rules - -Soft deletion includes strict business constraints: - -- A user **cannot delete themselves** β†’ `UserSelfDeleteNotAllowedException` -- A user with `OWNER` role cannot be deleted β†’ `OperationNotAllowedException` -- Deletion sets status to `DELETED` instead of removing the document - -```mermaid -flowchart TD - DeleteRequest["Soft Delete Request"] --> LoadUser["Load User"] - LoadUser --> SelfCheck{"Requester == Target?"} - SelfCheck -->|"Yes"| SelfError["Throw Self Delete Exception"] - SelfCheck -->|"No"| OwnerCheck{"Has OWNER Role?"} - OwnerCheck -->|"Yes"| OwnerError["Throw OperationNotAllowedException"] - OwnerCheck -->|"No"| MarkDeleted["Set Status = DELETED"] - MarkDeleted --> SaveDeleted["Save User"] - SaveDeleted --> PostDelete["UserProcessor.postProcessUserDeleted()"] -``` - -This ensures organizational integrity and prevents accidental tenant lockout. - ---- - -## 4. SSOConfigService - -`SSOConfigService` manages Single Sign-On provider configuration for the tenant. - -### 4.1 Responsibilities - -- List enabled providers -- Return editable configuration (including decrypted secret) -- Create or update provider config (upsert) -- Toggle provider enabled state -- Delete provider configuration -- Validate domain and provisioning constraints -- Trigger SSO lifecycle processors - -### 4.2 Core Dependencies - -- `SSOConfigRepository` -- `EncryptionService` -- `SSOProperties` -- `SSOConfigProcessor` -- `SSOConfigMapper` -- `DomainValidationService` - -### 4.3 SSO Configuration Flow - -```mermaid -flowchart TD - AdminRequest["Upsert SSOConfigRequest"] --> Validate["validateAutoProvision()"] - Validate --> Normalize["Normalize Domains"] - Normalize --> DomainValidation["DomainValidationService.validate*"] - DomainValidation --> Encrypt["Encrypt Client Secret"] - Encrypt --> SaveConfig["Save via SSOConfigRepository"] - SaveConfig --> PostProcess["SSOConfigProcessor.postProcessConfigSaved()"] - PostProcess --> ReturnResponse["Return SSOConfigResponse"] -``` - -### 4.4 Domain Validation Logic - -When `autoProvisionUsers` is enabled: - -- βœ… `allowedDomains` must not be empty -- βœ… Domains are normalized (trimmed, lowercased, deduplicated) -- βœ… Public/generic domains are rejected -- βœ… Domains must exist (pluggable validator) -- βœ… Microsoft provider requires `msTenantId` when auto-provisioning - -This ensures secure enterprise SSO onboarding. - -### 4.5 Secure Secret Handling - -- Client secrets are encrypted before persistence. -- Decryption occurs only when returning editable configuration. -- Encryption is delegated to `EncryptionService`. - ---- - -## 5. DefaultDomainExistenceValidator - -`DefaultDomainExistenceValidator` provides the baseline implementation for domain existence checks. - -### 5.1 Behavior - -```java -public boolean anyExists(List domains) { - return false; -} -``` - -### 5.2 Design Purpose - -- Acts as a **non-blocking default** implementation. -- Enabled via `@ConditionalOnMissingBean`. -- Allows SaaS or enterprise deployments to override with a stricter validator. - -```mermaid -flowchart LR - DomainService["DomainValidationService"] --> Interface["DomainExistenceValidator"] - Interface --> DefaultImpl["DefaultDomainExistenceValidator"] - Interface --> CustomImpl["SaaS Override Implementation"] -``` - -This pattern supports multi-tenant SaaS deployments where domain ownership must be verified. - ---- - -## 6. Cross-Cutting Patterns - -### 6.1 Processor Hook Pattern - -Services delegate side effects to processors after core operations: - -- `postProcessUserGet` -- `postProcessUserUpdated` -- `postProcessUserDeleted` -- `postProcessConfigSaved` -- `postProcessConfigDeleted` -- `postProcessConfigToggled` - -This keeps services focused on: - -- Validation -- Persistence -- Mapping - -While processors handle: - -- Audit logging -- External sync -- Notifications -- Policy enforcement - -```mermaid -sequenceDiagram - participant Controller - participant Service - participant Repository - participant Processor - - Controller->>Service: Update Request - Service->>Repository: Save Entity - Repository-->>Service: Saved Entity - Service->>Processor: postProcess(...) - Service-->>Controller: Response DTO -``` - ---- - -## 7. Integration with Other Modules - -The Services module integrates with: - -- API Service Core Controllers (REST & GraphQL) -- Mongo domain and repository modules -- Security and OAuth infrastructure -- Authorization Service for SSO flows -- Encryption and crypto services - -High-level integration view: - -```mermaid -flowchart TD - Client["Client Application"] --> Gateway["Gateway Service"] - Gateway --> ApiCore["API Service Core"] - ApiCore --> ServicesNode["Services Module"] - ServicesNode --> Mongo[("MongoDB")] - ServicesNode --> Authz["Authorization Service"] - ServicesNode --> Crypto["Encryption Service"] -``` - ---- - -## 8. Design Principles - -The Services module follows these architectural principles: - -- βœ… Clear separation of concerns -- βœ… Stateless service layer -- βœ… Repository-driven persistence -- βœ… DTO ↔ Entity mapping abstraction -- βœ… Post-operation extension hooks -- βœ… Pluggable domain validation -- βœ… Secure secret handling -- βœ… Defensive business rule enforcement - ---- - -# Summary - -The **Services** module forms the business logic core of user and SSO management within OpenFrame’s API Service Core. It ensures: - -- Secure and validated SSO onboarding -- Safe user lifecycle operations -- Tenant-aware domain validation -- Extensible post-processing -- Clean separation between API, business logic, and persistence layers - -It acts as the orchestration layer that binds controllers, repositories, processors, and security infrastructure into a cohesive, enterprise-ready service architecture. \ No newline at end of file diff --git a/docs/reference/architecture/authorization-service-core-auth-controllers-and-dtos/authorization-service-core-auth-controllers-and-dtos.md b/docs/reference/architecture/authorization-service-core-auth-controllers-and-dtos/authorization-service-core-auth-controllers-and-dtos.md deleted file mode 100644 index 2445546a4..000000000 --- a/docs/reference/architecture/authorization-service-core-auth-controllers-and-dtos/authorization-service-core-auth-controllers-and-dtos.md +++ /dev/null @@ -1,393 +0,0 @@ -# Authorization Service Core Auth Controllers And Dtos - -## Overview - -The **Authorization Service Core Auth Controllers And Dtos** module exposes the public HTTP interface for tenant onboarding, login, invitation acceptance, password reset, SSO discovery, and tenant discovery within the OpenFrame multi-tenant authorization system. - -It acts as the **edge layer** of the Authorization Service Core, translating HTTP requests into domain-level service calls and orchestrating redirects, cookies, and validation logic. - -This module is tightly integrated with: - -- The authorization server and tenant context layer: - [Authorization Service Core Server And Tenant](../authorization-service-core-server-and-tenant/authorization-service-core-server-and-tenant.md) -- The SSO flow and security utilities layer (redirect helpers, auth state utilities, registration constants) -- The persistence and key management layer (Mongo authorization service, client repositories, tenant keys) - ---- - -## Architectural Role - -This module sits at the boundary between: - -- 🌐 External clients (browser, SPA, CLI, third-party tools) -- πŸ” Spring Security & OAuth2 Authorization Server -- 🏒 Multi-tenant domain services - -It contains: - -- REST controllers -- MVC login controller -- Request/response DTOs -- Validation annotations - -### High-Level Request Flow - -```mermaid -flowchart LR - Client["Browser / Client App"] --> Controllers["Auth Controllers"] - Controllers --> Services["Domain Services"] - Services --> Persistence["Authorization & Tenant Persistence"] - Services --> Security["Spring Security / OAuth2"] - Security --> Client -``` - -The controllers remain intentionally thin. All business logic is delegated to service-layer components such as: - -- `InvitationRegistrationService` -- `SsoInvitationService` -- `TenantRegistrationService` -- `TenantDiscoveryService` -- `PasswordResetService` - ---- - -# Controllers - -## 1. InvitationRegistrationController - -**Base path:** `/invitations` - -Handles invitation-based onboarding, both standard and SSO-based. - -### Endpoints - -#### `POST /invitations/accept` -Registers a user using an invitation token. - -- Input: `InvitationRegistrationRequest` -- Output: `AuthUser` -- Delegates to: `InvitationRegistrationService` - -#### `GET /invitations/accept/sso` -Initiates SSO-based invitation acceptance. - -Flow: - -```mermaid -sequenceDiagram - participant Browser - participant Controller as InvitationRegistrationController - participant SSOService as SsoInvitationService - participant IdP as External IdP - - Browser->>Controller: GET /invitations/accept/sso - Controller->>SSOService: startAccept(request) - SSOService-->>Controller: SsoAuthorizeData - Controller->>Browser: Set-Cookie + 303 Redirect - Browser->>IdP: OAuth2 Authorization Request -``` - -Key behaviors: - -- Clears previous auth state using `AuthStateUtils` -- Sets secure, HTTP-only SSO invitation cookie -- Performs 303 redirect using `Redirects.seeOther` -- Encodes error message and redirects to configured `openframe.auth.error-url` on failure - ---- - -## 2. TenantRegistrationController - -**Base path:** `/oauth` - -Responsible for creating new tenants via: - -- Password-based registration -- SSO-based registration - -### `POST /oauth/register` - -Registers a new tenant and initial user. - -- Input: `TenantRegistrationRequest` -- Output: `Tenant` -- Delegates to: `TenantRegistrationService` - -### `GET /oauth/register/sso` - -Initiates SSO tenant registration. - -```mermaid -flowchart TD - Start["User Clicks Register with SSO"] --> Clear["Clear Auth State"] - Clear --> StartReg["SsoTenantRegistrationService.startRegistration()"] - StartReg --> Cookie["Set COOKIE_SSO_REG"] - Cookie --> Redirect["Redirect to IdP"] -``` - -Security properties: - -- Secure cookies -- HTTP-only flags -- Tenant-scoped redirect path -- Centralized error handling - ---- - -## 3. LoginController - -MVC controller (not REST). - -### Endpoints - -- `GET /login` β†’ renders login page -- `GET /` β†’ renders index page - -Responsibilities: - -- Inject error message when `?error` query param is present -- Expose optional password reset URL from configuration - -This controller integrates with Spring Security login processing. - ---- - -## 4. PasswordResetController - -**Base path:** `/password-reset` - -Handles password recovery flows. - -### `POST /password-reset/request` - -- Input: `ResetRequest` -- Normalizes email to lowercase -- Delegates to `PasswordResetService.createResetToken` -- Returns HTTP 202 (Accepted) - -### `POST /password-reset/confirm` - -- Input: `ResetConfirm` -- Enforces strong password pattern validation -- Delegates to `PasswordResetService.resetPassword` -- Returns HTTP 204 (No Content) - -Password validation rules: - -- Minimum 8 characters -- At least one uppercase -- At least one lowercase -- At least one digit -- At least one special character - ---- - -## 5. SsoDiscoveryController - -**Base path:** `/sso/providers` - -Used by frontend before redirecting to SSO provider. - -### `GET /sso/providers/invite` - -- Requires `invitationId` -- Loads invitation via `InvitationValidator` -- Returns effective providers for tenant - -### `GET /sso/providers/registration` - -- Returns system default SSO providers -- Delegates to `SSOConfigService` - -Response model: - -```text -{ - "providers": ["google", "microsoft"] -} -``` - ---- - -## 6. TenantDiscoveryController - -**Base path:** `/tenant` - -Supports multi-tenant login UX. - -### `GET /tenant/discover?email=` - -Returns: - -- Whether accounts exist -- Associated tenant ID -- Available authentication providers - -Flow: - -```mermaid -flowchart LR - User["User enters email"] --> Controller["TenantDiscoveryController"] - Controller --> Service["TenantDiscoveryService"] - Service --> Response["TenantDiscoveryResponse"] -``` - -Used to determine: - -- Password login vs SSO -- Redirect to tenant-specific issuer - ---- - -# DTO Layer - -The DTOs define the external API contract and enforce validation constraints. - -## InvitationRegistrationRequest - -Extends `CoreUserRequest`. - -Fields: - -- `invitationId` (required) -- `switchTenant` (optional) - ---- - -## TenantRegistrationRequest - -Extends `CoreUserRequest`. - -Fields: - -- `email` -- `tenantName` -- `tenantDomain` -- `accessCode` - -Validation ensures: - -- Proper tenant domain format -- Organization name pattern restrictions - ---- - -## SsoTenantRegistrationInitRequest - -Used before redirecting to IdP. - -Fields: - -- `email` -- `tenantName` -- `tenantDomain` -- `provider` -- `redirectTo` - -This ensures the system has enough context to: - -- Create tenant if needed -- Attach SSO configuration -- Prepare OAuth2 client registration - ---- - -## SsoInvitationAcceptRequest - -Fields: - -- `invitationId` -- `provider` -- `switchTenant` -- `redirectTo` - -Used to bootstrap SSO flow for invited users. - ---- - -## PasswordResetDtos - -Two nested DTOs: - -- `ResetRequest` -- `ResetConfirm` - -Encapsulates strict password validation rules. - ---- - -## TenantDiscoveryResponse - -```text -{ - "email": "user@example.com", - "has_existing_accounts": true, - "tenant_id": "tenant123", - "auth_providers": ["password", "google"] -} -``` - ---- - -## TenantAvailabilityResponse - -Indicates whether a tenant domain is available and suggests alternatives. - ---- - -# Multi-Tenant & Security Integration - -This module works in conjunction with: - -- Tenant context resolution -- OAuth2 authorization server -- SSO provider strategies (Google, Microsoft) -- Registration processors -- Token generation and persistence - -### Cross-Module Interaction - -```mermaid -flowchart TD - Controllers["Auth Controllers"] --> TenantLayer["Tenant Context Layer"] - Controllers --> SSOFlow["SSO Flow & Handlers"] - Controllers --> AuthServer["OAuth2 Authorization Server"] - AuthServer --> Persistence["Mongo Authorization Service"] -``` - -The controllers do not directly manipulate: - -- JWT generation -- Key pairs -- OAuth token storage -- Client registration persistence - -These responsibilities are delegated to the lower layers of the Authorization Service Core. - ---- - -# Design Principles - -βœ… Thin controllers -βœ… Strong DTO validation -βœ… Clear separation of concerns -βœ… Secure cookie handling -βœ… Explicit redirect logic -βœ… Multi-tenant aware flows -βœ… SSO-provider abstraction - ---- - -# Summary - -The **Authorization Service Core Auth Controllers And Dtos** module defines the complete public-facing contract of the OpenFrame Authorization Service. - -It: - -- Exposes login, registration, invitation, and password reset APIs -- Coordinates SSO bootstrap and redirect logic -- Enforces strict validation rules -- Supports multi-tenant discovery and onboarding -- Bridges HTTP requests to secure, tenant-aware domain services - -This module is the entry point into the OpenFrame multi-tenant identity and authorization infrastructure. \ No newline at end of file diff --git a/docs/reference/architecture/authorization-service-core-keys-and-authorization-persistence/authorization-service-core-keys-and-authorization-persistence.md b/docs/reference/architecture/authorization-service-core-keys-and-authorization-persistence/authorization-service-core-keys-and-authorization-persistence.md deleted file mode 100644 index abca1c21b..000000000 --- a/docs/reference/architecture/authorization-service-core-keys-and-authorization-persistence/authorization-service-core-keys-and-authorization-persistence.md +++ /dev/null @@ -1,355 +0,0 @@ -# Authorization Service Core Keys And Authorization Persistence - -## Overview - -The **Authorization Service Core Keys And Authorization Persistence** module is responsible for: - -- Generating and managing tenant-scoped RSA signing keys -- Converting RSA keys to and from PEM format -- Persisting OAuth2 registered clients -- Persisting OAuth2 authorizations (authorization codes, access tokens, refresh tokens) -- Preserving PKCE metadata across the authorization lifecycle - -This module forms the cryptographic and persistence backbone of the Authorization Server. It ensures that: - -- Each tenant has a secure signing key used for JWT issuance -- OAuth2 clients are stored and reconstructed correctly -- Authorization state (including PKCE parameters) survives restarts and distributed deployments - -It integrates tightly with Spring Authorization Server, MongoDB repositories, and the tenant-aware authorization server configuration. - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - ClientApp["OAuth2 Client Application"] --> AuthServer["Authorization Server"] - - subgraph key_layer["Tenant Key Layer"] - TenantKeyService["TenantKeyService"] --> KeyGenerator["RsaAuthenticationKeyPairGenerator"] - TenantKeyService --> PemUtil["PemUtil"] - TenantKeyService --> TenantKeyRepo["TenantKeyRepository"] - TenantKeyService --> EncryptionService["EncryptionService"] - end - - subgraph client_layer["Client Registration Layer"] - RegisteredRepo["MongoRegisteredClientRepository"] --> MongoClientRepo["RegisteredClientMongoRepository"] - end - - subgraph auth_layer["Authorization Persistence Layer"] - AuthService["MongoAuthorizationService"] --> AuthMapper["MongoAuthorizationMapper"] - AuthService --> MongoAuthRepo["MongoOAuth2AuthorizationRepository"] - AuthMapper --> MongoEntity["MongoOAuth2Authorization"] - end - - AuthServer --> TenantKeyService - AuthServer --> RegisteredRepo - AuthServer --> AuthService -``` - -### Responsibilities by Layer - -| Layer | Responsibility | -|--------|----------------| -| Tenant Key Layer | RSA key generation, encryption, and retrieval per tenant | -| Client Registration Layer | Persistent storage of OAuth2 RegisteredClient definitions | -| Authorization Persistence Layer | Persistent storage of authorization codes, access tokens, refresh tokens, and PKCE metadata | - ---- - -## Tenant RSA Key Management - -### Purpose - -The Authorization Service must sign JWT access tokens and ID tokens. In a multi-tenant system, each tenant requires its own signing key to: - -- Isolate cryptographic trust boundaries -- Support per-tenant key rotation -- Prevent cross-tenant token misuse - -### Core Components - -- `PemUtil` -- `RsaAuthenticationKeyPairGenerator` -- `TenantKeyService` - ---- - -### PemUtil - -`PemUtil` is a low-level utility that: - -- Parses PEM-encoded RSA public/private keys -- Serializes RSA keys into PEM format -- Wraps base64 content into 64-character lines - -Supported formats: - -```text ------BEGIN PUBLIC KEY----- -(base64) ------END PUBLIC KEY----- - ------BEGIN PRIVATE KEY----- -(base64) ------END PRIVATE KEY----- -``` - -This class ensures interoperability with external JWK and JWT libraries. - ---- - -### RsaAuthenticationKeyPairGenerator - -This Spring component generates RSA key pairs using configurable properties: - -- Algorithm (for example, RSA) -- Key size (for example, 2048 or 4096) -- SecureRandom algorithm - -Generation flow: - -```mermaid -flowchart TD - Start["Generate Key Pair"] --> InitRandom["Initialize SecureRandom"] - InitRandom --> InitGenerator["Initialize KeyPairGenerator"] - InitGenerator --> Generate["Generate RSA KeyPair"] - Generate --> ConvertPem["Convert to PEM"] - ConvertPem --> CreateKid["Create kid UUID"] - CreateKid --> Result["Return AuthenticationKeyPair"] -``` - -Each generated key pair includes: - -- RSAPublicKey -- RSAPrivateKey -- Public PEM -- Private PEM -- `kid` (Key ID) prefixed with `kid-` - -The `kid` is later embedded into JWT headers. - ---- - -### TenantKeyService - -`TenantKeyService` provides the runtime contract used by the Authorization Server to retrieve signing keys. - -#### Responsibilities - -1. Retrieve active key for a tenant -2. Generate and persist a key if none exists -3. Encrypt private key material before persistence -4. Return a Nimbus `RSAKey` for signing - -#### getOrCreateActiveKey Flow - -```mermaid -flowchart TD - Request["Request RSAKey for Tenant"] --> CheckActive["Check Active Key Count"] - CheckActive --> Found{"Active Key Exists?"} - Found -->|"Yes"| LoadDoc["Load TenantKey Document"] - Found -->|"No"| Create["Generate New Key"] - Create --> Encrypt["Encrypt Private PEM"] - Encrypt --> Save["Save TenantKey"] - Save --> LoadDoc - LoadDoc --> Parse["Parse PEM to RSA Keys"] - Parse --> BuildJwk["Build Nimbus RSAKey"] - BuildJwk --> Return["Return RSAKey"] -``` - -#### Security Model - -- Public key is stored in plaintext PEM. -- Private key is encrypted using `EncryptionService` before persistence. -- Only decrypted in-memory during signing operations. - -If multiple active keys are detected for a tenant, a warning is logged to prevent `kid` mismatches. - ---- - -## OAuth2 Client Persistence - -### MongoRegisteredClientRepository - -This class implements Spring Authorization Server's `RegisteredClientRepository` interface. - -It acts as a bridge between: - -- Spring `RegisteredClient` -- Mongo `MongoRegisteredClient` document - -### Save Mapping - -On save: - -- Authentication methods are converted to string values -- Grant types are converted to string values -- Token TTL values are converted to seconds -- PKCE requirement flags are persisted - -### Reconstruction Mapping - -On retrieval: - -- `ClientSettings` is rebuilt -- `TokenSettings` is rebuilt -- Authentication methods and grant types are rehydrated -- Redirect URIs and scopes are restored - -This ensures full compatibility with Spring Authorization Server runtime expectations. - ---- - -## OAuth2 Authorization Persistence - -### Core Components - -- `MongoAuthorizationService` -- `MongoAuthorizationMapper` - -These classes implement and support Spring's `OAuth2AuthorizationService`. - ---- - -## MongoAuthorizationMapper - -This class converts between: - -- `OAuth2Authorization` (Spring domain model) -- `MongoOAuth2Authorization` (Mongo document) - -It persists: - -- Authorization grant type -- Authorization code -- Access token -- Refresh token -- State -- PKCE parameters -- Authorization request snapshot - -### PKCE Preservation Strategy - -PKCE parameters (`code_challenge`, `code_challenge_method`) are captured from: - -- OAuth2AuthorizationRequest additional parameters -- Authorization code metadata - -They are normalized and stored in: - -- `arAdditional` -- `authorizationCodeMetadata` - -When reconstructing: - -- PKCE keys are restored with underscore format -- Both dot and underscore variants are normalized -- OAuth2AuthorizationRequest is rebuilt - -This ensures PKCE validation works correctly during token exchange. - ---- - -## MongoAuthorizationService - -Implements `OAuth2AuthorizationService`. - -### Save Flow - -```mermaid -flowchart TD - SaveCall["Save Authorization"] --> ExtractPkce["Extract PKCE Parameters"] - ExtractPkce --> MapEntity["Map to Mongo Entity"] - MapEntity --> Persist["Save to Mongo"] - Persist --> Verify["Debug Verify PKCE"] -``` - -### Token Lookup Flow - -```mermaid -flowchart TD - Lookup["Find By Token"] --> TypeCheck{"Token Type?"} - TypeCheck -->|"Access"| AccessQuery["Find By Access Token"] - TypeCheck -->|"Refresh"| RefreshQuery["Find By Refresh Token"] - TypeCheck -->|"Code"| CodeQuery["Find By Authorization Code"] - TypeCheck -->|"Null"| MultiQuery["Try All Token Fields"] - AccessQuery --> MapBack["Map to Domain"] - RefreshQuery --> MapBack - CodeQuery --> MapBack - MultiQuery --> MapBack - MapBack --> ReturnAuth["Return OAuth2Authorization"] -``` - -It supports lookup by: - -- Access token -- Refresh token -- Authorization code -- State - -After retrieval, the entity is mapped back to `OAuth2Authorization`, ensuring: - -- PKCE metadata is restored -- Principal placeholder is recreated -- Tokens are rebuilt with correct expiry - ---- - -## End-to-End Authorization Lifecycle - -```mermaid -sequenceDiagram - participant User - participant AuthServer - participant KeyService - participant AuthService - participant MongoDB - - User->>AuthServer: Authorization Request with PKCE - AuthServer->>AuthService: Save OAuth2Authorization - AuthService->>MongoDB: Persist Authorization Code - AuthServer->>KeyService: Get Tenant Signing Key - KeyService->>MongoDB: Load or Create TenantKey - KeyService->>AuthServer: Return RSAKey - AuthServer->>User: Issue JWT Access Token - User->>AuthServer: Token Exchange - AuthServer->>AuthService: Find Authorization by Code - AuthService->>MongoDB: Retrieve Authorization - AuthServer->>User: Return Access and Refresh Tokens -``` - ---- - -## Security Considerations - -1. Private Key Encryption - - Private RSA keys are encrypted before Mongo persistence. - - Decrypted only when building the runtime RSAKey. - -2. PKCE Integrity - - PKCE values are preserved exactly as provided. - - Underscore normalization avoids validation mismatches. - -3. Token Expiry - - TTL values are enforced through stored expiration timestamps. - - Expired authorizations can be pruned by repository-level TTL logic. - -4. Multi-Tenant Isolation - - Each tenant has an isolated signing key. - - Each authorization record references its registered client. - ---- - -## Summary - -The **Authorization Service Core Keys And Authorization Persistence** module provides: - -- Cryptographically secure tenant-aware RSA key management -- Robust Mongo-backed OAuth2 client persistence -- Reliable authorization and token persistence -- Full PKCE support with metadata normalization -- Seamless integration with Spring Authorization Server - -It is the foundation that guarantees secure JWT issuance, correct OAuth2 behavior, and multi-tenant isolation within the OpenFrame Authorization Server architecture. diff --git a/docs/reference/architecture/authorization-service-core-server-and-tenant/authorization-service-core-server-and-tenant.md b/docs/reference/architecture/authorization-service-core-server-and-tenant/authorization-service-core-server-and-tenant.md deleted file mode 100644 index 5fbba1505..000000000 --- a/docs/reference/architecture/authorization-service-core-server-and-tenant/authorization-service-core-server-and-tenant.md +++ /dev/null @@ -1,405 +0,0 @@ -# Authorization Service Core Server And Tenant - -## Overview - -The **Authorization Service Core Server And Tenant** module is the heart of OpenFrame's multi-tenant identity and OAuth2 infrastructure. It implements a fully multi-tenant OAuth2 Authorization Server using Spring Authorization Server, with per-tenant key material, dynamic client registration, and tenant-aware authentication flows. - -This module is responsible for: - -- Acting as the OAuth2 / OIDC Authorization Server -- Issuing tenant-scoped JWT access tokens and refresh tokens -- Managing per-tenant signing keys (JWKS) -- Resolving tenant context for every request -- Integrating with SSO and dynamic OAuth2 clients -- Enforcing tenant isolation at the authentication and token layers - -It works closely with: - -- Data Mongo Domain Model (users, OAuth clients, tokens) -- Security Core And OAuth BFF (shared security constants and utilities) -- Gateway Service Core Security And Routing (JWT validation and issuer routing) -- Authorization Service Core SSO Flow And Utils (SSO login, provider strategies) - ---- - -## High-Level Architecture - -The module is composed of five core building blocks: - -- Authorization Server Configuration -- Default Security Configuration -- Dynamic Client Registration -- Tenant Context Management -- Tenant-Aware Key Infrastructure - -```mermaid -flowchart TD - Client["Browser or API Client"] --> Gateway["Gateway Service"] - Gateway --> Authz["Authorization Service Core Server And Tenant"] - - subgraph authz_core["Authorization Layer"] - TenantFilter["TenantContextFilter"] - AuthServerConfig["AuthorizationServerConfig"] - SecurityConfigBean["SecurityConfig"] - DynamicClientRepo["DynamicClientRegistrationRepository"] - TenantKeySvc["TenantKeyService"] - end - - Authz --> TenantFilter - TenantFilter --> AuthServerConfig - TenantFilter --> SecurityConfigBean - AuthServerConfig --> TenantKeySvc - SecurityConfigBean --> DynamicClientRepo - - AuthServerConfig --> JWKS["Per-Tenant JWKS"] - AuthServerConfig --> JWT["Signed JWT Tokens"] -``` - ---- - -## Multi-Tenant Design Principles - -The Authorization Service Core Server And Tenant module is built around strict tenant isolation: - -1. Every request must resolve a tenant identifier. -2. Every JWT contains a `tenant_id` claim. -3. Every tenant has its own RSA key pair. -4. OAuth2 client registrations are resolved dynamically per tenant. -5. User authentication always queries tenant-scoped user records. - -Tenant identity flows through the entire request lifecycle using a thread-local context. - ---- - -## Tenant Context Management - -### TenantContext - -Component: -- `openframe-oss-lib.openframe-authorization-service-core.src.main.java.com.openframe.authz.config.tenant.TenantContext.TenantContext` - -A lightweight `ThreadLocal` holder storing the current tenant ID for the duration of a request. - -Responsibilities: - -- Store tenant ID per thread -- Provide `setTenantId`, `getTenantId`, and `clear` -- Ensure isolation between concurrent requests - -```mermaid -flowchart LR - Request["Incoming HTTP Request"] --> Filter["TenantContextFilter"] - Filter --> ThreadLocal["TenantContext ThreadLocal"] - ThreadLocal --> Services["UserService and Key Services"] -``` - ---- - -### TenantContextFilter - -Component: -- `openframe-oss-lib.openframe-authorization-service-core.src.main.java.com.openframe.authz.config.tenant.TenantContextFilter.TenantContextFilter` - -This filter executes early in the filter chain and resolves the tenant ID using: - -- URL path prefix (for example `/tenantId/oauth2/authorize`) -- Query parameter `tenant` -- Existing HTTP session attribute - -Key behaviors: - -- Stores tenant ID in both session and `TenantContext` -- Prevents unsafe cross-tenant session reuse -- Allows special onboarding tenant switching -- Clears context after request completion - -Tenant resolution directly influences: - -- JWT signing key selection -- OAuth2 client registration -- User lookup and authentication - ---- - -## Authorization Server Configuration - -### AuthorizationServerConfig - -Component: -- `openframe-oss-lib.openframe-authorization-service-core.src.main.java.com.openframe.authz.config.AuthorizationServerConfig.AuthorizationServerConfig` - -This class configures Spring Authorization Server with multi-issuer support. - -### Key Responsibilities - -- Enables OIDC support -- Allows multiple issuers (`multipleIssuersAllowed(true)`) -- Configures OAuth2 endpoints -- Sets up JWT encoder and decoder -- Defines token customization logic -- Registers tenant-aware JWK source - -### Security Filter Chain (Order 1) - -```mermaid -flowchart TD - Request["OAuth2 Endpoint Request"] --> Match["Authorization Endpoints Matcher"] - Match --> Authenticated["Require Authentication"] - Authenticated --> JWTValidation["JWT Resource Server Enabled"] - JWTValidation --> Response["OAuth2 Response"] -``` - -This chain applies only to Authorization Server endpoints. - ---- - -## Per-Tenant Key Infrastructure - -### JWKSource and TenantKeyService Integration - -The `jwkSource` bean dynamically selects the RSA key based on the current tenant: - -```mermaid -flowchart LR - JWTRequest["JWKS or Token Signing"] --> TenantId["TenantContext.getTenantId()"] - TenantId --> KeyService["TenantKeyService.getOrCreateActiveKey"] - KeyService --> RSAKey["Tenant RSAKey"] - RSAKey --> JWKSet["JWKSet Returned"] -``` - -Important properties: - -- Each tenant has its own active RSA key -- Key ID (`kid`) is logged and exposed via JWKS -- If tenant ID is missing, the request fails - -This guarantees: - -- Cryptographic tenant isolation -- Independent key rotation per tenant -- Safe multi-issuer JWT validation - ---- - -## JWT Token Customization - -The module injects tenant and user metadata into access tokens. - -### Custom Claims Added - -When issuing an `access_token`, the following claims are added: - -- `tenant_id` -- `userId` -- `roles` - -Role logic: - -- If user has `OWNER`, `ADMIN` is implicitly included -- Roles are emitted as string names - -```mermaid -flowchart TD - Auth["Authenticated Principal"] --> Lookup["UserService.findActiveByEmailAndTenant"] - Lookup --> Claims["Add Claims to JWT"] - Claims --> Token["Signed Access Token"] -``` - -Additionally: - -- Refresh token grants update `lastLogin` -- Claims are derived from tenant-scoped `AuthUser` - -This ensures that downstream services (Gateway, API Service) can enforce tenant and role-based authorization purely from JWT claims. - ---- - -## User Authentication Integration - -### UserDetailsService - -Provides tenant-aware authentication: - -- Looks up users by email + tenant ID -- Converts roles into `ROLE_*` authorities -- Uses BCrypt for password hashing - -### AuthenticationManager - -Uses a `DaoAuthenticationProvider` wired with: - -- Tenant-aware `UserDetailsService` -- BCrypt `PasswordEncoder` - -This enables: - -- Form login -- Programmatic authentication -- Password-based fallback for SSO users - ---- - -## Default Security Configuration - -### SecurityConfig - -Component: -- `openframe-oss-lib.openframe-authorization-service-core.src.main.java.com.openframe.authz.config.SecurityConfig.SecurityConfig` - -This filter chain handles **non-Authorization Server endpoints** (Order 2). - -Responsibilities: - -- Form login configuration -- OAuth2 login configuration -- SSO failure handling -- Auto-provisioning of users -- Microsoft multi-tenant issuer validation - ---- - -### OAuth2 Login and SSO Integration - -Key integrations: - -- `SsoAuthorizationRequestResolver` -- `AuthSuccessHandler` -- Custom `JwtDecoderFactory` -- Auto-provisioning logic - -```mermaid -flowchart TD - User["User Clicks SSO"] --> OAuthLogin["oauth2Login()"] - OAuthLogin --> Resolver["SsoAuthorizationRequestResolver"] - Resolver --> Provider["External IdP"] - Provider --> Callback["OIDC Callback"] - Callback --> OidcService["Custom OidcUserService"] - OidcService --> AutoProvision["Auto Provision If Needed"] - AutoProvision --> Success["AuthSuccessHandler"] -``` - ---- - -## Auto-Provisioning Logic - -When a user logs in via OIDC: - -1. Resolve tenant from context -2. Extract email and provider -3. Load per-tenant SSO configuration -4. Validate allowed domains -5. Register or reactivate user if necessary -6. Assign ADMIN role during SSO registration - -Safeguards: - -- Does not block login if provisioning fails -- Honors global domain policy fallback -- Avoids duplicate image sync - -This tightly integrates identity provider flows with OpenFrame's tenant-based user model. - ---- - -## Dynamic Client Registration - -### DynamicClientRegistrationRepository - -Component: -- `openframe-oss-lib.openframe-authorization-service-core.src.main.java.com.openframe.authz.config.DynamicClientRegistrationRepository.DynamicClientRegistrationRepository` - -Implements `ClientRegistrationRepository` dynamically per tenant. - -Behavior: - -- Resolves tenant ID from: - - `TenantContext` - - HTTP session -- Delegates to `DynamicClientRegistrationService` -- Returns null if tenant is unresolved - -```mermaid -flowchart LR - OAuthFlow["OAuth2 Login Request"] --> Repo["DynamicClientRegistrationRepository"] - Repo --> TenantCtx["TenantContext"] - TenantCtx --> DynamicSvc["DynamicClientRegistrationService"] - DynamicSvc --> ClientReg["ClientRegistration"] -``` - -This allows: - -- Per-tenant SSO provider configuration -- Dynamic Google and Microsoft client resolution -- Runtime onboarding of new tenants - ---- - -## Multi-Issuer and Gateway Compatibility - -The Authorization Service Core Server And Tenant module is designed for compatibility with the Gateway Service: - -- Each tenant may have its own issuer URL -- JWT includes `tenant_id` -- JWKS endpoint is tenant-aware -- Gateway validates JWT using issuer-specific configuration - -This ensures horizontal scalability and safe routing across multiple tenants. - ---- - -## Security Characteristics - -- Per-tenant RSA key pairs -- BCrypt password hashing -- OIDC ID token validation -- Microsoft multi-tenant issuer pattern validation -- Session invalidation on unsafe tenant switch -- Strict JWT claim injection -- CSRF disabled for OAuth endpoints only - ---- - -## Request Lifecycle Summary - -```mermaid -flowchart TD - Step1["Incoming Request"] --> Step2["TenantContextFilter Resolves Tenant"] - Step2 --> Step3{{"OAuth2 Endpoint?"}} - Step3 -->|"Yes"| Step4["AuthorizationServerConfig"] - Step3 -->|"No"| Step5["SecurityConfig"] - Step4 --> Step6["JWT Issued with Tenant Claims"] - Step5 --> Step7["Authenticated Session or SSO"] - Step6 --> EndNode["Response"] - Step7 --> EndNode -``` - ---- - -## How This Module Fits Into OpenFrame - -Within the broader OpenFrame architecture: - -- It is the authoritative identity provider. -- It enforces tenant boundaries at authentication time. -- It supplies signed JWTs used by: - - API Service Core - - Gateway Service Core - - External API Service Core -- It integrates with Mongo repositories for user and OAuth persistence. - -Without this module, the multi-tenant trust boundary would not exist. - ---- - -## Conclusion - -The **Authorization Service Core Server And Tenant** module provides: - -- Multi-tenant OAuth2 Authorization Server -- Tenant-aware JWT issuance -- Per-tenant RSA key management -- Dynamic client registration -- SSO auto-provisioning -- Strict tenant isolation guarantees - -It forms the cryptographic and identity backbone of OpenFrame, enabling secure, scalable, and tenant-isolated authentication across the entire platform. \ No newline at end of file diff --git a/docs/reference/architecture/authorization-service-core-sso-flow-and-utils/authorization-service-core-sso-flow-and-utils.md b/docs/reference/architecture/authorization-service-core-sso-flow-and-utils/authorization-service-core-sso-flow-and-utils.md deleted file mode 100644 index e5c3b06e0..000000000 --- a/docs/reference/architecture/authorization-service-core-sso-flow-and-utils/authorization-service-core-sso-flow-and-utils.md +++ /dev/null @@ -1,359 +0,0 @@ -# Authorization Service Core Sso Flow And Utils - -## Overview - -The **Authorization Service Core Sso Flow And Utils** module encapsulates the Single Sign-On (SSO) flow orchestration, post-authentication handling, provider registration strategies, extensibility processors, and supporting web/security utilities for the OpenFrame Authorization Server. - -It sits on top of the core authorization server and tenant infrastructure and focuses on: - -- Handling OAuth2/OIDC authentication success -- Driving SSO-based tenant registration and invitation flows -- Dynamically configuring Google and Microsoft OIDC providers -- Providing extensibility hooks for registration and user lifecycle events -- Offering security and redirect utilities for clean session handling - -This module is a critical bridge between: - -- The **Authorization Server Core** (multi-tenant, OAuth2, client registration) -- The **User and Tenant Services** -- External OIDC providers (Google, Microsoft) - ---- - -## Architectural Context - -At a high level, the module enhances the OAuth2 login pipeline and injects SSO-specific behaviors. - -```mermaid -flowchart TD - Browser["Browser"] -->|"OAuth2 Login"| AuthServer["Authorization Server"] - AuthServer -->|"Authentication Success"| AuthSuccessHandler["Auth Success Handler"] - - AuthSuccessHandler -->|"Delegate"| SsoFlowHandlers["SSO Flow Handlers"] - AuthSuccessHandler -->|"Update User"| UserService["User Service"] - - SsoFlowHandlers --> InviteHandler["Invite SSO Handler"] - SsoFlowHandlers --> TenantRegHandler["Tenant Registration SSO Handler"] - - InviteHandler --> InvitationService["Invitation Registration Service"] - TenantRegHandler --> TenantRegService["Tenant Registration Service"] - - AuthSuccessHandler --> RedirectsUtil["Redirect Utilities"] - AuthSuccessHandler --> AuthStateUtils["Auth State Utilities"] -``` - -The flow can be summarized as: - -1. User authenticates via OAuth2 (Google or Microsoft). -2. `AuthSuccessHandler` updates user metadata and delegates to SSO flow handlers. -3. Based on SSO cookies, the appropriate handler executes: - - Invitation-based onboarding - - Tenant self-registration -4. Registration services create or update tenant/user entities. -5. Cookies are cleared and user is redirected to the correct tenant context. - ---- - -## Authentication Success Orchestration - -### AuthSuccessHandler - -**Responsibility:** Central post-authentication entry point. - -Key behaviors: - -- Extracts tenant ID from `TenantContext` -- Resolves email from `Authentication` (OIDC or standard login) -- Updates `lastLogin` via `UserService` -- Optionally marks email as verified for trusted providers -- Delegates to SSO tenant registration success handler - -### Email Resolution Strategy - -Email resolution is provider-aware and handled via `OidcUserUtils`: - -```mermaid -flowchart TD - OidcUser["OIDC User Claims"] --> Email["email"] - OidcUser --> Preferred["preferred_username"] - OidcUser --> Upn["upn"] - OidcUser --> Unique["unique_name"] - - Email --> Resolved["Resolved Email"] - Preferred --> Resolved - Upn --> Resolved - Unique --> Resolved -``` - -Order of precedence: - -1. `email` -2. `preferred_username` -3. `upn` -4. `unique_name` - -This ensures compatibility with Google and Azure AD organizational accounts. - ---- - -## SSO Flow Handling - -The module supports two primary SSO-driven flows using cookie-based state management. - -### SsoRegistrationConstants - -Defines cookie names and special onboarding tenant identifiers: - -- `of_sso_reg` β†’ Tenant self-registration flow -- `of_sso_invite` β†’ Invitation-based onboarding -- `sso-onboarding` β†’ Temporary tenant context - ---- - -### InviteSsoHandler - -**Use Case:** User accepts an invitation and authenticates via SSO. - -Flow: - -```mermaid -flowchart TD - Cookie["Invite Cookie"] --> Decode["Decode Invite Payload"] - Oidc["OIDC User"] --> Extract["Extract Names and Email"] - Decode --> BuildReq["Build InvitationRegistrationRequest"] - Extract --> BuildReq - BuildReq --> Register["InvitationRegistrationService.registerByInvitation"] - Register --> Redirect["Clear Cookie and Redirect"] -``` - -Key characteristics: - -- Validates SSO session via `SsoCookieCodec` -- Generates a random password (UUID-based) -- Uses provider picture (if available) -- Optionally switches tenant -- Clears SSO cookie while preserving OAuth session continuity - ---- - -### TenantRegSsoHandler - -**Use Case:** New tenant self-registration via SSO. - -Flow: - -```mermaid -flowchart TD - Cookie["Registration Cookie"] --> Decode["Decode Tenant Payload"] - Oidc["OIDC User"] --> Email["Resolve Email"] - Decode --> Validate["Validate Tenant Name and Domain"] - Email --> BuildReq["Build TenantRegistrationRequest"] - Validate --> BuildReq - BuildReq --> Register["TenantRegistrationService.registerTenant"] - Register --> Redirect["Clear Cookie and Redirect"] -``` - -Key behaviors: - -- Requires tenant name and domain -- Normalizes domain to lowercase -- Generates secure random password -- Associates authenticated email with tenant - ---- - -## OIDC Provider Strategies - -The module uses a strategy pattern for dynamic client registration. - -### GoogleClientRegistrationStrategy - -- Provider ID: `google` -- Uses `GoogleSSOProperties` -- Extends base OIDC registration strategy -- Integrates with `SSOConfigService` - -### MicrosoftClientRegistrationStrategy - -- Provider ID: `microsoft` -- Uses `MicrosoftSSOProperties` -- Supports Azure AD OIDC configuration - -```mermaid -flowchart LR - SSOConfigService["SSO Config Service"] --> GoogleStrategy["Google Client Registration Strategy"] - SSOConfigService --> MicrosoftStrategy["Microsoft Client Registration Strategy"] - - GoogleStrategy --> GoogleProps["Google SSO Properties"] - MicrosoftStrategy --> MicrosoftProps["Microsoft SSO Properties"] -``` - -This design enables: - -- Per-tenant provider configuration -- Default provider fallbacks -- Clean extension for additional IdPs - ---- - -## Default Provider Configuration - -### GoogleDefaultProviderConfig -### MicrosoftDefaultProviderConfig - -Provide default client ID and secret values when no tenant-specific configuration exists. - -This ensures: - -- Zero-config local development -- Sensible fallback for multi-tenant environments - ---- - -## Domain Policy Lookup - -### NoopGlobalDomainPolicyLookup - -Fallback implementation for domain-based auto-provisioning. - -- Returns empty result by default -- Activated only if no custom `GlobalDomainPolicyLookup` bean exists -- Enables domain-to-tenant auto-binding customization - -This is a key extensibility point for enterprise auto-provisioning. - ---- - -## Registration and User Lifecycle Processors - -The module defines multiple processor extension points with default no-op implementations. - -### DefaultRegistrationProcessor - -Hooks: - -- `preProcessTenantRegistration` -- `postProcessTenantRegistration` -- `postProcessInvitationRegistration` -- `postProcessAutoProvision` - -Allows custom logic such as: - -- Audit logging -- License provisioning -- External system synchronization - -### DefaultUserDeactivationProcessor - -Triggered after user deactivation. - -### DefaultUserEmailVerifiedProcessor - -Triggered when user email is marked verified. - -These processors follow a conditional bean pattern: - -- If no custom bean exists β†’ default implementation is used -- If custom bean exists β†’ default is ignored - -This enables safe override without modifying core logic. - ---- - -## Security and Web Utilities - -### OidcUserUtils - -Provides claim resolution helpers: - -- Email resolution across providers -- Picture URL extraction -- Safe string claim parsing - -Ensures provider inconsistencies do not leak into business logic. - ---- - -### ResetTokenUtil - -Generates secure password reset tokens: - -- 32 random bytes -- URL-safe Base64 encoding without padding -- Uses `SecureRandom` - -This provides cryptographically strong reset tokens. - ---- - -### AuthStateUtils - -Handles authentication state cleanup: - -- Invalidates session -- Clears `JSESSIONID` -- Removes cookies at root and context path - -Used during logout or auth reset scenarios. - ---- - -### Redirects - -Utility for safe HTTP redirects: - -- 302 Found -- 303 See Other -- Root-based redirect support -- Context-aware URL construction - -Ensures: - -- Clean redirect semantics -- No manual URL concatenation -- Context-path safety - ---- - -## Complete SSO Registration Sequence - -```mermaid -sequenceDiagram - participant Browser - participant AuthServer as "Authorization Server" - participant Success as "Auth Success Handler" - participant Flow as "SSO Flow Handler" - participant Service as "Registration Service" - - Browser->>AuthServer: OAuth2 Login (Google or Microsoft) - AuthServer->>Success: onAuthenticationSuccess - Success->>Success: Update lastLogin - Success->>Flow: Delegate to flow handler - Flow->>Service: Register user or tenant - Service->>Flow: Return tenant context - Flow->>Browser: Redirect to target tenant -``` - ---- - -## Key Design Principles - -1. **Separation of concerns** – Authentication success handling is isolated from registration logic. -2. **Cookie-driven flow state** – SSO flow intent is persisted safely across OAuth redirects. -3. **Strategy pattern for IdPs** – Enables clean addition of new providers. -4. **Extensibility via processors** – Business-specific logic can be injected without modifying core. -5. **Provider-agnostic claim resolution** – Normalizes Google and Microsoft differences. -6. **Secure defaults** – Strong random token generation and strict cookie clearing. - ---- - -## Summary - -The **Authorization Service Core Sso Flow And Utils** module is the orchestration layer that transforms raw OAuth2/OIDC authentication into: - -- Tenant-aware onboarding -- Invitation-driven account activation -- Secure and extensible registration flows -- Clean post-authentication lifecycle updates - -It acts as the glue between authentication, tenant management, and user lifecycle processing, ensuring a secure, extensible, and multi-tenant-ready SSO experience across OpenFrame. \ No newline at end of file diff --git a/docs/reference/architecture/authorization-service-core/authorization-service-core.md b/docs/reference/architecture/authorization-service-core/authorization-service-core.md new file mode 100644 index 000000000..2348854f4 --- /dev/null +++ b/docs/reference/architecture/authorization-service-core/authorization-service-core.md @@ -0,0 +1,340 @@ +# Authorization Service Core + +The Authorization Service Core module is the central identity and access management (IAM) component of the OpenFrame platform. It provides: + +- OAuth2 Authorization Server (Spring Authorization Server) +- OpenID Connect (OIDC) support +- Multi-tenant authentication and issuer isolation +- Dynamic SSO provider registration (Google, Microsoft) +- Tenant onboarding and invitation flows +- JWT issuance with tenant-scoped signing keys +- Persistent authorization storage (MongoDB) + +It acts as the trust anchor for all other services such as the Gateway Service Core and API services, issuing and validating tokens that secure platform APIs. + +--- + +## 1. High-Level Architecture + +The Authorization Service Core sits between end users (browser / SPA), external identity providers, and internal OpenFrame services. + +```mermaid +flowchart LR + User["User Browser"] -->|"Login / OAuth2"| AuthServer["Authorization Service Core"] + AuthServer -->|"OIDC Redirect"| Google["Google OIDC"] + AuthServer -->|"OIDC Redirect"| Microsoft["Microsoft OIDC"] + AuthServer -->|"JWT Access Token"| Gateway["Gateway Service Core"] + Gateway -->|"Forward Request"| ApiService["API Service Core"] + AuthServer -->|"Persist Authorization"| Mongo[("MongoDB")] +``` + +Core responsibilities: + +- Issue JWT access and refresh tokens +- Expose `.well-known` OIDC metadata +- Manage per-tenant signing keys +- Resolve tenant context from request path/session +- Support SSO-based onboarding and invitation acceptance +- Persist OAuth2 authorizations and registered clients + +--- + +## 2. Multi-Tenancy Model + +Multi-tenancy is enforced at the authentication and token layer. + +### 2.1 Tenant Context Resolution + +`TenantContextFilter` extracts the tenant identifier from: + +- URL path segment (e.g. `/tenantA/oauth2/authorize`) +- Query parameter (`tenant=`) +- HTTP session + +It stores the tenant in a `ThreadLocal` via `TenantContext`. + +```mermaid +flowchart TD + Request["Incoming HTTP Request"] --> Filter["TenantContextFilter"] + Filter -->|"Extract tenantId"| Context["TenantContext (ThreadLocal)"] + Context --> AuthLogic["Auth & Token Logic"] + AuthLogic --> Clear["Clear Context After Request"] +``` + +Key classes: + +- `TenantContext` +- `TenantContextFilter` + +This design ensures that: + +- Every JWT is issued with the correct tenant context. +- Signing keys and user lookups are tenant-scoped. + +--- + +## 3. Authorization Server Configuration + +The `AuthorizationServerConfig` configures Spring Authorization Server with: + +- Multiple issuers enabled (`multipleIssuersAllowed(true)`) +- OIDC support +- Custom JWT encoder/decoder +- Token customization +- Per-tenant JWK resolution + +### 3.1 Per-Tenant JWK Source + +JWT signing keys are resolved dynamically using `TenantKeyService`. + +```mermaid +flowchart TD + TokenRequest["Token Request"] --> JwkSource["JWKSource"] + JwkSource -->|"Resolve tenantId"| TenantKeyService + TenantKeyService -->|"Load or Create Key"| MongoKeys[("TenantKey Collection")] + TenantKeyService --> RSAKey["RSAKey (with kid)"] + RSAKey --> JwtEncoder["NimbusJwtEncoder"] +``` + +If no active key exists for a tenant: + +- `RsaAuthenticationKeyPairGenerator` generates a new RSA key pair. +- The private key is encrypted. +- The public key is stored for JWKS exposure. + +Key classes: + +- `TenantKeyService` +- `RsaAuthenticationKeyPairGenerator` +- `PemUtil` + +--- + +## 4. JWT Customization + +`OAuth2TokenCustomizer` enriches access tokens with: + +- `tenant_id` +- `userId` +- `roles` + +```mermaid +flowchart LR + Auth["Authenticated Principal"] --> Customizer["Token Customizer"] + Customizer --> Claims["Add Claims"] + Claims -->|"tenant_id"| Jwt["JWT Access Token"] + Claims -->|"roles"| Jwt +``` + +Special logic: + +- OWNER role implicitly includes ADMIN. +- On refresh token flow, `lastLogin` is updated. + +This ensures downstream services can enforce RBAC using token claims without additional DB lookups. + +--- + +## 5. Security Configuration (Non-Authorization Endpoints) + +`SecurityConfig` handles: + +- Form login (`/login`) +- OAuth2 login (Google, Microsoft) +- SSO auto-provisioning +- Microsoft-specific issuer validation + +### 5.1 OAuth2 Login Flow + +```mermaid +sequenceDiagram + participant Browser + participant AuthServer as "Authorization Service" + participant IdP as "OIDC Provider" + + Browser->>AuthServer: GET /oauth2/authorization/{provider} + AuthServer->>IdP: Redirect with state + PKCE + IdP-->>AuthServer: Authorization code + AuthServer->>AuthServer: Exchange code for tokens + AuthServer->>AuthServer: Auto-provision user if enabled + AuthServer-->>Browser: Redirect to success URL +``` + +Features: + +- Microsoft issuer pattern validation +- Dynamic client resolution via `DynamicClientRegistrationRepository` +- Auto-provision users based on tenant SSO configuration +- Domain-based tenant mapping via `GlobalDomainPolicyLookup` + +--- + +## 6. SSO & Onboarding Flows + +The module supports two major SSO-driven flows: + +1. Invitation acceptance via SSO +2. Tenant registration via SSO + +### 6.1 Invitation Acceptance via SSO + +Controller: `InvitationRegistrationController` +Handler: `InviteSsoHandler` + +Flow: + +```mermaid +flowchart TD + InviteLink["Invitation Link"] --> Start["Start SSO Accept"] + Start --> Cookie["Set SSO Invite Cookie"] + Cookie --> IdP["Redirect to IdP"] + IdP --> Callback["SSO Success"] + Callback --> Register["Register User By Invitation"] + Register --> Redirect["Redirect To Target Tenant"] +``` + +Key components: + +- `SsoCookieCodec` +- `SsoRegistrationConstants` +- `InvitationRegistrationService` + +### 6.2 Tenant Registration via SSO + +Controller: `TenantRegistrationController` +Handler: `TenantRegSsoHandler` + +Flow: + +```mermaid +flowchart TD + StartReg["Start Tenant SSO Registration"] --> CookieReg["Set SSO Reg Cookie"] + CookieReg --> IdPReg["Redirect to IdP"] + IdPReg --> CallbackReg["SSO Success"] + CallbackReg --> CreateTenant["Register Tenant + Owner"] + CreateTenant --> RedirectTenant["Redirect to New Tenant Context"] +``` + +The onboarding pseudo-tenant (`sso-onboarding`) allows seamless transition to the newly created tenant without losing authentication context. + +--- + +## 7. OAuth2 Persistence Layer + +### 7.1 Authorization Persistence + +`MongoAuthorizationService` stores: + +- Authorization codes +- Access tokens +- Refresh tokens +- PKCE parameters +- Authorization request metadata + +```mermaid +flowchart LR + OAuthAuth["OAuth2Authorization"] --> Mapper["MongoAuthorizationMapper"] + Mapper --> MongoEntity["MongoOAuth2Authorization"] + MongoEntity --> Mongo[("MongoDB")] +``` + +PKCE parameters are carefully preserved and restored during: + +- Code issuance +- Token exchange +- Refresh token usage + +### 7.2 Registered Clients + +`MongoRegisteredClientRepository` stores OAuth clients with: + +- Grant types +- Redirect URIs +- Scopes +- Token TTL settings +- PKCE enforcement flags + +--- + +## 8. Password Reset & Email Verification + +### Password Reset + +Controller: `PasswordResetController` +Utility: `ResetTokenUtil` + +- Generates secure base64 URL-safe tokens +- Enforces strong password pattern + +### Email Verification via SSO + +`AuthSuccessHandler`: + +- Updates `lastLogin` +- Marks email as verified for trusted providers (Google, Microsoft) + +Extensible processors: + +- `DefaultUserEmailVerifiedProcessor` +- `DefaultUserDeactivationProcessor` + +--- + +## 9. Integration with Other Modules + +The Authorization Service Core is tightly integrated with: + +- [Gateway Service Core](../gateway-service-core/gateway-service-core.md) + - Validates JWTs + - Enforces route-level security + +- [Security OAuth and JWT](../security-oauth-and-jwt/security-oauth-and-jwt.md) + - Shared JWT and PKCE utilities + +- [Data Model and Repositories Mongo](../data-model-and-repositories-mongo/data-model-and-repositories-mongo.md) + - Persists users, tenants, keys, and authorizations + +It acts as the identity provider for: + +- API Service Core +- External API Service Core +- Management Service Core + +--- + +## 10. Extension Points + +The module is designed for extensibility: + +- `RegistrationProcessor` (pre/post registration hooks) +- `UserDeactivationProcessor` +- `UserEmailVerifiedProcessor` +- `GlobalDomainPolicyLookup` +- Default provider configs (`GoogleDefaultProviderConfig`, `MicrosoftDefaultProviderConfig`) + +Custom implementations can override default no-op beans via Spring conditional configuration. + +--- + +## 11. Security Highlights + +- Per-tenant signing keys (isolated JWKS) +- Encrypted private key storage +- PKCE enforcement support +- Microsoft multi-tenant issuer validation +- Strict password policy enforcement +- Session isolation when tenant changes + +--- + +# Summary + +The Authorization Service Core module provides a fully multi-tenant, extensible OAuth2 + OIDC authorization server tailored for OpenFrame. It: + +- Isolates tenants cryptographically and logically +- Supports dynamic SSO providers +- Enables seamless onboarding and invitation flows +- Persists complete OAuth2 state in MongoDB +- Issues enriched JWTs for downstream microservices + +It forms the identity backbone of the OpenFrame platform and underpins all authenticated communication across services. diff --git a/docs/reference/architecture/client-core-agent-ingress/client-core-agent-ingress.md b/docs/reference/architecture/client-core-agent-ingress/client-core-agent-ingress.md new file mode 100644 index 000000000..67c24f304 --- /dev/null +++ b/docs/reference/architecture/client-core-agent-ingress/client-core-agent-ingress.md @@ -0,0 +1,424 @@ +# Client Core Agent Ingress + +## Overview + +The **Client Core Agent Ingress** module is the primary entry point for endpoint agents and tool agents into the OpenFrame platform. It is responsible for: + +- Agent registration and reinstallation +- OAuth-based client token issuance +- Tool agent binary delivery (temporary stub implementation) +- Processing real-time machine lifecycle and tool events from NATS +- Transforming external tool-specific identifiers into OpenFrame-compatible identifiers +- Asynchronous processing of tool installations + +This module acts as the **ingress boundary between edge devices (agents) and the OpenFrame backend**, translating HTTP and messaging inputs into domain-level operations. + +--- + +## High-Level Responsibilities + +1. **Agent Onboarding** – Register and reinstall endpoint agents. +2. **Authentication** – Issue OAuth tokens for agents. +3. **Machine State Tracking** – Process heartbeats and connection events. +4. **Tool Integration Mapping** – Normalize external tool agent identifiers. +5. **Event-Driven Updates** – Consume NATS JetStream and Core NATS subjects. +6. **Async Execution** – Run installation-related tasks using virtual threads. + +--- + +## Architecture Overview + +```mermaid +flowchart TD + Agent["Endpoint Agent"] -->|"POST /api/agents/register"| AgentController["Agent Controller"] + Agent -->|"POST /oauth/token"| AuthController["Agent Auth Controller"] + + AgentController --> RegistrationService["Agent Registration Service"] + RegistrationService --> RegistrationProcessor["Agent Registration Processor"] + + Agent -->|"Heartbeat (NATS)"| HeartbeatListener["Machine Heartbeat Listener"] + Agent -->|"Installed Agent (JetStream)"| InstalledListener["Installed Agent Listener"] + Agent -->|"Tool Connection (JetStream)"| ToolListener["Tool Connection Listener"] + + HeartbeatListener --> MachineStatusService["Machine Status Service"] + InstalledListener --> InstalledAgentService["Installed Agent Service"] + ToolListener --> ToolConnectionService["Tool Connection Service"] + + ToolListener --> Transformer["Tool Agent ID Transformers"] +``` + +The module exposes HTTP endpoints and subscribes to messaging streams. It bridges incoming traffic into internal services and data models. + +--- + +# Configuration Layer + +## Async Configuration + +**Component:** `AsyncConfig` + +- Enables Spring asynchronous execution. +- Defines `toolInstallExecutor` using `Executors.newVirtualThreadPerTaskExecutor()`. + +### Why Virtual Threads? + +- Lightweight concurrency +- High parallelism for tool installation tasks +- Reduced thread contention under heavy load + +```mermaid +flowchart LR + Task["Tool Install Task"] --> Executor["Virtual Thread Executor"] --> Worker["Virtual Thread"] +``` + +--- + +## Password Encoding + +**Component:** `PasswordEncoderConfig` + +- Provides a `BCryptPasswordEncoder` bean. +- Used for secure credential handling (e.g., client secrets). + +--- + +# HTTP API Layer + +## 1. Agent Authentication + +**Controller:** `AgentAuthController` + +### Endpoint + +`POST /oauth/token` + +### Supported Inputs + +- `grant_type` +- `refresh_token` +- `client_id` +- `client_secret` + +### Behavior + +- Delegates to `AgentAuthService.issueClientToken(...)` +- Returns: + - 200 with `AgentTokenResponse` + - 401 for invalid credentials + - 400 for server errors + +```mermaid +sequenceDiagram + participant Agent + participant AuthController as "Agent Auth Controller" + participant AuthService as "Agent Auth Service" + + Agent->>AuthController: POST /oauth/token + AuthController->>AuthService: issueClientToken() + AuthService-->>AuthController: AgentTokenResponse + AuthController-->>Agent: 200 OK +``` + +--- + +## 2. Agent Registration + +**Controller:** `AgentController` + +### Endpoints + +- `POST /api/agents/register` +- `POST /api/agents/reinstall` + +### Required Headers + +- `X-Initial-Key` +- `X-Machine-Id` (reinstall only) +- `X-Client-Secret` (reinstall only) + +### Request Body + +`AgentRegistrationRequest` + +Includes: + +- Core identity (hostname, organizationId) +- Network info (IP, MAC, UUID) +- OS and hardware metadata +- Tags (`AgentRegistrationTagInput`) + +### Tag Auto-Creation + +If a tag key does not exist in the organization, it is auto-created as a CUSTOM tag. + +```mermaid +flowchart TD + Request["AgentRegistrationRequest"] --> Validate["Validation"] + Validate --> CreateMachine["Create or Update Machine"] + CreateMachine --> AssignTags["Assign or Create Tags"] + AssignTags --> PostProcess["Post Registration Processor"] +``` + +--- + +## 3. Tool Agent File Delivery (Temporary) + +**Controller:** `ToolAgentFileController` + +Endpoint: + +`GET /tool-agent/{assetId}?os=mac|windows` + +Current behavior: + +- Returns static classpath resources. +- Throws error for unsupported OS or missing assets. +- Intended to be replaced by GitHub artifact-based distribution. + +--- + +# DTO Layer + +## AgentRegistrationRequest + +Defines all metadata required to register a machine. + +Key fields: + +- `hostname` +- `organizationId` +- `agentVersion` +- `DeviceType` +- `DeviceStatus` +- `tags` + +Ensures strong validation with `@NotBlank` and nested `@Valid`. + +--- + +## AgentRegistrationTagInput + +Used during registration to: + +- Create tag if missing +- Assign multiple values + +--- + +## CreateClientRequest + +Defines OAuth client creation properties: + +- `grantTypes` +- `scopes` + +--- + +## MetricsMessage + +Represents machine metrics: + +- `machineId` +- `cpu` +- `memory` +- `timestamp` + +Used for telemetry ingestion pipelines. + +--- + +# Event-Driven Architecture + +The module subscribes to NATS subjects and JetStream streams. + +```mermaid +flowchart TD + NATS["NATS / JetStream"] --> Heartbeat["Machine Heartbeat Listener"] + NATS --> Installed["Installed Agent Listener"] + NATS --> ToolConn["Tool Connection Listener"] + + Heartbeat --> MachineStatus["Machine Status Service"] + Installed --> InstalledService["Installed Agent Service"] + ToolConn --> ToolService["Tool Connection Service"] +``` + +--- + +## 1. MachineHeartbeatListener + +Subject: + +`machine.*.heartbeat` + +Behavior: + +- Extracts machineId from subject +- Generates server-side timestamp +- Calls `MachineStatusService.processHeartbeat(...)` + +Characteristics: + +- Uses NATS dispatcher threads +- Graceful drain on shutdown + +--- + +## 2. ClientConnectionListener + +Handles: + +- Machine connected +- Machine disconnected + +Updates machine online/offline state via `MachineStatusService`. + +--- + +## 3. InstalledAgentListener (JetStream) + +Stream: `INSTALLED_AGENTS` + +Subject pattern: + +`machine.*.installed-agent` + +Features: + +- Durable consumer +- Explicit ACK policy +- Redelivery up to 50 attempts +- Last-attempt detection + +Processing flow: + +```mermaid +flowchart TD + Message["InstalledAgentMessage"] --> Parse["Deserialize JSON"] + Parse --> Extract["Extract machineId"] + Extract --> Process["InstalledAgentService.addInstalledAgent"] + Process --> Ack["message.ack()"] +``` + +If processing fails: + +- Message is not acknowledged +- JetStream redelivers + +--- + +## 4. ToolConnectionListener (JetStream) + +Stream: `TOOL_CONNECTIONS` + +Subject pattern: + +`machine.*.tool-connection` + +Responsibilities: + +- Deserialize `ToolConnectionMessage` +- Transform agent tool ID +- Delegate to `ToolConnectionService` +- Explicit ACK on success + +--- + +# Tool Agent ID Transformation + +The module provides pluggable `ToolAgentIdTransformer` implementations. + +## FleetMdmAgentIdTransformer + +Purpose: + +- Converts Fleet MDM UUID into numeric host ID. + +Steps: + +1. Fetch tool configuration via `IntegratedToolService`. +2. Resolve API URL via `ToolUrlService`. +3. Query Fleet MDM using `FleetMdmClient`. +4. Select best matching host. +5. Return numeric host ID. + +Fallback behavior: + +- Retry until max delivery reached. +- On last attempt, fallback to original UUID. + +--- + +## MeshCentralAgentIdTransformer + +Purpose: + +- Normalize MeshCentral node IDs. + +Modes: + +- Legacy: `node//{id}` +- Tenant-scoped: `node/{tenantId}/{id}` + +Configuration-driven behavior: + +- `openframe.cluster-id` +- `openframe.client.meshcentral.tenant-scoped-node-id` + +--- + +# Extension Points + +## AgentRegistrationProcessor + +`DefaultAgentRegistrationProcessor` provides a no-op implementation. + +Custom implementations can: + +- Enrich machine metadata +- Trigger additional provisioning logic +- Integrate with external systems + +Spring will use a custom bean if provided. + +--- + +# Concurrency & Reliability + +## Concurrency + +- Virtual thread executor for installation tasks +- NATS dispatcher-managed threads + +## Reliability + +- JetStream durable consumers +- Explicit acknowledgment +- Max delivery tracking +- Graceful shutdown (`@PreDestroy` cleanup) + +--- + +# Integration with the Wider System + +The Client Core Agent Ingress module integrates with: + +- Data models (Machine, Device, Tool documents) +- Tool services (Fleet MDM, MeshCentral) +- NATS messaging infrastructure +- Machine status and registration services + +It forms the **first processing boundary** for all endpoint-originated traffic before data flows deeper into domain services, persistence, analytics, and automation layers. + +--- + +# Summary + +The **Client Core Agent Ingress** module is a hybrid HTTP + messaging ingress service responsible for: + +- Secure agent onboarding +- Real-time lifecycle tracking +- Tool integration normalization +- Reliable event processing +- Async installation handling + +It ensures that edge devices are consistently authenticated, registered, tracked, and integrated into the broader OpenFrame ecosystem with high reliability and extensibility. \ No newline at end of file diff --git a/docs/reference/architecture/data-access-mongo-sync/data-access-mongo-sync.md b/docs/reference/architecture/data-access-mongo-sync/data-access-mongo-sync.md new file mode 100644 index 000000000..a0b59d190 --- /dev/null +++ b/docs/reference/architecture/data-access-mongo-sync/data-access-mongo-sync.md @@ -0,0 +1,423 @@ +# Data Access Mongo Sync + +## Overview + +The **Data Access Mongo Sync** module provides the synchronous MongoDB persistence layer for the OpenFrame platform. It is responsible for: + +- Configuring MongoDB infrastructure and converters +- Enabling tenant-aware and non-tenant repository modes +- Implementing custom query logic with cursor-based pagination +- Managing advanced filtering and aggregation for core domain entities +- Handling retry logic for optimistic locking scenarios +- Seeding and maintaining system-level reference data (e.g., ticket statuses) + +This module sits directly above the MongoDB data model (documents and base repositories) and below business services in modules such as API Service Core and Authorization Service Core. + +--- + +## Architectural Position + +Data Access Mongo Sync acts as the concrete Mongo-backed implementation layer for repository contracts defined in the shared data model module. + +```mermaid +flowchart TD + Controllers["REST / GraphQL Controllers"] --> Services["Business Services"] + Services --> Repos["Data Access Mongo Sync Repositories"] + Repos --> Mongo[("MongoDB")] + + Repos --> Docs["Mongo Documents"] + Docs --> Mongo +``` + +### Key Responsibilities + +1. Infrastructure configuration (MongoTemplate, converters, indexing) +2. Custom repository implementations using MongoTemplate +3. Cursor-based pagination and stable sorting +4. Aggregations and statistical queries +5. Multi-tenant repository activation +6. Retry handling for optimistic locking + +--- + +## Module Structure + +The module can be logically grouped into the following areas: + +```mermaid +flowchart LR + Config["Configuration Layer"] --> Repo["Custom Repositories"] + Repo --> Domain["Domain-Specific Data Access"] + Repo --> Ticketing["Ticketing & Notifications"] + Repo --> Identity["User & Tenant"] + Repo --> Tools["Scripts & Integrated Tools"] + Retry["Retry & Resilience"] --> Repo + Seed["Seed Catalogs"] --> Repo +``` + +--- + +# 1. Configuration Layer + +## MongoInfraConfig + +- Enables Mongo auditing (`@EnableMongoAuditing`) +- Customizes `MappingMongoConverter` +- Configures map key dot replacement (`__dot__`) to avoid Mongo field path conflicts + +This ensures safe serialization of nested maps and consistent document mapping. + +## MongoSyncConfig + +Activated when: + +- `spring.data.mongodb.enabled=true` +- `openframe.tenant-isolation.enabled=false` + +Enables Mongo repositories while excluding `TenantAwareRepository` annotated types. + +## TenantAwareSyncConfig + +Activated when both Mongo and tenant isolation are enabled. + +```mermaid +flowchart TD + A["spring.data.mongodb.enabled"] --> B{{"Tenant isolation?"}} + B -->|No| C["MongoSyncConfig"] + B -->|Yes| D["TenantAwareSyncConfig"] +``` + +This allows switching between: + +- Global repositories +- Tenant-scoped repositories + +without changing business code. + +## MongoIndexConfig + +Executed on startup (`@PostConstruct`): + +- Ensures compound indexes on `application_events` +- Drops stale legacy indexes (e.g., tag uniqueness constraints) + +This prevents schema drift and improves query performance. + +## NotificationContextJacksonConfig & NotificationContextMongoConfig + +These classes configure: + +- Polymorphic subtype registration for notification contexts +- Custom read/write converters +- A selective type mapper for Mongo + +```mermaid +flowchart TD + Descriptors["NotificationContextDescriptor"] --> Jackson["Subtype Registration"] + Jackson --> Converters["Read / Write Converters"] + Converters --> MongoConverter["MappingMongoConverter"] + MongoConverter --> Mongo[("MongoDB")] +``` + +This allows safe storage of heterogeneous notification context payloads. + +--- + +# 2. Custom Repository Implementations + +The module heavily uses `MongoTemplate` for advanced queries beyond basic CRUD. + +## Common Patterns + +Across repositories, the following patterns are consistent: + +- Cursor-based pagination using `_id` +- Deterministic secondary sorting +- Allowlisted sortable fields +- Aggregation pipelines for grouped counts +- Search via case-insensitive regex + +### Cursor Pagination Strategy + +```mermaid +flowchart TD + Client["Client Request"] --> Query["Build Mongo Query"] + Query --> Cursor{{"Cursor Provided?"}} + Cursor -->|Yes| Criteria["Add lt/gt _id Criteria"] + Cursor -->|No| NoCursor["First Page"] + Criteria --> Sort["Apply Sort + _id Tie Breaker"] + NoCursor --> Sort + Sort --> Limit["Apply Limit"] + Limit --> Result["Return Page"] +``` + +This ensures: + +- Stable ordering +- No duplicates between pages +- Efficient index usage + +--- + +# 3. Domain-Specific Data Access + +## Assignment + +**CustomItemAssignmentRepositoryImpl** + +- Cursor-based pagination +- Search by display name +- Aggregation grouped by `AssignmentTargetType` + +Supports fast statistics for assignment dashboards. + +--- + +## Devices (Machines) + +**CustomMachineRepositoryImpl** + +- Dynamic filtering via `MachineQueryFilter` +- Status, OS, organization, device type filters +- Search across hostname, IP, serial, manufacturer +- Cursor-based pagination + +Optimized for fleet views and RMM integrations. + +--- + +## Events + +### ExternalApplicationEventRepository +- Standard MongoRepository +- Time-range and tag-based queries + +### CustomEventRepositoryImpl +- Date-range filtering +- Distinct event types and user IDs +- Cursor pagination + +Used heavily in audit and activity feeds. + +--- + +## Knowledge Base + +**CustomKnowledgeBaseItemRepositoryImpl** + +Features: + +- Folder/article separation +- Draft and archived logic +- Combined `$and` + `$or` criteria merging +- Updated-at based cursor pagination + +This supports hierarchical article browsing and admin collaboration workflows. + +--- + +## Notifications + +### CustomNotificationRepositoryImpl + +Two-phase retrieval: + +1. Fetch read-state rows (ordered by notification ID) +2. Fetch notification documents +3. Merge while preserving order + +```mermaid +sequenceDiagram + participant Client + participant Repo + participant Mongo + + Client->>Repo: findPageForRecipient() + Repo->>Mongo: Query NotificationReadState + Mongo-->>Repo: ReadState rows + Repo->>Mongo: Query Notifications by _id + Mongo-->>Repo: Notification docs + Repo-->>Client: NotificationPage +``` + +### CustomNotificationReadStateRepositoryImpl + +- Bulk unordered insert +- Swallows duplicate-key errors safely + +Prevents race-condition failures when marking notifications read. + +--- + +# 4. Ticketing Subsystem + +The ticket repository is one of the most advanced in the module. + +## CustomTicketRepositoryImpl + +Capabilities: + +- Rich multi-criteria filtering +- Cursor pagination with composite tie-breaking +- Aggregations: + - Count by status + - Count by status kind + - Count by status ID + - Average resolution time +- Bulk status updates +- Status reassignment +- Partial updates (e.g., title) + +```mermaid +flowchart TD + Filter["TicketQueryFilter"] --> Build["Build Query"] + Build --> Cursor["Apply Cursor"] + Cursor --> Sort["Sort + _id"] + Sort --> Find["MongoTemplate.find()"] + Build --> Agg["Aggregation Pipelines"] +``` + +## TicketAttachmentRepository +- Find by ticket ID +- Batch queries +- Cascade delete support + +## TicketNoteRepository +- Sorted note retrieval +- Batch lookup + +## TicketStatusDefinitionRepository +- Ordered retrieval by position +- Query by kind +- Existence validation + +--- + +## TicketStatusSeedCatalog + +Defines system-level ticket statuses using LexoRank positioning. + +- AI Assistance +- Tech Required +- Resolved +- Archived +- Optional custom "On Hold" + +This ensures deterministic ordering and consistent initial system state. + +--- + +# 5. Tools and Scripts + +## CustomIntegratedToolRepositoryImpl + +- Filtering by enabled/type/category/platform +- Distinct field extraction +- Sortable-field validation + +## CustomScriptRepositoryImpl + +Tenant-scoped script retrieval with: + +- Status filtering (soft-delete aware) +- Platform filtering +- Shell filtering +- Case-insensitive tag matching +- Direction-aware cursor logic + +Special care is taken to correctly compare Mongo `ObjectId` values rather than raw strings. + +--- + +# 6. Organization and Identity + +## CustomOrganizationRepositoryImpl + +Features: + +- Contract validity filtering +- Employee range filters +- Status defaulting +- Composite AND/OR criteria merging +- Cursor pagination + +## CustomUserRepositoryImpl + +- Email regex filtering +- Name search (first/last) +- Status filtering +- Descending sort by creation date + +## TenantRepository + +Extends `MongoRepository` and `BaseTenantRepository`. + +Supports: + +- Domain lookup +- Domain existence checks +- Projection-based domain queries + +--- + +# 7. OAuth Token Persistence + +## OAuthTokenRepository + +Provides Mongo-based token storage: + +- Find by access token +- Find by refresh token + +Used by the Authorization Service Core. + +--- + +# 8. Retry & Resilience + +## OptimisticLockingRetryListener + +A Spring Retry listener that: + +- Logs retry attempts for optimistic locking +- Logs success after retry +- Logs failure after exhaustion + +```mermaid +flowchart TD + Operation["Optimistic Update"] --> Conflict{{"Version Conflict?"}} + Conflict -->|Yes| Retry["Retry"] + Retry --> Success{{"Success?"}} + Success -->|Yes| Done["Complete"] + Success -->|No| Exhausted["Error"] +``` + +Improves reliability under concurrent updates. + +--- + +# Design Principles + +The Data Access Mongo Sync module follows these core principles: + +1. Database-level filtering for performance +2. Explicit allowlists for sortable fields +3. Stable cursor pagination +4. Aggregation over in-memory counting +5. Graceful handling of malformed cursors +6. Multi-tenant compatibility via configuration switching +7. Idempotent bulk operations + +--- + +# Conclusion + +The **Data Access Mongo Sync** module is the backbone of OpenFrame's persistence layer. It translates rich domain queries into efficient MongoDB operations while supporting: + +- Multi-tenancy +- High-concurrency safety +- Advanced analytics +- Cursor-based pagination +- Complex filtering + +By centralizing Mongo-specific logic here, higher-level modules remain clean, domain-focused, and database-agnostic. \ No newline at end of file diff --git a/docs/reference/architecture/data-kafka-configuration-and-retry/data-kafka-configuration-and-retry.md b/docs/reference/architecture/data-kafka-configuration-and-retry/data-kafka-configuration-and-retry.md deleted file mode 100644 index 195ade979..000000000 --- a/docs/reference/architecture/data-kafka-configuration-and-retry/data-kafka-configuration-and-retry.md +++ /dev/null @@ -1,380 +0,0 @@ -# Data Kafka Configuration And Retry - -## Overview - -The **Data Kafka Configuration And Retry** module provides the foundational Kafka infrastructure for the OpenFrame OSS ecosystem. It encapsulates: - -- Multi-tenant Kafka configuration -- Custom Spring Boot auto-configuration for Kafka -- Topic provisioning and management -- Standardized Debezium message modeling -- Structured recovery handling for failed Kafka operations - -This module is designed to be reusable across services such as Stream Service, Management Service, and other components that rely on event-driven communication. - -It replaces default Spring Boot Kafka auto-configuration with a controlled, tenant-aware setup tailored for OpenFrame OSS. - ---- - -## Architectural Role in the Platform - -Within the overall system, this module acts as the **Kafka infrastructure layer** used by event-driven services. - -```mermaid -flowchart LR - AppService[Application Service] --> KafkaInfra[Data Kafka Configuration And Retry] - KafkaInfra --> KafkaCluster[Kafka Cluster] - KafkaCluster --> StreamService[Stream Service Core] - KafkaCluster --> OtherConsumers[Other Consumers] -``` - -### Responsibilities - -1. Provide tenant-aware Kafka configuration -2. Standardize producer and consumer factories -3. Enable dynamic topic auto-creation -4. Provide a canonical Debezium message wrapper -5. Handle Kafka publishing failures via recovery logic - ---- - -## Module Components - -The module consists of the following core components: - -| Component | Responsibility | -|------------|----------------| -| `KafkaTopicProperties` | Defines topic configuration and auto-creation behavior | -| `OssKafkaConfig` | Enables Kafka and disables default auto-configuration | -| `OssTenantKafkaAutoConfiguration` | Main bean wiring for producers, consumers, listeners, and admin | -| `OssTenantKafkaProperties` | Tenant-scoped Kafka configuration properties | -| `KafkaHeader` | Standardized Kafka header constants | -| `DebeziumMessage` | Generic wrapper for Debezium CDC events | -| `KafkaRecoveryHandlerImpl` | Structured recovery handler for failed Kafka publishes | - ---- - -# Configuration Architecture - -## Custom Kafka Bootstrapping - -The module disables Spring Boot's default `KafkaAutoConfiguration` and replaces it with a controlled configuration: - -```mermaid -flowchart TD - SpringBoot[Spring Boot] --> DefaultKafkaAutoConfig[KafkaAutoConfiguration] - DefaultKafkaAutoConfig -->|Excluded| OssKafkaConfig - OssKafkaConfig --> OssTenantKafkaAutoConfiguration - OssTenantKafkaAutoConfiguration --> ProducerFactory - OssTenantKafkaAutoConfiguration --> ConsumerFactory - OssTenantKafkaAutoConfiguration --> KafkaTemplate - OssTenantKafkaAutoConfiguration --> KafkaAdmin -``` - -### OssKafkaConfig - -- Enables `@EnableKafka` -- Excludes `KafkaAutoConfiguration` -- Ensures only OSS tenant configuration is active - -This prevents accidental configuration conflicts and guarantees predictable Kafka behavior. - ---- - -## Tenant-Aware Kafka Properties - -### OssTenantKafkaProperties - -Bound to: - -```text -spring.oss-tenant -``` - -This class wraps Spring's `KafkaProperties`, allowing full reuse of: - -- Producer properties -- Consumer properties -- Listener configuration -- Template configuration -- Admin configuration - -### Enabling Kafka - -Kafka is conditionally activated using: - -```text -spring.oss-tenant.enabled=true -``` - -If disabled, no Kafka beans are registered. - ---- - -# Topic Management - -## KafkaTopicProperties - -Bound to: - -```text -openframe.oss-tenant.kafka.topics -``` - -It supports: - -- Automatic topic creation -- Partition configuration -- Replication factor configuration - -### Structure - -```text -openframe: - oss-tenant: - kafka: - topics: - inbound: - device-events: - name: device.events - partitions: 3 - replication-factor: 2 -``` - -### Auto-Creation Flow - -```mermaid -flowchart TD - ConfigProps[KafkaTopicProperties] --> AdminBean[KafkaAdmin] - AdminBean --> TopicBuilder - TopicBuilder --> NewTopic - NewTopic --> KafkaCluster -``` - -If topic auto-creation is enabled: - -- `KafkaAdmin` registers `NewTopic` beans -- Topics are created on startup -- Logging confirms partition and replica configuration - ---- - -# Producer and Consumer Infrastructure - -## Producer Side - -### ProducerFactory - -- Key serializer: `StringSerializer` -- Value serializer: `JsonSerializer` -- Backed by Spring `KafkaProperties` - -### KafkaTemplate - -- Default topic configurable -- JSON payload support -- Used by higher-level producers - -### OssTenantKafkaProducer - -Created as a bean to provide a simplified abstraction over `KafkaTemplate`. - ---- - -## Consumer Side - -### ConsumerFactory - -- Key deserializer: `StringDeserializer` -- Value deserializer: `JsonDeserializer` -- Uses Spring-configured properties - -### Listener Container Factory - -Configurable: - -- Concurrency -- AckMode (defaults to RECORD) -- Poll timeout -- Idle event interval -- Container logging - -```mermaid -flowchart LR - KafkaCluster --> ListenerContainer - ListenerContainer --> ConsumerFactory - ConsumerFactory --> JsonDeserializer - ListenerContainer --> AckMode[RECORD Ack Mode] -``` - -The default acknowledgment mode ensures per-record reliability if not explicitly configured. - ---- - -# Debezium Event Modeling - -## DebeziumMessage - -The module defines a generic wrapper for Debezium Change Data Capture events. - -### Structure - -```mermaid -flowchart TD - DebeziumMessage --> Payload - Payload --> Before[Before State] - Payload --> After[After State] - Payload --> Source - Payload --> Operation - Payload --> Timestamp - Source --> Database - Source --> Table - Source --> Connector -``` - -### Supported Fields - -- `before` – state before change -- `after` – state after change -- `operation` – CDC operation (c, u, d) -- `ts_ms` – event timestamp -- `source` – connector metadata - - database - - table - - connector type - - schema - - snapshot flag - -This structure ensures: - -- Strong typing of CDC events -- Cross-service consistency -- Cleaner event enrichment in downstream consumers - ---- - -# Kafka Headers Standardization - -## KafkaHeader - -Defines common header keys used across producers and consumers. - -Currently: - -```text -message-type -``` - -This enables: - -- Event routing -- Polymorphic deserialization -- Handler selection in downstream services - ---- - -# Retry and Recovery Handling - -## KafkaRecoveryHandlerImpl - -This component implements structured error recovery for failed Kafka publish operations. - -### Behavior - -When publishing fails: - -- The exception is captured -- Topic and key are logged -- Error class and message are recorded -- Payload summary is logged -- Full stacktrace is attached - -```mermaid -flowchart TD - Producer --> KafkaCluster - KafkaCluster -->|Failure| RecoveryHandler - RecoveryHandler --> StructuredLog -``` - -### Logging Strategy - -The recovery handler logs: - -- `topic` -- `key` -- `errorClass` -- `errorMsg` -- `payload` - -This ensures observability without introducing immediate dead-letter queue complexity. - -It can later be extended to: - -- Publish to retry topics -- Send to dead-letter queues -- Trigger alerting systems - ---- - -# Integration with Other Modules - -The **Data Kafka Configuration And Retry** module serves as a foundational infrastructure layer for: - -- Stream processing services consuming Debezium events -- Management services publishing operational events -- Notification pipelines -- Cross-service event propagation - -It does not implement business logic itself. Instead, it standardizes: - -- How Kafka is configured -- How topics are created -- How messages are structured -- How failures are handled - ---- - -# Design Principles - -### 1. Tenant Isolation -Kafka configuration is explicitly scoped under: - -```text -spring.oss-tenant -``` - -This prevents accidental mixing with default Kafka configurations. - -### 2. Explicit Auto-Configuration -Default Spring Kafka auto-configuration is excluded to avoid: - -- Bean conflicts -- Hidden configuration overrides -- Environment-specific ambiguity - -### 3. Strong CDC Modeling -Debezium events are modeled as typed structures instead of raw maps. - -### 4. Safe Defaults -- Ack mode defaults to RECORD -- Admin auto-creation enabled unless disabled -- Kafka enabled by default but explicitly configurable - -### 5. Observability First -Failure handling prioritizes structured logging for operational visibility. - ---- - -# Summary - -The **Data Kafka Configuration And Retry** module provides a controlled, tenant-aware Kafka foundation for the OpenFrame OSS platform. - -It ensures: - -- Deterministic Kafka bootstrapping -- Configurable topic lifecycle management -- Structured Debezium message handling -- Safe producer/consumer defaults -- Clear recovery and error logging - -By centralizing Kafka configuration and retry behavior, the module guarantees consistency across all services that rely on event-driven communication. \ No newline at end of file diff --git a/docs/reference/architecture/data-model-and-repositories-mongo/data-model-and-repositories-mongo.md b/docs/reference/architecture/data-model-and-repositories-mongo/data-model-and-repositories-mongo.md new file mode 100644 index 000000000..85dbc6b4c --- /dev/null +++ b/docs/reference/architecture/data-model-and-repositories-mongo/data-model-and-repositories-mongo.md @@ -0,0 +1,490 @@ +# Data Model And Repositories Mongo + +## Overview + +The **Data Model And Repositories Mongo** module defines the canonical MongoDB domain model for OpenFrame OSS and provides base repository abstractions and tenant-aware infrastructure primitives. + +This module is the foundation for: + +- Multi-tenant persistence +- Domain entities (Users, Organizations, Devices, Tickets, Events, Notifications, OAuth, Tools) +- Query filter objects used by services and GraphQL +- Base repository contracts (technology-agnostic) +- Tenant resolution via `TenantIdProvider` + +It is consumed by higher-level modules such as API services, Authorization Server, Management services, Stream Processing, and Gateway layers. + +--- + +## Architectural Position + +```mermaid +flowchart TD + Controllers["REST Controllers / GraphQL"] --> Services["Business Services"] + Services --> Repositories["Mongo Repositories"] + Repositories --> Documents["Mongo Documents"] + Documents --> MongoDB[("MongoDB")] + + TenantProvider["TenantIdProvider"] --> Repositories +``` + +### Responsibilities + +| Layer | Responsibility | +|--------|----------------| +| Documents | Define Mongo collections and embedded models | +| Query Filters | Encapsulate search criteria without API-layer dependency | +| Base Repositories | Define cross-cutting repository contracts | +| Tenant Provider | Provide current tenant context | + +--- + +# Multi-Tenancy Model + +Multi-tenancy is enforced at the document and repository level. + +### TenantScoped Pattern + +Most aggregate root documents implement `TenantScoped` and include: + +- `tenantId` field +- Index on `tenantId` + +Example collections: + +- `users` +- `organizations` +- `devices` +- `tickets` +- `events` +- `notifications` +- `oauth_tokens` + +### Tenant Resolution + +```mermaid +flowchart LR + Request["Incoming Request"] --> Security["Security Layer"] + Security --> TenantContext["Tenant Context"] + TenantContext --> TenantProvider["TenantIdProvider"] + TenantProvider --> Repository["Mongo Repository"] +``` + +The default implementation is: + +- `DefaultTenantIdProvider` + +It reads the `TENANT_ID` environment variable (default: `oss`). + +--- + +# Core Domain Aggregates + +## 1. User & Authentication + +### User + +Collection: `users` + +Key fields: + +- `email` (indexed) +- `roles` +- `emailVerified` +- `status` +- `tenantId` + +Email is normalized to lowercase. + +### AuthUser + +Extends `User` and is used by the Authorization Server. + +Additional fields: + +- `passwordHash` +- `loginProvider` (LOCAL, GOOGLE, etc.) +- `externalUserId` +- `lastLogin` +- `imageUrl` + +Unique compound index: + +```text +{'tenantId': 1, 'email': 1} +``` + +This guarantees per-tenant email uniqueness. + +--- + +## 2. Organization + +Collection: `organizations` + +Key capabilities: + +- Soft delete via `status` (ACTIVE, ARCHIVED, DELETED) +- Immutable `organizationId` +- Contract lifecycle validation (`isContractActive()`) +- Contact information embedding + +Indexed fields: + +- `tenantId` +- `name` +- `organizationId` (unique) +- `isDefault` +- `status` + +### OrganizationQueryFilter + +Encapsulates: + +- Category +- Employee ranges +- Active contract flag +- Status + +Used by repository implementations to build dynamic queries. + +--- + +## 3. Device Domain + +Collection: `devices` + +Fields include: + +- `machineId` +- `serialNumber` +- `osVersion` +- `status` +- `lastCheckin` +- Embedded: + - `DeviceConfiguration` + - `DeviceHealth` + +### Embedded Substructures + +- `Alert` +- `SecurityAlert` +- `ComplianceRequirement` + +These are embedded and not stored as separate collections. + +--- + +## 4. Ticketing System + +### Ticket + +Collection: `tickets` + +Compound indexes include: + +```text +'tenant_ticketNumber_idx' +'status_order' +'status_kind' +'status_id_order' +``` + +Key characteristics: + +- Tenant-scoped numbering +- Owner polymorphism: + - `AdminTicketOwner` + - `ClientTicketOwner` +- Status evolution support: + - Legacy `TicketStatus` + - New lifecycle via `statusId` + `statusKind` + +### Related Collections + +- `ticket_attachments` +- `ticket_notes` +- `ticket_statuses` + +```mermaid +flowchart TD + Ticket["Ticket"] --> Attachment["TicketAttachment"] + Ticket --> Note["TicketNote"] + Ticket --> StatusDef["TicketStatusDefinition"] +``` + +### TicketQueryFilter + +Supports: + +- Status filters (legacy + lifecycle) +- Assignee filters +- Organization filters +- Device filters +- Creation source +- Date ranges + +--- + +## 5. Events + +### CoreEvent + +Collection: `events` + +Represents internal system events. + +Fields: + +- `type` +- `payload` +- `status` (CREATED, PROCESSING, COMPLETED, FAILED) +- `timestamp` + +### ExternalApplicationEvent + +Collection: `external_application_events` + +Adds metadata: + +- `source` +- `version` +- `tags` + +### EventQueryFilter + +Supports filtering by: + +- User IDs +- Event types +- Date range + +--- + +## 6. Notifications + +### Notification + +Collection: `notifications` + +Fields: + +- `severity` +- `title` +- `description` +- `context` + +### NotificationReadState + +Collection: `notification_read_states` + +Compound indexes enforce: + +- Unique recipient + notification +- Efficient unread queries + +```mermaid +flowchart LR + Notification --> ReadState["NotificationReadState"] +``` + +--- + +## 7. OAuth & Authorization + +### MongoRegisteredClient + +Collection: `oauth_registered_clients` + +Compound unique index: + +```text +{'tenantId':1,'clientId':1} +``` + +Stores: + +- Grant types +- Redirect URIs +- Token TTL configuration +- PKCE requirements + +### OAuthToken + +Collection: `oauth_tokens` + +Fields: + +- `accessToken` +- `refreshToken` +- Expiry timestamps +- `clientId` + +--- + +## 8. Tagging System + +### TagAssignment + +Collection: `tag_assignments` + +Compound unique index: + +```text +{'tenantId': 1, 'entityId': 1, 'tagId': 1, 'entityType': 1} +``` + +Supports: + +- Multi-value tags +- Multiple entity types + +### TagValidation + +Provides: + +- Regex enforcement +- Max length (64) +- Key/value validation helpers + +--- + +## 9. Tool & Agent Configuration + +### IntegratedToolAgentConfiguration + +Defines: + +- Version metadata +- Download configurations +- Assets +- Installation command arguments + +### ToolAgentAsset + +Includes: + +- Version +- Download configurations +- Local filename mapping +- Executable flag + +### ScriptEnvVar + +Embedded within Script documents. + +Supports future secret encryption strategy (currently plaintext). + +--- + +# Query Filter Pattern + +Query filter classes: + +- `UserQueryFilter` +- `OrganizationQueryFilter` +- `EventQueryFilter` +- `TicketQueryFilter` +- `ScriptQueryFilter` +- `ToolQueryFilter` + +### Design Principle + +These filters: + +- Mirror API input models +- Avoid dependency on API layer +- Keep repository layer modular + +```mermaid +flowchart LR + ApiInput["API Filter Input"] --> Mapper["Service Mapper"] + Mapper --> QueryFilter["*QueryFilter"] + QueryFilter --> Repository["Custom Repository Impl"] +``` + +--- + +# Base Repository Abstractions + +The module defines technology-agnostic repository contracts. + +## BaseUserRepository + +Defines: + +- `findByEmail` +- `existsByEmail` +- `existsByEmailAndStatus` + +## BaseTenantRepository + +Defines: + +- `findByDomain` +- `existsByDomain` + +## BaseIntegratedToolRepository + +Defines: + +- `findByType` +- `findByKey` + +## BaseApiKeyRepository + +Defines: + +- `findByIdAndUserId` +- `findByUserId` +- `findExpiredKeys` + +### Design Characteristics + +- Generic return wrappers (`Optional`, `Mono`, `Flux`) +- Enables both blocking and reactive implementations +- Prevents service layer coupling to Mongo implementation details + +--- + +# Indexing Strategy + +Common patterns used across documents: + +1. `@Indexed` for high-frequency lookups +2. `@CompoundIndex` for: + - Tenant + natural key uniqueness + - Lifecycle sorting + - Recipient uniqueness +3. Partial indexes for constrained uniqueness (e.g., TicketStatusDefinition) + +This ensures: + +- Tenant isolation +- Efficient filtering +- Strong consistency for natural keys + +--- + +# Design Principles + +1. Tenant-first modeling +2. Soft deletion instead of hard deletes +3. Embedded documents for high-cohesion substructures +4. Strict index discipline for performance +5. Separation of API DTOs and persistence filters +6. Technology-agnostic repository contracts + +--- + +# Summary + +The **Data Model And Repositories Mongo** module: + +- Defines the canonical MongoDB schema for OpenFrame OSS +- Enforces multi-tenant boundaries +- Provides reusable repository contracts +- Supports ticketing, devices, events, notifications, OAuth, tagging, and tool configuration +- Acts as the persistence foundation for all higher-level services + +It is the backbone of the platform’s state management and tenant isolation strategy. \ No newline at end of file diff --git a/docs/reference/architecture/data-mongo-base-repositories/data-mongo-base-repositories.md b/docs/reference/architecture/data-mongo-base-repositories/data-mongo-base-repositories.md deleted file mode 100644 index 85d255373..000000000 --- a/docs/reference/architecture/data-mongo-base-repositories/data-mongo-base-repositories.md +++ /dev/null @@ -1,256 +0,0 @@ -# Data Mongo Base Repositories - -## Overview - -The **Data Mongo Base Repositories** module defines technology-agnostic repository contracts for core MongoDB-backed domain entities in the OpenFrame platform. These base interfaces act as a unifying abstraction layer between: - -- The **domain model** (Mongo documents such as User, Tenant, API Key, Integrated Tool) -- The **blocking (imperative)** repository implementations -- The **reactive (Project Reactor)** repository implementations - -By introducing generic wrapper types (`T`, `B`, `L`, `ID`), this module ensures that the same repository contract can be reused across both blocking and reactive stacks without duplicating business semantics. - ---- - -## Architectural Role in the Data Layer - -Within the broader persistence architecture, this module sits between the domain documents and concrete repository implementations. - -```mermaid -flowchart TD - Domain["Mongo Domain Model"] --> BaseRepo["Data Mongo Base Repositories"] - BaseRepo --> Blocking["Blocking Repositories"] - BaseRepo --> Reactive["Reactive Repositories"] - - Blocking --> Services["Application Services"] - Reactive --> Services -``` - -### Responsibilities - -- Define **common repository operations** for core entities -- Provide **generic signatures** adaptable to Optional / List or Mono / Flux -- Ensure **consistent query semantics** across execution models -- Reduce duplication between blocking and reactive repository layers - ---- - -## Design Philosophy - -Each base repository follows a consistent pattern: - -```text -public interface BaseXRepository { - T findBy...(...); - B existsBy...(...); - L findAllBy...(...); -} -``` - -Where: - -- `T` β†’ Wrapper for single-result queries - - Blocking: `Optional` - - Reactive: `Mono` -- `B` β†’ Wrapper for boolean responses - - Blocking: `boolean` - - Reactive: `Mono` -- `L` β†’ Wrapper for multi-result queries (if applicable) - - Blocking: `List` - - Reactive: `Flux` -- `ID` β†’ Identifier type (typically `String`) - -This allows downstream modules to: - -- Implement blocking repositories using Spring Data Mongo -- Implement reactive repositories using Spring Data Reactive Mongo -- Preserve identical method semantics across implementations - ---- - -# Core Interfaces - -## 1. BaseApiKeyRepository - -**Purpose:** Defines common API key query operations. - -### Key Methods - -- `findByIdAndUserId(String keyId, String userId)` -- `findByUserId(String userId)` -- `findExpiredKeys(Instant currentTime)` - -### Architectural Context - -```mermaid -flowchart LR - ApiKey["API Key Document"] --> ApiKeyRepo["BaseApiKeyRepository"] - ApiKeyRepo --> UserScope["User-Scoped Queries"] - ApiKeyRepo --> Expiry["Expiration Queries"] -``` - -### Design Considerations - -- API keys are **user-scoped**. -- Expiration handling is centralized via `findExpiredKeys`. -- Enables scheduled cleanup and security enforcement logic. - -This repository is critical for authentication flows and API access management. - ---- - -## 2. BaseTenantRepository - -**Purpose:** Defines tenant lookup and domain validation operations. - -### Key Methods - -- `findByDomain(String domain)` -- `existsByDomain(String domain)` - -### Architectural Context - -```mermaid -flowchart TD - Tenant["Tenant Document"] --> TenantRepo["BaseTenantRepository"] - TenantRepo --> DomainLookup["Domain-Based Resolution"] - DomainLookup --> AuthLayer["Authorization & SSO"] -``` - -### Design Considerations - -- Domain-based lookup enables **multi-tenant resolution**. -- Used heavily in authentication and tenant-aware request routing. -- `existsByDomain` supports validation during registration and onboarding flows. - ---- - -## 3. BaseIntegratedToolRepository - -**Purpose:** Provides lookup functionality for integrated tools. - -### Key Methods - -- `findByType(String type)` - -### Architectural Context - -```mermaid -flowchart LR - Tool["Integrated Tool Document"] --> ToolRepo["BaseIntegratedToolRepository"] - ToolRepo --> TypeLookup["Type-Based Resolution"] - TypeLookup --> Gateway["Gateway & Integration Layer"] -``` - -### Design Considerations - -- Tool resolution is based on logical tool type. -- Enables integration routing and upstream configuration. -- Keeps tool lookup semantics consistent across sync and reactive stacks. - ---- - -## 4. BaseUserRepository - -**Purpose:** Defines foundational user lookup and existence checks. - -### Key Methods - -- `findByEmail(String email)` -- `existsByEmail(String email)` -- `existsByEmailAndStatus(String email, UserStatus status)` - -### Architectural Context - -```mermaid -flowchart TD - UserDoc["User Document"] --> UserRepo["BaseUserRepository"] - UserRepo --> EmailLookup["Email-Based Lookup"] - UserRepo --> StatusCheck["Status Validation"] - StatusCheck --> Security["Authentication & Access Control"] -``` - -### Design Considerations - -- Email is the primary identity attribute. -- Status validation supports enforcement of states such as ACTIVE or DISABLED. -- Provides a consistent existence-check abstraction across blocking and reactive implementations. - ---- - -# Cross-Cutting Patterns - -## 1. Technology Agnostic Contracts - -All repositories avoid direct dependencies on: - -- Spring Data interfaces -- Reactive types -- Mongo-specific annotations - -Instead, they rely on generic wrapper types, allowing different execution models without rewriting business contracts. - ---- - -## 2. Multi-Tenancy Alignment - -The BaseTenantRepository and BaseUserRepository are foundational for tenant-aware identity resolution. - -```mermaid -flowchart TD - Request["Incoming Request"] --> DomainExtract["Extract Domain"] - DomainExtract --> TenantRepo["BaseTenantRepository"] - TenantRepo --> TenantResolved["Tenant Context Established"] - TenantResolved --> UserRepo["BaseUserRepository"] - UserRepo --> AuthResult["User Validated"] -``` - -This separation ensures that: - -- Tenant resolution is independent from authentication logic. -- User validation remains consistent regardless of execution model. - ---- - -## 3. Expiration and Lifecycle Handling - -Repositories such as BaseApiKeyRepository expose lifecycle-oriented queries (e.g., expired keys). This supports: - -- Scheduled cleanup tasks -- Security enforcement policies -- Token rotation and revocation strategies - ---- - -# Interaction with Other Data Modules - -Although this module only defines interfaces, it enables: - -- Concrete Mongo repository implementations in synchronization modules. -- Reactive repository implementations in reactive data modules. -- Service-layer logic in API and authorization services. - -```mermaid -flowchart LR - Base["Data Mongo Base Repositories"] --> SyncImpl["Mongo Sync Implementations"] - Base --> ReactiveImpl["Mongo Reactive Implementations"] - SyncImpl --> ServiceLayer["Service Layer"] - ReactiveImpl --> ServiceLayer -``` - -The base repository layer ensures consistent semantics regardless of whether the application is running in blocking or reactive mode. - ---- - -# Summary - -The **Data Mongo Base Repositories** module is a foundational abstraction layer in the OpenFrame persistence architecture. - -It provides: - -- Unified repository contracts -- Execution-model independence (blocking vs reactive) -- Tenant-aware and security-aware query primitives -- Reduced duplication across repository implementations - -By separating repository semantics from implementation details, this module ensures long-term maintainability, flexibility, and architectural consistency across the OpenFrame data stack. diff --git a/docs/reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md b/docs/reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md deleted file mode 100644 index 77d406e97..000000000 --- a/docs/reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md +++ /dev/null @@ -1,471 +0,0 @@ -# Data Mongo Domain Model - -## Overview - -The **Data Mongo Domain Model** module defines the core MongoDB document structures used across the OpenFrame platform. It provides the canonical persistence model for: - -- Multi-tenant users and authentication -- Organizations and tenancy boundaries -- Devices and operational state -- Tickets and PSA workflows -- Events and audit trail -- Notifications and read states -- OAuth clients and tokens -- Tag assignments - -This module acts as the **persistence foundation** for API services, authorization services, stream processing, gateway routing, and management components. All higher-level services depend on these document definitions for consistent storage and indexing strategies. - ---- - -## Architectural Role in the Platform - -The Data Mongo Domain Model sits at the base of the application stack and is shared by: - -- API Service Core (REST + GraphQL) -- Authorization Service Core -- Stream Service Core (Kafka consumers) -- Management Service Core -- External API Service Core -- Gateway and Security components (indirectly via persistence) - -### High-Level Data Flow - -```mermaid -flowchart TD - Gateway["Gateway Service"] --> ApiService["API Service Core"] - ApiService --> DomainModel["Data Mongo Domain Model"] - AuthService["Authorization Service"] --> DomainModel - StreamService["Stream Service"] --> DomainModel - ManagementService["Management Service"] --> DomainModel - ExternalApi["External API Service"] --> DomainModel - - DomainModel --> MongoDB[("MongoDB")] -``` - -The module defines **document schemas and indexing strategies**, while repositories and services (in other modules) implement business logic on top of these models. - ---- - -## Core Domain Areas - -The domain model can be grouped into the following bounded contexts: - -1. Identity & Authentication -2. Organization & Tenancy -3. Device Management -4. Ticketing (PSA) -5. Events & Audit -6. Notifications -7. OAuth & Client Registration -8. Tagging System - ---- - -# 1. Identity & Authentication - -## User - -**Collection:** `users` - -Represents an application user within a tenant. - -Key characteristics: - -- Indexed by `email` -- Supports multiple roles (`UserRole`) -- Email normalization (lowercased, trimmed) -- Soft status control via `UserStatus` -- Audit fields (`createdAt`, `updatedAt`) - -### Responsibilities - -- Technician identity -- Role-based authorization support -- Assignment target for tickets -- Reporter reference for tickets - ---- - -## AuthUser - -Extends `User` for multi-tenant authorization server use. - -**Compound Index:** `(tenantId, email)` unique when tenant exists. - -Additional fields: - -- `tenantId` (multi-tenancy boundary) -- `passwordHash` -- `loginProvider` (LOCAL, GOOGLE, etc.) -- `externalUserId` -- `lastLogin` -- `imageUrl` (cached profile picture) - -### Purpose - -- Used by the Authorization Service -- Supports SSO and external identity providers -- Maintains domain-based tenancy - -### Identity Model Diagram - -```mermaid -classDiagram - class User { - id - email - firstName - lastName - roles - status - createdAt - updatedAt - } - - class AuthUser { - tenantId - passwordHash - loginProvider - externalUserId - lastLogin - imageUrl - } - - User <|-- AuthUser -``` - ---- - -# 2. Organization & Tenancy - -## Organization - -**Collection:** `organizations` - -Represents a company or tenant-scoped business entity. - -Key features: - -- Unique `organizationId` -- Indexed `name` -- `isDefault` flag -- Business metadata (revenue, employees, contract dates) -- Soft-delete via `OrganizationStatus` (ACTIVE, ARCHIVED, DELETED) -- Contract validation logic - -### Important Design Decision - -Organizations are **never hard-deleted** to preserve device and ticket references. Instead: - -- `ARCHIVED` β†’ hidden from standard queries -- `DELETED` β†’ soft-deleted but still referentially valid - ---- - -# 3. Device Management - -## Device - -**Collection:** `devices` - -Represents a managed endpoint. - -Key fields: - -- `machineId` (link to external Machine entity) -- `serialNumber`, `model`, `osVersion` -- `status` (ACTIVE, OFFLINE, MAINTENANCE) -- `DeviceType` -- `lastCheckin` -- `DeviceConfiguration` -- `DeviceHealth` - -### Device Lifecycle - -```mermaid -stateDiagram-v2 - [*] --> ACTIVE - ACTIVE --> OFFLINE - OFFLINE --> ACTIVE - ACTIVE --> MAINTENANCE - MAINTENANCE --> ACTIVE -``` - -Devices are heavily used by: - -- Ticketing -- Stream event ingestion -- Management schedulers -- External API integrations - ---- - -# 4. Ticketing (PSA Domain) - -Ticketing is modeled as a primary entity with related collections. - -## Ticket - -**Collection:** `tickets` - -Primary PSA entity. - -Key design elements: - -- Unique `ticketNumber` (auto-increment per tenant) -- Indexed status and assignment fields -- Linked to: - - `deviceId` - - `organizationId` - - `assignedTo` (User) - - `reporterId` -- Soft resolution via `resolvedAt` - -### Index Strategy - -Compound indexes support: - -- Status-based dashboards -- Assignment filtering -- Organization filtering -- Device filtering - ---- - -## TicketNote - -**Collection:** `ticket_notes` - -Technician-only internal notes. - -- Indexed by `ticketId` -- Sorted by `createdAt` -- Editable (tracked via `updatedAt`) - ---- - -## TicketAttachment - -**Collection:** `ticket_attachments` - -Metadata-only storage. - -Files are stored externally (S3/MinIO), while this document stores: - -- `ticketId` -- `fileName` -- `contentType` -- `fileSize` -- `storagePath` - ---- - -### Ticket Aggregate Model - -```mermaid -flowchart TD - Ticket["Ticket"] --> Note["TicketNote"] - Ticket --> Attachment["TicketAttachment"] - Ticket --> User["User (Assigned)"] - Ticket --> Device["Device"] - Ticket --> Org["Organization"] -``` - -Ticket owns metadata; attachments and notes are independent collections for scalability. - ---- - -# 5. Events & Audit - -## CoreEvent - -**Collection:** `events` - -Represents system-level or domain events. - -Fields: - -- `type` -- `payload` -- `timestamp` -- `userId` -- `status` (CREATED, PROCESSING, COMPLETED, FAILED) - -### Usage - -- Stream ingestion persistence -- Audit logs -- Async processing workflows - ---- - -# 6. Notifications - -## Notification - -**Collection:** `notifications` - -Represents a system notification. - -Fields: - -- `severity` -- `title` -- `description` -- `createdAt` -- `context` - ---- - -## NotificationReadState - -**Collection:** `notification_read_states` - -Tracks per-recipient state. - -Compound indexes ensure: - -- Unique (recipientId, recipientType, notificationId) -- Fast unread queries by status -- Filtering by category - -### Notification Model - -```mermaid -flowchart TD - Notification["Notification"] --> ReadState["NotificationReadState"] - ReadState --> Recipient["User or Organization"] -``` - -Notifications are immutable; read state is tracked separately. - ---- - -# 7. OAuth & Client Registration - -## MongoRegisteredClient - -**Collection:** `oauth_registered_clients` - -Represents OAuth2 clients. - -Key features: - -- Unique `clientId` -- Grant types -- Redirect URIs -- Scopes -- PKCE support -- Token TTL configuration - ---- - -## OAuthToken - -**Collection:** `oauth_tokens` - -Stores issued tokens. - -Fields: - -- `userId` -- `accessToken` -- `refreshToken` -- Expiry timestamps -- `clientId` -- `scopes` - -Used by Authorization Service for token validation and revocation. - ---- - -# 8. Tagging System - -## TagAssignment - -**Collection:** `tag_assignments` - -Provides a unified tagging mechanism across entities. - -Unique compound index: - -- `(entityId, tagId, entityType)` - -Supports: - -- Label-style tags -- Key-value tags with `values` -- Timestamped tagging (`taggedAt`) -- Attribution (`taggedBy`) - -### Cross-Entity Tag Model - -```mermaid -flowchart LR - TagAssignment["TagAssignment"] --> Device - TagAssignment --> Ticket - TagAssignment --> Organization -``` - -This design avoids embedding tags inside each entity and enables consistent querying. - ---- - -# Multi-Tenancy Considerations - -Multi-tenancy is enforced through: - -- `tenantId` in AuthUser -- Tenant-scoped repositories in other modules -- Unique constraints within tenant boundaries -- Per-tenant ticket numbering - -The domain model avoids hard coupling to tenant logic; enforcement is handled at repository and service layers. - ---- - -# Indexing Strategy Summary - -The module heavily leverages: - -- `@Indexed` for query acceleration -- `@CompoundIndex` for dashboard queries -- Unique constraints for integrity -- Partial indexes for multi-tenant isolation - -This ensures: - -- Fast filtering -- Scalable dashboard queries -- Safe uniqueness guarantees -- Reduced cross-tenant data leakage risk - ---- - -# Design Principles - -The Data Mongo Domain Model follows these principles: - -1. Document-per-aggregate boundary -2. Soft deletes over hard deletes -3. External storage for large binary data -4. Event-driven compatibility -5. Strong indexing strategy -6. Multi-tenant safe uniqueness -7. Clear separation of identity vs domain users - ---- - -# Conclusion - -The **Data Mongo Domain Model** is the foundational persistence layer of OpenFrame. It defines the canonical MongoDB structures that power: - -- Authentication and SSO -- Tenant isolation -- Device monitoring -- Ticketing workflows -- Notifications -- Event processing -- OAuth client management - -All higher-level services rely on these documents to provide consistent, scalable, and multi-tenant safe behavior across the platform. diff --git a/docs/reference/architecture/data-mongo-query-filters/data-mongo-query-filters.md b/docs/reference/architecture/data-mongo-query-filters/data-mongo-query-filters.md deleted file mode 100644 index da7d8fd77..000000000 --- a/docs/reference/architecture/data-mongo-query-filters/data-mongo-query-filters.md +++ /dev/null @@ -1,316 +0,0 @@ -# Data Mongo Query Filters - -## Overview - -The **Data Mongo Query Filters** module defines strongly-typed filter objects used to construct dynamic MongoDB queries across the OpenFrame platform. These filters act as the boundary between higher-level API input (REST and GraphQL) and low-level repository query execution. - -Rather than embedding query logic directly in controllers or services, this module provides composable filter models that: - -- Encapsulate domain-specific filtering criteria -- Support time-based and enum-based constraints -- Enable clean separation between API DTOs and MongoDB query construction -- Promote consistency across sync and reactive repositories - -This module is part of the data layer and works closely with: - -- [Data Mongo Domain Model](../data-mongo-domain-model/data-mongo-domain-model.md) -- [Data Mongo Base Repositories](../data-mongo-base-repositories/data-mongo-base-repositories.md) -- [Data Mongo Sync Config and Custom Repositories](../data-mongo-sync-config-and-custom-repositories/data-mongo-sync-config-and-custom-repositories.md) -- [Data Mongo Reactive Repositories](../data-mongo-reactive-repositories/data-mongo-reactive-repositories.md) - ---- - -## Architectural Role in the System - -The Data Mongo Query Filters module sits between API-level filter inputs and repository-level query execution. - -```mermaid -flowchart LR - ApiLayer["API Layer\nREST + GraphQL"] --> FilterDTOs["Filter Input DTOs"] - FilterDTOs --> QueryFilters["Data Mongo Query Filters"] - QueryFilters --> CustomRepos["Custom Mongo Repositories"] - CustomRepos --> MongoDB[("MongoDB")] - - DomainModel["Domain Documents"] --> CustomRepos -``` - -### Flow Explanation - -1. **API Layer** receives filter inputs (e.g., `TicketFilterInput`, `UserFilterInput`). -2. These inputs are mapped into internal **Query Filter** objects defined in this module. -3. Custom repositories interpret the filter object and build a MongoDB `Criteria` / query. -4. Queries are executed against collections defined in the domain model. - -This separation ensures: - -- API contracts can evolve without tightly coupling to MongoDB internals. -- Query construction remains centralized and testable. -- Business services remain independent from persistence logic. - ---- - -## Design Principles - -### 1. Immutable, Builder-Based Construction - -All filter classes use Lombok annotations: - -- `@Data` -- `@Builder` -- `@NoArgsConstructor` -- `@AllArgsConstructor` - -This enables safe, expressive creation: - -```java -TicketQueryFilter filter = TicketQueryFilter.builder() - .statuses(List.of(TicketStatus.OPEN)) - .organizationIds(List.of("org-1")) - .createdAtFrom(Instant.now().minus(30, ChronoUnit.DAYS)) - .build(); -``` - -### 2. Domain-Oriented Filtering - -Each filter aligns directly with a Mongo document type from the domain model: - -- Events β†’ `EventQueryFilter` -- Organizations β†’ `OrganizationQueryFilter` -- Tickets β†’ `TicketQueryFilter` -- Tools β†’ `ToolQueryFilter` -- Users β†’ `UserQueryFilter` - -### 3. Null-Safe Optional Criteria - -All fields are optional. Repositories interpret `null` values as "no constraint". - -This allows flexible query composition without requiring multiple overloaded methods. - ---- - -# Filter Classes - -## EventQueryFilter - -**Component:** -`openframe-oss-lib.openframe-data-mongo-common.src.main.java.com.openframe.data.document.event.filter.EventQueryFilter.EventQueryFilter` - -### Purpose - -Encapsulates filtering logic for querying event documents. - -### Fields - -- `List userIds` – Filter by user IDs -- `List eventTypes` – Restrict by event categories -- `LocalDate startDate` – Inclusive lower bound -- `LocalDate endDate` – Inclusive upper bound - -### Usage Context - -Used by event-related repositories to construct date-range and type-based queries. - -```mermaid -flowchart TD - Filter["EventQueryFilter"] --> ByUser["userIds"] - Filter --> ByType["eventTypes"] - Filter --> ByStart["startDate"] - Filter --> ByEnd["endDate"] -``` - ---- - -## OrganizationQueryFilter - -**Component:** -`openframe-oss-lib.openframe-data-mongo-common.src.main.java.com.openframe.data.document.organization.filter.OrganizationQueryFilter.OrganizationQueryFilter` - -### Purpose - -Defines structured filtering for organization queries. - -### Fields - -- `String category` -- `Integer minEmployees` -- `Integer maxEmployees` -- `Boolean hasActiveContract` -- `String status` - -### Typical Query Logic - -Repositories may: - -- Apply numeric range constraints for employee size -- Apply boolean checks for contract state -- Apply equality matching on category and status - -```mermaid -flowchart LR - OrgFilter["OrganizationQueryFilter"] --> SizeRange["Employee Range"] - OrgFilter --> Contract["Active Contract"] - OrgFilter --> Status["Organization Status"] -``` - ---- - -## TicketQueryFilter - -**Component:** -`openframe-oss-lib.openframe-data-mongo-common.src.main.java.com.openframe.data.document.ticket.filter.TicketQueryFilter.TicketQueryFilter` - -### Purpose - -Provides comprehensive filtering for ticket documents. - -### Fields - -- `List statuses` -- `List organizationIds` -- `List assigneeIds` -- `List labelIds` -- `List deviceIds` -- `List creationSources` -- `Instant createdAtFrom` -- `Instant createdAtTo` - -### Capabilities - -- Multi-status filtering -- Organization-scoped filtering -- Assignee-based filtering -- Label and device constraints -- Creation source classification -- Time-window filtering using `Instant` - -```mermaid -flowchart TD - TicketFilter["TicketQueryFilter"] --> Statuses["TicketStatus List"] - TicketFilter --> OrgScope["Organization IDs"] - TicketFilter --> Assignees["Assignee IDs"] - TicketFilter --> Labels["Label IDs"] - TicketFilter --> Devices["Device IDs"] - TicketFilter --> Source["Creation Sources"] - TicketFilter --> TimeFrom["createdAtFrom"] - TicketFilter --> TimeTo["createdAtTo"] -``` - -This filter is commonly used in custom repository implementations for advanced aggregation and dashboard queries. - ---- - -## ToolQueryFilter - -**Component:** -`openframe-oss-lib.openframe-data-mongo-common.src.main.java.com.openframe.data.document.tool.filter.ToolQueryFilter.ToolQueryFilter` - -### Purpose - -Filters integrated tool documents based on enablement and classification. - -### Fields - -- `Boolean enabled` -- `String type` -- `String category` -- `String platformCategory` - -### Common Use Cases - -- Retrieve only enabled tools -- Filter by tool type (e.g., RMM, MDM) -- Segment tools by platform category - ---- - -## UserQueryFilter - -**Component:** -`openframe-oss-lib.openframe-data-mongo-common.src.main.java.com.openframe.data.document.user.filter.UserQueryFilter.UserQueryFilter` - -### Purpose - -Defines flexible filtering for user queries. - -### Fields - -- `String emailRegex` -- `String nameRegex` -- `UserStatus status` - -### Query Semantics - -- Regex-based filtering for search capabilities -- Status-based filtering (e.g., ACTIVE, DISABLED) - -```mermaid -flowchart LR - UserFilter["UserQueryFilter"] --> Email["emailRegex"] - UserFilter --> Name["nameRegex"] - UserFilter --> Status["UserStatus"] -``` - ---- - -# Integration with Repositories - -The filters are interpreted by repository implementations in the sync and reactive Mongo modules. - -```mermaid -flowchart TD - ServiceLayer["Service Layer"] --> QueryFilter["Query Filter"] - QueryFilter --> CustomRepository["Custom Repository Impl"] - CustomRepository --> CriteriaBuilder["Mongo Criteria Builder"] - CriteriaBuilder --> Mongo[("MongoDB Collection")] -``` - -Repositories typically: - -1. Inspect each non-null filter field. -2. Add corresponding `Criteria` conditions. -3. Combine conditions using `andOperator` / `orOperator`. -4. Apply pagination and sorting. - ---- - -# Benefits to the Platform - -### Clean Separation of Concerns - -- API DTOs define external contract. -- Query filters define internal query semantics. -- Repositories handle persistence mechanics. - -### Extensibility - -Adding a new filter field: - -1. Extend the filter class. -2. Update repository criteria logic. -3. Expose through API input DTO if needed. - -No cross-layer coupling is introduced. - -### Consistency Across Domains - -All filters follow a uniform pattern: - -- Builder-based construction -- Optional fields -- Domain-aligned attributes -- Used exclusively by data layer - ---- - -# Summary - -The **Data Mongo Query Filters** module provides the structured, domain-driven filtering layer for MongoDB queries across OpenFrame. - -It: - -- Encapsulates filtering logic for events, organizations, tickets, tools, and users. -- Bridges API input models with Mongo repository implementations. -- Supports flexible, composable, and maintainable query construction. -- Enforces consistent filtering patterns across the data access layer. - -This module is a foundational building block in the persistence architecture, enabling scalable, expressive, and maintainable MongoDB querying throughout the platform. \ No newline at end of file diff --git a/docs/reference/architecture/data-mongo-reactive-repositories/data-mongo-reactive-repositories.md b/docs/reference/architecture/data-mongo-reactive-repositories/data-mongo-reactive-repositories.md deleted file mode 100644 index 618500f92..000000000 --- a/docs/reference/architecture/data-mongo-reactive-repositories/data-mongo-reactive-repositories.md +++ /dev/null @@ -1,287 +0,0 @@ -# Data Mongo Reactive Repositories - -## Overview - -The **Data Mongo Reactive Repositories** module provides reactive, non-blocking access to MongoDB for selected core domain entities in the OpenFrame platform. Built on **Spring Data Reactive MongoDB** and **Project Reactor**, this module enables asynchronous data access patterns using `Mono` and `Flux` types. - -It complements the synchronous repository layer by offering reactive alternatives for high-concurrency or event-driven use cases, particularly around: - -- OAuth client persistence -- User lookups and existence checks -- Security and authentication flows - -This module integrates closely with: - -- The domain model defined in **Data Mongo Domain Model** -- Shared base repository contracts from the synchronous layer -- Spring Boot auto-configuration and repository scanning - ---- - -## Core Components - -This module contains three primary components: - -- `MongoReactiveConfig` -- `ReactiveOAuthClientRepository` -- `ReactiveUserRepository` - ---- - -## Architecture Overview - -```mermaid -flowchart TD - App["Application Context"] --> Config["MongoReactiveConfig"] - Config -->|"@EnableReactiveMongoRepositories"| Scan["Reactive Repository Scan"] - - Scan --> OAuthRepo["ReactiveOAuthClientRepository"] - Scan --> UserRepo["ReactiveUserRepository"] - - OAuthRepo --> OAuthClient["OAuthClient Document"] - UserRepo --> UserDoc["User Document"] - - OAuthClient --> MongoDB[("MongoDB")] - UserDoc --> MongoDB -``` - -The configuration class enables reactive repository scanning. Spring automatically wires implementations for repository interfaces extending `ReactiveMongoRepository`. - ---- - -## Reactive Programming Model - -This module uses **Project Reactor** primitives: - -- `Mono` β€” zero or one result -- `Flux` β€” zero to many results - -Example semantics: - -```text -Mono β†’ Asynchronous single user lookup -Mono β†’ Asynchronous existence check -``` - -Unlike synchronous repositories that block on I/O, reactive repositories: - -- Use non-blocking MongoDB drivers -- Integrate with WebFlux and reactive security chains -- Support high-concurrency workloads efficiently - ---- - -## Component Details - -### MongoReactiveConfig - -**Class:** `MongoReactiveConfig` -**Package:** `com.openframe.data.config` - -```java -@Configuration -@EnableReactiveMongoRepositories(basePackages = "com.openframe.data.reactive.repository") -public class MongoReactiveConfig { -} -``` - -#### Responsibilities - -- Enables reactive MongoDB repository scanning -- Registers Spring Data reactive repository infrastructure -- Configures repositories located under: - -```text -com.openframe.data.reactive.repository -``` - -This class acts as the entry point for the reactive persistence layer. - ---- - -### ReactiveOAuthClientRepository - -**Interface:** `ReactiveOAuthClientRepository` -**Extends:** `ReactiveMongoRepository` - -```java -@Repository -public interface ReactiveOAuthClientRepository - extends ReactiveMongoRepository { - - Mono findByClientId(String clientId); -} -``` - -#### Responsibilities - -- Reactive CRUD operations for `OAuthClient` -- Lookup by `clientId` -- Used by authentication and OAuth flows - -#### Data Flow - -```mermaid -sequenceDiagram - participant Service as "Auth Service" - participant Repo as "ReactiveOAuthClientRepository" - participant DB as "MongoDB" - - Service->>Repo: findByClientId(clientId) - Repo->>DB: Reactive query - DB-->>Repo: OAuthClient document - Repo-->>Service: Mono -``` - -#### Typical Use Cases - -- OAuth client validation -- Client credentials flow -- Token issuance validation -- Dynamic client lookup - ---- - -### ReactiveUserRepository - -**Interface:** `ReactiveUserRepository` -**Extends:** -- `ReactiveMongoRepository` -- `BaseUserRepository, Mono, String>` - -```java -@Repository -public interface ReactiveUserRepository - extends ReactiveMongoRepository, - BaseUserRepository, Mono, String> { - - Mono findByEmail(String email); - - Mono existsByEmail(String email); - - Mono existsByEmailAndStatus(String email, UserStatus status); -} -``` - -#### Responsibilities - -- Reactive user retrieval by email -- Email existence checks -- Status-aware existence validation -- Compliance with shared `BaseUserRepository` contract - -#### Inheritance Model - -```mermaid -flowchart LR - BaseRepo["BaseUserRepository"] --> ReactiveRepo["ReactiveUserRepository"] - ReactiveMongo["ReactiveMongoRepository"] --> ReactiveRepo - ReactiveRepo --> UserDoc["User Document"] -``` - -The repository: - -- Inherits generic user contract behavior -- Adapts return types to `Mono` -- Ensures consistency across reactive and synchronous layers - -#### Example Reactive Pattern - -```text -findByEmail(email) - β†’ returns Mono - β†’ may emit value or complete empty -``` - -This integrates directly with: - -- Reactive security filters -- OAuth BFF layers -- WebFlux controllers - ---- - -## Relationship to Other Modules - -### Data Mongo Domain Model - -The reactive repositories operate on domain documents defined in: - -- `User` -- `OAuthClient` - -These are defined in the **Data Mongo Domain Model** module. - -(See the corresponding domain model documentation for detailed schema information.) - ---- - -### Data Mongo Base Repositories - -`ReactiveUserRepository` implements the generic `BaseUserRepository` contract, ensuring: - -- Cross-layer API consistency -- Shared query method definitions -- Type-safe generics for both blocking and reactive variants - -This pattern allows the platform to maintain consistent repository contracts while supporting different execution models. - ---- - -## Reactive vs Synchronous Repositories - -| Aspect | Reactive | Synchronous | -|--------|----------|------------| -| Driver | Reactive MongoDB driver | Blocking MongoDB driver | -| Return Types | `Mono`, `Flux` | `T`, `List` | -| Thread Model | Event-loop friendly | Thread-per-request | -| Backpressure | Supported | Not supported | - -The reactive repositories are ideal for: - -- High-concurrency authentication workloads -- Streaming integrations -- Event-driven services - ---- - -## Security and OAuth Integration - -Reactive repositories are particularly important in: - -- OAuth client validation -- Email-based login checks -- Account existence validation -- Status-based user access control - -Because these flows are frequently executed and latency-sensitive, non-blocking access improves scalability and throughput. - ---- - -## Extension Guidelines - -When adding new reactive repositories: - -1. Place them under: - -```text -com.openframe.data.reactive.repository -``` - -2. Extend `ReactiveMongoRepository` -3. Return `Mono` or `Flux` -4. If applicable, implement shared base repository contracts -5. Ensure the entity exists in the domain model module - ---- - -## Summary - -The **Data Mongo Reactive Repositories** module provides: - -- Spring Data reactive MongoDB integration -- Reactive user and OAuth client persistence -- Contract alignment with base repository interfaces -- Non-blocking access patterns for authentication-critical flows - -It forms a lightweight but critical foundation for scalable security, authentication, and identity-related operations across the OpenFrame platform. \ No newline at end of file diff --git a/docs/reference/architecture/data-mongo-sync-config-and-custom-repositories/data-mongo-sync-config-and-custom-repositories.md b/docs/reference/architecture/data-mongo-sync-config-and-custom-repositories/data-mongo-sync-config-and-custom-repositories.md deleted file mode 100644 index 4c8b166e1..000000000 --- a/docs/reference/architecture/data-mongo-sync-config-and-custom-repositories/data-mongo-sync-config-and-custom-repositories.md +++ /dev/null @@ -1,434 +0,0 @@ -# Data Mongo Sync Config And Custom Repositories - -## Overview - -The **Data Mongo Sync Config And Custom Repositories** module provides the synchronous MongoDB configuration and advanced repository implementations used across the OpenFrame platform. - -It is responsible for: - -- Bootstrapping synchronous MongoDB repositories -- Customizing Mongo mapping and auditing behavior -- Managing indexes and legacy index cleanup -- Implementing advanced filtering, cursor-based pagination, sorting, and aggregation logic -- Providing bulk and optimized update operations -- Supporting optimistic locking retry observability - -This module builds on: - -- [Data Mongo Domain Model](../data-mongo-domain-model/data-mongo-domain-model.md) -- [Data Mongo Query Filters](../data-mongo-query-filters/data-mongo-query-filters.md) -- [Data Mongo Base Repositories](../data-mongo-base-repositories/data-mongo-base-repositories.md) - -It acts as the **data-access execution layer** for API services, management services, and background processors. - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - ApiLayer["API Service Layer"] --> RepositoryLayer["Custom Repositories"] - RepositoryLayer --> MongoTemplate["MongoTemplate"] - MongoTemplate --> MongoDB[("MongoDB")] - - DomainModel["Domain Documents"] --> RepositoryLayer - QueryFilters["Query Filter DTOs"] --> RepositoryLayer - - Config["Mongo Sync Config"] --> RepositoryLayer - IndexConfig["Mongo Index Config"] --> MongoDB -``` - -### Key Responsibilities - -| Layer | Responsibility | -|--------|----------------| -| Mongo Sync Config | Enables repositories, auditing, mapping customization | -| Mongo Index Config | Ensures required indexes, removes legacy indexes | -| Custom Repositories | Filtering, cursor pagination, aggregations, bulk updates | -| Retry Listener | Observability for optimistic locking retries | - ---- - -# Configuration Layer - -## Mongo Sync Config - -**Component:** `MongoSyncConfig` - -Enables synchronous Mongo repositories and configures Mongo mapping behavior. - -### Features - -- Conditional activation via `spring.data.mongodb.enabled` -- Enables `@EnableMongoRepositories` for `com.openframe.data.repository` -- Enables `@EnableMongoAuditing` -- Custom `MappingMongoConverter` - -### Mapping Customization - -```mermaid -flowchart LR - Factory["MongoDatabaseFactory"] --> Resolver["DefaultDbRefResolver"] - Resolver --> Converter["MappingMongoConverter"] - Conversions["MongoCustomConversions"] --> Converter -``` - -Notable configuration: - -- `setMapKeyDotReplacement("__dot__")` - - Allows storing map keys containing dots (`.`) safely - -This is critical for metadata-heavy documents like events and external application integrations. - ---- - -## Mongo Index Config - -**Component:** `MongoIndexConfig` - -Executed at application startup via `@PostConstruct`. - -### Index Responsibilities - -- Ensures compound indexes on `application_events` -- Drops stale legacy indexes - -### Example Indexes - -- `{ userId: ASC, timestamp: DESC }` -- `{ type: ASC, metadata.tags: ASC }` - -### Legacy Cleanup - -```mermaid -flowchart TD - Startup["Application Startup"] --> EnsureIndexes["Ensure Required Indexes"] - EnsureIndexes --> DropLegacy["Drop Stale Indexes"] - DropLegacy --> LogResult["Log Success Or Skip"] -``` - -The cleanup ensures: - -- Schema evolution safety -- Removal of org-scoped uniqueness constraints -- Tenant-wide tag uniqueness enforcement consistency - ---- - -# Custom Repository Implementations - -This module implements advanced data-access behavior beyond standard Spring Data repositories. - -Common patterns across repositories: - -- Cursor-based pagination using `_id` -- Secondary sort field with `_id` tie-breaker -- Mongo `Criteria`-based dynamic filtering -- Aggregation pipelines for grouped metrics -- Distinct queries for filter options -- Bulk operations for efficiency - ---- - -## Cursor-Based Pagination Pattern - -Most repositories follow this strategy: - -```mermaid -flowchart TD - ClientRequest["Client Request With Cursor"] --> ParseCursor["Parse ObjectId"] - ParseCursor --> LoadCursorDoc["Load Cursor Document"] - LoadCursorDoc --> CompareSortField["Compare Sort Field"] - CompareSortField --> AddCriteria["Add $or Criteria"] - AddCriteria --> ApplySort["Sort By Field And _id"] - ApplySort --> Limit["Apply Limit"] -``` - -This ensures: - -- Stable ordering -- Deterministic pagination -- No duplicates or gaps -- Proper tie-breaking when sort field values are equal - ---- - -# Repository Categories - -## 1. Assignment Repository - -**Component:** `CustomItemAssignmentRepositoryImpl` - -Features: - -- Cursor-based pagination -- Search by display name -- Group-by aggregation per `AssignmentTargetType` - -Aggregation example: - -```mermaid -flowchart LR - Match["Match itemId"] --> Group["Group By targetType"] - Group --> Count["Count"] - Count --> MapResult["Map To EnumMap"] -``` - ---- - -## 2. Machine Repository - -**Component:** `CustomMachineRepositoryImpl` - -Capabilities: - -- Multi-field filtering (status, type, OS, organization) -- Regex-based search (hostname, IP, serial, model) -- Cursor pagination with compound sorting -- Count queries for pagination metadata - ---- - -## 3. Event Repositories - -### External Application Event Repository - -**Component:** `ExternalApplicationEventRepository` - -- Spring Data `MongoRepository` -- Time-range queries -- Custom `@Query` for metadata tag filtering - -### Custom Event Repository - -**Component:** `CustomEventRepositoryImpl` - -Features: - -- Date range filtering -- Distinct queries (userId, event type) -- Cursor pagination -- Search over type and data fields - ---- - -## 4. Knowledge Base Repository - -**Component:** `CustomKnowledgeBaseItemRepositoryImpl` - -Advanced behaviors: - -- Folder vs article separation -- Archived vs active filtering -- Combined `$or` search and cursor criteria using `$and` -- Draft visibility model - -### Composite Criteria Strategy - -Spring Data does not allow multiple root `$or` conditions. - -Solution: - -```mermaid -flowchart TD - SearchCriteria["Search OR Criteria"] --> Combine - CursorCriteria["Cursor OR Criteria"] --> Combine - Combine["Combine Using $and"] --> FinalQuery["Final Query"] -``` - -This avoids null-key collisions in the Mongo query builder. - ---- - -## 5. Notification Repositories - -### Custom Notification Repository - -**Component:** `CustomNotificationRepositoryImpl` - -Implements: - -- Recipient-based filtering -- Read/unread filtering -- Title search -- Forward/backward pagination -- Two-phase fetch (read states first, then notifications) - -```mermaid -flowchart TD - FetchReadStates["Fetch NotificationReadState"] --> ExtractIds - ExtractIds["Extract Notification IDs"] --> FetchNotifications - FetchNotifications["Fetch Notification Documents"] --> Merge["Merge With Status"] -``` - -### Custom Notification Read State Repository - -**Component:** `CustomNotificationReadStateRepositoryImpl` - -- Bulk unordered insert -- Gracefully swallows duplicate-key errors -- Improves idempotency in concurrent flows - ---- - -## 6. Ticket Repository - -**Component:** `CustomTicketRepositoryImpl` - -One of the most feature-rich repositories. - -Capabilities: - -- Complex filtering (status, org, assignee, device) -- Created-at range filtering -- Cursor-based pagination with tie-breaker -- Aggregation metrics: - - Count by status - - Average resolution time -- Bulk status update -- Partial field updates (title) - -### Average Resolution Time Aggregation - -```mermaid -flowchart LR - MatchResolved["Match Resolved Tickets"] --> ProjectDiff["Compute resolvedAt - createdAt"] - ProjectDiff --> GroupAvg["Average Resolution Time"] -``` - ---- - -## 7. Organization Repository - -**Component:** `CustomOrganizationRepositoryImpl` - -Features: - -- Default ACTIVE status filtering -- Contract validity evaluation -- Employee range filters -- Regex category matching -- Cursor pagination with multi-field sorting - ---- - -## 8. Integrated Tool Repository - -**Component:** `CustomIntegratedToolRepositoryImpl` - -Capabilities: - -- Enabled/type/category filtering -- Search over name and description -- Distinct queries for: - - type - - category - - platformCategory - ---- - -## 9. User Repository - -**Component:** `CustomUserRepositoryImpl` - -Focused on search: - -- Status filtering -- Email regex -- Name regex (first OR last name) -- Sorted by createdAt descending - ---- - -## 10. OAuth Token Repository - -**Component:** `OAuthTokenRepository` - -Simple Spring Data repository with: - -- `findByAccessToken` -- `findByRefreshToken` - -Used by authentication and token lifecycle management flows. - ---- - -## 11. Ticket Attachments And Notes - -- `TicketAttachmentRepository` -- `TicketNoteRepository` - -Provide standard CRUD plus: - -- Find by ticket ID -- Batch lookup -- Delete by ticket ID -- Sorted retrieval (notes) - ---- - -# Retry Observability - -## Optimistic Locking Retry Listener - -**Component:** `OptimisticLockingRetryListener` - -Provides structured logging for: - -- Retry attempts -- Retry success -- Retry exhaustion - -```mermaid -flowchart TD - RetryStart["Retry Attempt"] --> OnError["onError()"] - OnError --> LogWarn["Log Warning"] - RetryEnd["Retry Complete"] --> Close - Close --> SuccessOrFail["Log Success Or Error"] -``` - -This improves: - -- Visibility into concurrency conflicts -- Operational debugging -- Retry exhaustion tracing - ---- - -# Cross-Module Relationships - -The **Data Mongo Sync Config And Custom Repositories** module interacts heavily with: - -- **Domain documents** from Data Mongo Domain Model -- **Query filter DTOs** from Data Mongo Query Filters -- **API Services** and **Management Services** that orchestrate business logic -- **Authorization Service** when dealing with OAuth tokens - -It serves as the **core synchronous persistence engine** of the OpenFrame platform. - ---- - -# Design Principles - -1. Database-Level Filtering First -2. Deterministic Cursor Pagination -3. Safe Aggregations With Type Mapping -4. Bulk Operations For Performance -5. Graceful Handling Of Legacy Indexes -6. Idempotent Insert Strategies -7. Clear Separation Between Query Building And Execution - ---- - -# Summary - -The **Data Mongo Sync Config And Custom Repositories** module: - -- Configures MongoDB for synchronous workloads -- Ensures proper indexing and schema evolution safety -- Implements advanced query logic across all major domain entities -- Provides efficient aggregation and bulk operations -- Enables stable cursor-based pagination across the platform -- Improves concurrency observability via retry listeners - -It is a foundational infrastructure module that underpins all synchronous data access within the OpenFrame ecosystem. \ No newline at end of file diff --git a/docs/reference/architecture/data-nats-notifications/data-nats-notifications.md b/docs/reference/architecture/data-nats-notifications/data-nats-notifications.md deleted file mode 100644 index 5f5202f3a..000000000 --- a/docs/reference/architecture/data-nats-notifications/data-nats-notifications.md +++ /dev/null @@ -1,365 +0,0 @@ -# Data Nats Notifications - -## Overview - -The **Data Nats Notifications** module is responsible for broadcasting real-time notifications over NATS while ensuring durable persistence in MongoDB. It acts as the bridge between: - -- The persistent notification domain model (Mongo documents) -- The read-state tracking subsystem -- The NATS messaging infrastructure -- Downstream consumers such as WebSocket gateways, clients, and machine agents - -This module guarantees that: - -- Notifications are **persisted first** (source of truth). -- Read states are created for each recipient. -- Real-time delivery over NATS is **best-effort and non-blocking**. -- Clients can reconcile missed events via GraphQL or REST catch-up. - -It is feature-flagged and can operate in persistence-only mode when NATS is disabled. - ---- - -## Core Responsibilities - -The Data Nats Notifications module provides: - -1. βœ… Command validation and audience sanitation -2. βœ… Durable notification persistence -3. βœ… Read-state creation for users and machines -4. βœ… NATS subject-based publishing -5. βœ… Safe failure handling with rollback protection -6. βœ… Graceful degradation when NATS is unavailable - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - Caller["Application Service"] --> Command["NotificationCommand"] - Command --> Broadcaster["NotificationBroadcaster"] - - Broadcaster --> Repo["NotificationRepository"] - Broadcaster --> ReadState["NotificationReadStateService"] - Broadcaster --> Registry["NotificationContextDescriptorRegistry"] - - Broadcaster -->|"optional"| Publisher["NotificationNatsPublisher"] - Publisher --> Nats["NATS Server"] - - Repo --> Mongo[("MongoDB")] - ReadState --> Mongo -``` - -### Key Design Principle - -**Persistence is the source of truth.** - -NATS delivery is best-effort. If publishing fails, the notification remains persisted and recipients reconcile via API queries. - ---- - -## Notification Lifecycle - -```mermaid -flowchart TD - Start["Broadcast Request"] --> Validate["Validate NotificationCommand"] - Validate --> Persist["Persist Notification"] - Persist --> CreateReadStates["Create Read States"] - CreateReadStates --> Publish["Publish to NATS"] - Publish --> End["Complete"] - - CreateReadStates -->|"failure"| Rollback["Delete Notification"] -``` - -### Transactional Strategy - -- If **read-state creation fails**, the notification document is deleted. -- If **NATS publish fails**, no rollback occurs. -- Recipients catch up using stored notifications. - ---- - -# Core Components - -## 1. NotificationCommand - -Immutable command object representing a validated broadcast request. - -### Responsibilities - -- Enforces required fields: - - `title` - - `severity` - - `context` -- Ensures `context.type` is non-blank -- Sanitizes and validates: - - `adminAudience` - - `machineAudience` -- Requires at least one non-empty audience -- Produces immutable recipient sets - -### Validation Rules - -```text -- title must not be blank -- severity must not be null -- context must not be null -- context.type must not be blank -- at least one audience must be non-empty -- no blank entries in audience sets -``` - -This prevents invalid broadcasts from reaching persistence or NATS. - ---- - -## 2. NotificationBroadcaster - -The orchestration service of the module. - -### Dependencies - -- `NotificationRepository` -- `NotificationReadStateService` -- `NotificationContextDescriptorRegistry` -- Optional `NotificationNatsPublisher` - -### Feature Flag - -```text -openframe.features.notifications.enabled=false -``` - -If disabled: -- No persistence -- No read states -- No NATS publish -- Broadcast returns `null` - ---- - -### Broadcast Flow - -```mermaid -flowchart LR - Cmd["NotificationCommand"] --> Build["Build Notification"] - Build --> Save["Save to Repository"] - Save --> Category["Resolve Category"] - Category --> RS["Create Read States"] - RS --> Publish["Publish Per Recipient"] -``` - -### Read-State Creation - -Recipients are separated by type: - -- `RecipientType.USER` -- `RecipientType.MACHINE` - -The category is derived from: - -- `NotificationContextDescriptorRegistry` - -If read-state creation throws an exception: - -1. The persisted notification is deleted. -2. The exception is rethrown. -3. Caller must retry. - ---- - -### NATS Publishing Strategy - -Publishing is performed per-recipient: - -- Each admin user β†’ `user.{userId}.notification` -- Each machine β†’ `machine.{machineId}.notification` - -Failures are handled individually: - -- Logged as warnings -- No rollback -- Recipient reconciles via API - ---- - -## 3. NotificationNatsPublisher - -Encapsulates subject generation and NATS publishing. - -### Conditional Activation - -```text -@ConditionalOnProperty("spring.cloud.stream.enabled") -``` - -If Spring Cloud Stream is disabled: -- Publisher bean is not created -- Broadcaster logs persistence-only mode - ---- - -### Subject Templates - -```text -user.{userId}.notification -machine.{machineId}.notification -``` - -### Publish Safety Rules - -- `userId` and `machineId` must not be blank -- Notification must be persisted (must have `id`) -- `NatsException` is caught and logged -- No exception propagates to caller - ---- - -### Message Construction - -The domain `Notification` document is mapped to a lightweight transport model: - -```mermaid -flowchart TD - Doc["Notification Document"] --> Msg["NotificationMessage"] - - Msg --> Id["id"] - Msg --> Severity["severity"] - Msg --> Title["title"] - Msg --> Desc["description"] - Msg --> Created["createdAt"] - Msg --> Context["context"] -``` - -Only fields required by clients are included. - ---- - -## 4. NotificationMessage - -Transport DTO published to NATS. - -### Fields - -```text -id : String -severity : NotificationSeverity -title : String -description : String -createdAt : Instant -context : NotificationContext -``` - -Designed to: - -- Be serialization-friendly -- Avoid heavy domain coupling -- Provide enough context for real-time rendering - ---- - -# Reliability Model - -The module intentionally separates: - -- **Storage reliability** (strong) -- **Delivery reliability** (eventual) - -```mermaid -flowchart TD - Persist["Persist Notification"] --> Durable["Durable Storage"] - Durable --> RealTime["Real-Time Publish"] - - RealTime -->|"failure"| CatchUp["Client Catch-Up via API"] -``` - -### Guarantees - -| Concern | Guarantee | -|----------|-----------| -| Persistence | Strong (Mongo) | -| Read State | Strong (created before publish) | -| NATS Delivery | Best effort | -| Consistency | Eventual for clients | - ---- - -# Integration with the Platform - -Although self-contained, this module integrates with: - -- Mongo notification domain model -- Notification read-state services -- NATS infrastructure -- GraphQL notification queries -- Gateway WebSocket or client subscribers - -### Typical End-to-End Flow - -```mermaid -sequenceDiagram - participant Service - participant Broadcaster - participant Mongo - participant NATS - participant Client - - Service->>Broadcaster: broadcast(command) - Broadcaster->>Mongo: save(notification) - Broadcaster->>Mongo: createReadStates() - Broadcaster->>NATS: publish(subject, message) - NATS->>Client: deliver real-time event - Client->>Mongo: query for reconciliation -``` - ---- - -# Design Principles - -## 1. Fail-Safe Persistence - -No notification is published without being persisted. - -## 2. Controlled Rollback - -Only read-state failures trigger deletion. - -## 3. Optional Real-Time Layer - -System works fully without NATS. - -## 4. Recipient Isolation - -Failures per-recipient do not block others. - -## 5. Immutable Command Pattern - -Validation happens before orchestration. - ---- - -# Configuration Summary - -```text -openframe.features.notifications.enabled -spring.cloud.stream.enabled -``` - -| Property | Purpose | -|----------|----------| -| openframe.features.notifications.enabled | Master feature flag | -| spring.cloud.stream.enabled | Enables NATS publisher bean | - ---- - -# Conclusion - -The **Data Nats Notifications** module provides a robust, fault-tolerant notification broadcasting mechanism built on: - -- Strong persistence guarantees -- Read-state tracking -- Optional real-time messaging via NATS -- Safe error handling and graceful degradation - -It ensures that OpenFrame can deliver reliable notifications to both administrators and machine agents while maintaining consistency and operational safety across distributed services. \ No newline at end of file diff --git a/docs/reference/architecture/data-pinot-repositories/data-pinot-repositories.md b/docs/reference/architecture/data-pinot-repositories/data-pinot-repositories.md deleted file mode 100644 index c3a1457da..000000000 --- a/docs/reference/architecture/data-pinot-repositories/data-pinot-repositories.md +++ /dev/null @@ -1,308 +0,0 @@ -# Data Pinot Repositories - -## Overview - -The **Data Pinot Repositories** module provides analytical data access capabilities using **Apache Pinot** for high-performance, real-time OLAP queries. While transactional and operational data is stored in MongoDB, this module is responsible for: - -- Real-time log analytics -- Device-level aggregations and facet queries -- High-volume, time-series filtering -- Efficient count and distinct queries for UI filters - -It acts as the **analytics read layer** of the OpenFrame platform, optimized for dashboards, filters, and log exploration workloads. - ---- - -## Architectural Role in the Platform - -The platform follows a polyglot persistence architecture: - -- **MongoDB** β†’ transactional and domain persistence -- **Kafka / Stream Service** β†’ event ingestion and enrichment -- **Apache Pinot** β†’ analytical querying and aggregations - -The Data Pinot Repositories module sits on top of Pinot and exposes structured repository abstractions used by API and management services. - -```mermaid -flowchart LR - StreamService["Stream Service Core Kafka And Handlers"] -->|"Ingest Events"| PinotCluster["Apache Pinot Cluster"] - MongoDB["Data Mongo Domain Model"] -->|"Transactional Data"| Services["API & Management Services"] - PinotCluster -->|"Analytical Queries"| PinotRepos["Data Pinot Repositories"] - PinotRepos -->|"Facet Counts & Logs"| Services -``` - -### Upstream Dependencies - -- Event ingestion from **Stream Service Core Kafka And Handlers** -- Pinot cluster availability and schema configuration - -### Downstream Consumers - -- API Service (GraphQL and REST) -- External API Service -- Management Service (resync operations) - ---- - -## Core Components - -The module contains the following core components: - -| Component | Responsibility | -|------------|----------------| -| `PinotConfig` | Configures Pinot broker and controller connections | -| `PinotEventEntity` | Domain marker for Pinot-backed event projections | -| `PinotClientDeviceRepository` | Device analytics and facet queries | -| `PinotClientLogRepository` | Log search, filtering, and projection queries | - ---- - -## Configuration Layer - -### PinotConfig - -`PinotConfig` provides Spring-managed `Connection` beans for interacting with Pinot: - -- **Broker Connection** β†’ Used for query execution -- **Controller Connection** β†’ Used for cluster/controller operations - -Configuration properties: - -- `pinot.broker.url` -- `pinot.controller.url` -- `pinot.tables.devices.name` -- `pinot.tables.logs.name` - -```mermaid -flowchart TD - AppConfig["Spring Context"] --> PinotConfig["PinotConfig"] - PinotConfig --> BrokerConn["pinotBrokerConnection()"] - PinotConfig --> ControllerConn["pinotControllerConnection()"] - BrokerConn --> PinotCluster["Pinot Broker"] - ControllerConn --> PinotController["Pinot Controller"] -``` - -The broker connection is injected into repository implementations using `@Qualifier("pinotBrokerConnection")`. - ---- - -## Device Analytics Repository - -### PinotClientDeviceRepository - -The **PinotClientDeviceRepository** provides analytical queries over the `devices` Pinot table. - -It supports: - -- Faceted filter queries -- Aggregated device counts -- Multi-dimensional filtering -- Tag-based filtering - -### Supported Filters - -- Status (excluding `DELETED`) -- Device type -- OS type -- Organization -- Tags -- Tag key-value pairs - -### Facet Query Pattern - -Facet queries dynamically exclude the field being aggregated to ensure correct filter option computation. - -```mermaid -flowchart TD - Start["Facet Request"] --> BuildQuery["PinotQueryBuilder"] - BuildQuery --> ApplyFilters["applyDeviceFilters()"] - ApplyFilters --> ExcludeField{"Exclude Facet Field?"} - ExcludeField -->|"Yes"| SkipFilter["Skip That Filter"] - ExcludeField -->|"No"| ApplyFilter["Apply Filter"] - SkipFilter --> GroupBy["GROUP BY facetField"] - ApplyFilter --> GroupBy - GroupBy --> Execute["executeKeyCountQuery()"] -``` - -### Key Behaviors - -1. Always excludes devices with status `DELETED`. -2. Dynamically removes the active facet field from filters. -3. Returns `Map` for UI-ready filter counts. -4. Uses count queries for total filtered device count. - -This repository is optimized for dashboard filters and device inventory analytics. - ---- - -## Log Analytics Repository - -### PinotClientLogRepository - -The **PinotClientLogRepository** supports time-series log exploration and advanced filtering. - -### Capabilities - -- Date range filtering -- Cursor-based pagination -- Relevance search -- Sortable column validation -- Distinct filter options -- Organization option projections - -### Query Construction Flow - -```mermaid -flowchart LR - Request["Log Query Request"] --> Builder["PinotQueryBuilder"] - Builder --> DateRange["whereDateRange()"] - DateRange --> Filters["whereIn() / whereEquals()"] - Filters --> Search["whereRelevanceLogSearch()"] - Search --> Cursor["whereCursor()"] - Cursor --> Sorting["orderBySortInput()"] - Sorting --> Limit["limit()"] - Limit --> Execute["executeLogQuery()"] -``` - -### Sorting Controls - -The repository enforces a whitelist of sortable columns: - -- `eventTimestamp` -- `severity` -- `eventType` -- `toolType` -- `organizationId` -- `deviceId` -- `ingestDay` - -If a field is not sortable: - -- It falls back to `eventTimestamp`. - -This protects against invalid or unsafe query construction. - -### Projection Mapping - -Results are mapped into `LogProjection` objects using column index mapping derived from Pinot result sets. - -Important fields include: - -- `toolEventId` -- `eventTimestamp` -- `toolType` -- `eventType` -- `severity` -- `organizationId` -- `summary` - -The `eventTimestamp` is converted from epoch milliseconds to `Instant`. - ---- - -## Multi-Tenant Isolation - -All queries are built using a `tenantId` parameter via `PinotQueryBuilder`. - -This ensures: - -- Logical data isolation -- Tenant-safe aggregations -- Secure filtering across shared Pinot tables - -```mermaid -flowchart TD - Query["Incoming Query"] --> TenantFilter["Apply tenantId"] - TenantFilter --> PinotExec["Execute on Shared Table"] - PinotExec --> TenantScoped["Tenant Scoped Result"] -``` - ---- - -## Interaction with Other Modules - -### Stream Ingestion - -Events are ingested via Kafka and enriched by the stream layer before being indexed into Pinot. - -See: - -- [Stream Service Core Kafka And Handlers](../stream-service-core-kafka-and-handlers.md) - -### Device Pinot Resynchronization - -Management workflows may trigger re-synchronization of device data into Pinot. - -See: - -- [Management Service Core Initializers And Schedulers](../management-service-core-initializers-and-schedulers.md) - -### API Layer Consumption - -The API layer consumes Pinot repositories to provide: - -- Filter option endpoints -- Log search endpoints -- Aggregated dashboard counts - ---- - -## Design Principles - -### 1. Separation of Concerns - -- MongoDB β†’ transactional storage -- Pinot β†’ analytical read queries - -### 2. Query Builder Abstraction - -All query logic flows through a `PinotQueryBuilder`, ensuring: - -- Safe dynamic query construction -- Consistent filter application -- Centralized cursor logic - -### 3. Facet-First Design - -Repositories are optimized for UI filter experiences: - -- Count per facet value -- Distinct option lists -- Filter exclusion logic - -### 4. Performance-Oriented - -- Aggregations pushed to Pinot -- Minimal application-side computation -- Projection-based mapping - ---- - -## End-to-End Log Analytics Flow - -```mermaid -flowchart LR - Client["UI / API Request"] --> ApiService["API Service"] - ApiService --> LogRepo["PinotClientLogRepository"] - LogRepo --> PinotBroker["Pinot Broker"] - PinotBroker --> PinotServer["Pinot Servers"] - PinotServer --> PinotBroker - PinotBroker --> LogRepo - LogRepo --> ApiService - ApiService --> Client -``` - ---- - -## Summary - -The **Data Pinot Repositories** module provides the analytical backbone of the OpenFrame platform. - -It enables: - -- Real-time log exploration -- High-performance device filtering -- Multi-dimensional facet queries -- Secure tenant-scoped analytics - -By leveraging Apache Pinot, it ensures scalable, low-latency query execution for dashboard and filtering workloads while keeping transactional systems isolated in MongoDB. \ No newline at end of file diff --git a/docs/reference/architecture/data-redis-cache/data-redis-cache.md b/docs/reference/architecture/data-redis-cache/data-redis-cache.md deleted file mode 100644 index cb2f0e676..000000000 --- a/docs/reference/architecture/data-redis-cache/data-redis-cache.md +++ /dev/null @@ -1,240 +0,0 @@ -# Data Redis Cache - -The **Data Redis Cache** module provides Redis-based caching and key infrastructure for the OpenFrame platform. It integrates Spring Cache with Redis, configures synchronous and reactive Redis templates, and enforces tenant-aware key prefixing to ensure strict data isolation across organizations. - -This module acts as the foundational caching layer used by API services, management services, and other components that require high-performance, low-latency data access. - ---- - -## Purpose and Responsibilities - -The Data Redis Cache module is responsible for: - -- Enabling and configuring Spring Cache backed by Redis -- Providing a tenant-aware cache key prefix strategy -- Supplying `RedisTemplate` and `ReactiveRedisTemplate` beans -- Enforcing standardized key/value serialization -- Supporting conditional activation based on configuration - -Redis is enabled only when the property `spring.redis.enabled=true` is set, allowing flexible deployment topologies. - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - AppServices["Application Services"] --> CacheAbstraction["Spring Cache Abstraction"] - CacheAbstraction --> CacheManager["RedisCacheManager"] - CacheManager --> RedisConnection["RedisConnectionFactory"] - CacheManager --> KeyBuilder["OpenframeRedisKeyBuilder"] - RedisConnection --> RedisServer[("Redis Server")] - - ReactiveServices["Reactive Services"] --> ReactiveTemplate["ReactiveRedisTemplate"] - ReactiveTemplate --> ReactiveConnection["ReactiveRedisConnectionFactory"] - ReactiveConnection --> RedisServer -``` - -### Key Components - -| Component | Responsibility | -|------------|----------------| -| CacheConfig | Configures Spring Cache and RedisCacheManager | -| RedisConfig | Provides RedisTemplate and ReactiveRedisTemplate beans | -| OpenframeRedisKeyConfiguration | Registers OpenframeRedisKeyBuilder for tenant-aware key prefixes | - ---- - -# Core Configuration Classes - -## CacheConfig - -**Class:** `CacheConfig` - -This class enables Spring caching and configures the Redis-backed `CacheManager`. - -### Activation - -The configuration is loaded only if: - -```text -spring.redis.enabled=true -``` - -### Default Cache Behavior - -The default cache configuration includes: - -- Time-to-live (TTL): 6 hours -- Null values disabled -- String key serialization -- JSON value serialization using `GenericJackson2JsonRedisSerializer` -- Tenant-aware key prefixing - -### Tenant-Aware Key Prefixing - -All cache keys follow this structure: - -```text -::: -``` - -The prefix is computed via `OpenframeRedisKeyBuilder`, ensuring isolation between tenants. - -### Custom TTL Overrides - -Certain caches use shorter TTL values: - -| Cache Name | TTL | -|------------|------| -| fleetPolicyCache | 1 hour | -| fleetQueryCache | 1 hour | - -This prevents stale policy or query data from persisting too long. - -### Cache Initialization Flow - -```mermaid -flowchart TD - Start["Application Startup"] --> RedisEnabled{"Redis Enabled?"} - RedisEnabled -->|Yes| CreateManager["Create RedisCacheManager"] - RedisEnabled -->|No| Skip["Skip Redis Cache Config"] - CreateManager --> DefaultConfig["Apply Default TTL 6h"] - DefaultConfig --> FleetOverride["Override Fleet TTL 1h"] - FleetOverride --> Ready["CacheManager Ready"] -``` - ---- - -## RedisConfig - -**Class:** `RedisConfig` - -This class provides low-level Redis beans for both blocking and reactive usage. - -### Provided Beans - -| Bean | Type | Purpose | -|------|------|----------| -| redisTemplate | RedisTemplate | Standard Redis operations | -| reactiveStringRedisTemplate | ReactiveStringRedisTemplate | Reactive string-based operations | -| reactiveRedisTemplate | ReactiveRedisTemplate | Reactive key-value operations | - -### Serialization Strategy - -All templates use: - -- String serialization for keys -- String serialization for values (in template beans) -- JSON serialization for Spring Cache values (via CacheConfig) - -This ensures consistent behavior across imperative and reactive flows. - -### Repository Support - -`@EnableRedisRepositories` enables Redis-backed repositories under: - -```text -com.openframe.data.repository.redis -``` - -This allows future extensions such as token stores, distributed locks, or transient state persistence. - ---- - -## OpenframeRedisKeyConfiguration - -**Class:** `OpenframeRedisKeyConfiguration` - -This configuration registers the `OpenframeRedisKeyBuilder` bean. - -### Responsibilities - -- Binds `OpenframeRedisProperties` -- Constructs a reusable key builder -- Ensures consistent prefix generation across the platform - -The key builder centralizes key naming conventions, preventing: - -- Cross-tenant collisions -- Environment conflicts -- Inconsistent naming patterns - ---- - -# Multi-Tenant Key Strategy - -The Data Redis Cache module enforces strict tenant-aware isolation. - -```mermaid -flowchart LR - TenantA["Tenant A"] --> KeyBuilder - TenantB["Tenant B"] --> KeyBuilder - KeyBuilder --> RedisKeyA["tenantA:deviceCache::123"] - KeyBuilder --> RedisKeyB["tenantB:deviceCache::123"] - RedisKeyA --> RedisServer[("Redis")] - RedisKeyB --> RedisServer -``` - -Even when logical cache keys are identical, the computed prefix ensures physical separation in Redis. - ---- - -# Integration Within the Platform - -The Data Redis Cache module supports multiple platform layers: - -- API services for query result caching -- Authorization services for token or metadata caching -- Management services for transient synchronization data -- Gateway services for rate limiting or session metadata - -It works alongside: - -- Mongo repositories (primary persistence layer) -- Kafka streaming services (event-driven updates) -- Security modules (authentication and token validation) - -Redis acts as a performance optimization layer β€” never the source of truth. - ---- - -# Conditional Activation and Deployment - -Redis caching is optional and controlled by configuration. - -```text -spring.redis.enabled=true -``` - -If disabled: - -- CacheConfig is not loaded -- RedisConfig is not loaded -- No Redis repositories are enabled - -This design supports: - -- Local development without Redis -- Staging environments with partial caching -- Production environments with full caching enabled - ---- - -# Design Principles - -The Data Redis Cache module follows these principles: - -1. Tenant Isolation by Default -2. Safe Serialization Strategies -3. Sensible TTL Defaults with Domain Overrides -4. Reactive and Imperative Support -5. Conditional Bootstrapping - ---- - -# Summary - -The **Data Redis Cache** module provides a structured, tenant-aware, and extensible Redis caching layer for the OpenFrame platform. By centralizing Redis configuration, serialization policies, and key naming strategies, it ensures consistent behavior across services while preserving strong multi-tenant boundaries. - -It serves as the high-performance caching backbone of the platform while maintaining strict separation from the system's primary data stores. \ No newline at end of file diff --git a/docs/reference/architecture/eventing-and-messaging-kafka-nats/eventing-and-messaging-kafka-nats.md b/docs/reference/architecture/eventing-and-messaging-kafka-nats/eventing-and-messaging-kafka-nats.md new file mode 100644 index 000000000..bac2cf52a --- /dev/null +++ b/docs/reference/architecture/eventing-and-messaging-kafka-nats/eventing-and-messaging-kafka-nats.md @@ -0,0 +1,385 @@ +# Eventing And Messaging Kafka Nats + +## Overview + +The **Eventing And Messaging Kafka Nats** module provides the core messaging infrastructure for the OpenFrame platform. It standardizes how services publish and consume asynchronous events using: + +- **Apache Kafka** for durable, high-throughput event streaming and data integration. +- **NATS** for low-latency, real-time messaging between backend services and connected agents or users. + +This module acts as the foundational messaging layer that connects: + +- Stream processing pipelines +- Analytics ingestion (e.g., Pinot) +- Agent command and control +- Real-time notifications +- Cross-service event propagation + +It abstracts broker-specific configuration and exposes consistent models and publishers for upstream modules. + +--- + +## High-Level Architecture + +```mermaid +flowchart LR + ApiServices["API Services"] -->|"Publish Events"| KafkaLayer["Kafka Cluster"] + KafkaLayer -->|"Stream Processing"| StreamCore["Stream Processing Core"] + StreamCore -->|"Enriched Events"| Analytics["Analytics Pinot"] + + ApiServices -->|"Commands / Notifications"| NatsLayer["NATS Cluster"] + NatsLayer --> Agents["Connected Agents"] + NatsLayer --> Clients["User Clients"] + + Management["Management Services"] -->|"Topic Init"| KafkaLayer +``` + +### Responsibilities + +| Broker | Purpose | Characteristics | +|--------|----------|----------------| +| Kafka | Durable event streaming | Partitioned, replayable, analytics-oriented | +| NATS | Real-time messaging | Low latency, subject-based routing | + +--- + +# Kafka Integration + +Kafka in this module is designed for: + +- Tenant-aware configuration +- Topic auto-provisioning +- Structured event models +- Debezium change-data-capture integration +- Controlled producer/consumer lifecycle + +## Configuration Components + +### OssKafkaConfig + +Disables Spring Boot's default `KafkaAutoConfiguration` and enables manual configuration control via: + +- `@EnableKafka` +- Explicit producer/consumer factories + +This ensures strict separation between OSS tenant Kafka configuration and any default Spring Kafka behavior. + +--- + +### OssTenantKafkaProperties + +Configuration root bound to: + +```text +spring.oss-tenant +``` + +Wraps `KafkaProperties` and enables: + +- Bootstrap server configuration +- Producer/consumer tuning +- Listener concurrency and ack modes +- Template defaults + +--- + +### KafkaTopicProperties + +Bound to: + +```text +openframe.oss-tenant.kafka.topics +``` + +Supports: + +- Topic auto-creation +- Per-topic partitions +- Replication factor + +Example structure: + +```text +openframe: + oss-tenant: + kafka: + topics: + inbound: + machine-events: + name: machine-events + partitions: 3 + replicationFactor: 2 +``` + +--- + +## Auto-Configuration Flow + +`OssTenantKafkaAutoConfiguration` creates: + +- ProducerFactory +- KafkaTemplate +- ConsumerFactory +- ListenerContainerFactory +- KafkaAdmin +- Auto-created topics +- OssTenantKafkaProducer + +```mermaid +flowchart TD + Properties["OssTenantKafkaProperties"] --> ProducerFactory["ProducerFactory"] + Properties --> ConsumerFactory["ConsumerFactory"] + Properties --> KafkaAdmin["KafkaAdmin"] + + ProducerFactory --> KafkaTemplate["KafkaTemplate"] + ConsumerFactory --> ListenerFactory["ListenerContainerFactory"] + KafkaTemplate --> Producer["OssTenantKafkaProducer"] +``` + +### Key Features + +- JSON serialization via `JsonSerializer` +- Record-level acknowledgment default +- Optional admin topic creation +- Concurrency tuning via listener properties + +--- + +## Kafka Message Models + +### MachinePinotMessage + +Used to stream machine state updates toward analytics systems. + +Fields include: + +- `tenantId` +- `machineId` +- `organizationId` +- `deviceType` +- `status` +- `osType` +- `tags` +- `tagKeyValues` +- `ingestionTime` + +This message is typically consumed by the Stream Processing Core and forwarded to Pinot. + +--- + +### DebeziumMessage + +Generic wrapper for CDC events. + +Structure: + +```mermaid +flowchart TD + Root["DebeziumMessage"] --> Payload["Payload"] + Payload --> Before["before"] + Payload --> After["after"] + Payload --> Source["source"] + Payload --> Operation["operation"] +``` + +Supports: + +- Database change events +- Operation codes (create/update/delete) +- Source metadata (schema, table, collection) + +Used heavily in stream processing and data synchronization workflows. + +--- + +## Kafka Headers + +`KafkaHeader` defines: + +```text +message-type +``` + +This enables polymorphic message handling by allowing consumers to inspect event type metadata without parsing payload content. + +--- + +## Failure Handling + +### KafkaRecoveryHandlerImpl + +Handles producer-side failures by: + +- Logging structured error summaries +- Capturing topic, key, payload +- Attaching stack traces + +Current implementation logs and defers recovery to higher-level retry strategies. + +--- + +# NATS Integration + +NATS is used for real-time messaging between: + +- Backend services +- OpenFrame agents +- User-facing clients + +It is optimized for: + +- Command execution +- Notification broadcasting +- Tool installation updates +- Connection lifecycle events + +--- + +## Notification Flow + +```mermaid +flowchart LR + Service["Business Service"] --> Broadcaster["NotificationBroadcaster"] + Broadcaster --> Repository["NotificationRepository"] + Broadcaster --> NatsPublisher["NotificationNatsPublisher"] + NatsPublisher --> UserSubject["user.{id}.notification"] + NatsPublisher --> MachineSubject["machine.{id}.notification"] +``` + +### NotificationCommand + +Validates: + +- Non-empty title +- Non-null severity +- Valid context type +- At least one audience (admins or machines) + +Ensures consistency before persistence or publishing. + +--- + +### NotificationBroadcaster + +Responsibilities: + +1. Feature flag check +2. Persist notification +3. Create read-state entries +4. Publish to NATS subjects +5. Safe failure handling and cleanup + +If NATS publishing fails, clients reconcile state via GraphQL queries. + +--- + +### NotificationNatsPublisher + +Publishes to: + +- `user.{userId}.notification` +- `machine.{machineId}.notification` + +Ensures: + +- Notification is persisted before publish +- Topic validation +- Exception safety + +--- + +## Agent Command & Control + +### CommandMessage + +Sent to subject: + +```text +machine.{machineId}.command-execution +``` + +Payload includes: + +- `executionId` +- `code` +- `shell` +- `privilegeLevel` +- `timeout` + +Used for remote script execution. + +--- + +### CancelMessage + +Sent to abort an in-flight execution. + +Contains: + +- `executionId` + +--- + +## Tool Lifecycle Messages + +- `ToolInstallationMessage` +- `ToolAgentUpdateMessage` +- `ToolConnectionMessage` +- `InstalledAgentMessage` +- `ClientConnectionEvent` + +These support: + +- Tool distribution +- Agent upgrades +- Session management +- Asset updates + +--- + +# Cross-Module Relationships + +This module integrates closely with: + +- Stream Processing Core (Kafka consumers and processors) +- Analytics Pinot (MachinePinotMessage ingestion) +- Client Core Agent Ingress (NATS listeners) +- Management Service Core (topic initialization and stream configuration) + +Kafka supports durable, replayable pipelines. + +NATS supports interactive, real-time orchestration. + +Together they provide a hybrid eventing architecture that balances: + +- Durability +- Performance +- Scalability +- Multi-tenant isolation + +--- + +# Design Principles + +1. Broker abstraction with Spring auto-configuration +2. Tenant-aware messaging isolation +3. Fail-safe publishing strategies +4. Explicit message models +5. Clear separation between streaming and real-time messaging + +--- + +# Summary + +The **Eventing And Messaging Kafka Nats** module forms the backbone of asynchronous communication in OpenFrame. + +It provides: + +- Structured Kafka streaming configuration +- Debezium event support +- Topic lifecycle management +- Real-time NATS messaging +- Agent command execution +- Notification broadcasting + +By separating durable event streams from real-time agent messaging, the module ensures both reliability and responsiveness across the platform. \ No newline at end of file diff --git a/docs/reference/architecture/external-api-service-core/external-api-service-core.md b/docs/reference/architecture/external-api-service-core/external-api-service-core.md index 8a65298ac..1bc285b6c 100644 --- a/docs/reference/architecture/external-api-service-core/external-api-service-core.md +++ b/docs/reference/architecture/external-api-service-core/external-api-service-core.md @@ -2,200 +2,224 @@ ## Overview -The **External Api Service Core** module exposes a secure, API key–based REST interface for external integrations with the OpenFrame platform. +The **External Api Service Core** module exposes a public, API-key–secured REST interface for third-party systems and customer integrations to interact with the OpenFrame platform. It provides: -- Public REST endpoints under `/api/v1/**` -- Tool proxy endpoints under `/tools/**` -- OpenAPI (Swagger) documentation -- Cursor-based pagination, filtering, and sorting -- API key authentication via `X-API-Key` +- Public REST endpoints for devices, events, logs, organizations, and tools +- API-key based authentication (via `X-API-Key` header) +- Cursor-based pagination and filtering +- Tool API proxying for integrated platforms (RMM, MDM, etc.) +- OpenAPI (Swagger) documentation for discoverability -Unlike the internal GraphQL-based API service, this module is purpose-built for third-party systems, automation scripts, and integration partners. +This module acts as a controlled, external-facing boundary on top of internal services defined across: ---- +- API domain services (DeviceService, EventService, LogService, ToolService, Organization services) +- Mongo repositories and data model modules +- Stream processing and analytics modules (for logs/events) +- Gateway and authorization modules (for upstream security enforcement) -## Architectural Positioning +--- -The External Api Service Core sits at the boundary between external consumers and the internal platform services. +## High-Level Architecture ```mermaid -flowchart LR - Client["External Client"] -->|"X-API-Key"| ExternalAPI["External Api Service Core"] +flowchart TD + Client["External Client / Integration"] -->|"X-API-Key"| Gateway["Gateway Service Core"] + Gateway --> ExternalApi["External Api Service Core"] + + subgraph rest_layer["REST Controllers"] + DeviceCtrl["DeviceController"] + EventCtrl["EventController"] + LogCtrl["LogController"] + OrgCtrl["OrganizationController"] + ToolCtrl["ToolController"] + IntegrationCtrl["IntegrationController"] + end + + ExternalApi --> rest_layer + + rest_layer --> Services["Domain Services Layer"] + Services --> Mongo["Mongo Data Access"] + Services --> Pinot["Analytics (Pinot)"] + Services --> Kafka["Stream & Eventing"] + + IntegrationCtrl --> RestProxy["RestProxyService"] + RestProxy --> IntegratedTools["Integrated Tool APIs"] +``` - ExternalAPI --> DeviceService["Device Service"] - ExternalAPI --> EventService["Event Service"] - ExternalAPI --> LogService["Log Service"] - ExternalAPI --> OrganizationService["Organization Services"] - ExternalAPI --> ToolService["Tool Service"] +### Architectural Role - ExternalAPI --> ProxyService["Rest Proxy Service"] - ProxyService --> IntegratedTool["Integrated Tool"] +The External Api Service Core is: - DeviceService --> MongoDB[("MongoDB")] - EventService --> MongoDB - LogService --> Pinot[("Apache Pinot")] - ToolService --> MongoDB -``` +- **Northbound API layer** for external consumers +- **Stateless REST interface** over internal services +- **Security-aware boundary** relying on API keys +- **Proxy gateway** for integrated tool APIs -### Responsibilities +It does not implement deep business logic itself. Instead, it orchestrates: -- Authenticate requests using API keys -- Translate REST query parameters into internal filter criteria -- Apply cursor-based pagination -- Map domain models into external DTOs -- Proxy tool-specific requests to integrated tools +- Filter construction +- Pagination translation +- Sorting conversion +- DTO mapping +- Delegation to internal services --- -## Authentication Model +## Authentication & Security Model + +All endpoints require an API key. -All endpoints require an API key provided in the `X-API-Key` header. +### API Key Format ```text X-API-Key: ak_keyId.sk_secretKey ``` -Internally: +### Security Flow -- The API key is validated by upstream security filters -- `X-User-Id` and `X-API-Key-Id` headers are injected -- Controllers use those headers for auditing and authorization - -The module does **not** perform token-based OAuth authentication. It is explicitly designed for machine-to-machine API key usage. +```mermaid +sequenceDiagram + participant Client + participant Gateway + participant ExternalApi as ExternalApiService + participant Services + + Client->>Gateway: Request + X-API-Key + Gateway->>ExternalApi: Forward validated request + ExternalApi->>Services: Execute business logic + Services-->>ExternalApi: Domain result + ExternalApi-->>Gateway: REST response + Gateway-->>Client: HTTP response +``` ---- +Rate limits are enforced at the platform level and reflected in response headers: -## OpenAPI Configuration +- `X-RateLimit-Limit-Minute` +- `X-RateLimit-Remaining-Minute` +- `X-RateLimit-Limit-Hour` +- `X-RateLimit-Remaining-Hour` -### OpenApiConfig +--- -The `OpenApiConfig` class configures: +## Module Components -- API metadata (title, version, license) -- API key security scheme -- Grouped OpenAPI paths -- Server base path `/external-api` +### 1. OpenAPI Configuration -Documented path groups: +**Class:** `OpenApiConfig` -```text -Included: -- /tools/** -- /api/v1/** +Responsibilities: -Excluded: -- /actuator/** -- /api/core/** -``` +- Defines Swagger/OpenAPI metadata +- Documents authentication requirements +- Configures API grouping (`/tools/**`, `/api/v1/**`) +- Declares API key security scheme -Security scheme definition: +This ensures external consumers can: -```text -Type: APIKEY -In: HEADER -Header name: X-API-Key -``` +- Discover endpoints +- Understand rate limits +- View request/response schemas --- -# REST Controllers +## REST Controllers -All REST endpoints are versioned under `/api/v1` except integration proxy endpoints (`/tools/**`). +Each controller follows a consistent pattern: + +1. Parse query parameters +2. Build filter criteria DTO +3. Translate pagination via `CursorPaginationCriteria` +4. Apply sorting via `SortInput` +5. Delegate to domain service +6. Map domain result to external response DTO --- -## DeviceController +### DeviceController + +Base path: -**Base Path:** `/api/v1/devices` +```text +/api/v1/devices +``` + +#### Capabilities -### Capabilities +- List devices (paginated, filtered, searchable) +- Get device by machine ID +- Retrieve device filter options with counts +- Update device status (DELETED / ARCHIVED) +- Optional tag expansion via `includeTags` -- List devices with filtering and pagination -- Retrieve a device by machine ID -- Retrieve device filter options -- Update device status +#### Filtering Model -### Query Features +Uses: -The controller converts query parameters into `DeviceFilterCriteria`: +- `DeviceFilterCriteria` +- `CursorPaginationCriteria` +- `SortInput` -- Status filters -- Device type filters -- OS type filters -- Organization filters -- Tag filters -- Search -- Sorting -- Cursor-based pagination +#### Data Mapping Flow ```mermaid -flowchart TD - Request["GET /api/v1/devices"] --> Criteria["DeviceFilterCriteria"] - Criteria --> Pagination["CursorPaginationCriteria"] - Pagination --> Query["DeviceService.queryDevices()"] - Query --> Result["Query Result"] +flowchart LR + Request["GET /api/v1/devices"] --> Filter["DeviceFilterCriteria"] + Filter --> Service["DeviceService.queryDevices()"] + Service --> Result["Paged Machine Result"] Result --> Mapper["DeviceMapper"] Mapper --> Response["DevicesResponse"] ``` -### Tag Enrichment +Optional tag enrichment: -When `includeTags=true`, the controller: - -1. Extracts machine IDs -2. Loads tags via `TagService` -3. Returns enriched response - -Failures in tag loading fall back to non-enriched responses. +- Calls `TagService.getTagsForMachines()` +- Merges tag data into `DeviceResponse` --- -## EventController +### EventController + +Base path: -**Base Path:** `/api/v1/events` +```text +/api/v1/events +``` -### Capabilities +#### Capabilities -- Query events with filtering -- Retrieve event by ID +- List events (cursor-based pagination) +- Get event by ID - Create event - Update event -- Retrieve filter options +- Retrieve event filter metadata -### Filtering Dimensions +Uses: -- User IDs -- Event types -- Date range -- Search term -- Sorting -- Cursor pagination - -```mermaid -flowchart TD - EventRequest["GET /api/v1/events"] --> Filter["EventFilterCriteria"] - Filter --> Service["EventService.queryEvents()"] - Service --> Mapper["EventMapper"] - Mapper --> EventsResponse["EventsResponse"] -``` +- `EventFilterCriteria` +- `EventService` +- `EventMapper` -Create and update operations directly delegate to `EventService`. +Events represent domain-level occurrences (user actions, system events, integrations). --- -## LogController +### LogController -**Base Path:** `/api/v1/logs` +Base path: -### Capabilities +```text +/api/v1/logs +``` -- Query logs with filtering -- Retrieve log filter options -- Retrieve detailed log entry +#### Capabilities -### Filtering Dimensions +- Query logs with advanced filtering +- Retrieve filter metadata +- Get detailed log entry + +Filtering supports: - Date range - Tool type @@ -203,85 +227,75 @@ Create and update operations directly delegate to `EventService`. - Severity - Organization - Device ID -- Search -- Sorting -- Cursor pagination - -Log queries are executed via `LogService`, which may retrieve data from analytics stores such as Apache Pinot. +- Search text -Detailed log retrieval requires composite identifiers: - -```text -ingestDay -toolType -eventType -timestamp -toolEventId -``` +Logs are typically backed by analytics infrastructure (e.g., Pinot) and domain services. --- -## OrganizationController +### OrganizationController -**Base Path:** `/api/v1/organizations` +Base path: + +```text +/api/v1/organizations +``` -### Capabilities +#### Capabilities - List organizations with filtering -- Retrieve organization by database ID -- Retrieve organization by business identifier +- Get organization by database ID +- Get by business `organizationId` - Create organization - Update organization -- Update status (ACTIVE / ARCHIVED) -- Check archive eligibility - -### Query Delegation +- Archive validation (`can-archive`) +- Update organization status -The controller delegates: +Delegates to: -- Reads β†’ `OrganizationQueryService` -- Writes β†’ `OrganizationCommandService` -- Archival checks β†’ `OrganizationService` +- `OrganizationQueryService` +- `OrganizationCommandService` +- `OrganizationService` -```mermaid -flowchart LR - OrgRequest["Organization Request"] --> QueryService["OrganizationQueryService"] - OrgRequest --> CommandService["OrganizationCommandService"] - CommandService --> Validation["Archive Rules"] -``` - -Archiving is blocked when active devices exist. +Supports both business ID and database ID access patterns. --- -## ToolController +### ToolController -**Base Path:** `/api/v1/tools` +Base path: -### Capabilities +```text +/api/v1/tools +``` + +#### Capabilities - List integrated tools +- Filter by type, category, enabled state - Retrieve tool filter options -Filtering includes: +Uses: -- Enabled status -- Tool type -- Category -- Search -- Sorting +- `ToolFilterCriteria` +- `ToolService` +- `ToolMapper` -Delegates to `ToolService` and maps results using `ToolMapper`. +This controller provides metadata about connected integrations (RMM, MDM, etc.). --- -## IntegrationController +### IntegrationController (Tool API Proxy) -**Base Path:** `/tools/{toolId}/**` +Base path: -This controller proxies arbitrary HTTP requests to integrated tools. +```text +/tools/{toolId}/** +``` -Supported methods: +This controller enables full HTTP proxying to integrated tools. + +#### Supported Methods - GET - POST @@ -290,147 +304,155 @@ Supported methods: - DELETE - OPTIONS -```mermaid -flowchart TD - Client["External Client"] --> ProxyController["IntegrationController"] - ProxyController --> RestProxyService["RestProxyService"] - RestProxyService --> ToolRepo["IntegratedToolRepository"] - RestProxyService --> Resolver["ProxyUrlResolver"] - Resolver --> TargetTool["Integrated Tool API"] -``` +It delegates to `RestProxyService`. --- -# Rest Proxy Service - -The `RestProxyService` performs secure HTTP forwarding. +## RestProxyService -## Responsibilities +The **RestProxyService** is a core infrastructure component enabling dynamic upstream API routing. -1. Validate tool existence -2. Verify tool is enabled -3. Resolve upstream URL -4. Attach tool credentials -5. Forward request -6. Return upstream response +### Responsibilities -## Credential Injection +1. Resolve tool by ID via `IntegratedToolRepository` +2. Validate tool is enabled +3. Resolve correct API URL via `ToolUrlService` +4. Build headers (API key or bearer token) +5. Rewrite target URI via `ProxyUrlResolver` +6. Execute HTTP call using Apache HttpClient +7. Return proxied response -Based on `APIKeyType`: +### Proxy Flow -```text -HEADER β†’ Custom header injection -BEARER_TOKEN β†’ Authorization: Bearer -NONE β†’ No credential +```mermaid +flowchart TD + Incoming["/tools/{toolId}/..."] --> Lookup["Find IntegratedTool"] + Lookup --> Enabled{"Enabled?"} + Enabled -->|No| Reject["400 / 404"] + Enabled -->|Yes| ResolveUrl["Resolve ToolUrl (API)"] + ResolveUrl --> BuildHeaders["Attach APIKey / Bearer"] + BuildHeaders --> Rewrite["ProxyUrlResolver"] + Rewrite --> HttpCall["Execute HTTP Request"] + HttpCall --> Return["Return ResponseEntity"] ``` -## HTTP Client Configuration +### Credential Injection -- Connection timeout: 10 seconds -- Response timeout: 60 seconds -- Apache HttpClient 5 +Supports: -The service preserves: +- Header-based API keys +- Bearer tokens +- No-auth tools -- HTTP method -- Request body -- Response status code -- Response body +This allows uniform integration with heterogeneous external systems. --- -# Pagination Model +## External DTO Layer -All list endpoints use cursor-based pagination. +The module defines dedicated REST DTOs under `com.openframe.external.dto.*`. -Components: +These DTOs: -- `CursorPaginationCriteria.fromRest(cursor, limit)` -- `SortInput.from(sortField, sortDirection)` +- Decouple internal domain models from public contract +- Stabilize API response formats +- Provide Swagger annotations +- Support pagination metadata (`PageInfo`) -Benefits: +Examples: -- Stable pagination -- Scalable large dataset traversal -- No offset-based performance degradation +- `DeviceResponse`, `DevicesResponse` +- `EventResponse`, `EventsResponse` +- `LogResponse`, `LogsResponse`, `LogDetailsResponse` +- `OrganizationsResponse` +- `ToolResponse`, `ToolsResponse` + +This separation prevents leaking internal document structures. --- -# Error Handling +## Pagination & Sorting Model -Standard HTTP status codes are used: +The External Api Service Core standardizes pagination via cursor-based semantics. -```text -200 Success -201 Created -204 No Content -400 Bad Request -401 Unauthorized -403 Forbidden -404 Not Found -409 Conflict -429 Too Many Requests -500 Internal Server Error +```mermaid +flowchart LR + Client["cursor + limit"] --> CursorCriteria["CursorPaginationCriteria.fromRest()"] + CursorCriteria --> Service + Service --> PageInfo + PageInfo --> Response ``` -Domain-specific exceptions (e.g., `DeviceNotFoundException`, `OrganizationNotFoundException`) are translated into structured error responses. - ---- - -# Data Sources and Dependencies +Benefits: -The module integrates with: +- Scalable pagination for large datasets +- Stable sorting +- Efficient backend queries -- MongoDB (devices, organizations, tools) -- Apache Pinot (log analytics) -- Integrated tool APIs (via proxy) -- Core domain services from the API layer +Sorting is abstracted using: -It does not directly manage persistence; instead, it orchestrates existing services. +- `SortInput.from(field, direction)` --- -# Key Design Characteristics +## Error Handling -## 1. Separation of Concerns +Common HTTP responses: -- Controllers handle HTTP concerns -- Services perform business logic -- Mappers transform domain β†’ external DTO -- Proxy service handles external tool routing +- 200 – Success +- 201 – Created +- 204 – No Content +- 400 – Validation error +- 401 – Unauthorized +- 404 – Not Found +- 409 – Conflict +- 429 – Rate limit exceeded +- 500 – Internal error -## 2. External-First Contract +Errors return structured `ErrorResponse` payloads. -The REST surface is optimized for: +--- -- Predictable filtering -- Explicit query parameters -- Stable versioning (`/api/v1`) -- API key–based automation +## Integration Within the Platform -## 3. Observability +The External Api Service Core integrates with: -All controllers log: +- Domain services (device, event, log, organization, tool) +- Mongo repositories (data model modules) +- Analytics layer (for log querying) +- Stream/eventing infrastructure (for event generation) +- Gateway module (traffic routing & JWT/API key enforcement) +- Authorization module (tenant & identity management) -- Request parameters -- User ID -- API key ID -- Pagination and sorting +It is intentionally thin and orchestration-focused, serving as a: -This enables auditability and traceability. +- Public contract boundary +- Stable integration surface +- Secure proxy interface --- -# Summary +## Summary + +The **External Api Service Core** module provides: + +- Public REST APIs for core OpenFrame entities +- API-key based authentication model +- Cursor-based pagination & flexible filtering +- Tool API proxying for external integrations +- Strict DTO-based response contracts +- Full OpenAPI documentation -The **External Api Service Core** module provides a secure, API key–driven REST interface for third-party integrations. +It plays a critical role in enabling: -It: +- Third-party automation +- Customer integrations +- Ecosystem expansion +- Controlled external access to OpenFrame capabilities -- Exposes device, event, log, organization, and tool APIs -- Implements cursor-based pagination and rich filtering -- Proxies requests to integrated tools -- Enforces API key authentication -- Publishes OpenAPI documentation +This module should remain: -It acts as the official external integration boundary of the OpenFrame platform, enabling automation, ecosystem integrations, and partner access without exposing internal GraphQL or domain-layer complexity. +- Stateless +- Contract-stable +- Strictly layered over domain services +- Security-conscious diff --git a/docs/reference/architecture/gateway-service-core-security-and-routing/gateway-service-core-security-and-routing.md b/docs/reference/architecture/gateway-service-core-security-and-routing/gateway-service-core-security-and-routing.md deleted file mode 100644 index 8920d6d19..000000000 --- a/docs/reference/architecture/gateway-service-core-security-and-routing/gateway-service-core-security-and-routing.md +++ /dev/null @@ -1,368 +0,0 @@ -# Gateway Service Core Security And Routing - -## Overview - -The **Gateway Service Core Security And Routing** module is the reactive edge layer of the OpenFrame platform. It is responsible for: - -- Acting as the **entry point** for REST and WebSocket traffic -- Enforcing **JWT-based authentication and role-based authorization** -- Providing **API key authentication and rate limiting** for external APIs -- Dynamically resolving **multi-tenant issuers** -- Routing and proxying requests to integrated tools (e.g., MeshCentral, Tactical RMM) -- Handling WebSocket proxying and session decoration - -Built on **Spring Cloud Gateway + WebFlux + Netty**, it provides a fully reactive, non-blocking gateway optimized for high concurrency and real-time integrations. - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - Client["Client / Agent / UI"] --> Gateway["Gateway Service Core Security And Routing"] - - subgraph security_layer["Security Layer"] - AuthHeader["AddAuthorizationHeaderFilter"] - JwtConfig["JwtAuthConfig"] - SecurityConfig["GatewaySecurityConfig"] - ApiKeyFilter["ApiKeyAuthenticationFilter"] - OriginFilter["OriginSanitizerFilter"] - end - - subgraph routing_layer["Routing & Proxy Layer"] - IntegrationCtrl["IntegrationController"] - WsConfig["WebSocketGatewayConfig"] - UpstreamResolvers["ToolUpstreamResolvers"] - end - - Gateway --> security_layer - security_layer --> routing_layer - routing_layer --> Upstream["Integrated Tools / NATS / Services"] -``` - -The module is divided into four major concerns: - -1. **Reactive Server & Client Configuration (Netty + WebClient)** -2. **Security (JWT, API Keys, Roles, CORS, Origin)** -3. **WebSocket Routing & Decoration** -4. **Tool Upstream Resolution & Proxying** - ---- - -## Reactive Infrastructure Configuration - -### NettySocketConfig - -Configures low-level TCP behavior for both: - -- Embedded Netty web server -- Gateway outbound HTTP client -- Reactor Netty WebSocket client - -Key socket options: - -- `SO_LINGER = 0` (fast connection teardown) -- `TCP_NODELAY = true` (low-latency communication) - -This ensures optimal performance for high-throughput proxy and WebSocket traffic. - ---- - -### WebClientConfig - -Provides a preconfigured `WebClient.Builder` with: - -- 30s connection timeout -- 30s response timeout -- Read/write timeout handlers - -Used internally for REST proxying to upstream tools. - ---- - -## Security Architecture - -Security is layered and reactive. It supports: - -- Multi-tenant JWT validation -- Role-based access control -- API key authentication for external APIs -- Bearer token resolution from multiple sources -- CORS control -- Origin header sanitization - -### Security Flow - -```mermaid -flowchart TD - Request["Incoming Request"] --> Origin["OriginSanitizerFilter"] - Origin --> AuthHeader["AddAuthorizationHeaderFilter"] - AuthHeader --> JwtValidation["JWT Authentication"] - JwtValidation --> RoleCheck["Role Authorization Rules"] - RoleCheck --> ApiKeyCheck{"External API Path?"} - ApiKeyCheck -->|Yes| ApiKeyFilter["ApiKeyAuthenticationFilter"] - ApiKeyCheck -->|No| Continue["Continue Routing"] - ApiKeyFilter --> Continue - Continue --> Controller["Controller / Route"] -``` - ---- - -## JWT Authentication & Multi-Tenant Support - -### JwtAuthConfig - -- Uses a **Caffeine cache** to store `ReactiveAuthenticationManager` instances per issuer -- Dynamically builds JWT decoders per tenant -- Supports: - - Platform issuer (local public key) - - External issuers via OIDC discovery - -### IssuerUrlProvider - -Resolves allowed issuers dynamically from tenant configuration. - -- Reads tenants reactively -- Builds issuer URLs using: - - `allowed-issuer-base` - - Tenant ID - - Optional super-tenant ID -- Caches issuer URLs - -This enables strict issuer validation in multi-tenant deployments. - ---- - -## GatewaySecurityConfig - -Defines the reactive security filter chain. - -Key behavior: - -- Disables CSRF, HTTP Basic, form login -- Enables OAuth2 Resource Server -- Uses `ReactiveAuthenticationManagerResolver` for issuer-based validation -- Injects `AddAuthorizationHeaderFilter` before authentication - -### Role Rules - -- `/api/**` β†’ ADMIN -- `/tools/agent/**` β†’ AGENT -- `/ws/tools/agent/**` β†’ AGENT -- `/ws/nats` β†’ AGENT or ADMIN -- `/guide/**` β†’ ADMIN -- `/clients/**` β†’ AGENT - -All other paths default to `permitAll()` unless matched earlier. - ---- - -## AddAuthorizationHeaderFilter - -Ensures a Bearer token exists by resolving it from: - -1. Access token cookie -2. Custom header -3. Query parameter - -If found, it injects: - -```text -Authorization: Bearer -``` - -This enables WebSocket and browser flows that cannot always set headers directly. - ---- - -## API Key Authentication & Rate Limiting - -### ApiKeyAuthenticationFilter - -A `GlobalFilter` applied to `/external-api/**` endpoints. - -Flow: - -```mermaid -flowchart TD - Req["/external-api/**"] --> CheckKey["Check X-API-Key Header"] - CheckKey --> Valid{"Valid Key?"} - Valid -->|No| Reject401["Return 401"] - Valid -->|Yes| RateCheck["RateLimitService.isAllowed"] - RateCheck --> Allowed{"Allowed?"} - Allowed -->|No| Reject429["Return 429 + Retry-After"] - Allowed -->|Yes| AddHeaders["Add RateLimit Headers"] - AddHeaders --> AddContext["Inject X-User-Id + X-Api-Key-Id"] - AddContext --> Forward["Forward to External API"] -``` - -Capabilities: - -- Validates API key -- Increments usage stats -- Enforces minute/hour/day limits -- Adds rate limit headers -- Injects user context headers - ---- - -## WebSocket Gateway Configuration - -### WebSocketGatewayConfig - -Defines WebSocket routing using Spring Cloud Gateway. - -Routes: - -- `/ws/tools/agent/{toolId}/**` -- `/ws/tools/{toolId}/**` -- `/ws/nats` -- `/ws/nats-api` - -It also decorates: - -- `WebSocketClient` (optional proxy cleanup) -- `WebSocketService` (security + metrics decorator) - ---- - -### WebSocket Proxy URL Filters - -#### ToolAgentWebSocketProxyUrlFilter - -- Extracts `toolId` from agent path -- Resolves upstream WebSocket URL - -#### ToolApiWebSocketProxyUrlFilter - -- Extracts `toolId` from API path -- Removes `Origin` header -- Injects tool-specific API key headers - -These filters delegate actual resolution to the upstream resolver registry. - ---- - -## Tool Upstream Resolution - -The gateway uses a strategy pattern for resolving tool destinations. - -```mermaid -flowchart LR - Request["Tool Request"] --> Registry["ToolUpstreamResolverRegistry"] - Registry -->|Specific Tool| Mesh["MeshCentralUpstreamResolver"] - Registry -->|Specific Tool| Tactical["TacticalRmmUpstreamResolver"] - Registry -->|Fallback| Default["DefaultToolUpstreamResolver"] - Mesh --> Proxy - Tactical --> Proxy - Default --> Proxy -``` - -### DefaultToolUpstreamResolver - -- Reads tool URLs from MongoDB -- Uses `ToolUrlService` -- Resolves API and WS URLs - -### MeshCentralUpstreamResolver - -- Uses configuration properties -- Avoids DB lookup -- Supports path prefix injection - -### TacticalRmmUpstreamResolver - -- Routes REST to backend -- Routes WebSocket to: - - NATS (if path matches) - - Daphne (otherwise) - -This allows fine-grained routing without an intermediate nginx layer. - ---- - -## CORS & Origin Handling - -### CorsConfig - -- Enabled unless explicitly disabled -- Uses Spring Cloud Gateway global CORS properties - -### OriginSanitizerFilter - -- Removes `Origin: null` headers -- Prevents invalid origin propagation to upstream services - ---- - -## Internal Auth Probe - -### InternalAuthProbeController - -Conditional endpoint: - -``` -GET /internal/authz/probe -``` - -Used for health checks in internal deployments. - ---- - -## Path Constants - -`PathConstants` centralizes prefix definitions: - -- `/clients` -- `/api` -- `/tools` -- `/ws/tools` -- `/chat` -- `/guide` - -Ensures consistent routing and security matching. - ---- - -## Request Lifecycle Summary - -```mermaid -flowchart TD - Client --> Gateway - Gateway --> OriginFilter - OriginFilter --> AuthHeader - AuthHeader --> JwtResolver - JwtResolver --> RoleAuth - RoleAuth --> RouteDecision - RouteDecision -->|REST| IntegrationController - RouteDecision -->|WebSocket| WebSocketGatewayConfig - IntegrationController --> UpstreamResolver - WebSocketGatewayConfig --> UpstreamResolver - UpstreamResolver --> IntegratedTool -``` - ---- - -## Key Design Principles - -- Fully reactive (WebFlux + Reactor) -- Strict multi-tenant issuer validation -- Pluggable upstream resolution per tool -- Centralized security enforcement at the edge -- API key rate limiting for external APIs -- Optimized Netty socket configuration -- Clean separation between security, routing, and proxy concerns - ---- - -## Conclusion - -The **Gateway Service Core Security And Routing** module is the secure, reactive edge of the OpenFrame platform. It combines: - -- JWT-based multi-tenant authentication -- Role-based authorization -- API key validation and rate limiting -- WebSocket proxying and decoration -- Intelligent upstream resolution - -It ensures that all traffic entering the platform is authenticated, authorized, rate-limited, and routed efficiently to the correct internal or integrated service. \ No newline at end of file diff --git a/docs/reference/architecture/gateway-service-core/gateway-service-core.md b/docs/reference/architecture/gateway-service-core/gateway-service-core.md new file mode 100644 index 000000000..89ce07f27 --- /dev/null +++ b/docs/reference/architecture/gateway-service-core/gateway-service-core.md @@ -0,0 +1,492 @@ +# Gateway Service Core + +The **Gateway Service Core** module is the reactive edge layer of the OpenFrame platform. It acts as: + +- βœ… API Gateway (Spring Cloud Gateway) +- βœ… WebSocket Proxy (for tools, agents, and NATS) +- βœ… Security Enforcement Layer (JWT, API keys, role-based access) +- βœ… Multi-tenant issuer-aware resource server +- βœ… Tool-specific upstream router + +It sits between clients (UI, agents, external consumers) and internal platform services such as: + +- API Service Core +- External API Service Core +- Authorization Service Core +- Integrated Tools (MeshCentral, Tactical RMM, etc.) + +This module is fully reactive (Spring WebFlux + Reactor Netty) and optimized for high-throughput REST and WebSocket traffic. + +--- + +## 1. Architectural Overview + +The Gateway Service Core orchestrates routing, authentication, rate limiting, and upstream resolution. + +```mermaid +flowchart LR + Client["Browser / Agent / External API Client"] --> Gateway["Gateway Service Core"] + + Gateway --> ApiService["API Service Core"] + Gateway --> ExternalApi["External API Service Core"] + Gateway --> Authz["Authorization Service Core"] + Gateway --> Mesh["MeshCentral"] + Gateway --> Tactical["Tactical RMM"] + Gateway --> Nats["NATS WebSocket"] + + subgraph security["Security Layer"] + Jwt["JWT Validation"] + ApiKey["API Key Auth + Rate Limit"] + Roles["Role Based Access"] + end + + Gateway --> Jwt + Gateway --> ApiKey + Gateway --> Roles +``` + +### Responsibilities + +| Concern | Implemented By | +|----------|----------------| +| REST routing | Spring Cloud Gateway routes + controllers | +| WebSocket proxy | WebSocketGatewayConfig | +| JWT validation | JwtAuthConfig + GatewaySecurityConfig | +| Multi-issuer resolution | JwtIssuerReactiveAuthenticationManagerResolver | +| API key auth | ApiKeyAuthenticationFilter | +| Rate limiting | RateLimitService + RateLimitConstants | +| Tool upstream resolution | ToolUpstreamResolver implementations | +| Origin hardening | OriginSanitizerFilter | +| CORS configuration | CorsConfig / CorsDisableConfig | + +--- + +# 2. Core Layers of the Gateway + +The module can be understood in seven logical layers. + +```mermaid +flowchart TD + Netty["Netty & WebClient Config"] --> Security + Security["Security & JWT Layer"] --> Filters + Filters["Global & Web Filters"] --> Controllers + Controllers["REST Controllers"] --> Routing + Routing["WebSocket & Route Locator"] --> Upstream + Upstream["Tool Upstream Resolvers"] --> ExternalSystems + + ExternalSystems["Tools / NATS / Internal Services"] +``` + +--- + +# 3. Netty & Reactive HTTP Configuration + +## NettySocketConfig + +Optimizes TCP-level behavior: + +- `SO_LINGER = 0` +- `TCP_NODELAY = true` +- Custom WebSocket client bean + +Improves: +- Connection teardown performance +- Reduced latency for WebSocket frames +- Reduced packet buffering delay + +## WebClientConfig + +Creates a tuned `WebClient.Builder` with: + +- 30s connect timeout +- 30s response timeout +- Read/write timeout handlers +- Reactor Netty HTTP client connector + +This client is used for proxying REST calls to upstream tools. + +--- + +# 4. Security Layer + +The Gateway Service Core is a **Reactive OAuth2 Resource Server**. + +## 4.1 GatewaySecurityConfig + +Configures: + +- CSRF disabled +- CORS disabled at Spring Security level +- JWT resource server +- Role-based authorization rules +- AddAuthorizationHeaderFilter insertion + +### Role Mapping + +JWT claims are mapped as: + +- `roles` β†’ `ROLE_*` +- `scope` β†’ `SCOPE_*` +- `sub` β†’ principal + +### Path Authorization Model + +| Path | Required Role | +|------|---------------| +| `/api/**` | ADMIN | +| `/tools/agent/**` | AGENT | +| `/ws/tools/agent/**` | AGENT | +| `/ws/nats` | AGENT or ADMIN | +| `/content/**` | ADMIN | + +Path prefixes are centralized in `PathConstants`. + +--- + +## 4.2 JWT Multi-Issuer Support (JwtAuthConfig) + +The gateway supports multiple tenant issuers. + +```mermaid +flowchart TD + Request["Incoming Request"] --> ExtractIssuer["Extract iss claim"] + ExtractIssuer --> Cache["Caffeine Cache"] + Cache --> Manager["ReactiveAuthenticationManager"] + Manager --> Decoder["NimbusReactiveJwtDecoder"] + Decoder --> Validate["Strict Issuer Validation"] +``` + +Features: + +- Per-issuer authentication manager caching +- Public key loading for super tenant +- Dynamic issuer resolution via IssuerUrlProvider +- Strict issuer validation using allowed issuer list + +### DefaultIssuerUrlProvider + +OSS fallback implementation: + +- Single-tenant +- Issuer = `allowed-issuer-base + super-tenant-id` + +--- + +# 5. Pre-Authentication & Security Filters + +## 5.1 AddAuthorizationHeaderFilter + +Purpose: + +Ensures every private endpoint has a standard `Authorization: Bearer` header. + +Token resolution priority: + +1. Access token cookie +2. Custom `Access-Token` header +3. `authorization` query parameter + +If resolved β†’ header injected before authentication. + +--- + +## 5.2 ApiKeyAuthenticationFilter + +Global filter for `/external-api/**`. + +```mermaid +flowchart TD + Req["External API Request"] --> HasKey{"API Key Present?"} + HasKey -->|No| Reject["401 UNAUTHORIZED"] + HasKey -->|Yes| Validate["Validate API Key"] + Validate -->|Invalid| Reject + Validate -->|Valid| Rate{"Rate Limit OK?"} + Rate -->|No| Limit["429 RATE_LIMIT_EXCEEDED"] + Rate -->|Yes| AddHeaders["Add Context + Rate Headers"] + AddHeaders --> Continue["Forward to External API"] +``` + +### Responsibilities + +- Validate API key +- Increment usage statistics +- Enforce minute/hour/day limits +- Inject context headers: + - `X-API-KEY-ID` + - `X-USER-ID` +- Add standard rate limit headers + +Rate limit constants are defined in `RateLimitConstants`. + +--- + +## 5.3 OriginSanitizerFilter + +Removes invalid `Origin: null` header to avoid CORS misbehavior and upstream rejection. + +--- + +## 5.4 CORS Configuration + +Two modes: + +### CorsConfig + +- Standard Spring Cloud Gateway CORS configuration +- Controlled via `spring.cloud.gateway.globalcors` + +### CorsDisableConfig + +When `openframe.gateway.disable-cors=true`: + +- Allows all origins +- Enables credentials +- Intended for SaaS same-domain deployments + +--- + +# 6. REST Controllers + +## IntegrationController + +Handles tool REST proxying: + +| Endpoint | Description | +|----------|-------------| +| `/tools/{toolId}/health` | Health check | +| `/tools/{toolId}/test` | Connection test | +| `/tools/{toolId}/**` | Proxy API calls | +| `/tools/agent/{toolId}/**` | Proxy agent calls | + +Flow: + +1. Extract `toolId` +2. Resolve upstream +3. Forward via RestProxyService +4. Return reactive response + +--- + +## InternalAuthProbeController + +Conditional endpoint: + +- `/internal/authz/probe` +- Enabled only if `openframe.gateway.internal.enable=true` + +Used for internal service-to-service auth probing. + +--- + +# 7. WebSocket Gateway Layer + +Configured via `WebSocketGatewayConfig`. + +### Supported WebSocket Endpoints + +| Path | Purpose | +|------|----------| +| `/ws/tools/{toolId}/**` | Tool API WebSocket | +| `/ws/tools/agent/{toolId}/**` | Agent WebSocket | +| `/ws/nats` | NATS WebSocket | +| `/ws/nats-api` | NATS API WebSocket | + +```mermaid +flowchart LR + Client["WebSocket Client"] --> GatewayWS["WebSocket Gateway"] + + GatewayWS --> ToolApiFilter + GatewayWS --> ToolAgentFilter + GatewayWS --> NatsRoute + + ToolApiFilter --> UpstreamResolver + ToolAgentFilter --> UpstreamResolver +``` + +### ToolApiWebSocketProxyUrlFilter + +- Extracts toolId from path +- Resolves upstream +- Injects tool API key headers +- Removes `Origin` header + +### ToolAgentWebSocketProxyUrlFilter + +- Extracts toolId +- Resolves upstream +- No API key header mutation + +### Proxy Session Cleanup + +Optional wrapper around WebSocket client: + +- Enabled via property +- Cleans up dangling sessions +- Logs traffic metrics + +--- + +# 8. Tool Upstream Resolution + +The Gateway Service Core supports pluggable routing strategies. + +```mermaid +flowchart TD + Request --> Registry["ToolUpstreamResolverRegistry"] + Registry -->|MeshCentral| MeshResolver + Registry -->|Tactical RMM| TacticalResolver + Registry -->|Other| DefaultResolver + + MeshResolver --> ProxyUrlResolver + TacticalResolver --> ProxyUrlResolver + DefaultResolver --> ToolUrlService +``` + +## 8.1 DefaultToolUpstreamResolver + +Fallback strategy: + +- Reads ToolUrl from Mongo +- Uses ToolUrlService +- Supports REST and WebSocket + +--- + +## 8.2 MeshCentralUpstreamResolver + +Optimized routing: + +- Avoids Mongo lookup +- Reads static config from properties +- Supports API and WebSocket upstreams +- Supports optional tenant path prefix injection + +Special handling: +- Avoids double tenant path prefix +- Preserves raw query components + +--- + +## 8.3 TacticalRmmUpstreamResolver + +Multi-upstream routing logic: + +| Traffic Type | Upstream | +|-------------|----------| +| REST | Django backend | +| WS containing NATS prefix | NATS listener | +| Other WS | Daphne (ASGI) | + +Routing decision is path-based. + +--- + +# 9. Rate Limiting Model + +Integrated with API key validation. + +Headers returned (if enabled): + +- `X-Rate-Limit-Limit-Minute` +- `X-Rate-Limit-Remaining-Minute` +- `X-Rate-Limit-Limit-Hour` +- `X-Rate-Limit-Remaining-Hour` +- `X-Rate-Limit-Limit-Day` +- `X-Rate-Limit-Remaining-Day` + +On exceed: + +- HTTP 429 +- `Retry-After: 60` +- Structured JSON error + +--- + +# 10. End-to-End Request Lifecycle + +## REST API Request + +```mermaid +flowchart TD + Client --> OriginFilter + OriginFilter --> AddAuthHeader + AddAuthHeader --> JwtAuth + JwtAuth --> RoleCheck + RoleCheck --> Controller + Controller --> UpstreamResolver + UpstreamResolver --> Proxy + Proxy --> ToolService +``` + +## External API Request + +```mermaid +flowchart TD + Client --> ApiKeyFilter + ApiKeyFilter --> RateLimit + RateLimit --> ContextHeaders + ContextHeaders --> ExternalApiService +``` + +## WebSocket Request + +```mermaid +flowchart TD + Client --> SecurityDecorator + SecurityDecorator --> WsRoute + WsRoute --> UrlFilter + UrlFilter --> UpstreamResolver + UpstreamResolver --> WebSocketProxy +``` + +--- + +# 11. How It Fits in the Platform + +The Gateway Service Core is the **single public entry point** for: + +- UI requests +- Agent traffic +- Tool integrations +- External API consumers +- WebSocket communications + +It integrates with: + +- Authorization Service Core (JWT issuer, OAuth) +- API Service Core (admin APIs) +- External API Service Core (public APIs) +- Integrated Tools (MeshCentral, Tactical RMM) +- NATS messaging + +It provides: + +- Centralized security enforcement +- Multi-tenant isolation +- Tool abstraction +- Protocol bridging (HTTP ↔ WebSocket) +- Rate-limited external API access + +--- + +# Conclusion + +The **Gateway Service Core** module is the high-performance, security-aware edge layer of OpenFrame. + +It combines: + +- Reactive networking (Netty) +- OAuth2 JWT resource server +- API key + rate limiting +- Dynamic multi-tenant issuer resolution +- Pluggable upstream routing strategies +- WebSocket proxying + +This design enables OpenFrame to support: + +- SaaS multi-tenant deployments +- OSS single-tenant deployments +- Tool-agnostic integration architecture +- High concurrency WebSocket workloads + +The Gateway Service Core is foundational to the platform’s scalability, security, and extensibility. diff --git a/docs/reference/architecture/integrations-sdks/integrations-sdks.md b/docs/reference/architecture/integrations-sdks/integrations-sdks.md new file mode 100644 index 000000000..7acfaf5ed --- /dev/null +++ b/docs/reference/architecture/integrations-sdks/integrations-sdks.md @@ -0,0 +1,445 @@ +# Integrations Sdks + +## Overview + +The **Integrations Sdks** module provides typed, reusable Java SDKs for integrating OpenFrame with external RMM and device management platforms. It abstracts vendor-specific REST APIs into strongly typed models and Spring Boot auto-configurations, enabling other services in the platform to: + +- Authenticate against third-party systems +- Manage agents and hosts +- Execute remote scripts and queries +- Synchronize policies and schedules +- Parse and handle installation artifacts + +Currently, the module focuses on: + +- **Tactical RMM SDK** – agent lifecycle, script execution, and schedule management +- **Fleet MDM SDK** – host inventory, compliance policies, and live queries + +These SDKs are consumed by higher-level services such as gateway, management, and stream-processing layers, allowing OpenFrame to orchestrate actions across integrated tools. + +--- + +## High-Level Architecture + +```mermaid +flowchart LR + subgraph OpenFramePlatform["OpenFrame Platform Services"] + Management["Management Services"] + Gateway["Gateway Services"] + Stream["Stream Processing"] + end + + subgraph IntegrationsSdks["Integrations Sdks Module"] + TacticalSdk["Tactical RMM SDK"] + FleetSdk["Fleet MDM SDK"] + end + + subgraph ExternalSystems["External Systems"] + TacticalApi["Tactical RMM API"] + FleetApi["Fleet MDM API"] + end + + Management --> TacticalSdk + Management --> FleetSdk + Gateway --> TacticalSdk + Stream --> FleetSdk + + TacticalSdk --> TacticalApi + FleetSdk --> FleetApi +``` + +The Integrations Sdks module acts as a **boundary adapter layer** between OpenFrame internal services and external vendor APIs. + +--- + +## Design Principles + +### 1. Strongly Typed API Contracts +Each external endpoint is represented by: + +- Request models (e.g., `CreatePolicyRequest`, `RunScriptRequest`) +- Response models (e.g., `HostSearchResponse`, `AgentInfo`) +- Supporting value objects (e.g., `TaskAction`, `AssignedHost`) + +This ensures: + +- Compile-time validation +- Clear serialization rules via Jackson annotations +- Reduced coupling between internal services and raw JSON + +### 2. Spring Boot Auto-Configuration +The Tactical RMM SDK exposes: + +- `TacticalRmmConfig` – provides a `TacticalRmmClient` bean via Spring Boot auto-configuration + +This allows downstream services to inject and use the client without manual wiring. + +### 3. Vendor Isolation +Each vendor integration is encapsulated in its own namespace: + +- `com.openframe.sdk.tacticalrmm.*` +- `com.openframe.sdk.fleetmdm.*` + +This enables: + +- Independent evolution +- Clean separation of models +- Future addition of new SDKs without breaking existing integrations + +--- + +# Tactical RMM SDK + +## Purpose + +The Tactical RMM SDK enables OpenFrame to interact with Tactical RMM instances for: + +- Agent discovery and inspection +- Script execution +- Scheduled automation tasks +- Agent assignment management +- Registration secret extraction + +--- + +## Core Configuration + +### TacticalRmmConfig + +Registers a `TacticalRmmClient` as a Spring Bean: + +```mermaid +flowchart TD + AutoConfig["TacticalRmmConfig"] --> Bean["TacticalRmmClient Bean"] + Bean --> Services["Management or Gateway Services"] +``` + +This enables dependency injection across services that need to call Tactical RMM. + +--- + +## Agent Models + +### AgentInfo +Represents detailed agent information returned by Tactical RMM APIs. + +Key attributes: +- `agentId` +- `platform` +- `operatingSystem` +- `hostname` + +### AgentListItem +Lightweight representation for list endpoints (`detail=false`). + +Includes: +- Internal primary key (`id` / `pk` alias) +- Agent ID +- Hostname +- Site +- Client + +### AssignedAgent +Used for assigning agents to scheduled tasks. + +--- + +## Script Execution + +### RunScriptRequest +Represents payload for: + +- `POST /agents//runscript/` + +Supports: +- Script ID +- Arguments +- Environment variables +- Timeout +- Output mode (`wait`, `email`, `collector`) +- Optional server execution + +```mermaid +sequenceDiagram + participant Service + participant TacticalApi as "Tactical RMM API" + + Service->>TacticalApi: RunScriptRequest + TacticalApi-->>Service: Script execution result +``` + +--- + +## Scheduled Script Management + +### TacticalScheduledScript +Represents a scheduled task including: + +- Schedule metadata +- Recurrence intervals +- Supported platforms +- Assigned agents +- Associated actions + +### CreateScriptScheduleRequest / UpdateScriptScheduleRequest +Used to: + +- Create recurring scheduled scripts +- Modify recurrence, enabled state, or actions + +### TaskAction +Defines a single action inside a scheduled task: + +- Script reference +- Timeout +- Run-as-user flag +- Script arguments +- Environment variables + +### ScriptScheduleAgentsResult +Reports: + +- Number of assigned agents +- Task result rows created or deleted during sync + +--- + +## Registration Secret Handling + +### RegistrationSecretParser + +Extracts the `--auth` secret from a Tactical RMM install command string. + +This is critical when: + +- Parsing installation commands +- Synchronizing registration secrets +- Automating agent bootstrap flows + +```mermaid +flowchart TD + Command["Install Command String"] --> Parser["RegistrationSecretParser"] + Parser --> Secret["Extracted Auth Secret"] +``` + +The parser: +- Supports quoted and unquoted values +- Is case-insensitive +- Provides fallback pattern matching + +--- + +# Fleet MDM SDK + +## Purpose + +The Fleet MDM SDK enables OpenFrame to integrate with Fleet for: + +- Host inventory synchronization +- Compliance policy management +- Scheduled and live query execution +- Authentication and setup workflows + +--- + +## Authentication Models + +### LoginResponse +Returns: +- API token +- Available teams + +### SetupResponse +Returns: +- Initial token during setup + +### CreateUserResponse +Returns: +- Token for newly created user + +These tokens are used by higher-level services to authenticate subsequent API calls. + +--- + +## Host Management + +### Host +Represents a fully detailed Fleet host including: + +- Identification: `id`, `uuid`, `hostname` +- OS and platform metadata +- CPU, memory, and disk metrics +- Network information (IP, MAC) +- Enrollment and update timestamps +- Team metadata + +### HostSearchResponse +Wraps paginated host search results: + +- List of `Host` +- Page metadata +- Sort and filter information + +```mermaid +flowchart TD + Service["OpenFrame Service"] --> FleetApi["Fleet MDM API"] + FleetApi --> Response["HostSearchResponse"] + Response --> Hosts["List of Host"] +``` + +--- + +## Policy Management + +### Policy +Represents a compliance policy in Fleet: + +- Query logic +- Resolution guidance +- Author metadata +- Passing/failing host counts +- Team association +- Assigned hosts + +### CreatePolicyRequest / UpdatePolicyRequest +Used to: + +- Define compliance rules +- Modify name, query, description, and platform + +--- + +## Scheduled Queries + +### CreateScheduledQueryRequest / UpdateScheduledQueryRequest + +Defines recurring osquery-based checks: + +- Query text +- Interval +- Platform targeting + +--- + +## Live Queries + +### RunLiveQueryRequest + +Used to create distributed campaigns via: + +- Ad-hoc SQL queries +- Existing saved query IDs +- Target selection by: + - Host IDs + - Label IDs + - Team IDs + +### LiveQueryCampaign +Represents a created campaign: + +- Campaign ID +- Query +- Status +- User reference +- Creation timestamp + +```mermaid +sequenceDiagram + participant Service + participant FleetApi as "Fleet MDM API" + + Service->>FleetApi: RunLiveQueryRequest + FleetApi-->>Service: LiveQueryCampaign + Note over FleetApi: Results streamed asynchronously +``` + +--- + +# Cross-Cutting Concerns + +## JSON Mapping + +All models use: + +- `@JsonProperty` for field mapping +- `@JsonIgnoreProperties(ignoreUnknown = true)` for forward compatibility +- `@JsonInclude(JsonInclude.Include.NON_NULL)` for minimal payloads + +This ensures resilience against: + +- Vendor API version drift +- Backward-incompatible field additions + +--- + +## Error Isolation + +The SDK layer: + +- Does not embed business logic +- Avoids persistence concerns +- Focuses purely on transport and representation + +This separation allows: + +- Business services to remain vendor-agnostic +- Easy mocking in tests +- Swapping or upgrading integrations independently + +--- + +# Integration Flow Example + +Below is a typical flow where OpenFrame schedules a script in Tactical RMM: + +```mermaid +flowchart TD + ManagementService["Management Service"] --> TacticalClient["TacticalRmmClient"] + TacticalClient --> CreateSchedule["CreateScriptScheduleRequest"] + CreateSchedule --> TacticalApi["Tactical RMM API"] + TacticalApi --> ScheduledScript["TacticalScheduledScript"] + ScheduledScript --> ManagementService +``` + +And a Fleet compliance sync example: + +```mermaid +flowchart TD + SyncService["Compliance Sync Service"] --> FleetClient["Fleet SDK"] + FleetClient --> PolicyRequest["CreatePolicyRequest"] + PolicyRequest --> FleetApi["Fleet MDM API"] + FleetApi --> Policy["Policy"] + Policy --> SyncService +``` + +--- + +# Extensibility Strategy + +The Integrations Sdks module is designed for expansion: + +- Each new vendor gets a dedicated package namespace +- Models remain vendor-specific and isolated +- Auto-configuration classes register vendor clients +- Higher-level services depend only on SDK abstractions + +Future integrations can follow the same pattern: + +1. Create vendor-specific model classes +2. Provide a typed client +3. Register via Spring Boot auto-configuration +4. Keep business orchestration outside the SDK + +--- + +# Summary + +The **Integrations Sdks** module is a foundational integration layer within OpenFrame. It: + +- Encapsulates third-party APIs +- Provides strongly typed request/response contracts +- Supports Tactical RMM and Fleet MDM integrations +- Enables management, gateway, and stream services to orchestrate external tools +- Maintains strict separation between transport models and business logic + +By isolating external integrations behind structured SDKs, OpenFrame ensures maintainability, scalability, and clean architectural boundaries across its distributed services ecosystem. diff --git a/docs/reference/architecture/management-service-core-initializers-and-schedulers/management-service-core-initializers-and-schedulers.md b/docs/reference/architecture/management-service-core-initializers-and-schedulers/management-service-core-initializers-and-schedulers.md deleted file mode 100644 index 2d3ed0a48..000000000 --- a/docs/reference/architecture/management-service-core-initializers-and-schedulers/management-service-core-initializers-and-schedulers.md +++ /dev/null @@ -1,433 +0,0 @@ -# Management Service Core Initializers And Schedulers - -The **Management Service Core Initializers And Schedulers** module is responsible for bootstrapping, orchestrating, and maintaining background operational workflows across the OpenFrame platform. It ensures that required configuration artifacts, streams, secrets, scripts, and distributed jobs are initialized correctly at startup and remain consistent throughout runtime. - -This module acts as the operational backbone for: - -- System bootstrapping (initial secrets, client configuration, tool agents) -- Distributed scheduling with cluster-safe locking -- Stream and messaging initialization (NATS) -- External tool orchestration (e.g., Tactical RMM) -- Maintenance and retry workflows -- Pinot resynchronization and fleet setup tasks - ---- - -## Architectural Overview - -The module is structured into five major concerns: - -1. **Configuration Layer** – Enables scheduling, retries, and distributed locks. -2. **Initializers** – Bootstrap critical data and external integrations at startup. -3. **Schedulers** – Periodic maintenance and retry workflows. -4. **Controllers** – Operational endpoints for manual triggers. -5. **Processors & Services** – Extension points and domain orchestration logic. - -```mermaid -flowchart TD - subgraph ConfigLayer["Configuration Layer"] - MgmtConfig["ManagementConfiguration"] - RetryConfig["RetryConfiguration"] - ShedLock["ShedLockConfig"] - end - - subgraph Initializers["Startup Initializers"] - SecretInit["AgentRegistrationSecretInitializer"] - AgentInit["IntegratedToolAgentInitializer"] - NatsInit["NatsStreamConfigurationInitializer"] - ClientInit["OpenFrameClientConfigurationInitializer"] - TacticalInit["TacticalRmmScriptsInitializer"] - end - - subgraph Schedulers["Background Schedulers"] - VersionFallback["AgentVersionUpdatePublishFallbackScheduler"] - ApiKeySync["ApiKeyStatsSyncScheduler"] - Heartbeat["DeviceHeartbeatOfflineDetectionScheduler"] - FleetMdm["FleetMdmSetupScheduler"] - end - - subgraph Controllers["Operational Controllers"] - ToolCtrl["IntegratedToolController"] - PinotCtrl["DevicePinotResyncController"] - ReleaseCtrl["ReleaseVersionController"] - end - - MgmtConfig --> Initializers - ShedLock --> Schedulers - RetryConfig --> Schedulers - Controllers --> Schedulers - Initializers --> Schedulers -``` - ---- - -# 1. Configuration Layer - -## ManagementConfiguration - -Provides the base Spring configuration for the module. - -Key responsibilities: -- Component scanning across `com.openframe` -- Excludes `CassandraHealthIndicator` to avoid unnecessary health wiring -- Exposes a `PasswordEncoder` using BCrypt for secure hashing - -This ensures secure secret generation and compatibility with other security modules. - ---- - -## RetryConfiguration - -Enables Spring Retry using: - -```text -@EnableRetry -``` - -This allows services and schedulers to use retry semantics for transient failures (e.g., network, external systems, Redis, NATS). - ---- - -## ShedLockConfig - -Enables: - -- `@EnableScheduling` -- `@EnableSchedulerLock` - -Provides a Redis-backed distributed `LockProvider`. - -### Tenant-Scoped Locking - -Lock keys follow the pattern: - -```text -of:{tenantId}:job-lock:: -``` - -This ensures: -- No duplicate scheduler execution across clustered nodes -- Isolation per tenant -- Environment separation - -```mermaid -flowchart LR - Scheduler["Scheduled Job"] --> LockProvider["RedisLockProvider"] - LockProvider --> Redis[("Redis")] - Redis --> LockKey["Tenant Scoped Lock Key"] -``` - ---- - -# 2. Startup Initializers - -All initializers implement `ApplicationRunner`, meaning they execute after Spring Boot startup. - ---- - -## AgentRegistrationSecretInitializer - -Ensures an initial `AgentRegistrationSecret` exists at startup. - -Flow: - -```mermaid -flowchart TD - Start["Application Startup"] --> SecretInit["AgentRegistrationSecretInitializer"] - SecretInit --> Service["AgentRegistrationSecretManagementService"] - Service --> Create["createInitialSecret()"] - Create --> Processor["DefaultAgentRegistrationSecretManagementProcessor"] -``` - -Features: -- Idempotent secret creation -- Pluggable post-processing via `AgentRegistrationSecretManagementProcessor` -- Safe failure handling - ---- - -## IntegratedToolAgentInitializer - -Loads agent configurations from classpath resources. - -Responsibilities: -- Reads JSON configuration files -- Maps to `IntegratedToolAgentConfiguration` -- Updates persisted configuration via `IntegratedToolAgentService` -- Fails fast if no configuration is defined - -This guarantees consistent tool-agent configuration across environments. - ---- - -## NatsStreamConfigurationInitializer - -Pre-provisions required NATS streams: - -- TOOL_INSTALLATION -- CLIENT_UPDATE -- TOOL_UPDATE -- TOOL_CONNECTIONS -- INSTALLED_AGENTS - -Each stream defines: -- Subjects (wildcard-based routing) -- File storage -- Retention policy - -Also supports extension via `AdditionalStreamConfigurationProvider`. - -```mermaid -flowchart TD - Init["NatsStreamConfigurationInitializer"] --> DefaultStreams["Default Stream Configurations"] - Init --> Additional["AdditionalStreamConfigurationProvider"] - DefaultStreams --> NatsService["NatsStreamManagementService"] - Additional --> NatsService -``` - ---- - -## OpenFrameClientConfigurationInitializer - -Loads the default client configuration from: - -```text -agent-configurations/client-configuration.json -``` - -Steps: -1. Deserialize JSON -2. Assign default ID -3. Persist via `OpenFrameClientConfigurationService` - -Ensures the client always has a canonical configuration baseline. - ---- - -## TacticalRmmScriptsInitializer - -Bootstraps PowerShell scripts inside Tactical RMM. - -Workflow: - -```mermaid -flowchart TD - Startup["Application Startup"] --> LoadTool["Load Tactical RMM Tool"] - LoadTool --> FetchScripts["Fetch Existing Scripts"] - FetchScripts --> Compare["Compare by Name"] - Compare -->|"Missing"| Create["Create Script"] - Compare -->|"Exists"| Update["Update Script"] -``` - -Features: -- Loads script bodies from classpath -- Idempotent create-or-update behavior -- Uses API key stored in IntegratedTool -- Logs per-script success/failure - -This guarantees consistent automation availability for OpenFrame client updates. - ---- - -# 3. Background Schedulers - -Schedulers run conditionally based on configuration properties and use ShedLock for distributed safety when needed. - ---- - -## AgentVersionUpdatePublishFallbackScheduler - -Purpose: Retry publishing client and tool-agent updates when previous attempts failed. - -Key concepts: -- Checks `PublishState` -- Retries until `maxPublishAttempts` -- Publishes via NATS publishers - -```mermaid -flowchart TD - Tick["Scheduled Tick"] --> ClientCheck["Check Client PublishState"] - ClientCheck --> RetryClient["Publish If Needed"] - Tick --> AgentCheck["Check Tool Agents"] - AgentCheck --> RetryAgent["Publish If Needed"] -``` - -This ensures eventual consistency for version update propagation. - ---- - -## ApiKeyStatsSyncScheduler - -Synchronizes API key statistics from Redis to MongoDB. - -Features: -- Distributed lock (`apiKeyStatsSync`) -- Configurable interval -- Safe retry behavior - -Ensures durable persistence of usage metrics. - ---- - -## DeviceHeartbeatOfflineDetectionScheduler - -Periodically marks stale devices as offline. - -Flow: - -```mermaid -flowchart LR - Scheduler["Heartbeat Scheduler"] --> Service["DeviceHeartbeatOfflineDetectionService"] - Service --> Mongo[("MongoDB Device Records")] -``` - -This keeps device health states accurate. - ---- - -## FleetMdmSetupScheduler - -Ensures Fleet MDM server is configured and API tokens are generated when necessary. - -Logic: -- Looks up `fleetmdm-server` tool -- Runs setup logic if present -- Retries on failure - -This supports automated fleet provisioning. - ---- - -# 4. Operational Controllers - -These controllers provide operational endpoints for manual intervention. - ---- - -## IntegratedToolController - -Endpoint base: `/v1/tools` - -Responsibilities: -- Retrieve tool configurations -- Save tool configuration -- Conditionally create/update Debezium connectors -- Execute post-save hooks - -```mermaid -flowchart TD - Client["API Client"] --> Controller["IntegratedToolController"] - Controller --> Save["IntegratedToolService.saveTool()"] - Save --> TenantCheck["TenantIdProvider"] - TenantCheck -->|"Registered"| Debezium["DebeziumService"] - Save --> Hooks["IntegratedToolPostSaveHook"] -``` - -Key design detail: -- Debezium connectors are deferred until tenant registration. - ---- - -## DevicePinotResyncController - -Endpoint: `/v1/devices/pinot-resync` - -Purpose: -- Replays all machines to Pinot -- Uses `MachineTagEventService.processMachineSaveAll` - -This is typically used for analytics rehydration or recovery. - ---- - -## ReleaseVersionController - -Endpoint: `/v1/cluster-registrations` - -Delegates version processing to `ReleaseVersionService`. - -Intended to coordinate cluster-level version awareness and propagation. - ---- - -# 5. Services and Extension Points - -## OpenFrameClientVersionUpdateService - -Currently serves as a coordination entry point for client version publishing. - -Uses: -- `OpenFrameClientUpdatePublisher` - -Designed for future expansion to centralize version update orchestration. - ---- - -## IntegratedToolPostSaveHook - -A lightweight extension mechanism: - -```text -void onToolSaved(String toolId, IntegratedTool tool) -``` - -Advantages: -- No Spring event overhead -- Simple service-specific side effects -- Clean separation of concerns - ---- - -## DefaultAgentRegistrationSecretManagementProcessor - -Default implementation used when no custom processor is defined. - -Behavior: -- Logs secret creation -- Supports override via Spring bean replacement - ---- - -# Runtime Interaction Summary - -The module ensures: - -1. Startup is safe and idempotent. -2. Distributed jobs execute only once per cluster. -3. External systems (NATS, Tactical RMM, Debezium) are reconciled. -4. Retry and fallback logic guarantee eventual consistency. -5. Operational APIs allow manual recovery and resync. - -```mermaid -flowchart TD - Boot["Application Boot"] --> Initializers - Initializers --> Messaging["NATS Streams Ready"] - Messaging --> Schedulers - Schedulers --> External["External Systems"] - Controllers --> External -``` - ---- - -# Key Design Principles - -- **Idempotency First** – Initializers and schedulers tolerate repeated execution. -- **Distributed Safety** – ShedLock prevents duplicate execution. -- **Tenant Isolation** – Lock keys and processing are tenant-scoped. -- **Eventual Consistency** – Fallback schedulers ensure propagation. -- **Extensibility** – Hooks and processors enable modular extension. - ---- - -# Conclusion - -The **Management Service Core Initializers And Schedulers** module is the operational orchestrator of OpenFrame. It ensures that the system boots correctly, remains consistent across distributed nodes, maintains external integrations, and continuously reconciles state through scheduled processes. - -Without this module, OpenFrame would lack: - -- Safe distributed background processing -- Automated stream and tool provisioning -- Reliable publish retry mechanisms -- Operational recovery endpoints - -It is a foundational module enabling stable, self-healing, multi-tenant management operations. \ No newline at end of file diff --git a/docs/reference/architecture/management-service-core/management-service-core.md b/docs/reference/architecture/management-service-core/management-service-core.md new file mode 100644 index 000000000..a9a62a3ba --- /dev/null +++ b/docs/reference/architecture/management-service-core/management-service-core.md @@ -0,0 +1,450 @@ +# Management Service Core + +## Overview + +The **Management Service Core** module is the operational backbone of the OpenFrame platform. It is responsible for: + +- System bootstrapping and environment initialization +- Tool and agent configuration management +- Scheduled background orchestration +- Data migrations and lifecycle evolution +- Cross-system synchronization (MongoDB, Redis, NATS, Kafka Connect, Pinot) +- Operational endpoints for cluster and tool management + +Unlike API-facing modules, the Management Service Core focuses on **infrastructure orchestration, consistency enforcement, and platform lifecycle control**. + +It sits between: + +- Data layer modules (MongoDB, Redis, Kafka, NATS, Pinot) +- Integration SDKs (Tactical RMM, Fleet MDM) +- Messaging infrastructure +- Tenant-aware domain services + +--- + +## High-Level Architecture + +```mermaid +flowchart TD + Controllers["REST Controllers"] --> Services["Management Services"] + Services --> DataLayer["MongoDB / Redis Repositories"] + Services --> Messaging["NATS / Kafka Publishers"] + Services --> ExternalTools["Integrated Tools (RMM / MDM)"] + + Initializers["Application Initializers"] --> Services + Schedulers["Distributed Schedulers"] --> Services + Migrations["Mongock Change Units"] --> DataLayer + + Locking["ShedLock (Redis)"] --> Schedulers +``` + +### Responsibilities by Layer + +| Layer | Responsibility | +|--------|---------------| +| Controllers | Operational control endpoints | +| Initializers | System bootstrap and configuration seeding | +| Schedulers | Background reconciliation and retry jobs | +| Services | Business orchestration logic | +| Migrations | Data model evolution and backfills | +| Config | Retry, scheduling, locking, security setup | + +--- + +## Configuration Layer + +### ManagementConfiguration + +- Enables component scanning across `com.openframe` +- Excludes `CassandraHealthIndicator` +- Provides `PasswordEncoder` using `BCryptPasswordEncoder` + +This ensures secure hashing for management-related credentials. + +--- + +### RetryConfiguration + +```text +@EnableRetry +``` + +Enables Spring Retry support for resilience in external calls (e.g., tool integrations, connector creation, sync operations). + +--- + +### ShedLockConfig + +The module uses **ShedLock with Redis** for distributed scheduling safety. + +Key characteristics: + +- Enables Spring scheduling +- Uses Redis-based lock provider +- Tenant-aware lock prefix +- Environment-specific lock namespace + +Lock key pattern: + +```text +of:{tenantId}:job-lock:: +``` + +This prevents duplicate execution across clustered instances. + +--- + +## REST Controllers + +The Management Service Core exposes operational endpoints. + +### DevicePinotResyncController + +Endpoint: + +```text +POST /v1/devices/pinot-resync +``` + +Purpose: + +- Loads all machines from MongoDB +- Re-emits machine save events +- Triggers reprocessing into Pinot analytics + +Data Flow: + +```mermaid +flowchart LR + API["Resync Endpoint"] --> Repo["MachineRepository"] + Repo --> Service["MachineTagEventService"] + Service --> Pinot["Pinot Analytics"] +``` + +This is primarily used for recovery or analytics backfills. + +--- + +### IntegratedToolController + +Endpoint namespace: + +```text +/v1/tools +``` + +Responsibilities: + +- Retrieve tool configurations +- Create or update integrated tools +- Persist Debezium connector templates +- Trigger Kafka Connect updates (if tenant registered) +- Invoke post-save hooks + +Flow of a Save Operation: + +```mermaid +flowchart TD + Request["SaveToolRequest"] --> ToolService["IntegratedToolService"] + ToolService --> Mongo["MongoDB"] + ToolService --> Debezium["DebeziumService"] + ToolService --> Hooks["IntegratedToolPostSaveHook[]"] +``` + +Special behavior: + +- Connectors are deferred until tenant registration +- Hooks provide extension points without event bus complexity + +--- + +### ReleaseVersionController + +Endpoint: + +```text +POST /v1/cluster-registrations +``` + +Accepts: + +```text +ReleaseVersionRequest + imageTagVersion: String +``` + +Delegates to `ReleaseVersionService` for cluster version handling. + +--- + +## Application Initializers + +Initializers implement `ApplicationRunner` and execute at startup. + +### AgentRegistrationSecretInitializer + +- Ensures an initial agent registration secret exists +- Delegates to `AgentRegistrationSecretManagementService` +- Supports pluggable post-processing + +--- + +### IntegratedToolAgentInitializer + +- Loads agent configurations from classpath JSON files +- Deserializes into `IntegratedToolAgentConfiguration` +- Updates persistent configuration fields +- Fails fast if no configurations defined + +--- + +### NatsStreamConfigurationInitializer + +Creates NATS streams at startup. + +Configured streams include: + +- TOOL_INSTALLATION +- CLIENT_UPDATE +- TOOL_UPDATE +- TOOL_CONNECTIONS +- INSTALLED_AGENTS + +Each stream defines: + +- Subject patterns (e.g., `machine.*.tool-installation`) +- Storage type +- Retention policy + +```mermaid +flowchart TD + Init["NATS Stream Initializer"] --> Mgmt["NatsStreamManagementService"] + Mgmt --> NATS["NATS JetStream"] +``` + +--- + +### OpenFrameClientConfigurationInitializer + +- Loads `client-configuration.json` +- Updates stored OpenFrame client configuration +- Ensures consistent rollout behavior + +--- + +### TacticalRmmScriptsInitializer + +Synchronizes predefined scripts into Tactical RMM: + +1. Loads script content from classpath +2. Fetches existing scripts from Tactical RMM +3. Creates or updates scripts as needed + +```mermaid +flowchart TD + Start["Startup"] --> Tool["IntegratedToolService"] + Tool --> Tactical["TacticalRmmClient"] + Tactical --> Create["Create Script"] + Tactical --> Update["Update Script"] +``` + +This ensures platform-managed automation scripts remain consistent. + +--- + +## Data Migrations (Mongock) + +The module includes Mongock `@ChangeUnit` migrations. + +### BackfillDocumentVersionChangeUnit + +Backfills missing `documentVersion` fields in: + +- integrated_tool_agents +- openframe_client_configuration +- release_versions + +Sets default value: + +```text +0L +``` + +--- + +### BackfillTicketOrdersChangeUnit + +Backfills ticket ordering using **LexoRank**: + +- Groups by status +- Sorts by creation date +- Assigns incremental ranks + +```mermaid +flowchart TD + Query["Tickets Without Order"] --> Sort["Sort by CreatedAt DESC"] + Sort --> Rank["Generate LexoRank"] + Rank --> Update["Update Ticket Order Field"] +``` + +--- + +### MigrateTicketStatusesChangeUnit + +Performs lifecycle migration from legacy status field to new model: + +- Seeds system statuses +- Migrates legacy tickets +- Sets `statusId` and `statusKind` +- Removes legacy field +- Feature flag controlled + +This migration is tenant-aware and safe to rerun. + +--- + +## Distributed Schedulers + +Schedulers perform background consistency tasks. + +### AgentVersionUpdatePublishFallbackScheduler + +Purpose: + +- Retries publishing OpenFrame client updates +- Retries publishing tool agent updates +- Honors max publish attempts +- Uses ShedLock for distributed safety + +```mermaid +flowchart TD + Tick["Scheduled Tick"] --> Check["Check PublishState"] + Check --> PublishClient["Publish Client Update"] + Check --> PublishTool["Publish Tool Agent Update"] +``` + +--- + +### ApiKeyStatsSyncScheduler + +- Synchronizes API key usage stats from Redis to MongoDB +- Distributed lock protected +- Configurable interval + +--- + +### DeviceHeartbeatOfflineDetectionScheduler + +- Periodically detects stale device heartbeats +- Marks devices offline +- Controlled via feature property + +--- + +### FleetMdmSetupScheduler + +- Detects Fleet MDM server integration +- Runs setup if needed +- Retrieves and persists API token + +--- + +## Service Layer + +### OpenFrameClientVersionUpdateService + +Responsible for processing new release versions. + +Currently delegates publishing behavior via: + +- `OpenFrameClientUpdatePublisher` + +Designed to: + +- Trigger client version rollout +- Coordinate update distribution + +--- + +### DefaultAgentRegistrationSecretManagementProcessor + +Provides default post-processing for newly created agent secrets. + +Extensible via: + +```text +AgentRegistrationSecretManagementProcessor +``` + +Allows platform-specific behavior injection without modifying core logic. + +--- + +## Cross-Cutting Concerns + +### Multi-Tenancy + +- TenantIdProvider used across migrations and controllers +- Lock keys are tenant-scoped +- Queries are tenant-filtered + +--- + +### Messaging Integration + +The module integrates with: + +- NATS JetStream +- Kafka Connect (Debezium connectors) +- Redis +- MongoDB + +```mermaid +flowchart LR + Mongo["MongoDB"] --> Debezium["Kafka Connect"] + Services --> NATS["NATS Publisher"] + Redis["Redis"] --> Schedulers +``` + +--- + +### Resilience & Safety + +- Spring Retry enabled +- ShedLock distributed scheduling +- Idempotent migrations +- Defensive exception handling +- Feature-flag guarded rollouts + +--- + +## Execution Lifecycle + +```mermaid +flowchart TD + Boot["Application Boot"] --> Config["Configuration Beans"] + Config --> Initializers["ApplicationRunner Initializers"] + Initializers --> Streams["NATS Streams Created"] + Initializers --> Secrets["Secrets Created"] + Initializers --> ToolConfig["Tool Config Applied"] + + Boot --> Migrations["Mongock Change Units"] + Boot --> Schedulers["Background Jobs Activated"] +``` + +--- + +## Summary + +The **Management Service Core** module is responsible for: + +- Bootstrapping platform infrastructure +- Managing integrated tool lifecycle +- Orchestrating background reconciliation jobs +- Enforcing tenant-safe distributed execution +- Performing data migrations and schema evolution +- Synchronizing cross-system configurations + +It acts as the **operational control plane** of OpenFrame, ensuring that configuration, integrations, and distributed tasks remain consistent across tenants and clustered deployments. + +This module is foundational for platform stability, lifecycle management, and integration governance. \ No newline at end of file diff --git a/docs/reference/architecture/security-core-and-oauth-bff/security-core-and-oauth-bff.md b/docs/reference/architecture/security-core-and-oauth-bff/security-core-and-oauth-bff.md deleted file mode 100644 index 95246761a..000000000 --- a/docs/reference/architecture/security-core-and-oauth-bff/security-core-and-oauth-bff.md +++ /dev/null @@ -1,351 +0,0 @@ -# Security Core And Oauth Bff - -## Overview - -The **Security Core And Oauth Bff** module provides the foundational security building blocks for the OpenFrame platform. It combines: - -- JWT encoding and decoding infrastructure -- RSA key configuration and loading -- PKCE (Proof Key for Code Exchange) utilities -- OAuth2 Backend-for-Frontend (BFF) endpoints -- Redirect resolution and state management - -This module acts as the bridge between the platform’s Authorization Server, Gateway, and frontend clients. It centralizes token handling, cookie management, and OAuth flows while remaining configurable and extensible. - ---- - -## Architectural Role in the Platform - -At runtime, Security Core And Oauth Bff sits between: - -- Frontend clients (browser-based applications) -- The Authorization Server -- The Gateway layer - -It is responsible for: - -- Initiating OAuth2 authorization flows -- Managing PKCE and state -- Exchanging authorization codes for tokens -- Storing tokens securely in HTTP-only cookies -- Refreshing and revoking tokens -- Providing JWT encoder/decoder beans for internal services - -### High-Level Architecture - -```mermaid -flowchart LR - Browser["Browser Client"] -->|"GET /oauth/login"| Bff["OAuth BFF Controller"] - Bff -->|"Redirect to authorize"| AuthServer["Authorization Server"] - AuthServer -->|"Callback with code"| Bff - Bff -->|"Exchange code for tokens"| AuthServer - Bff -->|"Set HttpOnly Cookies"| Browser - - Browser -->|"API calls with cookies"| Gateway["Gateway Service"] - Gateway -->|"Validate JWT"| JwtDecoder["JwtDecoder Bean"] -``` - ---- - -## Core Components - -### 1. JwtSecurityConfig - -**Class:** `JwtSecurityConfig` - -This Spring configuration class provides the core JWT infrastructure: - -- `JwtEncoder` bean using `NimbusJwtEncoder` -- `JwtDecoder` bean using `NimbusJwtDecoder` -- RSA-based signing and verification - -#### Responsibilities - -- Build a JWK set from configured RSA keys -- Expose a `JwtEncoder` for signing tokens -- Expose a `JwtDecoder` for validating tokens - -### JWT Bean Configuration Flow - -```mermaid -flowchart TD - JwtConfig["JwtConfig"] -->|"loadPublicKey()"| PublicKey["RSAPublicKey"] - JwtConfig -->|"loadPrivateKey()"| PrivateKey["RSAPrivateKey"] - - PublicKey --> RsaKey["RSAKey Builder"] - PrivateKey --> RsaKey - - RsaKey --> JwkSet["JWKSet"] - JwkSet --> JwtEncoder["NimbusJwtEncoder"] - - PublicKey --> JwtDecoder["NimbusJwtDecoder"] -``` - -This design ensures: - -- Strong asymmetric cryptography (RSA) -- Compatibility with Spring Security OAuth2 Resource Server -- Clean separation of configuration and usage - ---- - -### 2. JwtConfig - -**Class:** `JwtConfig` - -`JwtConfig` is a Spring `@ConfigurationProperties` service that binds properties under the `jwt` prefix. - -#### Properties - -- `publicKey` -- `privateKey` -- `issuer` -- `audience` - -#### Key Responsibilities - -- Load RSA public key -- Parse and decode PEM-encoded private key -- Produce `RSAPublicKey` and `RSAPrivateKey` instances - -The private key loading process: - -```mermaid -flowchart TD - PemString["PEM Private Key"] --> StripHeaders["Remove BEGIN/END markers"] - StripHeaders --> Base64Decode["Base64 Decode"] - Base64Decode --> KeySpec["PKCS8EncodedKeySpec"] - KeySpec --> KeyFactory["KeyFactory RSA"] - KeyFactory --> RsaPrivate["RSAPrivateKey"] -``` - -This encapsulation prevents direct cryptographic handling throughout the codebase and centralizes key logic. - ---- - -### 3. SecurityConstants - -**Class:** `SecurityConstants` - -Defines standardized names used across OAuth flows: - -- `authorization` query parameter -- `access_token` -- `refresh_token` -- `Access-Token` header -- `Refresh-Token` header - -This prevents string duplication and ensures consistent token propagation between: - -- BFF controller -- Cookie service -- Gateway filters - ---- - -### 4. PKCEUtils - -**Class:** `PKCEUtils` - -Utility class implementing PKCE for secure OAuth2 Authorization Code flows. - -#### Provided Methods - -- `generateState()` – 128-bit random value -- `generateCodeVerifier()` – 256-bit random value -- `generateCodeChallenge()` – SHA-256 based challenge -- `urlEncode()` – Safe redirect parameter encoding - -### PKCE Flow - -```mermaid -flowchart TD - Verifier["Code Verifier (random 32 bytes)"] --> Hash["SHA-256"] - Hash --> Challenge["Base64URL Encode"] - - Challenge -->|"Sent to Authorization Server"| AuthServer["Authorization Server"] - Verifier -->|"Stored by BFF"| BffStore["State JWT / Cookie"] -``` - -This ensures: - -- Protection against authorization code interception -- CSRF mitigation via state parameter -- Strong cryptographic randomness via `SecureRandom` - ---- - -## OAuth Backend-For-Frontend (BFF) - -### OAuthBffController - -**Class:** `OAuthBffController` - -This is the central HTTP interface for browser-based authentication. - -It is conditionally enabled via: - -- `openframe.gateway.oauth.enable=true` - -### Exposed Endpoints - -| Endpoint | Method | Purpose | -|-----------|--------|----------| -| `/oauth/login` | GET | Start OAuth flow | -| `/oauth/continue` | GET | Continue flow without clearing session | -| `/oauth/callback` | GET | Handle authorization code | -| `/oauth/refresh` | POST | Refresh tokens | -| `/oauth/logout` | GET | Revoke refresh token and clear cookies | -| `/oauth/dev-exchange` | GET | Exchange development ticket | - ---- - -## End-to-End Login Flow - -```mermaid -sequenceDiagram - participant Browser - participant BFF as OAuth BFF Controller - participant Auth as Authorization Server - - Browser->>BFF: GET /oauth/login - BFF->>Auth: Redirect with state and PKCE challenge - Auth->>Browser: Login UI - Auth->>BFF: Callback with code and state - BFF->>Auth: Exchange code for tokens - Auth->>BFF: Access and Refresh tokens - BFF->>Browser: Set HttpOnly cookies and redirect -``` - -### Key Mechanics - -1. State is signed as a JWT. -2. State cookie is stored with configurable TTL. -3. Tokens are returned as HTTP-only cookies. -4. Optional development ticket injection. -5. Errors redirect to configured error URL. - ---- - -## Token Refresh Flow - -```mermaid -sequenceDiagram - participant Browser - participant BFF - participant Auth as Authorization Server - - Browser->>BFF: POST /oauth/refresh - BFF->>Auth: Refresh token request - Auth->>BFF: New tokens - BFF->>Browser: Set new cookies -``` - -Features: - -- Supports refresh via cookie or header -- Supports tenant-based lookup -- Returns 401 if token missing or invalid - ---- - -## Logout Flow - -```mermaid -sequenceDiagram - participant Browser - participant BFF - participant Auth as Authorization Server - - Browser->>BFF: GET /oauth/logout - BFF->>Auth: Revoke refresh token - BFF->>Browser: Clear auth cookies -``` - -Logout ensures: - -- Refresh token revocation -- Cookie invalidation -- Stateless cleanup - ---- - -## Redirect Resolution - -### DefaultRedirectTargetResolver - -Provides fallback redirect resolution logic: - -1. Use explicit `redirectTo` parameter if provided -2. Fallback to `Referer` header -3. Default to `/` - -```mermaid -flowchart TD - Requested["Requested redirectTo"] -->|"Has value"| UseRequested["Use requested value"] - Requested -->|"Empty"| RefererCheck["Check Referer header"] - RefererCheck -->|"Present"| UseReferer["Use Referer"] - RefererCheck -->|"Missing"| DefaultRoot["Use /"] -``` - -This allows safe continuation after login without tight frontend coupling. - ---- - -## Security Design Principles - -Security Core And Oauth Bff follows these principles: - -- **Asymmetric cryptography** for JWT signing -- **Short-lived state tokens** for CSRF prevention -- **PKCE enforcement** for public clients -- **HttpOnly cookie storage** for tokens -- **Tenant-aware flows** -- **Reactive non-blocking controllers** using Project Reactor - ---- - -## Configuration Overview - -### JWT Properties - -```text -jwt.publicKey.value= -jwt.privateKey.value= -jwt.issuer=openframe -jwt.audience=openframe-api -``` - -### OAuth BFF Properties - -```text -openframe.gateway.oauth.enable=true -openframe.gateway.oauth.state-cookie-ttl-seconds=180 -openframe.gateway.oauth.dev-ticket-enabled=true -openframe.auth.error-url=/auth/error -``` - ---- - -## Extension Points - -Security Core And Oauth Bff is designed to be extended: - -- Replace `RedirectTargetResolver` -- Customize cookie behavior via `CookieService` -- Extend `OAuthBffService` -- Plug in alternative key loading mechanisms - ---- - -## Summary - -The **Security Core And Oauth Bff** module provides: - -- JWT cryptographic infrastructure -- OAuth2 BFF endpoints -- PKCE and CSRF protection -- Token refresh and revocation logic -- Redirect safety and cookie management - -It is a foundational module that secures the entire OpenFrame request lifecycle, enabling multi-tenant, OAuth2-compliant, and browser-safe authentication flows. \ No newline at end of file diff --git a/docs/reference/architecture/security-oauth-and-jwt/security-oauth-and-jwt.md b/docs/reference/architecture/security-oauth-and-jwt/security-oauth-and-jwt.md new file mode 100644 index 000000000..289ed9c83 --- /dev/null +++ b/docs/reference/architecture/security-oauth-and-jwt/security-oauth-and-jwt.md @@ -0,0 +1,373 @@ +# Security Oauth And Jwt + +## Overview + +The **Security Oauth And Jwt** module provides the foundational security building blocks for the OpenFrame platform. It is responsible for: + +- JWT encoding and decoding using RSA keys +- OAuth2 login and token lifecycle management (Authorization Code + PKCE) +- Backend-for-Frontend (BFF) OAuth orchestration +- Secure cookie handling for access and refresh tokens +- Development ticket exchange for local or debug scenarios + +This module acts as the glue between: + +- The Authorization Server +- The Gateway Service +- API Services +- Frontend applications (via BFF endpoints) + +It centralizes token generation, validation, and OAuth flow coordination while remaining reusable across services. + +--- + +## Architectural Context + +At a high level, Security Oauth And Jwt sits between client-facing services and the Authorization Server. + +```mermaid +flowchart LR + Browser["Browser / Frontend"] -->|"/oauth/login"| BffController["OAuthBffController"] + BffController -->|"Redirect to IdP"| AuthServer["Authorization Server"] + AuthServer -->|"Authorization Code"| BffController + BffController -->|"Token Exchange"| AuthServer + BffController -->|"Set Auth Cookies"| Browser + + ApiService["API Services"] -->|"Validate JWT"| JwtDecoder["JwtDecoder"] + JwtEncoder["JwtEncoder"] -->|"Signs Tokens"| JwtDecoder +``` + +### Key Responsibilities + +1. **JWT Infrastructure** – RSA-based signing and verification +2. **OAuth BFF Flow** – Secure Authorization Code + PKCE orchestration +3. **Token Storage Strategy** – HttpOnly cookies and optional dev headers +4. **PKCE & State Security** – CSRF mitigation and proof-of-possession + +--- + +## Module Components + +The module is divided into two main areas: + +- Core JWT configuration (security-core) +- OAuth BFF orchestration (security-oauth) + +--- + +# JWT Infrastructure + +## JwtSecurityConfig + +**Component:** `JwtSecurityConfig` + +This configuration class wires the Spring Security JWT encoder and decoder. + +### Responsibilities + +- Creates a `JwtEncoder` using an RSA key pair +- Creates a `JwtDecoder` using the RSA public key +- Uses Nimbus JOSE implementation + +```mermaid +flowchart TD + JwtConfig["JwtConfig"] -->|"loadPublicKey()"| JwtSecurityConfig + JwtConfig -->|"loadPrivateKey()"| JwtSecurityConfig + + JwtSecurityConfig --> JwtEncoder["NimbusJwtEncoder"] + JwtSecurityConfig --> JwtDecoder["NimbusJwtDecoder"] +``` + +### Security Model + +- **Private key** β†’ used for signing tokens +- **Public key** β†’ used for verifying tokens +- Decoder does not require private key + +This separation enables safe public key distribution to downstream services. + +--- + +## JwtConfig + +**Component:** `JwtConfig` + +Configuration properties bound under the `jwt` prefix. + +### Responsibilities + +- Loads RSA public key +- Loads RSA private key (PKCS8) +- Exposes issuer and audience +- Converts PEM values to `RSAPublicKey` and `RSAPrivateKey` + +```mermaid +flowchart TD + AppConfig["application.yml jwt.*"] --> JwtConfig + JwtConfig -->|"Base64 Decode"| KeyFactory + KeyFactory --> RSAPublic["RSAPublicKey"] + KeyFactory --> RSAPrivate["RSAPrivateKey"] +``` + +### Security Considerations + +- Private key material is stripped of headers and whitespace +- Uses Java `KeyFactory` with RSA algorithm +- Designed for externalized secret management + +--- + +## SecurityConstants + +Defines shared OAuth and token header names: + +- `ACCESS_TOKEN` +- `REFRESH_TOKEN` +- `ACCESS_TOKEN_HEADER` +- `REFRESH_TOKEN_HEADER` +- `AUTHORIZATION_QUERY_PARAM` + +This avoids hard-coded strings across services. + +--- + +# PKCE Support + +## PKCEUtils + +Implements Proof Key for Code Exchange utilities. + +### Responsibilities + +- Generate cryptographically secure state parameter +- Generate code verifier (256-bit entropy) +- Generate SHA-256 code challenge +- Base64URL encoding (no padding) + +```mermaid +flowchart TD + GenerateVerifier["generateCodeVerifier()"] --> Hash["SHA-256"] + Hash --> Encode["Base64URL"] + Encode --> Challenge["code_challenge"] + + GenerateState["generateState()"] --> Encode +``` + +### Security Impact + +- Protects against authorization code interception +- Mitigates CSRF via state parameter +- Uses `SecureRandom` for entropy + +--- + +# OAuth BFF Layer + +The OAuth BFF layer implements a secure Backend-for-Frontend pattern. + +## OAuthBffController + +Primary entry point for OAuth operations. + +Base path: `/oauth` + +### Endpoints + +| Endpoint | Purpose | +|-----------|----------| +| `/login` | Initiates OAuth login with PKCE + state | +| `/continue` | Continues flow without clearing cookies | +| `/callback` | Handles authorization code exchange | +| `/refresh` | Refreshes access token | +| `/logout` | Revokes refresh token and clears cookies | +| `/dev-exchange` | Exchanges dev ticket for tokens | + +--- + +## OAuth Login Flow + +```mermaid +sequenceDiagram + participant Browser + participant BFF as OAuthBffController + participant Auth as AuthorizationServer + + Browser->>BFF: GET /oauth/login + BFF->>BFF: Generate state and PKCE + BFF->>Auth: Redirect to authorize endpoint + Auth->>Browser: Login page + Browser->>Auth: Submit credentials + Auth->>BFF: Redirect with code and state + BFF->>Auth: Exchange code for tokens + BFF->>Browser: Set auth cookies and redirect +``` + +### Security Behavior + +- Clears existing auth cookies before login +- Creates signed state JWT +- Stores state in secure cookie +- Validates state during callback +- Exchanges code for tokens +- Stores tokens in HttpOnly cookies + +--- + +## Token Refresh + +Flow: + +1. Extract refresh token from cookie or header +2. Lookup tenant if required +3. Call OAuth service refresh endpoint +4. Replace cookies + +If token missing or invalid β†’ returns 401. + +--- + +## Logout + +- Clears auth cookies +- Revokes refresh token +- Returns 204 No Content + +Prevents reuse of refresh tokens. + +--- + +## Development Ticket Flow + +### InMemoryOAuthDevTicketStore + +Used when dev ticket feature is enabled. + +```mermaid +flowchart TD + Tokens["TokenResponse"] --> CreateTicket + CreateTicket --> Store["ConcurrentHashMap"] + DevClient["Dev Client"] -->|"ticket"| Consume + Consume --> Store +``` + +### Purpose + +- Allows secure token handoff in development +- Avoids exposing cookies in cross-origin environments +- Stores tokens temporarily in memory + +This component is conditionally loaded if no other `OAuthDevTicketStore` bean is present. + +--- + +## Redirect Resolution + +### DefaultRedirectTargetResolver + +Determines final redirect target after login. + +Priority order: + +1. Explicit `redirectTo` parameter +2. HTTP `Referer` header +3. Fallback to `/` + +Designed to prevent redirect confusion and ensure safe defaults. + +--- + +## Forwarded Header Handling + +### NoopForwardedHeadersContributor + +Provides a no-op implementation when no custom forwarded header contributor exists. + +This prevents missing bean errors in reactive environments and allows downstream customization. + +--- + +# Token Lifecycle + +```mermaid +flowchart TD + Login["Login"] --> AccessToken["Access Token"] + Login --> RefreshToken["Refresh Token"] + + AccessToken --> ApiCall["API Call"] + RefreshToken --> Refresh["/oauth/refresh"] + + Refresh --> AccessToken + Logout["Logout"] --> Revoke["Revoke Refresh Token"] +``` + +--- + +# Integration Points + +Security Oauth And Jwt integrates with: + +- Authorization Service (token issuing authority) +- Gateway Service (JWT validation at edge) +- API Services (resource server validation) +- CookieService (secure token storage) + +It does not implement user storage or credential validation directly β€” those responsibilities belong to the Authorization Server module. + +--- + +# Security Design Principles + +## 1. Asymmetric Cryptography + +- RSA key pair +- Public key distribution allowed +- Private key never exposed + +## 2. PKCE Everywhere + +- Code verifier (256-bit entropy) +- SHA-256 challenge +- Base64URL without padding + +## 3. HttpOnly Cookie Strategy + +- Tokens not exposed to JavaScript +- Refresh tokens stored securely +- Cookies cleared on logout + +## 4. Reactive and Stateless + +- WebFlux compatible +- Stateless JWT validation +- No session storage required + +--- + +# Configuration Summary + +Typical configuration properties: + +```text +jwt.public-key.value +jwt.private-key.value +jwt.issuer +jwt.audience +openframe.gateway.oauth.enable +openframe.gateway.oauth.state-cookie-ttl-seconds +openframe.gateway.oauth.dev-ticket-enabled +openframe.auth.error-url +``` + +--- + +# Summary + +The **Security Oauth And Jwt** module provides: + +- RSA-based JWT encoding and decoding +- PKCE-secured OAuth2 flows +- Backend-for-Frontend orchestration +- Token lifecycle management +- Development token exchange support + +It forms the core security backbone of the OpenFrame platform, ensuring secure, scalable, and standards-compliant authentication across services. \ No newline at end of file diff --git a/docs/reference/architecture/stream-processing-core/stream-processing-core.md b/docs/reference/architecture/stream-processing-core/stream-processing-core.md new file mode 100644 index 000000000..6a1b448e0 --- /dev/null +++ b/docs/reference/architecture/stream-processing-core/stream-processing-core.md @@ -0,0 +1,364 @@ +# Stream Processing Core + +The **Stream Processing Core** module is the real-time event ingestion and transformation engine of the OpenFrame platform. It consumes Change Data Capture (CDC) events and tool-generated messages from Kafka, normalizes them into a unified event model, enriches them with tenant and device context, and routes them to downstream systems such as Cassandra and Kafka topics. + +This module acts as the bridge between: + +- Integrated tools (MeshCentral, Tactical RMM, Fleet MDM) +- Kafka (CDC + tool event streams) +- Data enrichment services (Redis, tenant resolution) +- Downstream persistence layers (e.g., Cassandra) +- Unified event taxonomy used across the platform + +--- + +## High-Level Responsibilities + +1. **Kafka & Kafka Streams configuration** +2. **Tool-specific event deserialization** +3. **Event normalization into unified types** +4. **Tenant and device enrichment** +5. **Debezium CDC operation handling (C/R/U/D)** +6. **Downstream dispatch to storage or Kafka topics** +7. **Fleet MDM activity stream joins using Kafka Streams** + +--- + +# Architecture Overview + +```mermaid +flowchart TD + KafkaTopics["Kafka Topics\n(Integrated Tool Events)"] --> JsonListener["JsonKafkaListener"] + JsonListener --> Processor["GenericJsonMessageProcessor"] + Processor --> Deserializers["Tool Event Deserializers"] + Deserializers --> Enrichment["IntegratedToolDataEnrichmentService"] + Enrichment --> Mapping["EventTypeMapper"] + Mapping --> Handlers["DebeziumMessageHandler"] + Handlers --> CassandraHandler["DebeziumCassandraMessageHandler"] + Handlers --> TenantKafkaHandler["TenantDebeziumKafkaMessageHandler"] + + subgraph streams_layer["Kafka Streams Enrichment"] + Activities["Fleet Activities Topic"] --> Joiner["ActivityEnrichmentService"] + HostActivities["Fleet Host Activities Topic"] --> Joiner + Joiner --> EnrichedTopic["Enriched Fleet Events Topic"] + end +``` + +The module consists of two main pipelines: + +- **Listener-based CDC processing** (via `@KafkaListener`) +- **Kafka Streams topology** for Fleet MDM activity enrichment + +--- + +# Core Processing Flow + +## 1. Kafka Consumption + +`JsonKafkaListener` listens to multiple inbound topics: + +- MeshCentral events +- Tactical RMM events +- Tactical task result events +- Fleet MDM activity events +- Fleet policy membership events +- Fleet query result events + +It extracts the `MessageType` header and forwards the payload to a generic processor. + +```mermaid +sequenceDiagram + participant Kafka + participant Listener as JsonKafkaListener + participant Processor as GenericJsonMessageProcessor + participant Deserializer + participant Enrichment + participant Handler + + Kafka->>Listener: CommonDebeziumMessage + MessageType + Listener->>Processor: process(message, type) + Processor->>Deserializer: Deserialize by type + Deserializer->>Enrichment: Enrich data + Enrichment->>Handler: Handle operation +``` + +--- + +# Configuration Layer + +## KafkaConfig + +Provides: + +- `Converter` +- Converts Kafka header bytes into `MessageType` enum + +This enables header-driven routing of events. + +## KafkaStreamsConfig + +Enables Kafka Streams when `kafka.stream.enabled=true`. + +Key features: + +- Dynamic `application.id` (includes cluster ID if present) +- JSON Serdes for: + - `ActivityMessage` + - `HostActivityMessage` +- At-least-once processing +- Stream idle configuration to allow window closing +- Controlled batching and producer tuning + +--- + +# Deserialization Layer + +All tool-specific deserializers extend a shared base: + +- `IntegratedToolEventDeserializer` + +Each implementation extracts: + +- Agent ID +- Tool event ID +- Source event type +- Timestamp +- Message +- Details / Result / Error + +## Implemented Deserializers + +### Fleet MDM + +- `FleetEventDeserializer` +- `FleetPolicyActivityDeserializer` +- `FleetPolicyMembershipEventDeserializer` +- `FleetQueryResultEventDeserializer` + +Features: + +- Policy and query cache lookups +- Conditional cache eviction on policy mutations +- JSON-safe result and error construction +- Activity type to human-readable message mapping + +### MeshCentral + +- `MeshCentralEventDeserializer` + +Features: + +- Handles nested JSON strings +- Builds `etype.action` composite event types +- Extracts tenant from `domain` field + +### Tactical RMM + +- `TrmmAgentHistoryEventDeserializer` +- `TrmmAuditEventDeserializer` +- `TrmmTaskResultEventDeserializer` + +Features: + +- Agent primary key resolution via cache +- Script/task metadata lookup +- Result and error normalization + +--- + +# Unified Event Mapping + +## EventTypeMapper + +Maps: + +- `(IntegratedToolType, sourceEventType)` + +To: + +- `UnifiedEventType` + +If no mapping exists β†’ defaults to `UNKNOWN`. + +```mermaid +flowchart LR + SourceEvent["toolType + sourceEventType"] --> Mapper["EventTypeMapper"] + Mapper --> Unified["UnifiedEventType"] +``` + +This ensures all integrated tool events conform to a consistent taxonomy used by: + +- Cassandra storage +- Analytics systems +- Notification services + +--- + +# Data Enrichment + +## IntegratedToolDataEnrichmentService + +Adds contextual data to deserialized events: + +### Machine Enrichment + +Uses `MachineIdCacheService`: + +- Machine ID +- Hostname +- Organization ID +- Organization Name + +### Tenant Resolution + +Two modes: + +1. **Tenant cluster mode** β†’ Uses `TenantIdProvider` +2. **Shared cluster mode** β†’ Uses `ClusterTenantIdResolver` + +This guarantees that every event has a resolved `tenantId`. + +--- + +# Debezium Message Handling + +## GenericMessageHandler + +Abstract base class implementing: + +- Transform β†’ Push flow +- Operation routing (CREATE / READ / UPDATE / DELETE) + +```mermaid +flowchart TD + Message --> Transform["transform()"] + Transform --> Operation["getOperationType()"] + Operation --> Dispatch["handleCreate / handleUpdate / handleDelete"] +``` + +## DebeziumMessageHandler + +Adds: + +- CDC operation type resolution (`c`, `r`, `u`, `d`) + +## DebeziumCassandraMessageHandler + +Transforms enriched events into: + +- `UnifiedLogEvent` + +Writes to: + +- Cassandra via `UnifiedLogEventRepository` + +The event key includes: + +- tenantId +- ingestDay +- toolType +- unifiedEventType +- eventTimestamp +- toolEventId + +## TenantDebeziumKafkaMessageHandler + +Publishes enriched events to: + +- Tenant-scoped outbound Kafka topic + +## TenantIdRequiredDebeziumEventValidator + +Drops events if: + +- `tenantId` is missing or blank + +This enforces strict tenant isolation. + +--- + +# Kafka Streams: Fleet Activity Join + +## ActivityEnrichmentService + +Purpose: + +Join Fleet `activities` and `host_activities` topics to attach `hostId` to activities. + +### Join Strategy + +- Left join +- 5-second window +- Activity ID used as join key + +```mermaid +flowchart TD + ActivitiesTopic["fleet activities"] --> ReKey["Re-key by activityId"] + HostActivitiesTopic["fleet host activities"] --> ReKeyHost["Re-key by activityId"] + ReKey --> Join["Left Join (5s window)"] + ReKeyHost --> Join + Join --> HeaderAdd["Add MessageType Header"] + HeaderAdd --> OutputTopic["enriched fleet events"] +``` + +Additional behavior: + +- Dynamically resolves message type: + - `FLEET_MDM_EVENT` + - `FLEET_MDM_POLICY_ACTIVITY_EVENT` +- Adds Kafka headers before publishing + +--- + +# Timestamp Handling + +## TimestampParser + +Utility for parsing ISO-8601 timestamps produced by Debezium. + +- Converts to epoch milliseconds +- Logs warnings for invalid formats + +Ensures consistent event-time semantics across tools. + +--- + +# Operational Modes + +The module adapts to deployment mode: + +| Mode | Behavior | +|------|----------| +| Tenant cluster | Direct tenant resolution via provider | +| Shared SaaS cluster | Tenant resolved from tool-specific domain | +| Cassandra enabled | Writes to Cassandra | +| Kafka Streams enabled | Activity enrichment topology activated | + +--- + +# Integration Points with Other Modules + +The Stream Processing Core interacts with: + +- **Eventing and Messaging (Kafka/NATS)** for topic configuration and producers +- **Data Model and Repositories (Mongo/Cassandra)** for unified event storage +- **Security (OAuth/JWT)** for tenant context propagation +- **Analytics (Pinot)** which consumes unified events + +It acts as the real-time ingestion backbone of the OpenFrame ecosystem. + +--- + +# Summary + +The **Stream Processing Core** module provides: + +- Multi-tool event ingestion +- Unified taxonomy mapping +- Strict tenant isolation +- Real-time enrichment via Redis and caches +- Kafka Streams-based correlation +- Cassandra persistence +- Forwarding to outbound Kafka topics + +It transforms heterogeneous integrated-tool events into a normalized, enriched, multi-tenant event stream that powers analytics, logging, automation, and auditing across the platform. diff --git a/docs/reference/architecture/stream-service-core-kafka-and-handlers/stream-service-core-kafka-and-handlers.md b/docs/reference/architecture/stream-service-core-kafka-and-handlers/stream-service-core-kafka-and-handlers.md deleted file mode 100644 index cc6d0eb16..000000000 --- a/docs/reference/architecture/stream-service-core-kafka-and-handlers/stream-service-core-kafka-and-handlers.md +++ /dev/null @@ -1,356 +0,0 @@ -# Stream Service Core Kafka And Handlers - -## Overview - -The **Stream Service Core Kafka And Handlers** module is the event ingestion, normalization, enrichment, and distribution backbone of the OpenFrame platform. - -It is responsible for: - -- Consuming Debezium CDC events from integrated tools (MeshCentral, Tactical RMM, Fleet MDM) -- Deserializing tool-specific payloads into unified internal models -- Enriching events with tenant, organization, and device metadata -- Mapping tool-specific event types to unified event types -- Persisting events (e.g., Cassandra) -- Republishing normalized events to outbound Kafka topics -- Running Kafka Streams pipelines for activity enrichment - -This module acts as the **bridge between external tool ecosystems and the unified OpenFrame event model** used across API, analytics, notifications, and management services. - ---- - -## High-Level Architecture - -```mermaid -flowchart LR - subgraph Tools["Integrated Tools"] - MeshCentral["MeshCentral"] - Tactical["Tactical RMM"] - Fleet["Fleet MDM"] - end - - subgraph KafkaIn["Inbound Kafka Topics"] - InTopics["Debezium Topics"] - end - - subgraph StreamCore["Stream Service Core Kafka And Handlers"] - Listener["JsonKafkaListener"] - Processor["GenericJsonMessageProcessor"] - Deserializer["Tool Event Deserializers"] - Enrichment["IntegratedToolDataEnrichmentService"] - Mapper["EventTypeMapper"] - Handler["DebeziumMessageHandler"] - CassandraHandler["DebeziumCassandraMessageHandler"] - TenantKafkaHandler["TenantDebeziumKafkaMessageHandler"] - end - - subgraph Storage["Storage & Downstream"] - Cassandra["Cassandra UnifiedLogEvent"] - KafkaOut["Outbound Kafka Topics"] - end - - MeshCentral --> InTopics - Tactical --> InTopics - Fleet --> InTopics - - InTopics --> Listener - Listener --> Processor - Processor --> Deserializer - Deserializer --> Enrichment - Enrichment --> Mapper - Mapper --> Handler - - Handler --> CassandraHandler - Handler --> TenantKafkaHandler - - CassandraHandler --> Cassandra - TenantKafkaHandler --> KafkaOut -``` - ---- - -## Processing Flow - -The module follows a structured event pipeline: - -```mermaid -flowchart TD - A["Kafka Debezium Event"] --> B["JsonKafkaListener"] - B --> C["GenericJsonMessageProcessor"] - C --> D["Tool-Specific Deserializer"] - D --> E["Unified DeserializedDebeziumMessage"] - E --> F["IntegratedToolDataEnrichmentService"] - F --> G["EventTypeMapper"] - G --> H["DebeziumMessageHandler"] - H --> I["Destination Handler"] - I --> J["Cassandra or Kafka"] -``` - -### Key Stages - -1. **Kafka Consumption** – `JsonKafkaListener` consumes multi-topic integrated tool events. -2. **Deserialization** – Tool-specific deserializers convert raw CDC JSON into structured internal messages. -3. **Enrichment** – Tenant, machine, and organization metadata are resolved via Redis cache services. -4. **Type Mapping** – `EventTypeMapper` maps tool-specific source types into `UnifiedEventType`. -5. **Handling & Dispatch** – Generic and specialized handlers route events to storage or outbound topics. - ---- - -## Kafka Configuration Layer - -### KafkaConfig - -Provides: - -- `Converter` for resolving message type headers. -- Integration with Spring Kafka infrastructure. - -This enables dynamic routing of heterogeneous tool events through a unified processing layer. - -### KafkaStreamsConfig - -Enables Kafka Streams processing when `kafka.stream.enabled=true`. - -Key responsibilities: - -- Configures `application.id` (namespaced by cluster ID) -- Defines custom `Serde` for: - - `ActivityMessage` - - `HostActivityMessage` -- Configures state store, threading, and processing guarantees - -Processing guarantee: `AT_LEAST_ONCE` - ---- - -## Kafka Streams: Activity Enrichment - -`ActivityEnrichmentService` builds a Kafka Streams topology that: - -- Consumes: - - Fleet activity topic - - Fleet host activity topic -- Performs a time-windowed left join -- Enriches activity events with host information -- Adds required Kafka headers -- Publishes enriched events back to an inbound topic - -```mermaid -flowchart LR - ActivityTopic["Fleet Activities"] --> Join - HostActivityTopic["Fleet Host Activities"] --> Join - Join["Left Join (5s Window)"] --> Enriched["Enriched ActivityMessage"] - Enriched --> HeaderAdder["Add MESSAGE_TYPE_HEADER"] - HeaderAdder --> OutputTopic["Fleet MDM Events Topic"] -``` - -This allows Fleet policy and activity events to be normalized before entering the main Debezium event processing path. - ---- - -## Tool-Specific Deserializers - -Each integrated tool has a dedicated deserializer extending `IntegratedToolEventDeserializer`. - -### Fleet MDM - -- `FleetEventDeserializer` -- `FleetPolicyActivityDeserializer` -- `FleetPolicyMembershipEventDeserializer` -- `FleetQueryResultEventDeserializer` - -Responsibilities: - -- Extract agent ID -- Resolve policy/query metadata via cache services -- Build structured `result` or `error` JSON -- Normalize timestamps using `TimestampParser` -- Assign correct `MessageType` - -### Tactical RMM - -- `TrmmAgentHistoryEventDeserializer` -- `TrmmAuditEventDeserializer` -- `TrmmTaskResultEventDeserializer` - -Responsibilities: - -- Resolve primary key IDs to logical agent IDs via cache -- Parse script results and command outputs -- Convert execution results into structured JSON payloads - -### MeshCentral - -- `MeshCentralEventDeserializer` - -Responsibilities: - -- Parse nested JSON payloads -- Extract `etype.action` composite source type -- Extract tenant domain for shared cluster resolution - ---- - -## Unified Event Mapping - -`EventTypeMapper` maps: - -- `IntegratedToolType` -- Source event type string - -To: - -- `UnifiedEventType` - -```mermaid -flowchart LR - ToolType["IntegratedToolType"] --> Key - SourceType["Source Event Type"] --> Key - Key["tool:sourceType"] --> Mapper["EventTypeMapper"] - Mapper --> Unified["UnifiedEventType"] -``` - -If no mapping exists, the system falls back to `UNKNOWN`. - -This ensures: - -- Cross-tool normalization -- Consistent severity assignment -- Unified downstream analytics - ---- - -## Data Enrichment Layer - -`IntegratedToolDataEnrichmentService` enriches deserialized events with: - -- Machine ID -- Hostname -- Organization ID and name -- Tenant ID - -It integrates with: - -- Redis cache via `MachineIdCacheService` -- Tenant resolution via `TenantIdProvider` -- Optional `ClusterTenantIdResolver` (shared cluster mode) - -```mermaid -flowchart TD - Event["DeserializedDebeziumMessage"] --> MachineLookup["MachineIdCacheService"] - MachineLookup --> OrgLookup["Organization Cache"] - OrgLookup --> TenantResolution["TenantIdProvider or ClusterTenantIdResolver"] - TenantResolution --> Enriched["IntegratedToolEnrichedData"] -``` - -This ensures that all downstream consumers receive fully contextualized events. - ---- - -## Message Handling Framework - -### GenericMessageHandler - -Provides a template pattern: - -- Validate message -- Transform to destination model -- Resolve operation type (CREATE, UPDATE, DELETE, READ) -- Route to appropriate handler method - -### DebeziumMessageHandler - -Specialization for CDC events: - -- Extracts operation code from Debezium payload (`c`, `u`, `d`, `r`) -- Maps to `OperationType` - -### DebeziumCassandraMessageHandler - -Transforms events into `UnifiedLogEvent` and writes to Cassandra. - -Responsibilities: - -- Construct partition key -- Populate severity and unified type -- Persist to `UnifiedLogEventRepository` - -### TenantDebeziumKafkaMessageHandler - -Publishes validated events to tenant-scoped outbound Kafka topics using `OssTenantRetryingKafkaProducer`. - ---- - -## Validation Layer - -`TenantIdRequiredDebeziumEventValidator` ensures: - -- All events have a resolved tenant ID -- Events without tenant context are dropped - -This prevents cross-tenant contamination and guarantees multi-tenant isolation. - ---- - -## Timestamp Normalization - -`TimestampParser` standardizes ISO-8601 timestamps into epoch milliseconds. - -All integrated tools rely on Debezium ISO format, ensuring: - -- Consistent event ordering -- Reliable partitioning -- Accurate time-based queries - ---- - -## Multi-Tenant and Cluster Modes - -The module supports two deployment modes: - -1. **Tenant Cluster Mode** - - One tenant per Kafka cluster - - `TenantIdProvider` resolves tenant ID directly - -2. **Shared Cluster Mode** - - Multiple tenants share Kafka - - `ClusterTenantIdResolver` maps tool-specific domain identifiers to canonical tenant IDs - -This design ensures compatibility across SaaS and dedicated deployments. - ---- - -## Integration with Other Modules - -This module integrates tightly with: - -- Data Mongo domain model (for unified event storage references) -- Data Cassandra repositories (for log persistence) -- Data Kafka configuration and retry (for producer reliability) -- Data Redis cache (for machine and organization lookups) -- Management and API services (downstream consumers of normalized events) - -It serves as the **event normalization boundary** between external tools and the internal OpenFrame domain model. - ---- - -## Summary - -The **Stream Service Core Kafka And Handlers** module provides: - -- Reliable Kafka ingestion -- Tool-specific deserialization -- Unified event type normalization -- Tenant-aware enrichment -- Cassandra persistence -- Kafka republishing -- Kafka Streams activity joins - -It is a critical infrastructure layer enabling: - -- Cross-tool observability -- Unified audit trails -- Analytics and reporting -- Real-time notification pipelines -- Multi-tenant isolation guarantees - -Without this module, the OpenFrame platform would lack a consistent, normalized, and tenant-safe event backbone. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CancelDispatchResponse.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CancelDispatchResponse.md new file mode 100644 index 000000000..bf7505688 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CancelDispatchResponse.md @@ -0,0 +1,25 @@ + +A simple Data Transfer Object (DTO) representing the response payload returned after a dispatch cancellation request. + +## Key Components + +| Element | Type | Description | +|---------|------|-------------| +| `executionId` | `String` | Unique identifier of the cancelled dispatch execution | + +## Usage Example + +```java +// Build a cancellation response with the execution ID +CancelDispatchResponse response = CancelDispatchResponse.builder() + .executionId("exec-abc-12345") + .build(); + +// Access the execution ID +String id = response.getExecutionId(); +``` + +## Notes + +- Annotated with Lombok `@Data` β€” auto-generates getters, setters, `equals`, `hashCode`, and `toString`. +- Annotated with Lombok `@Builder` β€” enables the fluent builder pattern for construction. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CancelExecutionInput.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CancelExecutionInput.md new file mode 100644 index 000000000..769c28e18 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CancelExecutionInput.md @@ -0,0 +1,22 @@ + +Data Transfer Object (DTO) for capturing the required inputs to cancel an ongoing command execution. + +## Key Components + +| Field | Type | Constraint | Description | +|-------|------|------------|-------------| +| `machineId` | `String` | `@NotBlank` | Identifies the target machine running the execution | +| `executionId` | `String` | `@NotBlank` | Identifies the specific execution to cancel | + +## Usage Example + +```java +CancelExecutionInput input = new CancelExecutionInput(); +input.setMachineId("machine-abc123"); +input.setExecutionId("exec-xyz789"); + +// Pass to command service +commandService.cancelExecution(input); +``` + +> Both fields are mandatory β€” validation will reject any request where either `machineId` or `executionId` is null or blank. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CommandDispatchResponse.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CommandDispatchResponse.md new file mode 100644 index 000000000..bb77e362c --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CommandDispatchResponse.md @@ -0,0 +1,20 @@ + +A lightweight DTO representing the server's response after dispatching a command for execution. + +## Key Components + +- **`executionId`** (`String`) β€” Unique identifier assigned to the dispatched command execution, used to track or query its status. +- **`@Data`** β€” Lombok annotation generating getters, setters, `equals`, `hashCode`, and `toString`. +- **`@Builder`** β€” Lombok annotation enabling the builder pattern for instance construction. + +## Usage Example + +```java +// Building a response after dispatching a command +CommandDispatchResponse response = CommandDispatchResponse.builder() + .executionId("exec-7f3a1b92-4c2d-11ef-a1b2-0242ac120002") + .build(); + +// Accessing the execution ID +String id = response.getExecutionId(); +``` \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.RunCommandInput.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.RunCommandInput.md new file mode 100644 index 000000000..272b5d177 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.RunCommandInput.md @@ -0,0 +1,28 @@ + +DTO representing the GraphQL input payload for dispatching an ad-hoc shell command to a remote agent via the OpenFrame RMM API. + +## Key Components + +| Field | Type | Constraint | Description | +|-------|------|-----------|-------------| +| `machineId` | `String` | `@NotBlank` | Target machine identifier | +| `command` | `String` | `@NotBlank` | Shell command to execute | +| `shell` | `ScriptShell` | `@NotNull` | Shell type (e.g. `PowerShell`, `Bash`) | +| `privilegeLevel` | `PrivilegeLevel` | `@NotNull` | Execution privilege (e.g. `ADMIN`, `USER`) | +| `timeoutSeconds` | `Integer` | `@Positive` | Optional execution timeout in seconds | + +## Usage Example + +```java +RunCommandInput input = new RunCommandInput(); +input.setMachineId("machine-abc123"); +input.setCommand("Get-Process | Select-Object Name, CPU"); +input.setShell(ScriptShell.POWERSHELL); +input.setPrivilegeLevel(PrivilegeLevel.ADMIN); +input.setTimeoutSeconds(30); + +// Pass to GraphQL mutation resolver +commandService.runCommand(input); +``` + +> **Note:** `timeoutSeconds` is optional β€” if omitted or `null`, the agent applies its default execution timeout. All other fields are required and validated at the GraphQL layer before the command is dispatched. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.CreateScriptInput.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.CreateScriptInput.md new file mode 100644 index 000000000..3d556ff14 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.CreateScriptInput.md @@ -0,0 +1,37 @@ + +Data Transfer Object (DTO) representing the request payload for creating a new script in the OpenFrame RMM module. Tenant attribution is resolved server-side from the authenticated principal and must not be supplied by the client. + +## Key Components + +| Field | Type | Constraints | Description | +|-------|------|-------------|-------------| +| `name` | `String` | `@NotBlank` | Display name of the script | +| `description` | `String` | β€” | Optional human-readable description | +| `shell` | `ScriptShell` | `@NotNull` | Target shell/interpreter (e.g. PowerShell, Bash) | +| `scriptBody` | `String` | `@NotBlank` | Raw script content | +| `tag` | `String` | β€” | Optional categorization tag | +| `supportedPlatforms` | `List` | β€” | Target OS platforms | +| `defaultTimeoutSeconds` | `Integer` | `@Positive` | Execution timeout; must be > 0 if provided | +| `defaultArgs` | `List` | β€” | Default arguments passed at runtime | +| `envVars` | `List` | `@Valid` | Environment variables with cascaded validation | + +## Usage Example + +```java +CreateScriptInput input = new CreateScriptInput(); +input.setName("Disk Cleanup"); +input.setDescription("Clears temp files on Windows endpoints"); +input.setShell(ScriptShell.POWERSHELL); +input.setScriptBody("Remove-Item -Path $env:TEMP\\* -Recurse -Force"); +input.setTag("maintenance"); +input.setSupportedPlatforms(List.of(ScriptPlatform.WINDOWS)); +input.setDefaultTimeoutSeconds(120); +input.setDefaultArgs(List.of("--silent")); + +ScriptEnvVarInput envVar = new ScriptEnvVarInput(); +envVar.setKey("LOG_LEVEL"); +envVar.setValue("INFO"); +input.setEnvVars(List.of(envVar)); +``` + +> **Security note:** Never include a tenant ID in this payload. Tenant attribution is automatically derived from the authenticated principal at the resolver/controller layer. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptEnvVarInput.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptEnvVarInput.md new file mode 100644 index 000000000..164faf9ab --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptEnvVarInput.md @@ -0,0 +1,30 @@ + +Symmetric DTO representing a script environment variable, used for both create/update input and API response output with an identical field shape in both directions. + +## Key Components + +| Field | Type | Description | +|-------|------|-------------| +| `name` | `String` | Environment variable name β€” required, must not be blank (`@NotBlank`) | +| `value` | `String` | Environment variable value β€” optional | +| `secret` | `boolean` | Marks `value` as sensitive (passwords, API keys, tokens); values are currently stored in plaintext pending full secret-management implementation | + +## Usage Example + +```java +// Build a plain environment variable +ScriptEnvVarInput javaHome = ScriptEnvVarInput.builder() + .name("JAVA_HOME") + .value("/usr/lib/jvm/java-21") + .secret(false) + .build(); + +// Build a sensitive environment variable (stored plaintext until encryption lands) +ScriptEnvVarInput apiKey = ScriptEnvVarInput.builder() + .name("API_KEY") + .value("sk-abc123") + .secret(true) + .build(); +``` + +> **⚠️ Known Limitation:** Secret values are currently stored in plaintext. UI layers, logs, and audit trails are responsible for masking and redacting `secret = true` fields until encryption-at-rest and secure agent delivery are implemented. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptFilterInput.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptFilterInput.md new file mode 100644 index 000000000..3e8f611c3 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptFilterInput.md @@ -0,0 +1,33 @@ + +Data Transfer Object (DTO) for filtering RMM scripts by shell type, status, platform, and tag criteria. + +## Key Components + +| Field | Type | Description | +|-------|------|-------------| +| `shells` | `List` | Filter by shell types (e.g., PowerShell, Bash) | +| `statuses` | `List` | Filter by script status; excludes `DELETED` by default when null/empty | +| `supportedPlatforms` | `List` | Matches scripts supporting **any** of the specified platforms | +| `tag` | `String` | Case-insensitive exact tag match | + +Lombok annotations (`@Data`, `@Builder`, `@NoArgsConstructor`, `@AllArgsConstructor`) provide getters/setters, builder pattern, and both no-arg and all-arg constructors. + +## Usage Example + +```java +// Builder pattern β€” filter for active PowerShell scripts on Windows +ScriptFilterInput filter = ScriptFilterInput.builder() + .shells(List.of(ScriptShell.POWERSHELL)) + .statuses(List.of(ScriptStatus.ACTIVE)) + .supportedPlatforms(List.of(ScriptPlatform.WINDOWS)) + .tag("patch-management") + .build(); + +// No-arg constructor + setters β€” filter active/draft Bash scripts on Linux/macOS +ScriptFilterInput filter = new ScriptFilterInput(); +filter.setShells(List.of(ScriptShell.BASH)); +filter.setStatuses(List.of(ScriptStatus.ACTIVE, ScriptStatus.DRAFT)); +filter.setSupportedPlatforms(List.of(ScriptPlatform.LINUX, ScriptPlatform.MACOS)); +``` + +> **Note:** Leaving `statuses` null or empty applies a default filter that automatically excludes `DELETED` scripts β€” pass an explicit status list to override this behaviour. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptResponse.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptResponse.md new file mode 100644 index 000000000..03a4d6263 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptResponse.md @@ -0,0 +1,40 @@ + +Data Transfer Object (DTO) representing a script resource returned by the OpenFrame API, intentionally omitting the internal `tenantId` since tenant context is resolved from authentication. + +## Key Components + +| Field | Type | Description | +|-------|------|-------------| +| `id` | `String` | Unique script identifier | +| `name` | `String` | Human-readable script name | +| `shell` | `String` | Shell type enum name (e.g. `POWERSHELL`) | +| `scriptBody` | `String` | Raw script content | +| `supportedPlatforms` | `List` | Platform enum names (e.g. `WINDOWS`) | +| `defaultTimeoutSeconds` | `Integer` | Execution timeout | +| `defaultArgs` | `List` | Default arguments passed at runtime | +| `envVars` | `List` | Environment variable definitions | +| `status` | `String` | Lifecycle status enum name (e.g. `ACTIVE`) | +| `statusChangedAt` / `createdAt` / `updatedAt` | `Instant` | Audit timestamps | + +Uses Lombok `@Data` and `@Builder` β€” no boilerplate constructors, getters, or setters required. + +## Usage Example + +```java +ScriptResponse response = ScriptResponse.builder() + .id("scr_abc123") + .name("Disk Cleanup") + .shell("POWERSHELL") + .scriptBody("Get-PSDrive -PSProvider FileSystem") + .supportedPlatforms(List.of("WINDOWS")) + .defaultTimeoutSeconds(60) + .defaultArgs(List.of("-Verbose")) + .status("ACTIVE") + .createdAt(Instant.now()) + .build(); +``` + +## Notes + +- `tenantId` is **deliberately excluded** β€” it is resolved server-side from the authenticated request context and never exposed on the wire. +- All enum-typed fields (`shell`, `status`, platform entries) are serialized as their **enum name strings**, keeping the API loosely coupled to internal enum definitions. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.UpdateScriptInput.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.UpdateScriptInput.md new file mode 100644 index 000000000..e2f66b2e1 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.UpdateScriptInput.md @@ -0,0 +1,36 @@ + +Data Transfer Object (DTO) representing the full-replacement payload for updating an existing script resource via PUT semantics. All writable fields overwrite stored values, including `null`s which clear optional fields. + +## Key Components + +| Field | Type | Constraint | Description | +|-------|------|------------|-------------| +| `name` | `String` | `@NotBlank` | Script display name (required) | +| `shell` | `ScriptShell` | `@NotNull` | Target shell/interpreter (required) | +| `scriptBody` | `String` | `@NotBlank` | Script source content (required) | +| `description` | `String` | β€” | Optional human-readable description | +| `tag` | `String` | β€” | Optional categorization tag | +| `supportedPlatforms` | `List` | β€” | OS/platform compatibility list | +| `defaultTimeoutSeconds` | `Integer` | `@Positive` | Execution timeout in seconds | +| `defaultArgs` | `List` | β€” | Default argument list | +| `envVars` | `List` | `@Valid` | Environment variable definitions | + +## Usage Example + +```java +// Build a full replacement payload for PUT /scripts/{id} +UpdateScriptInput input = new UpdateScriptInput(); +input.setName("Disk Cleanup"); +input.setShell(ScriptShell.POWERSHELL); +input.setScriptBody("Get-PSDrive | Where-Object { $_.Used -gt 1GB }"); +input.setDescription("Reports drives exceeding 1 GB used"); +input.setTag("maintenance"); +input.setSupportedPlatforms(List.of(ScriptPlatform.WINDOWS)); +input.setDefaultTimeoutSeconds(60); +input.setDefaultArgs(List.of("-Verbose")); + +// Optional: clear environment variables by omitting or passing null +input.setEnvVars(null); // clears any previously stored env vars +``` + +> **PUT semantics:** This DTO enforces full-replace behavior. Omitting optional fields (passing `null`) explicitly clears them on the stored document. For partial updates, use a PATCH endpoint with a separate input DTO. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/mapper/.ScriptMapper.md b/openframe-api-lib/src/main/java/com/openframe/api/mapper/.ScriptMapper.md new file mode 100644 index 000000000..1e79bfa4d --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/mapper/.ScriptMapper.md @@ -0,0 +1,43 @@ + +Handles bidirectional mapping between `Script` domain entities and their corresponding DTOs (`CreateScriptInput`, `UpdateScriptInput`, `ScriptResponse`), including nested environment variable and platform mappings. + +## Key Components + +| Method | Description | +|--------|-------------| +| `toEntity(tenantId, input)` | Converts a `CreateScriptInput` DTO into a new `Script` entity scoped to the given tenant | +| `updateEntity(existing, input)` | Mutates an existing `Script` entity in-place from an `UpdateScriptInput` DTO | +| `toResponse(entity)` | Converts a `Script` entity to a `ScriptResponse` DTO, serializing enums (`shell`, `status`, `platforms`) to their string names | +| `mapEnvVarsToEntity(...)` | Private helper β€” maps `ScriptEnvVarInput` list to `ScriptEnvVar` entity list | +| `mapEnvVarsToResponse(...)` | Private helper β€” maps `ScriptEnvVar` entity list back to `ScriptEnvVarInput` list (secret masking pending) | +| `mapPlatformsToResponse(...)` | Private helper β€” maps `ScriptPlatform` enum list to string names | + +> **Note:** GraphQL-specific concerns (cursor pagination, Relay Connection/Edge envelope) are intentionally kept in `GraphQLScriptMapper` alongside the DGS resolver. This mapper is transport-agnostic and reusable across GraphQL, REST, and messaging layers. + +## Usage Example + +```java +@Service +public class ScriptService { + + private final ScriptMapper scriptMapper; + private final ScriptRepository scriptRepository; + + // Create a new script entity from API input + public ScriptResponse createScript(String tenantId, CreateScriptInput input) { + Script entity = scriptMapper.toEntity(tenantId, input); + Script saved = scriptRepository.save(entity); + return scriptMapper.toResponse(saved); + } + + // Apply partial updates to an existing script + public ScriptResponse updateScript(String scriptId, UpdateScriptInput input) { + Script existing = scriptRepository.findById(scriptId).orElseThrow(); + scriptMapper.updateEntity(existing, input); + Script saved = scriptRepository.save(existing); + return scriptMapper.toResponse(saved); + } +} +``` + +> ⚠️ Secret env var values are currently returned as-is. Masking for `secret == true` fields is pending secret-management integration. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/service/.KnowledgeBaseTempAttachmentService.md b/openframe-api-lib/src/main/java/com/openframe/api/service/.KnowledgeBaseTempAttachmentService.md index 6df506d03..5e7a7f4a9 100644 --- a/openframe-api-lib/src/main/java/com/openframe/api/service/.KnowledgeBaseTempAttachmentService.md +++ b/openframe-api-lib/src/main/java/com/openframe/api/service/.KnowledgeBaseTempAttachmentService.md @@ -1,52 +1,49 @@ - -Manages temporary file uploads for Knowledge Base articles before they are permanently saved, mirroring the ticket-based `TempAttachmentService` pattern. + +Manages temporary file uploads for Knowledge Base articles before they are saved, providing a two-phase upload flow where files are staged in GCS and then promoted to permanent storage upon article save. ## Key Components | Method | Description | |--------|-------------| -| `createUploadUrl()` | Creates a `TempAttachment` record and generates a unique GCS storage path for a pending upload | -| `generateUploadUrl()` | Returns a presigned GCS PUT URL for direct frontend-to-storage upload | +| `createUploadUrl()` | Creates a `TempAttachment` record and generates a unique GCS staging path under `temp/{uuid}/{fileName}` | +| `generateUploadUrl()` | Returns a presigned GCS PUT URL (configurable expiry, default 15 min) for direct browser-to-GCS upload | | `deleteTempAttachment()` | Deletes a temp attachment from GCS and the database, enforcing uploader ownership | -| `linkTempAttachmentsToArticle()` | Moves temp files to permanent `kb-attachments/` paths and creates `KnowledgeBaseItemAttachment` records | -| `moveToArticle()` *(private)* | Handles per-file GCS move, temp record cleanup, and permanent attachment creation | - -**Configuration property:** `openframe.kb.presigned-url-expiration-minutes` (default: `15`) - -## Upload Flow - -```mermaid -sequenceDiagram - participant U as User - participant S as KBTempAttachmentService - participant GCS as GCS Storage - participant DB as Database - - U->>S: createUploadUrl(fileName, contentType) - S->>DB: Save TempAttachment (temp/{id}/file) - S->>U: Return TempAttachment + presigned PUT URL - U->>GCS: Upload file directly - U->>S: linkTempAttachmentsToArticle(articleId, tempIds) - S->>GCS: moveFile(temp path β†’ kb-attachments path) - S->>DB: Delete TempAttachment - S->>DB: Save KnowledgeBaseItemAttachment +| `linkTempAttachmentsToArticle()` | Promotes a list of temp files to permanent `kb-attachments/{articleId}/{fileName}` paths and creates `KnowledgeBaseItemAttachment` records | +| `moveToArticle()` (private) | Moves a single file in GCS, deletes the `TempAttachment`, and builds the permanent attachment entity | + +**Storage path patterns:** + +```text +Temporary: temp/{uuid}/{fileName} +Permanent: kb-attachments/{articleId}/{fileName} +``` + +**Configuration:** + +```text +openframe.kb.presigned-url-expiration-minutes=15 (default) ``` ## Usage Example ```java -// Step 1: Request an upload URL +// Step 1 β€” Request upload URL TempAttachment temp = kbTempAttachmentService.createUploadUrl( - userId, "diagram.png", "image/png", 204800L + userId, "architecture.png", "image/png", 204800L ); -String presignedUrl = kbTempAttachmentService.generateUploadUrl(temp); -// Frontend uploads directly to presignedUrl via HTTP PUT +String presignedPutUrl = kbTempAttachmentService.generateUploadUrl(temp); +// Frontend uploads directly to presignedPutUrl via HTTP PUT -// Step 2: On article save, promote temp files to permanent storage +// Step 2 β€” On article save, promote temp files to permanent storage List attachments = kbTempAttachmentService.linkTempAttachmentsToArticle( articleId, List.of(temp.getId()), userId ); -``` \ No newline at end of file + +// Cancel an upload (user removed a file before saving) +kbTempAttachmentService.deleteTempAttachment(userId, temp.getId()); +``` + +> **Note:** Each step in `linkTempAttachmentsToArticle` is fault-tolerant β€” if an individual GCS move fails, that attachment is skipped and logged rather than rolling back the entire batch. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/service/rmm/.ScriptService.md b/openframe-api-lib/src/main/java/com/openframe/api/service/rmm/.ScriptService.md new file mode 100644 index 000000000..8fc24126a --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/service/rmm/.ScriptService.md @@ -0,0 +1,44 @@ + +Service layer for RMM script management, providing CRUD operations with tenant-scoped isolation, soft-delete semantics, and cursor-based pagination. + +## Key Components + +| Method | Description | +|--------|-------------| +| `create(CreateScriptInput)` | Creates a new script; throws `ConflictException` on duplicate name within tenant | +| `get(String id)` | Fetches a single visible (non-deleted) script by ID | +| `list(...)` | Cursor-paginated, filterable, searchable script listing with sort support | +| `update(String id, UpdateScriptInput)` | Full replacement (PUT semantics); enforces name uniqueness across tenant | +| `delete(String id)` | Soft-deletes by transitioning status to `DELETED`; idempotent | +| `loadVisibleOrThrow(...)` | Private helper treating `DELETED` status as non-existent | +| `loadOrThrow(...)` | Private helper loading any script regardless of status (used by delete) | +| `buildPageInfo(...)` | Constructs `PageInfo` with encoded cursors for forward/backward pagination | + +## Usage Example + +```java +// Create a script +CreateScriptInput input = CreateScriptInput.builder() + .name("Restart Service") + .shell(ShellType.POWERSHELL) + .build(); +ScriptResponse created = scriptService.create(input); + +// List with filter, search, and cursor pagination +ScriptFilterInput filter = ScriptFilterInput.builder() + .statuses(List.of(ScriptStatus.ACTIVE)) + .build(); +SortInput sort = SortInput.builder() + .field("name") + .direction(SortDirection.ASC) + .build(); +CursorPaginationCriteria pagination = CursorPaginationCriteria.of(20, null); + +GenericQueryResult results = + scriptService.list(filter, "restart", sort, pagination); + +// Soft-delete (idempotent) +scriptService.delete("script-id-123"); +``` + +> **Tenant scoping** is resolved automatically via `TenantIdProvider` (DB-per-tenant architecture). Callers never pass `tenantId` directly β€” this prevents super-admin JWT claims from misdirecting writes to a foreign tenant's data. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.CommandDataFetcher.md b/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.CommandDataFetcher.md new file mode 100644 index 000000000..467d46740 --- /dev/null +++ b/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.CommandDataFetcher.md @@ -0,0 +1,46 @@ + +GraphQL mutation resolver for RMM (Remote Monitoring & Management) ad-hoc command dispatch operations. Bridges GraphQL API calls to the `CommandDispatchService` for executing and cancelling remote commands on managed devices. + +## Key Components + +| Element | Description | +|---|---| +| `runCommand` | Mutation that dispatches an ad-hoc command to a remote device via `RunCommandInput` | +| `cancelExecution` | Mutation that cancels an in-progress command dispatch via `CancelExecutionInput` | +| `CommandDispatchService` | Injected service handling the underlying RMM command execution logic | + +## Usage Example + +```java +// GraphQL mutation β€” run a command +mutation { + runCommand(input: { + deviceId: "dev-123", + command: "Get-Service | Where-Object { $_.Status -eq 'Running' }" + }) { + dispatchId + status + } +} + +// GraphQL mutation β€” cancel an in-progress execution +mutation { + cancelExecution(input: { + dispatchId: "dispatch-abc-456" + }) { + success + message + } +} +``` + +```java +// Service delegation (internal) +@DgsMutation +public CommandDispatchResponse runCommand( + @InputArgument @Valid RunCommandInput input) { + return commandDispatchService.runCommand(input); +} +``` + +> **Validation:** Both mutations use `@Valid` + `@Validated` to enforce bean validation constraints on input DTOs before reaching the service layer, rejecting malformed requests early. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.NotificationDataFetcher.md b/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.NotificationDataFetcher.md index bbd5f16df..64e3dc25e 100644 --- a/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.NotificationDataFetcher.md +++ b/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.NotificationDataFetcher.md @@ -1,53 +1,48 @@ - -GraphQL DGS data fetcher that handles all notification-related queries and mutations for authenticated ADMIN and AGENT users, including listing, read-state management, and deletion operations. + +GraphQL DGS data fetcher that exposes notification queries and mutations for authenticated `ADMIN` and `AGENT` users, resolving the current recipient from the JWT principal before delegating to `NotificationService` and `NotificationReadStateService`. ## Key Components -| Member | Type | Description | -|--------|------|-------------| -| `notificationNodeId` | `@DgsData` | Encodes the notification's raw ID into a Relay-compliant global ID | -| `notifications` | `@DgsQuery` | Returns a paginated, cursor-based connection of `NotificationView` items with optional filter and search | -| `hasUnreadNotifications` | `@DgsQuery` | Returns `true` if the current recipient has any unread notifications | -| `unreadCountsByCategory` | `@DgsQuery` | Returns per-category unread counts as a list of `UnreadCategoryCount` | -| `markNotificationAsRead` | `@DgsMutation` | Marks a single notification (by Relay global ID) as read | -| `markAllNotificationsAsRead` | `@DgsMutation` | Marks all notifications as read; returns count affected | -| `deleteNotification` | `@DgsMutation` | Deletes a single notification by Relay global ID | -| `deleteAllReadNotifications` | `@DgsMutation` | Deletes all read notifications; returns count deleted | -| `currentRecipient()` | Private helper | Resolves the caller to a `Recipient` record β€” `MACHINE` for AGENT actors, `USER` otherwise | -| `decodeNotificationId()` | Private helper | Decodes and validates a Relay global ID, asserting the `Notification` type | +| Element | Description | +|---|---| +| `notifications()` | Paginated, filterable query returning a Relay-compliant `GenericConnection` | +| `hasUnreadNotifications()` | Boolean query indicating whether the current recipient has any unread notifications | +| `unreadCountsByCategory()` | Returns per-category unread counts as `List` | +| `markNotificationAsRead()` | Mutation to mark a single notification read by its Relay global ID | +| `markAllNotificationsAsRead()` | Mutation to mark all notifications read, returning affected count | +| `deleteNotification()` | Mutation to delete a single notification by its Relay global ID | +| `deleteAllReadNotifications()` | Mutation to delete all read notifications, returning affected count | +| `currentRecipient()` | Resolves `AGENT` principals to `RecipientType.MACHINE` (via `machineId`) and all others to `RecipientType.USER` | +| `decodeNotificationId()` | Decodes and validates a Relay global ID, enforcing the `Notification` type | +| `notificationNodeId()` | Field resolver that encodes the raw `id` into a Relay global ID on the `Notification` type | ## Usage Example ```java -# GraphQL β€” List notifications with pagination and filter +# GraphQL β€” paginated notification query query { - notifications(first: 10, filter: { read: false }) { + notifications(first: 10, filter: { read: false }, sort: { field: "createdAt", direction: DESC }) { edges { node { - id + id # Relay global ID title category + createdAt } } - pageInfo { - hasNextPage - endCursor - } + pageInfo { hasNextPage endCursor } } } -# GraphQL β€” Mark a single notification as read +# Mark a single notification as read mutation { - markNotificationAsRead(notificationId: "Notification:abc123") + markNotificationAsRead(notificationId: "Tm90aWZpY2F0aW9uOjEyMw==") } -# GraphQL β€” Unread counts per category -query { - unreadCountsByCategory { - category - count - } +# Delete all read notifications +mutation { + deleteAllReadNotifications # returns affected count } ``` -> All operations require a valid JWT and the `ADMIN` or `AGENT` authority. AGENT actors are resolved to their `machine_id` claim; all other authenticated users resolve to their `userId`. \ No newline at end of file +> All operations require a valid JWT with either `ADMIN` or `AGENT` authority. `AGENT` principals must carry a `machine_id` claim; missing or blank values throw `UnauthorizedException`. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.ScriptDataFetcher.md b/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.ScriptDataFetcher.md new file mode 100644 index 000000000..318ff15f7 --- /dev/null +++ b/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.ScriptDataFetcher.md @@ -0,0 +1,48 @@ + +GraphQL data fetcher (DGS component) that exposes CRUD operations for RMM scripts, delegating all business logic and tenant scoping to `ScriptService`. + +## Key Components + +| Member | Type | Description | +|--------|------|-------------| +| `script` | `@DgsQuery` | Fetches a single script by ID | +| `scripts` | `@DgsQuery` | Returns a cursor-paginated, filterable, searchable connection of scripts | +| `createScript` | `@DgsMutation` | Creates a new script from a validated `CreateScriptInput` | +| `updateScript` | `@DgsMutation` | Updates an existing script by ID with a validated `UpdateScriptInput` | +| `deleteScript` | `@DgsMutation` | Deletes a script by ID, returns `true` on success | +| `scriptService` | Dependency | Handles all business logic and tenant-scoped data access | +| `scriptMapper` | Dependency | Converts `ConnectionArgs` to pagination criteria and query results to GraphQL connections | + +## Usage Example + +```java +// Query β€” fetch a single script +ScriptResponse script = scriptDataFetcher.script("script-uuid-123"); + +// Query β€” list scripts with pagination, filter, and search +GenericConnection> page = scriptDataFetcher.scripts( + ScriptFilterInput.builder().platform("WINDOWS").build(), + "cleanup", // free-text search + SortInput.builder().field("name").direction("ASC").build(), + 10, // first + null, // after cursor + null, // last + null // before cursor +); + +// Mutation β€” create +ScriptResponse created = scriptDataFetcher.createScript( + CreateScriptInput.builder().name("Disk Cleanup").content("...").build() +); + +// Mutation β€” update +ScriptResponse updated = scriptDataFetcher.updateScript( + "script-uuid-123", + UpdateScriptInput.builder().name("Disk Cleanup v2").build() +); + +// Mutation β€” delete +boolean deleted = scriptDataFetcher.deleteScript("script-uuid-123"); +``` + +> **Note:** Tenant scoping is resolved internally by `ScriptService` via `TenantIdProvider`. Role-based authorization enforcement is pending the finalized RMM role model. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/mapper/.GraphQLScriptMapper.md b/openframe-api-service-core/src/main/java/com/openframe/api/mapper/.GraphQLScriptMapper.md new file mode 100644 index 000000000..5aae6f52f --- /dev/null +++ b/openframe-api-service-core/src/main/java/com/openframe/api/mapper/.GraphQLScriptMapper.md @@ -0,0 +1,38 @@ + +Assembles Relay-style GraphQL Connection envelopes for script queries and converts `ConnectionArgs` into `CursorPaginationCriteria` for pagination. + +## Key Components + +| Method | Description | +|--------|-------------| +| `toCursorPaginationCriteria(ConnectionArgs)` | Delegates to `CursorPaginationCriteria.fromConnectionArgs()` to translate Relay pagination args into internal criteria | +| `toConnection(GenericQueryResult)` | Maps a paginated query result into a Relay `Connection` envelope, encoding each item's ID as a Base64 cursor and attaching `PageInfo` | + +## Usage Example + +```java +@Component +public class ScriptDataFetcher { + + private final GraphQLScriptMapper graphQLScriptMapper; + private final ScriptService scriptService; + + public GenericConnection> getScripts(ConnectionArgs args) { + // Convert Relay args to internal pagination criteria + CursorPaginationCriteria criteria = + graphQLScriptMapper.toCursorPaginationCriteria(args); + + // Fetch paginated results from service + GenericQueryResult result = + scriptService.findScripts(criteria); + + // Wrap in Relay Connection envelope with cursors and pageInfo + return graphQLScriptMapper.toConnection(result); + } +} +``` + +## Design Notes + +- Intentionally thin β€” raw entity↔DTO mapping is delegated to `ScriptMapper` in `openframe-api-lib`, keeping this class free of DGS/Relay dependencies for non-GraphQL callers. +- Follows the same mapper-split pattern as `GraphQLDeviceMapper` and `GraphQLNotificationMapper` for consistency across the API layer. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolAgentUpdateService.md b/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolAgentUpdateService.md index fecda7810..7091acc79 100644 --- a/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolAgentUpdateService.md +++ b/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolAgentUpdateService.md @@ -1,38 +1,38 @@ - -Orchestrates forced tool agent update operations across managed machines by validating requests, fetching agent configurations, and publishing update events via NATS messaging. + +Handles forced tool agent update operations across managed machines by validating requests, resolving agent configurations, and dispatching update messages via NATS publisher. ## Key Components | Member | Type | Description | |--------|------|-------------| -| `process()` | Public Method | Handles update requests for a specific list of machine IDs | -| `processAll()` | Public Method | Triggers updates for all registered machines in the repository | -| `processMachine()` | Private Method | Fetches the tool agent config and publishes the update event for a single machine | -| `buildResponseItem()` | Private Method | Constructs a `ForceToolAgentUpdateResponseItem` with machine ID, agent ID, and status | -| `validateToolAgentId()` | Private Method | Guards against blank tool agent IDs | -| `validateMachineIds()` | Private Method | Guards against empty machine ID lists | -| `toolAgentUpdateUpdatePublisher` | Dependency | NATS publisher that dispatches the update message to the tool agent | +| `process()` | `public` | Triggers a forced update for a specified list of machine IDs | +| `processAll()` | `public` | Triggers a forced update across **all** registered machines | +| `processMachines()` | `private` | Streams machine IDs and delegates per-machine processing | +| `processMachine()` | `private` | Resolves the `IntegratedToolAgent` config and publishes the update event; returns `PROCESSED` or `FAILED` status | +| `buildResponseItem()` | `private` | Constructs a `ForceToolAgentUpdateResponseItem` with machine ID, agent ID, and status | +| `validateToolAgentId()` | `private` | Guards against blank tool agent IDs | +| `validateMachineIds()` | `private` | Guards against null or empty machine ID lists | ## Usage Example ```java // Force update a specific set of machines ForceToolAgentUpdateRequest request = new ForceToolAgentUpdateRequest(); -request.setToolAgentId("agent-123"); +request.setToolAgentId("agent-zabbix-01"); request.setMachineIds(List.of("machine-001", "machine-002")); ForceToolAgentUpdateResponse response = forceToolAgentUpdateService.process(request); response.getItems().forEach(item -> - log.info("Machine: {}, Status: {}", item.getMachineId(), item.getStatus()) + log.info("Machine {} β†’ {}", item.getMachineId(), item.getStatus()) ); -// Force update ALL registered machines for a given agent -ForceToolAgentUpdateResponse allResponse = forceToolAgentUpdateService.processAll("agent-123"); +// Force update across all registered machines +ForceToolAgentUpdateResponse allResponse = + forceToolAgentUpdateService.processAll("agent-zabbix-01"); ``` -## Status Outcomes +## Notes -Each processed machine returns one of two statuses from `ForceAgentStatus`: - -- **`PROCESSED`** β€” Update event successfully published via NATS -- **`FAILED`** β€” Exception occurred (agent not found or publish error); logged and gracefully captured per machine \ No newline at end of file +- Per-machine failures are **isolated** β€” one failure does not abort the remaining machines; the item is marked `FAILED` and processing continues. +- `processAll()` dynamically fetches the current machine list from `MachineRepository` at invocation time, so newly registered machines are always included. +- Update events are dispatched via `ToolAgentUpdateUpdatePublisher` (NATS), decoupling the HTTP request lifecycle from the actual agent update execution. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolInstallationService.md b/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolInstallationService.md index 043ad134b..9a24d7361 100644 --- a/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolInstallationService.md +++ b/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolInstallationService.md @@ -1,35 +1,39 @@ - -A Spring service that orchestrates forced tool agent installations and reinstallations across machines, providing batch processing capabilities for both specific machine sets and all available machines. + +Orchestrates forced installation and reinstallation of tool agents across managed machines, processing requests for specific machines or all machines in the system. ## Key Components -- **`process(ForceToolInstallationRequest)`** - Installs a tool agent on specified machines -- **`processAll(String toolAgentId)`** - Installs a tool agent on all available machines -- **`processReinstall(ForceToolInstallationRequest)`** - Forces reinstallation on specified machines -- **`processReinstallAll(String toolAgentId)`** - Forces reinstallation on all machines -- **`processMachines(List, String, boolean)`** - Internal batch processor for machine operations -- **`processMachine(String, String, boolean)`** - Handles individual machine installations with error handling +| Method | Description | +|--------|-------------| +| `process(request)` | Installs a tool agent on a specified list of machines | +| `processAll(toolAgentId)` | Installs a tool agent across all registered machines | +| `processAll(toolAgentId, reinstall)` | Installs or reinstalls a tool agent on all machines | +| `processReinstall(request)` | Reinstalls a tool agent on a specified list of machines | +| `processReinstallAll(toolAgentId)` | Reinstalls a tool agent across all registered machines | +| `processMachine(...)` | Resolves tool agent config and delegates to `ToolInstallationService`; returns `PROCESSED` or `FAILED` per machine | +| `buildResponseItem(...)` | Constructs a `ForceToolAgentInstallationResponseItem` with machine ID, tool agent ID, and status | + +**Dependencies:** +- `IntegratedToolAgentService` β€” resolves tool agent configuration by key +- `ToolInstallationService` β€” executes the actual installation logic +- `MachineRepository` β€” fetches all registered machines for bulk operations ## Usage Example ```java -@Autowired -private ForceToolInstallationService forceInstallationService; - -// Install tool on specific machines +// Force install on specific machines ForceToolInstallationRequest request = new ForceToolInstallationRequest(); -request.setToolAgentId("my-tool-agent"); -request.setMachineIds(Arrays.asList("machine-1", "machine-2")); - -ForceToolAgentInstallationResponse response = forceInstallationService.process(request); +request.setToolAgentId("zabbix-agent"); +request.setMachineIds(List.of("machine-001", "machine-002")); -// Install on all machines -ForceToolAgentInstallationResponse allResponse = - forceInstallationService.processAll("my-tool-agent"); +ForceToolAgentInstallationResponse response = forceToolInstallationService.process(request); +response.getItems().forEach(item -> + log.info("Machine {} β†’ {}", item.getMachineId(), item.getStatus()) +); -// Force reinstall on specific machines -ForceToolAgentInstallationResponse reinstallResponse = - forceInstallationService.processReinstall(request); +// Force reinstall across all machines +ForceToolAgentInstallationResponse allResponse = + forceToolInstallationService.processReinstallAll("zabbix-agent"); ``` -The service validates inputs, retrieves tool agent configurations, and delegates to `ToolInstallationService` while providing comprehensive error handling and status reporting through response items. \ No newline at end of file +Each machine in the response is independently marked as `PROCESSED` or `FAILED`, allowing partial success without aborting the full batch. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/service/.NotificationService.md b/openframe-api-service-core/src/main/java/com/openframe/api/service/.NotificationService.md index c48e59efe..f6d0a9663 100644 --- a/openframe-api-service-core/src/main/java/com/openframe/api/service/.NotificationService.md +++ b/openframe-api-service-core/src/main/java/com/openframe/api/service/.NotificationService.md @@ -1,44 +1,41 @@ - -Provides paginated listing of notifications for a recipient, with support for read/unread filtering, search normalization, and cursor-based pagination. + +Handles paginated retrieval of notifications for a given recipient, supporting cursor-based pagination, read-status filtering, search, and sort direction resolution. ## Key Components -| Member | Type | Description | -|--------|------|-------------| -| `SEARCH_MIN_LENGTH` | Constant | Minimum character threshold (2) before a search term is applied | -| `list()` | Method | Fetches a cursor-paginated page of `NotificationView` items for a given recipient | -| `normalizeSearch()` | Private Method | Trims and nullifies search strings shorter than `SEARCH_MIN_LENGTH` | -| `buildPageInfo()` | Private Method | Constructs `PageInfo` with cursor positions and forward/backward pagination flags | +| Member | Description | +|--------|-------------| +| `list(...)` | Overloaded public method that fetches a paginated, optionally sorted page of `NotificationView` items for a recipient | +| `normalizeSearch(String)` | Trims and nullifies search strings shorter than `SEARCH_MIN_LENGTH` (2 chars) | +| `resolveSortDirection(SortInput)` | Maps a `SortInput` to a Spring `Sort.Direction`, falling back to `DESC` and warning on invalid field names | +| `buildPageInfo(...)` | Constructs `PageInfo` with cursor-encoded start/end cursors and correct `hasNextPage`/`hasPreviousPage` flags for both forward and backward pagination | +| `ALLOWED_SORT_FIELDS` | Allowlist restricting sort fields to `id` and `createdAt` | ## Usage Example ```java -// Fetch first page of unread notifications for a user -NotificationFilter filter = new NotificationFilter(/* read= */ false, /* search= */ null); -CursorPaginationCriteria pagination = CursorPaginationCriteria.builder() - .limit(20) - .build(); +// Inject the service +@Autowired +private NotificationService notificationService; +// Build filter and pagination +NotificationFilter filter = new NotificationFilter(/* read= */ false, /* search= */ "alert"); +CursorPaginationCriteria pagination = CursorPaginationCriteria.of(20, null, false); +SortInput sort = new SortInput("createdAt", SortDirection.DESC); + +// Fetch paginated notifications for a user GenericQueryResult result = notificationService.list( - "user-123", - RecipientType.USER, - filter, - pagination + "user-123", + RecipientType.USER, + filter, + pagination, + sort ); -// Access results -List notifications = result.getItems(); -PageInfo pageInfo = result.getPageInfo(); - -// Fetch next page using end cursor -CursorPaginationCriteria nextPage = CursorPaginationCriteria.builder() - .limit(20) - .cursor(pageInfo.getEndCursor()) - .build(); +result.items().forEach(n -> System.out.println(n.id() + " β€” " + n.title())); +PageInfo page = result.pageInfo(); +System.out.println("Has next: " + page.hasNextPage()); +System.out.println("End cursor: " + page.endCursor()); ``` -## Pagination Behavior - -- Uses a **limit + 1** fetch strategy to determine if a next/previous page exists without a separate count query. -- Result order is **reversed** automatically when paginating backward. -- Cursors are encoded via `CursorCodec` using the notification's ID as the key. \ No newline at end of file +> **Note:** Requests one extra item (`limit + 1`) from the repository to determine `hasMore` without a separate count query. Backward pagination reverses the result list before mapping to views. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/service/.ReleaseVersionQueryService.md b/openframe-api-service-core/src/main/java/com/openframe/api/service/.ReleaseVersionQueryService.md index bbf54cb82..d21d84246 100644 --- a/openframe-api-service-core/src/main/java/com/openframe/api/service/.ReleaseVersionQueryService.md +++ b/openframe-api-service-core/src/main/java/com/openframe/api/service/.ReleaseVersionQueryService.md @@ -1,13 +1,13 @@ - -Provides read-only access to the current release version of the OpenFrame platform by querying the underlying data store. + +Retrieves the current release version from the database, providing a lightweight query service for accessing `ReleaseVersion` document records. ## Key Components | Element | Description | |---|---| -| `ReleaseVersionQueryService` | Spring `@Service` bean for querying release version data | -| `getReleaseVersion()` | Fetches the singleton `ReleaseVersion` document using `ReleaseVersion.DEFAULT_ID` | -| `ReleaseVersionRepository` | Injected repository (via `@RequiredArgsConstructor`) used to perform the lookup | +| `ReleaseVersionQueryService` | Spring `@Service` bean for release version lookups | +| `getReleaseVersion()` | Returns the first available `ReleaseVersion` entry wrapped in an `Optional` | +| `ReleaseVersionRepository` | Injected repository delegate handling the actual data access | ## Usage Example @@ -27,4 +27,4 @@ public class VersionController { } ``` -> **Note:** `getReleaseVersion()` returns an `Optional`, so callers should handle the case where no version document has been persisted yet (e.g., on a fresh deployment). \ No newline at end of file +> **Note:** `getReleaseVersion()` uses `findFirstBy()` under the hood, meaning it assumes at most one version record exists in the datastore at a time. If no record is found, an empty `Optional` is returned β€” callers should handle the absent case explicitly. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/service/rmm/.CommandDispatchService.md b/openframe-api-service-core/src/main/java/com/openframe/api/service/rmm/.CommandDispatchService.md new file mode 100644 index 000000000..73aa53b4b --- /dev/null +++ b/openframe-api-service-core/src/main/java/com/openframe/api/service/rmm/.CommandDispatchService.md @@ -0,0 +1,36 @@ + +Dispatches ad-hoc shell commands and cancellation requests from the dashboard to target agents via core NATS (fire-and-forget), without persisting any state on the backend. + +## Key Components + +| Member | Description | +|--------|-------------| +| `CommandNatsPublisher` | Injected publisher used to send NATS messages to the target agent | +| `runCommand(RunCommandInput)` | Generates a UUID execution ID, builds a `CommandMessage`, and publishes it to the agent's NATS subject | +| `cancelExecution(CancelExecutionInput)` | Builds a `CancelMessage` and publishes a cancellation request for an in-flight execution | + +## Usage Example + +```java +// Dispatch a shell command to a remote agent +RunCommandInput input = RunCommandInput.builder() + .machineId("machine-abc-123") + .command("Get-Process") + .shell("powershell") + .privilegeLevel("admin") + .timeoutSeconds(30) + .build(); + +CommandDispatchResponse response = commandDispatchService.runCommand(input); +String executionId = response.getExecutionId(); // UUID for tracking the execution + +// Cancel an in-flight execution +CancelExecutionInput cancelInput = CancelExecutionInput.builder() + .machineId("machine-abc-123") + .executionId(executionId) + .build(); + +CancelDispatchResponse cancelResponse = commandDispatchService.cancelExecution(cancelInput); +``` + +> **Note:** This service is pure transport. It does not persist commands or results β€” agent responses arrive on a separate NATS subject and are handled by the execution service. \ No newline at end of file diff --git a/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.NotificationDataFetcherIT.md b/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.NotificationDataFetcherIT.md index 302f985bb..f6e40eb54 100644 --- a/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.NotificationDataFetcherIT.md +++ b/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.NotificationDataFetcherIT.md @@ -1,46 +1,52 @@ - -Integration test suite for the `NotificationDataFetcher` GraphQL layer, verifying end-to-end notification query and mutation behavior against a real MongoDB instance using DGS and Spring Boot test infrastructure. + +Integration test suite for the `NotificationDataFetcher` GraphQL data fetcher, verifying end-to-end notification queries and mutations against a real MongoDB instance. ## Key Components -| Component | Description | -|-----------|-------------| -| `NotificationDataFetcherIT` | Main test class extending `BaseMongoIntegrationTest`; runs only when `-Dintegration.tests=true` is set | -| `DgsQueryExecutor` | Executes raw GraphQL query/mutation strings against the full DGS schema | -| `NotificationReadStateService` | Service under test; used to seed `NotificationReadState` documents before assertions | -| `MongoTemplate` | Used in `@BeforeEach` to drop and reset `Notification` and `NotificationReadState` collections | -| `loginAsAdmin` / `loginAsAgent` | Helpers that inject a synthetic `JwtAuthenticationToken` into the `SecurityContextHolder` for role-based test isolation | +| Component | Role | +|-----------|------| +| `DgsQueryExecutor` | Executes raw GraphQL query/mutation strings against the DGS runtime | +| `NotificationReadStateService` | Service under test β€” creates, marks, and deletes read-state rows | +| `NotificationFixtures` | Factory for seeding `Notification` documents in MongoDB | +| `BaseMongoIntegrationTest` | Base class providing embedded Mongo lifecycle and auth helpers (`loginAsAdmin`, `loginAsAgent`) | +| `Relay.toGlobalId()` | Converts raw MongoDB IDs to Relay-spec global IDs for mutation input | ## Test Coverage | Test | Scenario | |------|----------| -| `admin_lists_own_rows` | ADMIN JWT resolves USER-addressed notification rows via `notifications` query | -| `agent_lists_machine_rows` | AGENT JWT with `machine_id` claim resolves MACHINE-addressed rows | -| `has_unread` | `hasUnreadNotifications` flips from `true` β†’ `false` after `markRead` | -| `mark_as_read` | `markNotificationAsRead` mutation accepts a Relay global ID and returns `true` | -| `mark_all_as_read` | `markAllNotificationsAsRead` returns count of UNREAD rows updated | -| `delete_notification` | `deleteNotification` mutation soft-deletes a single row by Relay global ID | -| `delete_all_read` | `deleteAllReadNotifications` transitions only READ rows to DELETED | -| `unread_counts_by_category` | `unreadCountsByCategory` returns one entry per category in UNREAD rows | +| `admin_lists_own_rows` | USER-addressed read-state visible to matching JWT principal | +| `agent_lists_machine_rows` | MACHINE-addressed read-state resolved from `machine_id` JWT claim | +| `has_unread` | `hasUnreadNotifications` flips `true β†’ false` after `markRead` | +| `mark_as_read` | `markNotificationAsRead` mutation accepts Relay global ID | +| `mark_all_as_read` | `markAllNotificationsAsRead` returns count of updated rows | +| `delete_notification` | `deleteNotification` soft-deletes a single row | +| `delete_all_read` | `deleteAllReadNotifications` only transitions READ rows; UNREAD rows untouched | +| `admin_lists_oldest_first_with_sort_asc` | `sort: { direction: ASC }` returns oldest-first order | +| `admin_lists_newest_first_by_default` | Default (no sort arg) returns newest-first | ## Usage Example ```java -// Run integration tests (requires a running MongoDB; enable via system property) -./mvnw verify -Dintegration.tests=true - -// Example query exercised in tests -queryExecutor.execute(""" - query { notifications(first: 10) { edges { node { id title read } } } } -"""); - -// Example mutation with Relay global ID -String globalId = new Relay().toGlobalId("Notification", n.getId()); -queryExecutor.execute( - "mutation($id: ID!) { markNotificationAsRead(notificationId: $id) }", - Map.of("id", globalId) -); +// Only runs when -Dintegration.tests=true is set +@Test +void mark_as_read() { + loginAsAdmin(ALICE); + Notification n = mongoTemplate.save(NotificationFixtures.basic()); + readStateService.createForAudience( + n.getId(), NotificationCategory.TICKETS, + "title", RecipientType.USER, Set.of(ALICE) + ); + + String globalId = RELAY.toGlobalId("Notification", n.getId()); + ExecutionResult res = queryExecutor.execute( + "mutation($id: ID!) { markNotificationAsRead(notificationId: $id) }", + Map.of("id", globalId) + ); + assertThat(res.getErrors()).isEmpty(); + assertThat(res.>getData() + .get("markNotificationAsRead")).isEqualTo(true); +} ``` -> **Note:** Tests are gated by `@EnabledIfSystemProperty(named = "integration.tests", matches = "true")` and tagged `@Tag("integration")`. Cloud Stream auto-configurations are excluded to prevent messaging infrastructure from being required at test startup. \ No newline at end of file +> Tests are gated by `@EnabledIfSystemProperty(named = "integration.tests", matches = "true")` and require a running MongoDB instance provided by the `BaseMongoIntegrationTest` lifecycle. \ No newline at end of file diff --git a/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.TestApprovalContext.md b/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.TestApprovalContext.md index 9cc6371c9..2ae9ba745 100644 --- a/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.TestApprovalContext.md +++ b/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.TestApprovalContext.md @@ -1,24 +1,34 @@ - -A data model class representing the context payload for test approval notifications, extending the base `NotificationContext` with ticket and approval request identifiers. + +A notification context implementation for test approval workflows, carrying ticket and approval request identifiers needed to process approval-related notifications. ## Key Components | Member | Type | Description | |--------|------|-------------| -| `ticketId` | `String` | Identifier of the associated support ticket | -| `approvalRequestId` | `String` | Identifier of the approval request being tested | +| `TYPE` | `static final String` | Constant sourced from `TestApprovalContextDescriptor.TYPE`; identifies this context variant | +| `ticketId` | `String` | Reference to the associated ticket | +| `approvalRequestId` | `String` | Reference to the specific approval request | +| `getType()` | `@Override` | Returns the `TYPE` constant for serialization/deserialization routing | -Inherits all fields and behavior from `NotificationContext` via Lombok's `@SuperBuilder`, `@EqualsAndHashCode(callSuper = true)`, and `@ToString(callSuper = true)`. +## Design Notes + +- Extends `NotificationContext` using Lombok's `@SuperBuilder` for fluent construction that includes parent fields +- `@JsonIgnoreProperties(value = "type", allowGetters = true)` ensures the `type` field is excluded during deserialization but still serialized via `getType()`, supporting polymorphic JSON dispatch ## Usage Example ```java -// Build a TestApprovalContext using the generated builder -TestApprovalContext context = TestApprovalContext.builder() - .ticketId("TKT-1234") +// Build via SuperBuilder +TestApprovalContext ctx = TestApprovalContext.builder() + .ticketId("TICK-1234") .approvalRequestId("APR-5678") .build(); -// Pass context to a notification dispatcher -notificationService.sendTestApproval(context); +// Type check / routing +if (ctx.getType().equals(TestApprovalContext.TYPE)) { + notificationService.dispatch(ctx); +} + +// Serialized JSON will include "type" as a read-only property: +// { "type": "test_approval", "ticketId": "TICK-1234", "approvalRequestId": "APR-5678" } ``` \ No newline at end of file diff --git a/openframe-api-service-core/src/test/java/com/openframe/api/integration/service/.NotificationServiceIT.md b/openframe-api-service-core/src/test/java/com/openframe/api/integration/service/.NotificationServiceIT.md index 06ca94d6d..772fee488 100644 --- a/openframe-api-service-core/src/test/java/com/openframe/api/integration/service/.NotificationServiceIT.md +++ b/openframe-api-service-core/src/test/java/com/openframe/api/integration/service/.NotificationServiceIT.md @@ -1,50 +1,48 @@ - -Integration test suite for `NotificationService.list()`, validating pagination behavior, read-flag merging, audience isolation, search normalization, and soft-delete exclusion against a real MongoDB instance. + +Integration test suite for `NotificationService.list()`, verifying cursor pagination, read-flag merging, audience isolation, search normalization, soft-delete exclusion, and sort behavior against a real MongoDB instance. ## Key Components -| Component | Role | -|---|---| -| `NotificationService` | Service under test β€” handles `list()` with filters and cursor pagination | -| `NotificationReadStateService` | Creates and mutates read-state records used by `list()` | -| `BaseMongoIntegrationTest` | Base class providing embedded/real MongoDB wiring | -| `NotificationFixtures` | Helper that constructs `Notification` documents for seeding | -| `NotificationFilter` | Filter DTO accepting `read` boolean and search string | -| `CursorPaginationCriteria` | Pagination input (limit-based cursor) | +- **`NotificationServiceIT`** β€” Main test class extending `BaseMongoIntegrationTest`, conditionally enabled via the `integration.tests` system property +- **`reset()`** β€” Drops `Notification` and `NotificationReadState` collections before each test to ensure a clean state +- **`page10()`** β€” Helper that builds a default `CursorPaginationCriteria` with a limit of 10 +- **`seedFive()`** β€” (truncated) Helper that seeds multiple notifications for sort-order tests ## Test Coverage -| Test | Scenario | +| Test | Behavior Verified | |---|---| -| `list_user_with_read_flag` | Read state is merged into `NotificationView.read` | -| `list_machine_recipient` | `RecipientType.MACHINE` audience visibility works | -| `audience_isolation_by_id` | Notifications scoped to other users are hidden | -| `audience_isolation_by_type` | Same ID, different `RecipientType` β†’ isolated pages | -| `read_filter_narrows_page` | `read=true` / `read=false` filters return correct rows | -| `short_search_normalized_to_no_search` | 1-char search terms are ignored | -| `whitespace_search_normalized_to_no_search` | Whitespace-only search is treated as no-op | +| `list_user_with_read_flag` | Read status is derived from `NotificationReadState`, not the filter | +| `list_machine_recipient` | `MACHINE` recipient type resolves correctly | +| `audience_isolation_by_id` | Only the addressed recipient sees the notification | +| `audience_isolation_by_type` | `USER` vs `MACHINE` type disambiguates same-ID recipients | +| `read_filter_narrows_page` | `NotificationFilter(read=true/false)` returns only matching rows | +| `short_search_normalized_to_no_search` | Single-character search terms are ignored | +| `whitespace_search_normalized_to_no_search` | Whitespace-only search terms are trimmed and ignored | | `search_no_match_returns_empty_page` | Non-matching search returns empty page with `hasNextPage=false` | -| `deleted_excluded` | Soft-deleted read states are excluded from listing | +| `deleted_excluded` | Soft-deleted read states are excluded from listings | +| `sort_asc_returns_oldest_first` | Explicit `ASC` sort returns oldest notifications first | +| `sort_null_defaults_to_desc` | `null` sort defaults to `DESC` (newest-first) | +| `sort_invalid_field_falls_back_to_direction_only` | Unknown sort field is ignored; direction is still honored | ## Usage Example ```java -// Seed a notification and mark it read for a user -Notification n = mongoTemplate.save(NotificationFixtures.basic("welcome")); -readStateService.createForAudience( - n.getId(), NotificationCategory.TICKETS, "title", RecipientType.USER, Set.of("user-alice") -); -readStateService.markRead("user-alice", RecipientType.USER, n.getId()); - -// List with no filter β€” returns the notification with read=true -GenericQueryResult result = notificationService.list( - "user-alice", - RecipientType.USER, - NotificationFilter.EMPTY, - CursorPaginationCriteria.builder().limit(10).build() -); - -assertThat(result.getItems().get(0).read()).isTrue(); +// Requires -Dintegration.tests=true and a running MongoDB instance +// Tests are executed via the standard test lifecycle: + +@Test +void list_user_with_read_flag() { + Notification n = mongoTemplate.save(NotificationFixtures.basic("welcome")); + readStateService.createForAudience(n.getId(), NotificationCategory.TICKETS, "title", U, Set.of(ALICE)); + readStateService.markRead(ALICE, U, n.getId()); + + GenericQueryResult result = + notificationService.list(ALICE, U, NotificationFilter.EMPTY, page10()); + + assertThat(result.getItems()).hasSize(1); + assertThat(result.getItems().get(0).read()).isTrue(); +} ``` -> **Note:** Tests are gated by the system property `integration.tests=true` and require a live MongoDB connection. Operations on `markRead`, `markAllAsRead`, `delete`, `hasUnread`, and counts are covered in `NotificationReadStateServiceIT`. \ No newline at end of file +> **Note:** `markRead / markAllAsRead / delete / hasUnread / counts` are covered in `NotificationReadStateServiceIT`, not here. \ No newline at end of file diff --git a/openframe-api-service-core/src/test/java/com/openframe/api/mapper/.ScriptMapperTest.md b/openframe-api-service-core/src/test/java/com/openframe/api/mapper/.ScriptMapperTest.md new file mode 100644 index 000000000..b2e181994 --- /dev/null +++ b/openframe-api-service-core/src/test/java/com/openframe/api/mapper/.ScriptMapperTest.md @@ -0,0 +1,46 @@ + +Unit test class verifying the PUT (full-replace) semantics of `ScriptMapper#updateEntity`, ensuring all writable fields are overwritten β€” including explicit `null`s that clear previously-set values β€” while internal fields like `id` and `tenantId` remain untouched. + +## Key Components + +| Test Method | Purpose | +|---|---| +| `updateEntity_nullsInInput_clearFieldsOnEntity` | Verifies all writable fields are set to `null` when input has no values set | +| `updateEntity_doesNotTouchInternalFields` | Confirms `id` and `tenantId` are never modified by the mapper | +| `updateEntity_fullyPopulatedInput_overwritesAllFields` | Validates every writable field is correctly overwritten from a fully-populated `UpdateScriptInput` | +| `updateEntity_emptyListInput_clearsListField` | Ensures empty lists (`[]`) explicitly replace previously-populated list fields | +| `fullyPopulated()` | Private helper that builds a fully-hydrated `Script` entity for use as the pre-update baseline | + +## Usage Example + +```java +// Demonstrates the PUT semantics being tested: +ScriptMapper mapper = new ScriptMapper(); + +// Existing entity with data +Script existing = Script.builder() + .id("65f4a800...") + .tenantId("tenant-1") + .name("Restart Spooler") + .shell(ScriptShell.POWERSHELL) + .build(); + +// Input with new values β€” nulls will clear fields +UpdateScriptInput input = new UpdateScriptInput(); +input.setName("New Name"); +input.setShell(ScriptShell.BASH); +// description left null β†’ clears existing.description + +mapper.updateEntity(existing, input); + +// existing.getName() β†’ "New Name" +// existing.getShell() β†’ BASH +// existing.getId() β†’ "65f4a800..." (unchanged) +// existing.getDescription() β†’ null (cleared) +``` + +## Notes + +- Complements `ScriptServiceTest`, which mocks the mapper β€” this class tests the **real mapper instance** +- Empty list (`List.of()`) and `null` both result in a cleared collection, though they are technically distinct values +- Follows standard JUnit 5 + AssertJ patterns used across the OpenFrame test suite \ No newline at end of file diff --git a/openframe-api-service-core/src/test/java/com/openframe/api/service/rmm/.CommandDispatchServiceTest.md b/openframe-api-service-core/src/test/java/com/openframe/api/service/rmm/.CommandDispatchServiceTest.md new file mode 100644 index 000000000..0b5a7b438 --- /dev/null +++ b/openframe-api-service-core/src/test/java/com/openframe/api/service/rmm/.CommandDispatchServiceTest.md @@ -0,0 +1,50 @@ + +Unit test suite for `CommandDispatchService`, verifying NATS-based command dispatch and cancellation behavior for remote machine management (RMM) operations. + +## Key Components + +| Test Method | Purpose | +|---|---| +| `runCommand_publishesAndReturnsExecutionId` | Validates execution ID generation, correct NATS publish target, and full payload shape (`command`, `shell`, `privilegeLevel`, `timeout`) | +| `runCommand_forwardsPrivilegeLevelVerbatim` | Ensures `PrivilegeLevel` (`USER`/`ADMIN`) is passed through exactly as declared β€” no backend override | +| `runCommand_forwardsOptionalFieldsVerbatim` | Confirms optional `timeoutSeconds` is relayed to the agent unchanged | +| `runCommand_generatesDistinctExecutionIds` | Guards against global state leakage by asserting uniqueness across multiple invocations | +| `cancelExecution_publishesCancelAndEchoesExecutionId` | Verifies `CancelMessage` is published to the correct machine and the caller-supplied `executionId` is echoed back | +| `cancelExecution_doesNotGenerateNewExecutionId` | Asserts the service relays β€” never mints β€” execution IDs on cancellation | +| `cancelExecution_doesNotPublishCommand` | Enforces NATS path separation: cancel and run use distinct subjects (`publishCancel` vs `publishCommand`) | + +## Usage Example + +```java +// Test wiring β€” Mockito injects a mocked CommandNatsPublisher into the service +@ExtendWith(MockitoExtension.class) +class CommandDispatchServiceTest { + + @Mock + private CommandNatsPublisher commandNatsPublisher; + + @InjectMocks + private CommandDispatchService commandDispatchService; + + @Test + void runCommand_publishesAndReturnsExecutionId() { + RunCommandInput input = new RunCommandInput(); + input.setMachineId("machine-abc"); + input.setShell(ScriptShell.BASH); + input.setCommand("df -h"); + input.setPrivilegeLevel(PrivilegeLevel.ADMIN); + + CommandDispatchResponse response = commandDispatchService.runCommand(input); + + // Execution ID must be generated and non-blank + assertThat(response.getExecutionId()).isNotBlank(); + + // Verify exact NATS payload forwarded to the agent + ArgumentCaptor captor = ArgumentCaptor.forClass(CommandMessage.class); + verify(commandNatsPublisher).publishCommand(eq("machine-abc"), captor.capture()); + assertThat(captor.getValue().getCode()).isEqualTo("df -h"); + } +} +``` + +> **Design contract enforced:** The backend is a transparent relay β€” it generates execution IDs for `runCommand` but never overrides `privilegeLevel`, `timeout`, or `executionId` on cancel. NATS publish paths for run and cancel are strictly separated. \ No newline at end of file diff --git a/openframe-api-service-core/src/test/java/com/openframe/api/service/rmm/.ScriptServiceTest.md b/openframe-api-service-core/src/test/java/com/openframe/api/service/rmm/.ScriptServiceTest.md new file mode 100644 index 000000000..8d0cff73a --- /dev/null +++ b/openframe-api-service-core/src/test/java/com/openframe/api/service/rmm/.ScriptServiceTest.md @@ -0,0 +1,50 @@ + +Unit test suite for `ScriptService`, validating CRUD operations and cursor-based pagination behavior for RMM script management within a multi-tenant context. + +## Key Components + +| Test Method | What It Verifies | +|---|---| +| `create_whenNameUnique_persistsAndReturnsResponse` | Persists entity and returns mapped response on unique name | +| `create_whenNameAlreadyExists_throwsConflict` | Throws `ConflictException` without saving when name collides | +| `create_whenInputHasEnvVars_persistsThemThroughTheMapper` | Env vars (including secrets) pass through mapper unchanged | +| `get_whenExists_returnsResponse` | Returns mapped response for a valid tenant/script lookup | +| `get_whenNotFound_throwsNotFound` | Throws `NotFoundException` on missing script | +| `list_forwardFirstPage_dropsSentinelAndReportsHasNext` | Fetches `limit+1`, drops sentinel, sets `hasNextPage=true` | +| `list_forwardLastPage_hasNoNext` | No sentinel returned β†’ `hasNextPage=false`, `hasPreviousPage=true` | +| `list_backwardPagination_reversesItemsAndFlipsHasMore` | Reverses items for newest-first display; maps `hasMore` to `hasPreviousPage` | +| `list_emptyResult_returnsEmptyPageInfo` | Both cursors null, both page flags false on empty result | +| `list_normalisesPaginationCriteria` | Applies `CursorPaginationCriteria.normalize()` before repository fetch | + +**Mocked Dependencies:** `ScriptRepository`, `ScriptMapper`, `TenantIdProvider` + +## Usage Example + +```java +// Running all tests in this suite +@ExtendWith(MockitoExtension.class) +class ScriptServiceTest { + + @Mock ScriptRepository scriptRepository; + @Mock ScriptMapper scriptMapper; + @Mock TenantIdProvider tenantIdProvider; + + @InjectMocks ScriptService scriptService; + + @Test + void create_whenNameUnique_persistsAndReturnsResponse() { + when(scriptRepository.existsByTenantIdAndName(TENANT_ID, "Restart Spooler")) + .thenReturn(false); + when(scriptMapper.toEntity(TENANT_ID, createInput)).thenReturn(mapped); + when(scriptRepository.save(mapped)).thenReturn(saved); + when(scriptMapper.toResponse(saved)).thenReturn(response); + + ScriptResponse result = scriptService.create(createInput); + + assertThat(result).isSameAs(response); + verify(scriptRepository).save(mapped); + } +} +``` + +> **Note on secrets:** Env var secrets are currently stored in plaintext through the mapper. Encrypted storage is deferred to a future secret-management implementation. \ No newline at end of file diff --git a/openframe-authorization-service-core/src/main/java/com/openframe/authz/exception/.OwnerCannotSwitchTenantException.md b/openframe-authorization-service-core/src/main/java/com/openframe/authz/exception/.OwnerCannotSwitchTenantException.md new file mode 100644 index 000000000..02009c0dc --- /dev/null +++ b/openframe-authorization-service-core/src/main/java/com/openframe/authz/exception/.OwnerCannotSwitchTenantException.md @@ -0,0 +1,15 @@ + +A specialized exception thrown when a tenant owner attempts to switch to a different tenant, which is a prohibited operation in OpenFrame's authorization model. + +## Key Components + +- **`OwnerCannotSwitchTenantException`** β€” Extends `ConflictException` (HTTP 409), using the `ErrorCode.OWNER_CANNOT_SWITCH_TENANT` error code and embedding the offending email address in the message. + +## Usage Example + +```java +// Throw when a tenant owner attempts a tenant switch +if (user.isOwner()) { + throw new OwnerCannotSwitchTenantException(user.getEmail()); +} +``` \ No newline at end of file diff --git a/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/processor/.RegistrationProcessor.md b/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/processor/.RegistrationProcessor.md index 28862a828..63b754a80 100644 --- a/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/processor/.RegistrationProcessor.md +++ b/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/processor/.RegistrationProcessor.md @@ -1,43 +1,43 @@ - -Defines a hook-based interface for intercepting registration lifecycle events across tenant, invitation, and SSO auto-provisioning flows. + +Defines the `RegistrationProcessor` interface, providing pre/post processing hooks for tenant registration, invitation-based registration, SSO auto-provisioning, and tenant ID reservation. ## Key Components -| Method | Trigger Point | -|---|---| -| `preProcessTenantRegistration` | Before tenant registration logic executes | -| `postProcessTenantRegistration` | After a tenant and user are successfully created | -| `postProcessInvitationRegistration` | After a user registers via invitation link | -| `postProcessAutoProvision` | After SSO first-login provisioning or subsequent SSO profile refresh | +| Method | Description | +|--------|-------------| +| `preProcessTenantRegistration` | Hook called before tenant registration executes | +| `postProcessTenantRegistration` | Hook called after a tenant and user are successfully created | +| `postProcessInvitationRegistration` | Hook called after a user registers via invitation link | +| `postProcessAutoProvision` | Hook called after SSO first-login provisioning or subsequent SSO logins (supports profile picture sync) | +| `reserveTenantIdForRegistration` | Returns the tenant ID to assign; OSS default generates a UUID, SaaS implementations atomically claim a pre-provisioned cluster ID | -All methods provide default no-op implementations, making partial implementation safe β€” only override the hooks relevant to your use case. +All methods provide default no-op implementations, making partial override safe without requiring a full implementation. ## Usage Example ```java @Component -public class AuditRegistrationProcessor implements RegistrationProcessor { - - private final AuditService auditService; +public class CustomRegistrationProcessor implements RegistrationProcessor { @Override public void postProcessTenantRegistration( - Tenant tenant, - AuthUser user, - TenantRegistrationRequest request) { - auditService.logTenantCreated(tenant.getId(), user.getId()); + Tenant tenant, AuthUser user, TenantRegistrationRequest request) { + // Send welcome email, provision default resources, etc. + emailService.sendWelcome(user.getEmail(), tenant.getName()); } @Override public void postProcessAutoProvision(AuthUser user, String pictureUrl) { + // Sync SSO profile picture on every login if (pictureUrl != null) { - auditService.logProfilePictureUpdated(user.getId(), pictureUrl); + profileService.updateAvatar(user.getId(), pictureUrl); } } - // preProcessTenantRegistration and postProcessInvitationRegistration - // intentionally left as no-ops via default implementations + @Override + public String reserveTenantIdForRegistration(TenantRegistrationRequest request) { + // SaaS: claim a pre-provisioned cluster slot + return clusterPool.claimNextReady().getTenantId(); + } } -``` - -> Implement this interface to inject custom business logic β€” such as audit logging, welcome emails, or CRM sync β€” at any stage of the registration pipeline without modifying core auth flows. \ No newline at end of file +``` \ No newline at end of file diff --git a/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/user/.InvitationRegistrationService.md b/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/user/.InvitationRegistrationService.md index 38704311e..a9ee2d222 100644 --- a/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/user/.InvitationRegistrationService.md +++ b/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/user/.InvitationRegistrationService.md @@ -1,36 +1,52 @@ - -Service that handles user registration through invitation links in the OpenFrame multi-tenant authentication system. It manages tenant switching, user deactivation, and invitation acceptance workflows. + +Handles the full registration flow for invited users, including tenant resolution, existing-user conflict detection, and post-registration processing. ## Key Components -- **registerByInvitation()** - Main entry point that processes invitation registration requests -- **handleExistingActiveUser()** - Manages scenarios where user already exists in same or different tenant -- **createUserForInvitation()** - Creates new user account from invitation details -- **acceptInvitation()** - Marks invitation as accepted and triggers post-processing -- **resolveTargetTenantId()** - Determines target tenant based on local tenant configuration +| Member | Type | Description | +|--------|------|-------------| +| `registerByInvitation` | Public Method | Entry point β€” validates invitation, resolves tenant, creates or reuses user, then accepts the invitation | +| `resolveTargetTenantId` | Private Method | Returns the local tenant ID (single-tenant mode) or the invitation's tenant ID | +| `handleExistingActiveUser` | Private Method | Resolves conflicts when an active user already exists: reuse, deactivate-and-switch, or throw | +| `createUserForInvitation` | Private Method | Delegates new-user creation to `UserService` using invitation metadata | +| `acceptInvitation` | Private Method | Marks the invitation `ACCEPTED`, persists it, and triggers post-processing hooks | +| `markVerifiedQuietly` | Private Method | Marks the user's email as verified without blocking the flow on failure | ## Usage Example ```java +// Inject the service @Autowired -private InvitationRegistrationService registrationService; - -public void processUserInvitation() { - InvitationRegistrationRequest request = InvitationRegistrationRequest.builder() - .invitationId("inv-123") - .firstName("John") - .lastName("Doe") - .password("securePassword123") - .switchTenant(true) // Allow switching from current tenant - .build(); - - try { - AuthUser newUser = registrationService.registerByInvitation(request); - log.info("User registered successfully: {}", newUser.getEmail()); - } catch (UserActiveInAnotherTenantException e) { - log.error("User already active in different tenant: {}", e.getMessage()); - } -} +private InvitationRegistrationService invitationRegistrationService; + +// Build the request from a frontend payload +InvitationRegistrationRequest request = InvitationRegistrationRequest.builder() + .invitationId("inv-abc123") + .firstName("Jane") + .lastName("Doe") + .password("SecureP@ss1") + .switchTenant(false) // set true to allow cross-tenant migration + .build(); + +// Register β€” throws if invitation is invalid, expired, or tenant conflict exists +AuthUser newUser = invitationRegistrationService.registerByInvitation(request); ``` -The service supports tenant switching scenarios - if a user is already active in another tenant and `switchTenant=true`, it deactivates the existing user account before creating a new one in the target tenant. For local tenant deployments, it automatically resolves to the first available tenant. \ No newline at end of file +## Tenant-Switch Behaviour + +```mermaid +graph TD + A["registerByInvitation()"] --> B["Validate invitation"] + B --> C["Resolve target tenant"] + C --> D{"Active user\nalready exists?"} + D -->|"No"| E["Create new user"] + D -->|"Same tenant"| F["Reuse existing user"] + D -->|"Different tenant"| G{"switchTenant\n= true?"} + G -->|"Yes + OWNER role"| H["OwnerCannotSwitchTenantException"] + G -->|"Yes + non-owner"| I["Deactivate old user\nCreate new user"] + G -->|"No"| J["UserActiveInAnotherTenantException"] + E --> K["acceptInvitation()"] + F --> K + I --> K + K --> L["Return AuthUser"] +``` \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/config/.AsyncConfig.md b/openframe-client-core/src/main/java/com/openframe/client/config/.AsyncConfig.md new file mode 100644 index 000000000..7b4a1d6d0 --- /dev/null +++ b/openframe-client-core/src/main/java/com/openframe/client/config/.AsyncConfig.md @@ -0,0 +1,34 @@ + +Configures asynchronous task execution for the OpenFrame client application, enabling Spring's `@Async` support with a virtual thread-based executor for tool installation operations. + +## Key Components + +| Element | Description | +|---|---| +| `@EnableAsync` | Activates Spring's annotation-driven async processing | +| `TOOL_INSTALL_EXECUTOR` | Bean name constant (`"toolInstallExecutor"`) used to reference the executor | +| `toolInstallExecutor()` | Bean providing a `TaskExecutorAdapter` backed by a virtual thread-per-task executor | + +## Usage Example + +Inject the executor by name in any Spring-managed component to run tool installation tasks asynchronously: + +```java +import org.springframework.scheduling.annotation.Async; +import org.springframework.stereotype.Service; + +import static com.openframe.client.config.AsyncConfig.TOOL_INSTALL_EXECUTOR; + +@Service +public class ToolInstallService { + + @Async(TOOL_INSTALL_EXECUTOR) + public CompletableFuture installTool(String toolName) { + // Runs on a lightweight virtual thread + performInstall(toolName); + return CompletableFuture.completedFuture(null); + } +} +``` + +> **Note:** The executor uses `Executors.newVirtualThreadPerTaskExecutor()` (Java 21+), spawning a new virtual thread per task. This makes it ideal for I/O-bound tool installation workflows where high concurrency is needed without the overhead of a fixed thread pool. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/controller/.AgentController.md b/openframe-client-core/src/main/java/com/openframe/client/controller/.AgentController.md index 2303b4c4c..da3444d11 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/controller/.AgentController.md +++ b/openframe-client-core/src/main/java/com/openframe/client/controller/.AgentController.md @@ -1,31 +1,44 @@ - -A Spring Boot REST controller that handles agent registration functionality for the OpenFrame client system through a secure API endpoint. + +REST controller handling agent registration and reinstallation endpoints for the OpenFrame platform, exposing two authenticated operations under `/api/agents`. ## Key Components -- **AgentController**: Main REST controller class that exposes agent-related endpoints -- **register()**: POST endpoint that handles agent registration with initial key authentication -- **AgentRegistrationService**: Injected service that contains the business logic for agent registration -- **Security Headers**: Uses `X-Initial-Key` header for authentication during registration +| Element | Description | +|---|---| +| `AgentController` | `@RestController` mapped to `/api/agents` | +| `POST /api/agents/register` | Registers a new agent using an initial key | +| `POST /api/agents/reinstall` | Reinstalls an existing agent using machine ID and client secret | +| `AgentRegistrationService` | Injected service containing registration/reinstall business logic | +| `X-Initial-Key` | Required header for both endpoints β€” authenticates the registration request | +| `X-Machine-Id` | Required header for reinstall β€” identifies the target machine | +| `X-Client-Secret` | Required header for reinstall β€” authenticates the existing agent | ## Usage Example -```java -// Example request to register an agent -@PostMapping("/register") -public ResponseEntity register( - @RequestHeader("X-Initial-Key") String initialKey, - @Valid @RequestBody AgentRegistrationRequest request) { - - // Service handles validation and registration logic - AgentRegistrationResponse response = agentRegistrationService.register(initialKey, request); - return ResponseEntity.ok(response); -} - -// Example client usage (conceptual) -// POST /api/agents/register -// Headers: X-Initial-Key: your-initial-key -// Body: AgentRegistrationRequest JSON payload +**Register a new agent:** + +```bash +curl -X POST https:///api/agents/register \ + -H "X-Initial-Key: " \ + -H "Content-Type: application/json" \ + -d '{ + "hostname": "workstation-01", + "platform": "linux" + }' +``` + +**Reinstall an existing agent:** + +```bash +curl -X POST https:///api/agents/reinstall \ + -H "X-Initial-Key: " \ + -H "X-Machine-Id: " \ + -H "X-Client-Secret: " \ + -H "Content-Type: application/json" \ + -d '{ + "hostname": "workstation-01", + "platform": "linux" + }' ``` -The controller uses constructor injection via Lombok's `@RequiredArgsConstructor` and validates incoming requests with `@Valid`. All agent registration requests require an initial authentication key provided through the `X-Initial-Key` header for security purposes. \ No newline at end of file +Both endpoints return an `AgentRegistrationResponse` containing credentials or tokens needed for subsequent agent communication. Request bodies are validated via `@Valid` before reaching the service layer. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/dto/agent/.AgentRegistrationRequest.md b/openframe-client-core/src/main/java/com/openframe/client/dto/agent/.AgentRegistrationRequest.md index e1d54a2d3..2c9d6679e 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/dto/agent/.AgentRegistrationRequest.md +++ b/openframe-client-core/src/main/java/com/openframe/client/dto/agent/.AgentRegistrationRequest.md @@ -1,59 +1,49 @@ - -Data Transfer Object (DTO) used to capture all device metadata submitted by an agent during the registration process with the OpenFrame platform. + +Data Transfer Object (DTO) for agent registration requests, carrying device identification, network, hardware, and OS metadata from an OpenFrame agent to the server. ## Key Components -| Field | Type | Description | -|-------|------|-------------| -| `hostname` | `String` | Device hostname | -| `organizationId` | `String` | Target organization for the device | -| `ip` | `String` | Device IP address | -| `macAddress` | `String` | Network MAC address | -| `osUuid` | `String` | OS-level unique identifier | -| `agentVersion` | `String` | **Required** β€” agent version string | -| `status` | `DeviceStatus` | Current device status enum | -| `displayName` | `String` | Human-readable device name | -| `serialNumber` | `String` | Hardware serial number | -| `manufacturer` | `String` | Device manufacturer | -| `model` | `String` | Device model identifier | -| `type` | `DeviceType` | Device type enum | -| `osType` / `osVersion` / `osBuild` | `String` | OS details | -| `timezone` | `String` | Device timezone | -| `tags` | `List` | Tags to create and assign at registration | +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `hostname` | `String` | βœ… | Unique device hostname | +| `organizationId` | `String` | βœ… | Tenant/org scoping identifier | +| `agentVersion` | `String` | βœ… | Installed agent version string | +| `osType` | `String` | βœ… | Operating system type (e.g. `Windows`, `Linux`) | +| `ip` | `String` | ❌ | Primary IP address | +| `macAddress` | `String` | ❌ | Network adapter MAC | +| `osUuid` | `String` | ❌ | OS-level unique identifier | +| `status` | `DeviceStatus` | ❌ | Initial device status enum | +| `type` | `DeviceType` | ❌ | Device category enum | +| `osVersion` / `osBuild` | `String` | ❌ | OS version and build metadata | +| `serialNumber` / `manufacturer` / `model` | `String` | ❌ | Hardware inventory fields | +| `timezone` | `String` | ❌ | Device local timezone | +| `tags` | `List` | ❌ | Tags to create and assign at registration time | ## Usage Example ```java AgentRegistrationRequest request = new AgentRegistrationRequest(); -// Core identification -request.setHostname("workstation-01"); +// Required fields +request.setHostname("workstation-042"); request.setOrganizationId("org_abc123"); +request.setAgentVersion("2.4.1"); +request.setOsType("Windows"); -// Network -request.setIp("192.168.1.100"); +// Optional enrichment +request.setIp("192.168.1.42"); request.setMacAddress("AA:BB:CC:DD:EE:FF"); -request.setAgentVersion("2.4.1"); // @NotBlank β€” required - -// Hardware request.setManufacturer("Dell"); -request.setModel("Latitude 5520"); -request.setSerialNumber("SN-987654"); - -// OS -request.setType(DeviceType.WORKSTATION); -request.setOsType("Windows"); +request.setModel("Latitude 5540"); request.setOsVersion("11"); request.setOsBuild("22H2"); -request.setTimezone("America/New_York"); +request.setType(DeviceType.WORKSTATION); +request.setStatus(DeviceStatus.ONLINE); -// Tags (validated with @Valid) +// Tags assigned at registration AgentRegistrationTagInput tag = new AgentRegistrationTagInput(); tag.setName("managed"); request.setTags(List.of(tag)); ``` -## Validation Notes - -- `agentVersion` is the only `@NotBlank` enforced field β€” registration will be rejected if omitted. -- `tags` list is validated recursively via `@Valid`, delegating constraint checks to `AgentRegistrationTagInput`. \ No newline at end of file +> **Validation:** `@NotBlank` enforces `hostname`, `organizationId`, `agentVersion`, and `osType`. Nested `tags` are validated via `@Valid`, delegating to `AgentRegistrationTagInput` constraints. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/exception/.InvalidClientSecretException.md b/openframe-client-core/src/main/java/com/openframe/client/exception/.InvalidClientSecretException.md new file mode 100644 index 000000000..cd1d8b71b --- /dev/null +++ b/openframe-client-core/src/main/java/com/openframe/client/exception/.InvalidClientSecretException.md @@ -0,0 +1,18 @@ + +Signals that a client-provided secret is invalid during authentication, extending `UnauthorizedException` to produce a 401 Unauthorized response. + +## Key Components + +- **`InvalidClientSecretException`** β€” Extends `UnauthorizedException`; thrown when a client secret fails validation in the OpenFrame authentication flow. +- **Constructor** β€” Accepts an `ErrorCode` (for structured error identification) and a `String` message (for human-readable context), delegating both to the parent class. + +## Usage Example + +```java +if (!secretEncoder.matches(providedSecret, storedHash)) { + throw new InvalidClientSecretException( + ErrorCode.INVALID_CLIENT_SECRET, + "The provided client secret does not match our records." + ); +} +``` \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/listener/.MachineHeartbeatListener.md b/openframe-client-core/src/main/java/com/openframe/client/listener/.MachineHeartbeatListener.md index 35c1b778c..8e74deac8 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/listener/.MachineHeartbeatListener.md +++ b/openframe-client-core/src/main/java/com/openframe/client/listener/.MachineHeartbeatListener.md @@ -1,36 +1,34 @@ - -A Spring component that listens for machine heartbeat messages via NATS messaging system and processes them to track machine status. + +Subscribes to NATS machine heartbeat messages and delegates processing to `MachineStatusService`, managing the NATS dispatcher lifecycle from application startup through graceful shutdown. ## Key Components -- **`@EventListener(ApplicationReadyEvent.class)`** - Automatically subscribes to heartbeat messages when the application starts -- **`subscribeToMachineHeartbeats()`** - Sets up NATS dispatcher to listen on `machine.*.heartbeat` subject pattern -- **`handleMessage(Message)`** - Processes incoming heartbeat messages by extracting machine ID and timestamp -- **`@PreDestroy cleanup()`** - Gracefully shuts down the NATS dispatcher on application shutdown +| Member | Description | +|--------|-------------| +| `SUBJECT` | Wildcard NATS subject pattern `machine.*.heartbeat` matching all machine heartbeats | +| `subscribeToMachineHeartbeats()` | Creates a NATS dispatcher and subscribes on `ApplicationReadyEvent` | +| `handleMessage(Message)` | Extracts `machineId` from the subject, captures a server-side timestamp, and calls `machineStatusService.processHeartbeat()` | +| `cleanup()` | Drains the dispatcher with a 5-second timeout on bean destruction via `@PreDestroy` | -## Dependencies +## Usage Example -- **`MachineStatusService`** - Processes heartbeat data for machine status tracking -- **`NatsTopicMachineIdExtractor`** - Extracts machine ID from NATS topic patterns -- **NATS Connection** - Handles message subscription and delivery +The listener is auto-registered as a Spring `@Component` and requires no manual wiring. Ensure the following beans are available in the application context: -## Usage Example +```java +@Bean +public Connection natsConnection() throws Exception { + return Nats.connect("nats://localhost:4222"); +} +``` + +Once the application is ready, any message published to a subject matching `machine.*.heartbeat` is automatically handled: ```java -// The listener automatically starts when the Spring application is ready -// It subscribes to subjects like: -// - machine.001.heartbeat -// - machine.production-line-A.heartbeat -// - machine.sensor-xyz.heartbeat - -// When a heartbeat message arrives, it: -// 1. Extracts machine ID from the subject -// 2. Generates current timestamp -// 3. Calls machineStatusService.processHeartbeat(machineId, timestamp) - -// Configuration in application.yml: -// nats: -// url: nats://localhost:4222 +// Published by a remote agent or monitoring service +natsConnection.publish("machine.abc-123.heartbeat", new byte[0]); + +// Internally handled as: +// machineStatusService.processHeartbeat("abc-123", Instant.now()); ``` -The component uses Spring's lifecycle management to ensure clean startup and shutdown, with proper error handling and logging throughout the message processing pipeline. \ No newline at end of file +> **Note:** The `machineId` is extracted from the subject wildcard segment using `NatsTopicMachineIdExtractor`, not from the message payload β€” keeping the heartbeat payload schema-free and the timestamp authoritative on the server side. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/listener/.ToolConnectionListener.md b/openframe-client-core/src/main/java/com/openframe/client/listener/.ToolConnectionListener.md index 953be7312..f7e287701 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/listener/.ToolConnectionListener.md +++ b/openframe-client-core/src/main/java/com/openframe/client/listener/.ToolConnectionListener.md @@ -1,42 +1,44 @@ - -A Spring component that listens for tool connection events on a NATS JetStream subject, managing durable consumer lifecycle and delegating processing to `ToolConnectionService`. + +A Spring component that subscribes to a NATS JetStream push consumer on the `machine.*.tool-connection` subject, processes incoming tool connection messages, and delegates to `ToolConnectionService`. ## Key Components -| Member | Type | Description | -|--------|------|-------------| -| `STREAM_NAME` | Constant | Target JetStream stream (`TOOL_CONNECTIONS`) | -| `SUBJECT` | Constant | Wildcard subject pattern (`machine.*.tool-connection`) | -| `CONSUMER_NAME` | Constant | Durable consumer identifier (`tool-connection-processor-v2`) | -| `MAX_DELIVER` | Constant | Maximum redelivery attempts before message is dead-lettered (50) | -| `ACK_WAIT` | Constant | Acknowledgement timeout window (30 seconds) | -| `subscribeToToolConnections()` | Method | Bootstraps JetStream push subscription on `ApplicationReadyEvent` | -| `buildConsumerConfig()` | Method | Creates or updates the durable consumer with delivery group support | -| `handleMessage()` | Method | Deserializes and routes each `ToolConnectionMessage` | -| `cleanup()` | Method | Gracefully unsubscribes and drains the dispatcher on shutdown | +| Member | Description | +|--------|-------------| +| `STREAM_NAME` | JetStream stream name: `TOOL_CONNECTIONS` | +| `CONSUMER_NAME` | Durable consumer name: `tool-connection-processor-v2` | +| `MAX_DELIVER` / `ACK_WAIT` | Retry cap of 50 deliveries with a 30-second ack timeout | +| `subscribeToToolConnections()` | `@EventListener(ApplicationReadyEvent.class)` β€” creates the dispatcher and push subscription on startup | +| `buildConsumerConfig()` | Upserts the durable consumer; handles 404 (first-run) by creating it fresh | +| `handleMessage(Message)` | Deserializes `ToolConnectionMessage`, extracts `machineId` from the subject, calls `ToolConnectionService.addToolConnection()`, then acks on success or leaves unacked for redelivery on failure | +| `isLastAttempt(long)` | Returns `true` when `deliveredCount == MAX_DELIVER`, signaling the final retry | +| `cleanup()` | `@PreDestroy` β€” unsubscribes and drains the dispatcher with a 5-second timeout | ## Usage Example -```java -// Bean is auto-configured by Spring on startup. -// Subscription is triggered automatically via ApplicationReadyEvent. - -// Internally, messages are published to the subject pattern: -// machine..tool-connection - -// Example payload consumed by this listener: -{ - "toolType": "antivirus", - "agentToolId": "tool-abc-123" -} +The listener is auto-registered by Spring on startup β€” no manual wiring needed. -// On success: message.ack() is called. -// On failure: message is left unacked for redelivery (up to MAX_DELIVER=50 attempts). -// On last attempt: lastAttempt=true is passed to ToolConnectionService. +```java +// Automatically triggered when ApplicationReadyEvent fires. +// Internally, the subscription is equivalent to: + +ConsumerConfiguration config = ConsumerConfiguration.builder() + .durable("tool-connection-processor-v2") + .ackPolicy(AckPolicy.Explicit) + .deliverPolicy(DeliverPolicy.All) + .ackWait(Duration.ofSeconds(30)) + .maxDeliver(50) + .filterSubject("machine.*.tool-connection") + .deliverSubject("machine.tool-connection.delivery") + .deliverGroup("tool-connection") + .build(); + +PushSubscribeOptions options = PushSubscribeOptions.builder() + .stream("TOOL_CONNECTIONS") + .configuration(config) + .build(); + +js.subscribe("machine.*.tool-connection", dispatcher, this::handleMessage, false, options); ``` -## Notes - -- The `v2` consumer suffix was introduced during a hotfix to support delivery groups, which cannot be applied to existing consumers retroactively. -- Uses explicit ACK policy β€” failed processing never silently discards messages. -- `@PreDestroy` ensures clean drain of in-flight messages during shutdown with a 5-second grace period. \ No newline at end of file +> **Note on `v2` consumer suffix:** The consumer was renamed from `tool-connection-processor` to `tool-connection-processor-v2` to introduce a delivery group β€” NATS does not allow adding a delivery group to an existing consumer in-place, requiring a new consumer name for backward compatibility with already-deployed environments. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/service/.MachineStatusService.md b/openframe-client-core/src/main/java/com/openframe/client/service/.MachineStatusService.md index 5ddc7d454..193379acc 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/service/.MachineStatusService.md +++ b/openframe-client-core/src/main/java/com/openframe/client/service/.MachineStatusService.md @@ -1,37 +1,36 @@ - -Manages machine (device) connectivity status transitions within the OpenFrame platform, handling online/offline updates, heartbeat processing, and first-connection event detection with stale-event protection. + +Manages machine/device connectivity status updates within the OpenFrame platform, handling online/offline transitions, heartbeat processing, and first-connection event publishing with stale-event protection. ## Key Components | Method | Description | |--------|-------------| -| `updateToOnline()` | Marks a machine as `ONLINE` for a given timestamp | -| `updateToOffline()` | Marks a machine as `OFFLINE` for a given timestamp | -| `processHeartbeat()` | Treats an incoming heartbeat as an `ONLINE` status update | -| `update()` *(private)* | Core logic: loads the machine, validates event freshness, applies status change | -| `isEventNewer()` *(private)* | Guards against out-of-order events by comparing timestamps | -| `applyStatusUpdate()` *(private)* | Persists the new status and fires `DeviceFirstConnectedEvent` on `PENDING β†’ ONLINE/OFFLINE` transition | -| `logStaleEvent()` *(private)* | Warns when an older event arrives after a newer one has already been applied | - -## Behavior Notes - -- **Stale-event protection** β€” events with a timestamp older than `machine.lastSeen` are silently dropped with a warning log. -- **First-connection detection** β€” when a device transitions out of `PENDING` status for the first time, a `DeviceFirstConnectedEvent` is published via Spring's `ApplicationEventPublisher`. -- Throws `MachineNotFoundException` if the `machineId` does not exist in the repository. +| `updateToOnline(machineId, eventTimestamp)` | Marks a machine as `ONLINE` | +| `updateToOffline(machineId, eventTimestamp)` | Marks a machine as `OFFLINE` | +| `processHeartbeat(machineId, eventTimestamp)` | Treats a heartbeat as an `ONLINE` status update | +| `applyStatusUpdate(...)` | Persists the status change and fires `DeviceFirstConnectedEvent` when a `PENDING` device connects for the first time | +| `isEventNewer(...)` | Guards against out-of-order events by comparing timestamps | +| `logStaleEvent(...)` | Warns when an event is older than the machine's last recorded timestamp | ## Usage Example ```java -// Injected by Spring +// Injected via Spring DI @Autowired private MachineStatusService machineStatusService; -// Device comes online +// Mark a machine online when it connects machineStatusService.updateToOnline("machine-abc-123", Instant.now()); -// Device goes offline +// Process a periodic heartbeat +machineStatusService.processHeartbeat("machine-abc-123", Instant.now()); + +// Mark offline on disconnect machineStatusService.updateToOffline("machine-abc-123", Instant.now()); +``` -// Heartbeat received from agent -machineStatusService.processHeartbeat("machine-abc-123", Instant.now()); -``` \ No newline at end of file +## Notes + +- Throws `MachineNotFoundException` if the `machineId` does not exist in the repository. +- Stale events (timestamps older than `lastSeen`) are silently dropped with a `WARN` log β€” no persistence occurs. +- A `DeviceFirstConnectedEvent` is published exactly once when a device transitions out of the `PENDING` state, enabling downstream provisioning workflows. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationService.md b/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationService.md index 9a9a43fe5..62417a082 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationService.md +++ b/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationService.md @@ -1,35 +1,38 @@ - -Handles the end-to-end registration of OpenFrame agents, orchestrating OAuth client creation, machine persistence, tag assignment, tool installation, and post-processing within a single transaction. + +Handles agent registration and reinstallation within the OpenFrame platform, orchestrating OAuth client creation, machine persistence, tag assignment, and post-registration processing for managed devices. ## Key Components | Member | Type | Description | |--------|------|-------------| -| `register()` | Method | Entry point β€” validates the initial key, generates credentials, resolves the org, persists all entities, and returns the agent credentials | -| `saveOAuthClient()` | Method | Creates and persists an `OAuthClient` with hashed secret and `AGENT` role using client-credentials grant | -| `saveMachine()` | Method | Builds and persists a `Machine` document with `PENDING` status and `DESKTOP` type | -| `resolveOrganizationId()` | Method | Uses the requested org ID if it exists, otherwise falls back to the tenant's default organization | -| `saveInstalledAgent()` | Method | Optionally records the agent version via `InstalledAgentService` (feature-flagged by `openframe.feature.save-installed-agent-on-registration`) | -| `AGENT_ROLE` | Constant | Role assigned to all registered agent OAuth clients (`"AGENT"`) | -| `CLIENT_ID_TEMPLATE` | Constant | Pattern for client IDs: `agent_{machineId}` | +| `register()` | `public` | Validates the initial key, generates credentials, creates an `OAuthClient` and `Machine`, then triggers post-processing | +| `reinstall()` | `public` | Re-registers an existing agent by validating credentials, updating machine state, and re-running post-processing | +| `createOAuthClient()` | `private` | Persists a new `OAuthClient` with encoded secret, `CLIENT_CREDENTIALS` grant type, and `AGENT` role | +| `createMachine()` | `private` | Builds and saves a new `Machine` document with `PENDING` status and resolved organization ID | +| `updateMachine()` | `private` | Overwrites machine fields on reinstall and resets status to `PENDING` | +| `postProcessMachine()` | `private` | Saves installed agent record, assigns tags, triggers tool installation, and calls `AgentRegistrationProcessor` | +| `applyRegistrationRequestFields()` | `private` | Maps hostname, OS type, agent version, and `lastSeen` from the registration request onto a `Machine` | ## Usage Example ```java -// Called from a registration endpoint, typically with a pre-shared initial key +// Initial agent registration AgentRegistrationRequest request = new AgentRegistrationRequest(); request.setHostname("workstation-01"); -request.setIp("192.168.1.50"); request.setOsType("WINDOWS"); -request.setAgentVersion("1.4.2"); -request.setOrganizationId("org-abc123"); // optional; falls back to default org +request.setAgentVersion("2.4.1"); +request.setOrganizationId("org-abc"); +request.setTags(List.of("prod", "us-east")); AgentRegistrationResponse response = agentRegistrationService.register(initialKey, request); -// Response contains credentials the agent uses for subsequent API calls -String machineId = response.getMachineId(); -String clientId = response.getClientId(); // e.g. "agent_" -String clientSecret = response.getClientSecret(); +String machineId = response.getMachineId(); // e.g. "mch_a1b2c3" +String clientId = response.getClientId(); // e.g. "agent_mch_a1b2c3" +String clientSecret = response.getClientSecret(); // raw secret (store securely) + +// Reinstall on an already-registered machine +AgentRegistrationResponse reinstallResponse = + agentRegistrationService.reinstall(initialKey, machineId, clientSecret, request); ``` -> **Note:** The entire `register()` flow runs inside a `@Transactional` boundary. A TODO marks a pending two-phase commit strategy for the NATS integration to handle distributed consistency. \ No newline at end of file +> **Security note:** `clientSecret` is returned in plaintext only at registration time. It is stored hashed via `PasswordEncoder` and cannot be recovered β€” treat it like a password and store it immediately. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationToolInstallationService.md b/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationToolInstallationService.md index 0a4107be7..08df7c37b 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationToolInstallationService.md +++ b/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationToolInstallationService.md @@ -1,34 +1,28 @@ - -A service that coordinates tool agent installation during the agent registration process by retrieving all enabled tool agents and installing them on a specified machine. + +Handles asynchronous tool installation on newly registered agents by retrieving all enabled integrated tool agents and triggering their installation process on the specified machine. ## Key Components -- **process(String machineId)** - Main orchestration method that retrieves all enabled tool agents and triggers installation for each one -- **integratedToolAgentService** - Service dependency for managing integrated tool agent data -- **toolInstallationService** - Service dependency that handles the actual tool installation process +- **`process(String machineId)`** β€” Async method (executed on `TOOL_INSTALL_EXECUTOR` thread pool) that fetches all enabled `IntegratedToolAgent` records and invokes `ToolInstallationService.process()` for each one against the target machine. ## Usage Example ```java -@Autowired -private AgentRegistrationToolInstallationService installationService; - -// Install all enabled tools on a newly registered machine -public void registerNewAgent(String machineId) { - // ... other registration logic - - // Install all available tools - installationService.process(machineId); - - log.info("Tool installation completed for machine: {}", machineId); -} +// Called during agent registration flow +@Service +public class AgentRegistrationService { + + private final AgentRegistrationToolInstallationService toolInstallationService; -// Example in a registration controller -@PostMapping("/register") -public ResponseEntity registerAgent(@RequestParam String machineId) { - installationService.process(machineId); - return ResponseEntity.ok("Agent registered with tools installed"); + public void onAgentRegistered(String machineId) { + // Non-blocking: executes on TOOL_INSTALL_EXECUTOR thread pool + toolInstallationService.process(machineId); + } } ``` -The service follows a simple orchestration pattern where it retrieves all enabled tool agents from the data layer and delegates the actual installation work to the `ToolInstallationService` for each tool-machine combination. \ No newline at end of file +## Notes + +- Runs asynchronously via `@Async(AsyncConfig.TOOL_INSTALL_EXECUTOR)`, so it does not block the agent registration request thread. +- Installation is applied to **all enabled** tool agents β€” no per-machine filtering at this layer. +- Ordering and error handling per tool agent are delegated to `ToolInstallationService`. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.OrganizationIdResolver.md b/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.OrganizationIdResolver.md new file mode 100644 index 000000000..888806142 --- /dev/null +++ b/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.OrganizationIdResolver.md @@ -0,0 +1,36 @@ + +Resolves the target organization ID during agent registration, falling back to the default organization when the requested ID is absent or not found. + +## Key Components + +- **`resolve(String requestedOrganizationId)`** β€” Public entry point. Returns the requested organization ID if it exists, otherwise logs a warning and falls back to the default organization ID. +- **`existsRequested(String)`** *(private)* β€” Validates that the provided ID is non-blank and matches a persisted organization via `OrganizationService`. +- **`getDefaultOrganizationId()`** *(private)* β€” Fetches the default organization and extracts its ID; throws `IllegalStateException` if no default organization exists. + +## Usage Example + +```java +@Service +@RequiredArgsConstructor +public class AgentRegistrationService { + + private final OrganizationIdResolver organizationIdResolver; + + public void registerAgent(AgentRegistrationRequest request) { + // Resolves to the provided ID if valid, or falls back to the default org + String organizationId = organizationIdResolver.resolve(request.getOrganizationId()); + + // Proceed with registration using the resolved organization ID + agentRepository.save(new Agent(organizationId, request.getAgentName())); + } +} +``` + +## Behavior Summary + +| Scenario | Result | +|---|---| +| Valid, existing `organizationId` provided | Returns the provided ID | +| Non-blank ID provided but not found in DB | Logs a warning, returns default org ID | +| Blank/null `organizationId` provided | Returns default org ID silently | +| No default organization exists | Throws `IllegalStateException` | \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/service/validator/.ClientSecretValidator.md b/openframe-client-core/src/main/java/com/openframe/client/service/validator/.ClientSecretValidator.md new file mode 100644 index 000000000..7586169fc --- /dev/null +++ b/openframe-client-core/src/main/java/com/openframe/client/service/validator/.ClientSecretValidator.md @@ -0,0 +1,35 @@ + +Validates OAuth client secrets by checking for presence and verifying against the stored encoded secret using a `PasswordEncoder`. + +## Key Components + +| Element | Description | +|---|---| +| `ClientSecretValidator` | Spring `@Component` responsible for OAuth client secret validation | +| `validate(OAuthClient, String)` | Core method that enforces secret presence and BCrypt match | +| `InvalidClientSecretException` | Thrown on empty (`CLIENT_SECRET_EMPTY`) or mismatched (`CLIENT_SECRET_INVALID`) secrets | +| `PasswordEncoder` | Injected Spring Security encoder used for hash comparison | + +## Usage Example + +```java +@Service +@RequiredArgsConstructor +public class OAuthTokenService { + + private final ClientSecretValidator clientSecretValidator; + private final OAuthClientRepository clientRepository; + + public TokenResponse issueToken(String clientId, String clientSecret) { + OAuthClient client = clientRepository.findByClientId(clientId) + .orElseThrow(() -> new ClientNotFoundException("Unknown client")); + + // Throws InvalidClientSecretException if empty or hash mismatch + clientSecretValidator.validate(client, clientSecret); + + return generateToken(client); + } +} +``` + +> **Note:** The raw `clientSecret` is never stored β€” only the encoded form retrieved from `OAuthClient.getClientSecret()` is compared, keeping credentials safe at rest. \ No newline at end of file diff --git a/openframe-client-core/src/test/java/com/openframe/client/controller/.AgentControllerTest.md b/openframe-client-core/src/test/java/com/openframe/client/controller/.AgentControllerTest.md index 13e32af0b..362e1ade6 100644 --- a/openframe-client-core/src/test/java/com/openframe/client/controller/.AgentControllerTest.md +++ b/openframe-client-core/src/test/java/com/openframe/client/controller/.AgentControllerTest.md @@ -1,47 +1,48 @@ - -Unit test suite for the `AgentController`, validating HTTP behavior of the `/api/agents/register` endpoint using MockMvc with mocked service dependencies. + +Unit tests for `AgentController`, verifying the HTTP layer behavior of agent registration and reinstallation endpoints using MockMvc with mocked `AgentRegistrationService`. ## Key Components -| Component | Description | -|-----------|-------------| -| `AgentController` | Controller under test, wired with a mocked `AgentRegistrationService` | -| `AgentRegistrationService` | Mocked service handling agent registration logic | -| `AgentRegistrationRequest` | DTO with hostname, IP, MAC address, OS UUID, and agent version | -| `AgentRegistrationResponse` | DTO returning `machineId`, `clientId`, and `clientSecret` | -| `BaseGlobalExceptionHandler` | Applied via `setControllerAdvice` to validate error response shapes | -| `TestAuthenticationManager` | Stub authentication manager injected via `BasicAuthenticationFilter` | +- **`AgentControllerTest`** β€” Main test class using `@ExtendWith(MockitoExtension.class)` with standalone MockMvc setup +- **`AgentRegistrationService`** (mock) β€” Mocked service dependency injected into `AgentController` +- **`TestAuthenticationManager`** β€” Custom auth manager added as a `BasicAuthenticationFilter` for test context +- **`BaseGlobalExceptionHandler`** β€” Wired as controller advice to validate error response shapes ## Test Coverage -| Test | Expected Outcome | -|------|-----------------| -| `register_WithValidRequest_ReturnsOk` | `200 OK` with full credential payload | -| `register_MissingHeader_ReturnsBadRequest` | `400 BAD_REQUEST` when `X-Initial-Key` is absent | -| `register_WithoutInitialKey_ReturnsBadRequest` | `400 BAD_REQUEST` (duplicate missing-header scenario) | -| `register_WithInvalidInitialKey_ReturnsUnauthorized` | `401 UNAUTHORIZED` on secret validation failure | -| `register_WithDuplicateMachineId_ReturnsConflict` | `409 CONFLICT` when machine is already registered | -| `register_WithValidRequest_ReturnsCredentials` | `200 OK` with `clientId` and `clientSecret` | -| `register_WithValidRequest_StoresAgentInfo` | Verifies service receives all request fields correctly | +| Scenario | Endpoint | Expected Status | +|---|---|---| +| Valid registration request | `POST /api/agents/register` | `200 OK` | +| Missing `X-Initial-Key` header | `POST /api/agents/register` | `400 Bad Request` | +| Invalid initial key | `POST /api/agents/register` | `401 Unauthorized` | +| Duplicate machine registration | `POST /api/agents/register` | `409 Conflict` | +| Missing hostname field | `POST /api/agents/register` | `400 Validation Error` | +| Valid reinstall request | `POST /api/agents/reinstall` | `200 OK` | +| Missing `X-Machine-Id` header | `POST /api/agents/reinstall` | `400 Bad Request` | +| Missing `X-Client-Secret` header | `POST /api/agents/reinstall` | `400 Bad Request` | ## Usage Example ```java -// Example: how the service mock is configured and verified -when(agentRegistrationService.register(eq("test-key"), any(AgentRegistrationRequest.class))) - .thenReturn(registrationResponse); +// Verify service receives correct registration payload +verify(agentRegistrationService).register( + eq("test-key"), + argThat(request -> + request.getHostname().equals("test-host") && + request.getIp().equals("192.168.1.1") && + request.getMacAddress().equals("00:11:22:33:44:55") + ) +); -mockMvc.perform(post("/api/agents/register") +// Reinstall with all required headers +mockMvc.perform(post("/api/agents/reinstall") .header("X-Initial-Key", "test-key") + .header("X-Machine-Id", "m-1") + .header("X-Client-Secret", "secret-1") .contentType(MediaType.APPLICATION_JSON) .content(objectMapper.writeValueAsString(registrationRequest))) .andExpect(status().isOk()) .andExpect(jsonPath("$.machineId").value("test-machine-id")); +``` -// Verify all fields are passed through to the service -verify(agentRegistrationService).register( - eq("test-key"), - argThat(req -> req.getHostname().equals("test-host") && - req.getMacAddress().equals("00:11:22:33:44:55")) -); -``` \ No newline at end of file +> **Note:** The test class is truncated β€” `reinstall_WithInvalidClientSecret_*` and any subsequent tests are not shown but follow the same pattern of header validation and service exception mapping. \ No newline at end of file diff --git a/openframe-client-core/src/test/java/com/openframe/client/service/.AgentRegistrationServiceTest.md b/openframe-client-core/src/test/java/com/openframe/client/service/.AgentRegistrationServiceTest.md index 71c0c8f8e..aa3ce6684 100644 --- a/openframe-client-core/src/test/java/com/openframe/client/service/.AgentRegistrationServiceTest.md +++ b/openframe-client-core/src/test/java/com/openframe/client/service/.AgentRegistrationServiceTest.md @@ -1,40 +1,36 @@ - -Unit test suite for `AgentRegistrationService`, validating agent registration flows including new machine creation, duplicate detection, and tag assignment. + +Unit test suite for `AgentRegistrationService`, validating agent registration and reinstallation workflows including credential generation, machine persistence, tag assignment, and validation ordering. ## Key Components | Test Method | Purpose | |---|---| -| `registerAgent_WithNewMachine_ReturnsCredentials` | Verifies full registration lifecycle: credential generation, OAuth client persistence, and machine document creation | -| `registerAgent_WithExistingMachine_ThrowsException` | Ensures duplicate machine detection throws `IllegalStateException` and prevents any saves | -| `registerAgent_WithTags_AssignsTagsToDevice` | Confirms tag assignment is delegated to `RegistrationTagAssignmentService` when tags are present | -| `registerAgent_WithoutTags_DoesNotAssignTags` | Confirms `assignTags` is still called with a `null` tag list when no tags are provided | +| `registerAgent_WithNewMachine_ReturnsCredentials` | Verifies full registration flow: ID generation, OAuth client/machine persistence, credential response, and installed agent tracking | +| `registerAgent_WithExistingMachine_ThrowsException` | Ensures duplicate machine IDs are rejected with `IllegalStateException` and no save operations occur | +| `registerAgent_WithTags_AssignsTagsToDevice` | Confirms tag assignment is delegated to `RegistrationTagAssignmentService` when tags are provided | +| `registerAgent_WithoutTags_DoesNotAssignTags` | Validates that `assignTags` is called with `null` when the request contains no tags | +| `reinstall_WithValidKeyAndSecret_OverwritesMachineAndReturnsExistingCreds` | Verifies reinstall reuses existing credentials, resets machine state to `PENDING`, resolves new org ID, and enforces validator ordering | -**Mocked dependencies:** `OAuthClientRepository`, `MachineRepository`, `OrganizationService`, `AgentRegistrationSecretValidator`, `AgentSecretGenerator`, `MachineIdGenerator`, `PasswordEncoder`, `RegistrationTagAssignmentService`, `InstalledAgentService`, `AgentRegistrationToolInstallationService`, `AgentRegistrationProcessor` +**Mocked Dependencies:** `OAuthClientRepository`, `MachineRepository`, `AgentRegistrationSecretValidator`, `ClientSecretValidator`, `AgentSecretGenerator`, `MachineIdGenerator`, `OrganizationIdResolver`, `RegistrationTagAssignmentService`, `InstalledAgentService`, `PasswordEncoder`, `AgentRegistrationProcessor`, `AgentRegistrationToolInstallationService` ## Usage Example ```java -// Simulate a new agent registration with tags -AgentRegistrationRequest request = new AgentRegistrationRequest(); -request.setHostname("workstation-01"); -request.setIp("10.0.0.5"); -request.setMacAddress("AA:BB:CC:DD:EE:FF"); -request.setOsUuid("os-uuid-xyz"); -request.setAgentVersion("2.1.0"); -request.setTags(List.of( - AgentRegistrationTagInput.builder() - .key("site") - .values(List.of("NEW_YORK")) - .build() -)); - -// Expected: response contains machineId, clientId ("agent_"), and clientSecret -AgentRegistrationResponse response = agentRegistrationService.register("initial-key", request); +// Typical new agent registration assertions pattern +when(machineIdGenerator.generate()).thenReturn(MACHINE_ID); +when(oauthClientRepository.existsByMachineId(MACHINE_ID)).thenReturn(false); +when(agentSecretGenerator.generate()).thenReturn(CLIENT_SECRET); +when(passwordEncoder.encode(CLIENT_SECRET)).thenReturn("encoded-secret"); + +AgentRegistrationResponse response = agentRegistrationService.register(INITIAL_KEY, request); + +assertEquals("agent_" + MACHINE_ID, response.getClientId()); +assertEquals(CLIENT_SECRET, response.getClientSecret()); + +// Reinstall enforces validator ordering +InOrder inOrder = inOrder(agentRegistrationSecretValidator, clientSecretValidator); +inOrder.verify(agentRegistrationSecretValidator).validate(INITIAL_KEY); +inOrder.verify(clientSecretValidator).validate(existingClient, CLIENT_SECRET); ``` -**Key assertions validated across tests:** -- `clientId` is always prefixed as `agent_` -- Saved `OAuthClient` uses `client_credentials` grant type and `AGENT` role -- Saved `Machine` defaults to `DeviceStatus.PENDING` with a non-null `lastSeen` timestamp -- Secret validator is always invoked before any persistence \ No newline at end of file +> **Key invariants tested:** Reinstall never generates new machine IDs or secrets (`machineIdGenerator` and `agentSecretGenerator` are never called), machine status resets to `DeviceStatus.PENDING`, and the initial key is always validated before the client secret. \ No newline at end of file diff --git a/openframe-client-core/src/test/java/com/openframe/client/service/agentregistration/.OrganizationIdResolverTest.md b/openframe-client-core/src/test/java/com/openframe/client/service/agentregistration/.OrganizationIdResolverTest.md new file mode 100644 index 000000000..6ccb58abb --- /dev/null +++ b/openframe-client-core/src/test/java/com/openframe/client/service/agentregistration/.OrganizationIdResolverTest.md @@ -0,0 +1,44 @@ + +Unit tests for `OrganizationIdResolver`, verifying organization ID resolution logic including fallback-to-default and error handling behaviors. + +## Key Components + +| Element | Description | +|---------|-------------| +| `resolve_WithExistingRequestedOrg_ReturnsIt` | Confirms a known org ID is returned directly, skipping default lookup | +| `resolve_WithUnknownRequestedOrg_FallsBackToDefault` | Verifies fallback to the default organization when the requested org doesn't exist | +| `resolve_WithBlank_ReturnsDefaultWithoutLookup` | Ensures blank/whitespace input skips the org lookup and returns the default | +| `resolve_WhenNoDefaultOrganization_Throws` | Asserts `IllegalStateException` is thrown when no default organization is configured | +| `organization()` | Helper factory method building a minimal `Organization` stub for test assertions | + +## Usage Example + +```java +// Typical test flow: known org returns directly +when(organizationService.getOrganizationByOrganizationId("org-1")) + .thenReturn(Optional.of(organization("org-1"))); + +String result = resolver.resolve("org-1"); +assertEquals("org-1", result); +verify(organizationService, never()).getDefaultOrganization(); + +// Fallback: unknown org resolves to default +when(organizationService.getOrganizationByOrganizationId("org-x")) + .thenReturn(Optional.empty()); +when(organizationService.getDefaultOrganization()) + .thenReturn(Optional.of(organization("default-uuid"))); + +String result = resolver.resolve("org-x"); +assertEquals("default-uuid", result); +``` + +## Resolution Behavior Summary + +```text +Input β†’ Lookup attempted? β†’ Result +───────────────────────────────────────────── +Known org ID β†’ Yes β†’ Requested org ID +Unknown org β†’ Yes, then default β†’ Default org ID +Blank/null β†’ No β†’ Default org ID +No default β†’ β€” β†’ IllegalStateException +``` \ No newline at end of file diff --git a/openframe-client-core/src/test/java/com/openframe/client/service/validator/.ClientSecretValidatorTest.md b/openframe-client-core/src/test/java/com/openframe/client/service/validator/.ClientSecretValidatorTest.md new file mode 100644 index 000000000..693831ae6 --- /dev/null +++ b/openframe-client-core/src/test/java/com/openframe/client/service/validator/.ClientSecretValidatorTest.md @@ -0,0 +1,39 @@ + +Unit tests for `ClientSecretValidator`, verifying secret validation logic across three scenarios: matching secrets, blank input, and mismatched secrets. + +## Key Components + +| Component | Description | +|-----------|-------------| +| `ClientSecretValidator` | Subject under test β€” validates raw client secrets against stored encoded secrets | +| `PasswordEncoder` | Mocked Spring Security component used for BCrypt/hash comparison | +| `InvalidClientSecretException` | Expected exception carrying an `ErrorCode` on validation failure | +| `ErrorCode.CLIENT_SECRET_EMPTY` | Error code asserted when the provided secret is blank/whitespace | +| `ErrorCode.CLIENT_SECRET_INVALID` | Error code asserted when the secret does not match the stored hash | + +## Test Coverage + +| Test Method | Scenario | Expected Outcome | +|-------------|----------|-----------------| +| `validate_WithMatchingSecret_Passes` | Raw secret matches encoded secret | No exception thrown | +| `validate_WithBlankSecret_ThrowsEmpty` | Secret is blank (`" "`) | `InvalidClientSecretException` with `CLIENT_SECRET_EMPTY`; `PasswordEncoder.matches` never called | +| `validate_WithWrongSecret_ThrowsInvalid` | Raw secret does not match hash | `InvalidClientSecretException` with `CLIENT_SECRET_INVALID` | + +## Usage Example + +```java +// Mirrors how ClientSecretValidator is exercised in tests +OAuthClient client = new OAuthClient(); +client.setClientSecret("$2a$10$encodedHash..."); + +// Passes silently when secrets match +validator.validate(client, "correct-raw-secret"); + +// Throws InvalidClientSecretException (CLIENT_SECRET_EMPTY) for blank input +validator.validate(client, " "); + +// Throws InvalidClientSecretException (CLIENT_SECRET_INVALID) for wrong secret +validator.validate(client, "wrong-secret"); +``` + +> **Note:** The blank-secret test also asserts that `PasswordEncoder.matches` is **never invoked**, confirming the validator short-circuits before hash comparison when input is empty. \ No newline at end of file diff --git a/openframe-data-device-aspect/src/main/java/com/openframe/data/service/impl/.MachineTagEventServiceImpl.md b/openframe-data-device-aspect/src/main/java/com/openframe/data/service/impl/.MachineTagEventServiceImpl.md index 9c3540bb2..e18d97ded 100644 --- a/openframe-data-device-aspect/src/main/java/com/openframe/data/service/impl/.MachineTagEventServiceImpl.md +++ b/openframe-data-device-aspect/src/main/java/com/openframe/data/service/impl/.MachineTagEventServiceImpl.md @@ -1,18 +1,22 @@ - -Handles machine tag lifecycle events by processing save and delete operations for machines, tag assignments, and tags, then publishing enriched `MachinePinotMessage` payloads to a Kafka topic for downstream analytics. + +Handles machine tag events by processing changes to machines, tag assignments, and tags, then publishing the resulting `MachinePinotMessage` payloads to a Kafka topic for downstream consumption. ## Key Components | Method | Description | |---|---| -| `processMachineSave` | Publishes a single machine event to Kafka with its current tag state | -| `processMachineSaveAll` | Iterates and publishes Kafka events for a batch of machines | -| `processTagAssignmentSave` | Handles a single `DEVICE`-type tag assignment, publishing the updated machine state | -| `processTagAssignmentSaveAll` | Batch processes tag assignments, deduplicating by `entityId` to avoid redundant messages | -| `processTagSave` | Propagates a tag update to all machines that carry that tag | -| `processTagSaveAll` | Batch processes tag updates across all affected machines | -| `processTagAssignmentDelete` | Rebuilds and publishes a machine's tag state after a specific tag assignment is removed | -| `processTagAssignmentDeleteByTagId` | Bulk recomputes and publishes updated state for all machines affected by a tag deletion | +| `processMachineSave` | Sends a Kafka event when a single `Machine` is saved | +| `processMachineSaveAll` | Iterates and sends Kafka events for a batch of saved machines | +| `processTagAssignmentSave` | Publishes a machine update event when a `DEVICE`-type `TagAssignment` is saved | +| `processTagAssignmentSaveAll` | Batch-processes tag assignments, deduplicating by `entityId` | +| `processTagSave` | Propagates tag updates to all machines that hold the tag | +| `processTagSaveAll` | Batch version of `processTagSave` | +| `processTagAssignmentDelete` | Publishes an updated machine message after removing a specific tag assignment | +| `processTagAssignmentDeleteByTagId` | Bulk-removes a tag from all affected machines and republishes each machine message | +| `sendMachineEventToKafka` *(private)* | Fetches current tags and builds/publishes a `MachinePinotMessage` | +| `buildMachinePinotMessageFromParts` *(private)* | Constructs a `MachinePinotMessage` from pre-fetched machine, tags, and assignments | + +**Activation:** Enabled only when `openframe.device.aspect.enabled=true` (default: enabled). ## Usage Example @@ -21,15 +25,15 @@ Handles machine tag lifecycle events by processing save and delete operations fo @Autowired private MachineTagEventService machineTagEventService; -// Publish a machine event after save +// Trigger Kafka event after saving a machine Machine machine = machineRepository.save(newMachine); machineTagEventService.processMachineSave(machine); -// Publish updated state after removing a tag from a machine +// Trigger Kafka event after deleting a tag assignment machineTagEventService.processTagAssignmentDelete(machineId, tagId); -// Propagate tag metadata changes to all machines using that tag +// Propagate tag metadata change to all affected machines machineTagEventService.processTagSave(updatedTag); ``` -> **Note:** This service is conditionally enabled via `openframe.device.aspect.enabled=true` (defaults to `true`). It is typically invoked from AOP aspects or repository event listeners rather than called directly. \ No newline at end of file +> **Note:** All public methods catch exceptions internally and log errors without rethrowing, ensuring individual failures do not block batch operations. \ No newline at end of file diff --git a/openframe-data-kafka/src/main/java/com/openframe/kafka/producer/.GenericKafkaProducer.md b/openframe-data-kafka/src/main/java/com/openframe/kafka/producer/.GenericKafkaProducer.md index ae0551fdc..8f4c7d678 100644 --- a/openframe-data-kafka/src/main/java/com/openframe/kafka/producer/.GenericKafkaProducer.md +++ b/openframe-data-kafka/src/main/java/com/openframe/kafka/producer/.GenericKafkaProducer.md @@ -1,16 +1,16 @@ - -Abstract base class for Kafka message producers, providing both async (fire-and-forget) and synchronous (blocking) send operations with structured error classification, logging, and exception handling. + +Abstract base class for Kafka producers providing both async (fire-and-forget) and synchronous (blocking) message sending with structured error classification, retry signaling, and safe logging. ## Key Components | Member | Description | -|---|---| -| `sendAsync()` | Non-blocking send that returns a `CompletableFuture`; logs success/failure via `whenComplete` without altering the future's state | -| `sendAndAwait()` | Blocking send that joins the async future and classifies exceptions into `NonRetryableKafkaException` (fatal) or `TransientKafkaSendException` (retryable) | -| `handleSendFailure()` | Pattern-matched switch that logs authorization errors, oversized records, timeouts, retriable failures, and generic errors with appropriate log levels | +|--------|-------------| +| `sendAsync()` | Non-blocking send; returns a `CompletableFuture` and logs success/failure via `whenComplete` without altering the future's state | +| `sendAndAwait()` | Blocking send that joins the future and classifies exceptions into `NonRetryableKafkaException` (fatal) or `TransientKafkaSendException` (retryable) | +| `handleSendFailure()` | Pattern-matched async logging for `TopicAuthorizationException`, `RecordTooLargeException`, `TimeoutException`, `RetriableException`, and generic failures | | `validateTopic()` | Guards against null/blank topic names, throwing `NonRetryableKafkaException` immediately | -| `redact()` | Masks message keys in logs, exposing only the first 3 characters to avoid leaking sensitive data | -| `abbreviate()` | Truncates payload strings to 500 characters in log output | +| `redact()` | Masks message keys in logs (shows only first 3 chars) to avoid leaking sensitive routing data | +| `abbreviate()` | Truncates payload strings to 500 chars in log output | ## Usage Example @@ -24,27 +24,17 @@ public class OrderEventProducer extends GenericKafkaProducer { super(kafkaTemplate); } - // Fire-and-forget - public CompletableFuture> publishAsync(OrderEvent event) { + // Fire-and-forget β€” caller handles the future + public CompletableFuture> publishAsync(OrderCreatedEvent event) { return sendAsync(TOPIC, event.getOrderId(), event); } - // Blocking β€” suitable for use with @Retryable + // Blocking β€” use with @Retryable for transient failures @Retryable(retryFor = TransientKafkaSendException.class, maxAttempts = 3) - public void publishAndAwait(OrderEvent event) { + public void publish(OrderCreatedEvent event) { sendAndAwait(TOPIC, event.getOrderId(), event); } } ``` -## Exception Classification - -```text -CompletionException cause -β”œβ”€β”€ RecordTooLargeException β†’ NonRetryableKafkaException (fatal, do not retry) -β”œβ”€β”€ AuthorizationException β†’ NonRetryableKafkaException (fatal, do not retry) -β”œβ”€β”€ SerializationException β†’ NonRetryableKafkaException (fatal, do not retry) -β”œβ”€β”€ InvalidTopicException β†’ NonRetryableKafkaException (fatal, do not retry) -β”œβ”€β”€ UnknownTopicOrPartitionException β†’ TransientKafkaSendException (topic may be auto-creating) -└── Everything else β†’ TransientKafkaSendException (retryable) -``` \ No newline at end of file +> **Exception contract:** `sendAndAwait` throws `NonRetryableKafkaException` for fatal conditions (`RecordTooLargeException`, `AuthorizationException`, `SerializationException`, `InvalidTopicException`) and `TransientKafkaSendException` for all retryable cases, including `UnknownTopicOrPartitionException` (broker auto-create in progress). Pair with Spring Retry's `@Retryable` targeting `TransientKafkaSendException` to avoid retrying fatal errors. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/.TenantScoped.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/.TenantScoped.md new file mode 100644 index 000000000..fd506aa2e --- /dev/null +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/.TenantScoped.md @@ -0,0 +1,35 @@ + +A marker interface for domain objects that belong to a specific tenant, enabling multi-tenancy support across OpenFrame data models. + +## Key Components + +| Member | Type | Description | +|--------|------|-------------| +| `getTenantId()` | `String` | Returns the tenant identifier for this document | +| `setTenantId(String tenantId)` | `void` | Assigns a tenant identifier to this document | + +## Usage Example + +```java +@Document +public class Ticket implements TenantScoped { + private String tenantId; + + @Override + public String getTenantId() { + return tenantId; + } + + @Override + public void setTenantId(String tenantId) { + this.tenantId = tenantId; + } +} + +// Enforce tenant isolation in queries +public List findForCurrentTenant(String tenantId) { + return ticketRepository.findAllByTenantId(tenantId); +} +``` + +> Any document class implementing `TenantScoped` can be automatically filtered by tenant context, ensuring data isolation between MSP clients in the OpenFrame platform. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/agent/.AgentRegistrationSecret.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/agent/.AgentRegistrationSecret.md index 72f96fed2..96178f4f5 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/agent/.AgentRegistrationSecret.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/agent/.AgentRegistrationSecret.md @@ -1,33 +1,32 @@ - -A MongoDB document entity representing a registration secret used to authenticate and register agents within the OpenFrame platform. + +MongoDB document entity representing a secret key used for agent registration, scoped to a specific tenant. ## Key Components -| Field | Type | Description | -|-------|------|-------------| -| `id` | `String` | Primary identifier (`@Id`) for the document | -| `secretKey` | `String` | Unique indexed secret key used for agent registration | -| `createdAt` | `Instant` | Timestamp recording when the secret was created | -| `active` | `boolean` | Flag indicating whether the registration secret is currently valid | - -- **Collection:** `agent_registration_secrets` -- **Annotations:** Lombok `@Data` auto-generates getters, setters, `equals`, `hashCode`, and `toString` +| Element | Description | +|---|---| +| `@Document` | Maps to the `agent_registration_secrets` MongoDB collection | +| `@CompoundIndex` | Unique index on `tenantId` + `secretKey` to prevent duplicate secrets per tenant | +| `TenantScoped` | Interface ensuring all queries are tenant-isolated | +| `secretKey` | The registration secret presented by agents during onboarding | +| `active` | Flag to enable/disable a secret without deletion | +| `createdAt` | Audit timestamp recorded when the secret is issued | ## Usage Example ```java -// Creating a new agent registration secret AgentRegistrationSecret secret = new AgentRegistrationSecret(); +secret.setTenantId("tenant-abc"); secret.setSecretKey(UUID.randomUUID().toString()); secret.setCreatedAt(Instant.now()); secret.setActive(true); -// Persisting via a Spring Data MongoDB repository -AgentRegistrationSecret saved = agentRegistrationSecretRepository.save(secret); +// Persist via repository +agentRegistrationSecretRepository.save(secret); -// Deactivating an existing secret -saved.setActive(false); -agentRegistrationSecretRepository.save(saved); +// Lookup by tenant + key during agent handshake +Optional found = agentRegistrationSecretRepository + .findByTenantIdAndSecretKeyAndActiveTrue("tenant-abc", incomingKey); ``` -> **Note:** The `secretKey` field is enforced as unique at the database index level, preventing duplicate registration secrets across agents in the OpenFrame platform. \ No newline at end of file +> Secrets should be rotated periodically. Set `active = false` to revoke a secret without removing audit history. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKey.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKey.md index d1bd1d4a8..9864c603b 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKey.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKey.md @@ -1,42 +1,45 @@ - -MongoDB document entity representing an API key stored in the `api_keys` collection, including metadata, permission scopes, and expiry logic. + +MongoDB document entity representing an API key with tenant scoping, permission management, and lifecycle state tracking. ## Key Components | Member | Type | Description | |--------|------|-------------| -| `keyId` | `String` | MongoDB `@Id` β€” primary identifier for the key | -| `hashedKey` | `String` | Bcrypt/hashed representation of the raw API key | -| `name` / `description` | `String` | Human-readable label and description | -| `userId` | `String` | Indexed owner reference linking the key to a user | -| `scopes` | `List` | Fine-grained permission scopes *(TODO: enforced)* | -| `roles` | `List` | Role assignments *(TODO: enforced)* | -| `enabled` | `boolean` | Soft-disable flag, defaults to `true` | -| `createdAt` / `updatedAt` / `expiresAt` | `Instant` | Lifecycle timestamps | +| `keyId` | `String` | Primary identifier (`@Id`) for the API key document | +| `tenantId` | `String` | Indexed tenant reference (implements `TenantScoped`) | +| `hashedKey` | `String` | Stored hash of the raw API key secret | +| `userId` | `String` | Indexed owner/creator of the key | +| `scopes` / `roles` | `List` | Permission grants (TODO: enforcement pending) | +| `enabled` | `boolean` | Manual activation flag, defaults to `true` | +| `expiresAt` | `Instant` | Optional expiration timestamp | | `isExpired()` | `boolean` | Returns `true` if `expiresAt` is set and in the past | -| `isActive()` | `boolean` | Returns `true` if `enabled && !isExpired()` | +| `isActive()` | `boolean` | Returns `true` only when `enabled && !isExpired()` | ## Usage Example ```java -// Build a new API key document +// Create a new API key document ApiKey apiKey = ApiKey.builder() - .keyId(UUID.randomUUID().toString()) - .hashedKey(hashService.hash(rawKey)) + .tenantId("tenant-abc") + .userId("user-123") + .hashedKey(hashService.hash(rawSecret)) .name("CI Integration Key") - .description("Used by the CI pipeline") - .userId("user_abc123") - .scopes(List.of("read:tickets", "write:tickets")) + .description("Used by GitHub Actions pipeline") + .scopes(List.of("tickets:read", "tickets:write")) .roles(List.of("technician")) .enabled(true) .createdAt(Instant.now()) - .expiresAt(Instant.now().plus(Duration.ofDays(90))) + .expiresAt(Instant.now().plus(365, ChronoUnit.DAYS)) .build(); -// Check validity before processing a request -if (!apiKey.isActive()) { - throw new UnauthorizedException("API key is disabled or expired"); +// Lifecycle checks +if (apiKey.isActive()) { + // proceed with authenticated request +} + +if (apiKey.isExpired()) { + // reject or prompt renewal } ``` -> **Note:** `scopes` and `roles` fields are defined but not yet enforced β€” permission checks are marked as TODO and will be wired into the authorization layer in a future release. \ No newline at end of file +> **Note:** `hashedKey` stores only the hash β€” the raw key is never persisted. Scope and role enforcement is marked TODO and not yet implemented. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKeyStats.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKeyStats.md index bea8c95e2..4d9d14947 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKeyStats.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKeyStats.md @@ -1,28 +1,30 @@ - -MongoDB document entity for tracking API key usage statistics, storing request counts and last activity timestamp in the `api_key_stats` collection. + +MongoDB document entity that tracks usage statistics for API keys, including request counts and last activity timestamp. ## Key Components | Field | Type | Description | |-------|------|-------------| | `id` | `String` | MongoDB document identifier (`@Id`) | -| `totalRequests` | `Long` | Cumulative count of all requests made with the API key | +| `tenantId` | `String` | Indexed tenant reference (implements `TenantScoped`) | +| `totalRequests` | `Long` | Cumulative count of all requests made with the key | | `successfulRequests` | `Long` | Count of requests that completed successfully | -| `failedRequests` | `Long` | Count of requests that resulted in errors | +| `failedRequests` | `Long` | Count of requests that resulted in failure | | `lastUsed` | `LocalDateTime` | Timestamp of the most recent API key usage | ## Usage Example ```java -// Build a new stats document +// Building a new stats document for a tenant's API key ApiKeyStats stats = ApiKeyStats.builder() - .totalRequests(150L) - .successfulRequests(142L) - .failedRequests(8L) - .lastUsed(LocalDateTime.now()) - .build(); + .tenantId("tenant-abc-123") + .totalRequests(0L) + .successfulRequests(0L) + .failedRequests(0L) + .lastUsed(LocalDateTime.now()) + .build(); -// Update on each API call +// Updating stats after a successful request stats.setTotalRequests(stats.getTotalRequests() + 1); stats.setSuccessfulRequests(stats.getSuccessfulRequests() + 1); stats.setLastUsed(LocalDateTime.now()); @@ -30,6 +32,7 @@ stats.setLastUsed(LocalDateTime.now()); ## Notes -- Lombok annotations (`@Data`, `@Builder`, `@NoArgsConstructor`, `@AllArgsConstructor`) auto-generate boilerplate (getters, setters, builder pattern, constructors). -- Persisted via Spring Data MongoDB; the `@Document` annotation maps this class to the `api_key_stats` collection. -- Designed to complement an API key entity, providing a dedicated stats document to avoid bloating the primary key record. \ No newline at end of file +- Stored in the `api_key_stats` MongoDB collection +- `tenantId` is indexed for efficient per-tenant queries +- Implements `TenantScoped` to enforce multi-tenant data isolation +- Uses Lombok (`@Data`, `@Builder`, `@NoArgsConstructor`, `@AllArgsConstructor`) to reduce boilerplate \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/assignment/.ItemAssignment.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/assignment/.ItemAssignment.md index 7db0e2778..741140249 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/assignment/.ItemAssignment.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/assignment/.ItemAssignment.md @@ -1,42 +1,39 @@ - -MongoDB document model representing the assignment relationship between items (e.g., scripts, templates) and their targets (e.g., users, devices), stored in the `item_assignments` collection. + +MongoDB document entity representing the assignment relationship between an item (e.g., asset, service) and a target entity (e.g., user, organization) within a tenant-scoped context. ## Key Components -| Member | Type | Description | -|---|---|---| -| `id` | `String` | MongoDB document identifier (`@Id`) | -| `itemId` | `String` | Reference to the assigned item | -| `itemType` | `AssignmentItemType` | Enum classifying the item (e.g., script, template) | -| `targetType` | `AssignmentTargetType` | Enum classifying the assignment target (e.g., user, device) | -| `targetId` | `String` | Reference to the target entity (indexed) | -| `displayName` | `String` | Human-readable label for the assignment | -| `createdAt` | `Instant` | Auto-populated creation timestamp via `@CreatedDate` | - -### Indexes - -| Index Name | Fields | Unique | -|---|---|---| -| `item_target_unique` | `itemId`, `targetType`, `targetId` | βœ… Yes β€” prevents duplicate assignments | -| `target_lookup` | `targetType`, `targetId` | ❌ No β€” optimizes queries by target | -| *(single field)* | `targetId` | ❌ No β€” supports direct target lookups | +| Element | Description | +|---|---| +| `ItemAssignment` | Main document class mapped to the `item_assignments` MongoDB collection | +| `TenantScoped` | Interface implemented to enforce multi-tenancy isolation via `tenantId` | +| `AssignmentItemType` | Enum defining the type of item being assigned | +| `AssignmentTargetType` | Enum defining the type of target receiving the assignment | +| `item_target_unique` | Compound unique index on `itemId`, `targetType`, and `targetId` to prevent duplicate assignments | +| `target_lookup` | Compound index on `targetType` and `targetId` for efficient target-based queries | ## Usage Example ```java -// Building a new item assignment ItemAssignment assignment = ItemAssignment.builder() - .itemId("script-abc123") - .itemType(AssignmentItemType.SCRIPT) - .targetType(AssignmentTargetType.DEVICE) - .targetId("device-xyz789") - .displayName("Patch Tuesday Script β†’ Workstation 42") + .tenantId("tenant-123") + .itemId("item-456") + .itemType(AssignmentItemType.ASSET) + .targetType(AssignmentTargetType.USER) + .targetId("user-789") + .displayName("Laptop - John Doe") .build(); -// Querying assignments for a specific target -List assignments = mongoTemplate.find( - Query.query(Criteria.where("targetType").is(AssignmentTargetType.DEVICE) - .and("targetId").is("device-xyz789")), - ItemAssignment.class -); -``` \ No newline at end of file +// Persist via repository +itemAssignmentRepository.save(assignment); + +// Query all assignments for a specific target +List assignments = itemAssignmentRepository + .findByTargetTypeAndTargetId(AssignmentTargetType.USER, "user-789"); +``` + +## Notes + +- `createdAt` is automatically populated by Spring Data via `@CreatedDate` +- The unique compound index on `(itemId, targetType, targetId)` ensures an item cannot be assigned to the same target more than once +- Lombok annotations (`@Data`, `@Builder`, `@NoArgsConstructor`, `@AllArgsConstructor`) handle boilerplate generation \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthInvitation.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthInvitation.md index d624d9f79..5552d5577 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthInvitation.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthInvitation.md @@ -1,30 +1,30 @@ - -Represents a MongoDB document for authentication-based invitations, extending the base `Invitation` class with tenant-scoped uniqueness enforcement. + +MongoDB document entity representing an authentication invitation scoped to a tenant, combining tenant-aware persistence with invitation data through class inheritance. ## Key Components -| Component | Description | -|-----------|-------------| -| `AuthInvitation` | MongoDB document class extending `Invitation` for auth-specific invite records | -| `tenantId` | Indexed field scoping invitations to a specific tenant | -| `@CompoundIndex` | Enforces unique `tenantId + email` combinations where `tenantId` exists | +| Element | Description | +|---------|-------------| +| `AuthInvitation` | Extends `Invitation` with tenant-scoped auth context | +| `TenantScoped` | Inherited interface ensuring tenant isolation | +| `CompoundIndex` | Compound MongoDB index on `tenantId` + `email` for query performance | +| Lombok annotations | `@Data`, `@SuperBuilder`, `@NoArgsConstructor`, `@EqualsAndHashCode` for boilerplate reduction | ## Usage Example ```java -// Build a new auth invitation for a specific tenant +// Build a new AuthInvitation using the SuperBuilder pattern AuthInvitation invitation = AuthInvitation.builder() - .tenantId("tenant-abc-123") - .email("user@example.com") - .build(); + .tenantId("tenant-abc") + .email("technician@example.com") + .build(); -// Compound index ensures no duplicate email per tenant -// { tenantId: "tenant-abc-123", email: "user@example.com" } β†’ unique -``` +// Persist via a Spring Data MongoDB repository +authInvitationRepository.save(invitation); -## Notes +// Query by tenant and email (leverages compound index) +Optional found = authInvitationRepository + .findByTenantIdAndEmail("tenant-abc", "technician@example.com"); +``` -- Inherits base fields (e.g., `email`, invite metadata) from `Invitation` -- The `@CompoundIndex` uses a **partial filter** (`tenantId: { $exists: true }`), meaning global invitations without a `tenantId` are excluded from the uniqueness constraint -- `@Indexed` on `tenantId` enables efficient tenant-scoped queries -- Uses Lombok `@SuperBuilder` to support builder inheritance from the parent `Invitation` class \ No newline at end of file +> The compound index on `{'tenantId': 1, 'email': 1}` ensures efficient lookups when validating invitation uniqueness per tenant, which is critical in multi-tenant MSP environments where the same email may exist across different tenants. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthUser.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthUser.md index 8c5c82c6b..b2b314a93 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthUser.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthUser.md @@ -1,47 +1,44 @@ - -MongoDB document model representing an authenticated user in OpenFrame's multi-tenant Authorization Server, extending the base `User` document with auth-specific fields and tenant isolation. + +Represents a user entity for the multi-tenant Authorization Server, extending the base `User` document with authentication-specific fields such as password hash, login provider, and SSO profile data. ## Key Components | Member | Type | Description | |--------|------|-------------| -| `tenantId` | `String` | Indexed tenant identifier; combined with `email` as a unique compound index | -| `passwordHash` | `String` | Hashed password for `LOCAL` authentication | -| `loginProvider` | `String` | Authentication provider (e.g., `LOCAL`, `GOOGLE`) | +| `passwordHash` | `String` | Hashed password for local authentication | +| `loginProvider` | `String` | Authentication strategy (e.g., `LOCAL`, `GOOGLE`) | | `externalUserId` | `String` | User ID from the external OAuth/SSO provider | | `lastLogin` | `Instant` | Timestamp of the user's most recent login | -| `imageUrl` | `String` | Cached profile picture URL synced from tenant cluster via `USER_UPDATED` events | -| `getFullName()` | `String` | Derives display name from first/last name, falling back to email | +| `imageUrl` | `String` | Cached profile picture URL synced via `USER_UPDATED` events | +| `getFullName()` | `String` | Resolves display name from first/last name, falling back to email | + +**MongoDB Index:** Compound unique index on `{tenantId, email}` enforces per-tenant email uniqueness. ## Usage Example ```java -// Build a new local AuthUser for a specific tenant AuthUser authUser = AuthUser.builder() - .email("jane@acme.com") - .firstName("Jane") - .lastName("Doe") - .tenantId("acme-corp") - .passwordHash(hashService.hash("s3cur3P@ss")) - .loginProvider("LOCAL") - .build(); - -// Build an SSO user (e.g., Google) -AuthUser ssoUser = AuthUser.builder() - .email("john@acme.com") - .tenantId("acme-corp") - .loginProvider("GOOGLE") - .externalUserId("google-uid-9876") - .imageUrl("https://lh3.googleusercontent.com/photo.jpg") - .build(); - -// Resolve display name with fallback chain + .email("technician@acme.com") + .firstName("Jane") + .lastName("Doe") + .tenantId("acme-corp") + .passwordHash(BCrypt.hashpw(rawPassword, BCrypt.gensalt())) + .loginProvider("LOCAL") + .lastLogin(Instant.now()) + .build(); + +// Resolves to "Jane Doe" String displayName = authUser.getFullName(); -// β†’ "Jane Doe" -``` -## Notes +// For a Google SSO user with no name set, falls back to email +AuthUser ssoUser = AuthUser.builder() + .email("user@gmail.com") + .loginProvider("GOOGLE") + .externalUserId("google-uid-123") + .imageUrl("https://lh3.googleusercontent.com/photo.jpg") + .build(); + +String displayName = ssoUser.getFullName(); // β†’ "user@gmail.com" +``` -- The compound index on `{tenantId, email}` enforces per-tenant email uniqueness with a partial filter (`$exists: true`), allowing global users without a `tenantId`. -- `imageUrl` is a cache β€” a `null` value indicates no picture has been synced yet, not that the tenant cluster lacks one. -- Inherits base fields (`email`, `firstName`, `lastName`, etc.) from the `User` superclass via `@SuperBuilder`. \ No newline at end of file +> **Note:** `imageUrl` is a cache populated by `USER_UPDATED` events from the tenant cluster. A `null` value means the picture is not yet known locally β€” not that the user has no profile image. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/clientconfiguration/.OpenFrameClientConfiguration.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/clientconfiguration/.OpenFrameClientConfiguration.md index f133e40f4..50d1352a1 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/clientconfiguration/.OpenFrameClientConfiguration.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/clientconfiguration/.OpenFrameClientConfiguration.md @@ -1,39 +1,41 @@ - -MongoDB document entity representing a versioned OpenFrame client configuration record, stored in the `openframe_client_configuration` collection. + +Represents a MongoDB document storing per-tenant OpenFrame client configuration, including download settings and publish state metadata. ## Key Components | Field | Type | Description | -|---|---|---| +|-------|------|-------------| | `id` | `String` | MongoDB document identifier (`@Id`) | +| `tenantId` | `String` | Unique tenant identifier (`@Indexed(unique = true)`) | | `documentVersion` | `Long` | Optimistic locking version (`@Version`) | -| `version` | `String` | Client configuration version label | -| `downloadConfiguration` | `List` | List of download configuration entries | -| `createdAt` | `LocalDateTime` | Auto-populated creation timestamp (`@CreatedDate`) | -| `updatedAt` | `LocalDateTime` | Auto-populated last modified timestamp (`@LastModifiedDate`) | -| `publishState` | `PublishState` | Current publish state of the configuration | +| `version` | `String` | Client configuration version string | +| `downloadConfiguration` | `List` | List of download config entries for the tenant | +| `publishState` | `PublishState` | Current publish lifecycle state | +| `createdAt` / `updatedAt` | `LocalDateTime` | Audit timestamps (auto-managed) | + +## Key Components + +- **`TenantScoped`** β€” Implements tenant isolation interface, ensuring configurations are scoped per tenant +- **`@Document`** β€” Maps to the `openframe_client_configuration` MongoDB collection +- **Lombok** β€” `@Data`, `@Builder`, `@NoArgsConstructor`, `@AllArgsConstructor` reduce boilerplate +- **Optimistic locking** β€” `@Version` prevents concurrent write conflicts at the document level ## Usage Example ```java -// Build and save a new client configuration +// Build a new client configuration for a tenant OpenFrameClientConfiguration config = OpenFrameClientConfiguration.builder() - .version("1.0.0") + .tenantId("tenant-abc-123") + .version("2.1.0") .publishState(PublishState.DRAFT) .downloadConfiguration(List.of( DownloadConfiguration.builder() .platform("windows") - .url("https://downloads.openframe.ai/client/1.0.0/win") + .url("https://downloads.openframe.ai/client/2.1.0/win") .build() )) .build(); -mongoTemplate.save(config); - -// Retrieve and inspect state -OpenFrameClientConfiguration saved = mongoTemplate.findById(id, OpenFrameClientConfiguration.class); -System.out.println(saved.getPublishState()); // DRAFT -System.out.println(saved.getDocumentVersion()); // 1 (auto-incremented) -``` - -> **Note:** This entity relies on Spring Data MongoDB auditing (`@CreatedDate`, `@LastModifiedDate`) being enabled via `@EnableMongoAuditing` in the application context. Optimistic concurrency control is enforced through the `@Version` field. \ No newline at end of file +// Persist via Spring Data repository +clientConfigurationRepository.save(config); +``` \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/connector/.ConnectorAlert.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/connector/.ConnectorAlert.md index f7d7d4ace..c80926498 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/connector/.ConnectorAlert.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/connector/.ConnectorAlert.md @@ -1,38 +1,39 @@ - -A MongoDB document model representing an alert generated by a connector, tracking error state, retry attempts, and resolution status within the OpenFrame platform. + +MongoDB document entity representing an alert generated by a connector, storing error details, retry attempts, and resolution state within a tenant-scoped context. ## Key Components -| Field | Type | Description | -|-------|------|-------------| -| `id` | `String` | MongoDB document identifier (`@Id`) | -| `connectorName` | `String` | Name of the connector that triggered the alert | -| `errorType` | `ConnectorAlertType` | Enum categorizing the alert type | -| `errorMessage` | `String` | Human-readable error description | -| `attempts` | `int` | Number of retry attempts made | -| `createdAt` | `Instant` | Timestamp when the alert was first raised | -| `resolved` | `boolean` | Whether the alert has been resolved | -| `resolvedAt` | `Instant` | Timestamp when the alert was resolved | - -Stored in the `connector_alerts` MongoDB collection. Lombok annotations (`@Data`, `@Builder`, `@NoArgsConstructor`, `@AllArgsConstructor`) handle boilerplate generation. +| Element | Type | Description | +|---|---|---| +| `ConnectorAlert` | `@Document` | MongoDB entity mapped to `connector_alerts` collection | +| `TenantScoped` | Interface | Enforces multi-tenancy via indexed `tenantId` field | +| `connectorName` | `String` | Identifies the source connector that triggered the alert | +| `errorType` | `ConnectorAlertType` | Enum classifying the category of alert/error | +| `errorMessage` | `String` | Human-readable description of the error | +| `attempts` | `int` | Number of retry attempts made before alerting | +| `resolved` / `resolvedAt` | `boolean` / `Instant` | Tracks alert resolution status and timestamp | ## Usage Example ```java -// Creating a new unresolved connector alert +// Create a new connector alert ConnectorAlert alert = ConnectorAlert.builder() - .connectorName("psa-autotask") - .errorType(ConnectorAlertType.AUTH_FAILURE) - .errorMessage("Invalid API credentials provided") + .tenantId("tenant-abc") + .connectorName("zabbix-sync") + .errorType(ConnectorAlertType.CONNECTION_FAILURE) + .errorMessage("Unable to reach Zabbix host: timeout after 30s") .attempts(3) .createdAt(Instant.now()) .resolved(false) .build(); -// Marking an existing alert as resolved +// Mark an existing alert as resolved alert.setResolved(true); alert.setResolvedAt(Instant.now()); -// Saving via Spring Data MongoDB repository -connectorAlertRepository.save(alert); -``` \ No newline at end of file +// Query example (Spring Data MongoDB repository) +List openAlerts = alertRepository + .findByTenantIdAndResolvedFalse("tenant-abc"); +``` + +> **Note:** The `tenantId` field is indexed via `@Indexed` to ensure efficient per-tenant queries across the `connector_alerts` collection. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Device.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Device.md index 4b5795118..eafbd6b72 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Device.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Device.md @@ -1,36 +1,40 @@ - -Represents a managed device entity stored in the MongoDB `devices` collection, serving as the core data model for tracking physical and virtual machines across the OpenFrame platform. + +MongoDB document entity representing a managed device in the OpenFrame platform, storing hardware identity, OS info, health state, and tenant association. ## Key Components -| Field | Type | Description | -|-------|------|-------------| -| `id` | `String` | MongoDB document identifier (`@Id`) | -| `machineId` | `String` | Foreign reference linking to the Machine entity | -| `serialNumber` | `String` | Unique hardware serial identifier | -| `model` | `String` | Device model name/number | -| `osVersion` | `String` | Operating system version string | -| `status` | `String` | Lifecycle state: `ACTIVE`, `OFFLINE`, or `MAINTENANCE` | -| `type` | `DeviceType` | Enum classifying device category (e.g., `DESKTOP`, `LAPTOP`, `SERVER`) | -| `lastCheckin` | `Instant` | UTC timestamp of the most recent agent heartbeat | -| `configuration` | `DeviceConfiguration` | Embedded configuration details | -| `health` | `DeviceHealth` | Embedded health metrics and diagnostics | +| Element | Type | Description | +|---|---|---| +| `Device` | `@Document` class | MongoDB entity mapped to the `devices` collection | +| `TenantScoped` | Interface | Enforces multi-tenancy via `tenantId` | +| `id` | `String` | MongoDB primary key (`@Id`) | +| `tenantId` | `String` | Indexed tenant reference for scoped queries | +| `machineId` | `String` | Foreign reference to the `Machine` entity | +| `status` | `String` | Lifecycle state: `ACTIVE`, `OFFLINE`, `MAINTENANCE` | +| `type` | `DeviceType` | Enum classifying the device (e.g. `DESKTOP`, `LAPTOP`, `SERVER`) | +| `lastCheckin` | `Instant` | Timestamp of the most recent device heartbeat | +| `configuration` | `DeviceConfiguration` | Embedded configuration sub-document | +| `health` | `DeviceHealth` | Embedded health metrics sub-document | ## Usage Example ```java +// Create and persist a new device Device device = new Device(); -device.setMachineId("machine-abc123"); -device.setSerialNumber("SN-987654"); -device.setModel("Dell OptiPlex 7090"); -device.setOsVersion("Windows 11 22H2"); +device.setTenantId("tenant-abc"); +device.setMachineId("machine-xyz"); +device.setSerialNumber("SN-00123"); +device.setModel("Dell XPS 15"); +device.setOsVersion("Windows 11 23H2"); +device.setType(DeviceType.LAPTOP); device.setStatus("ACTIVE"); -device.setType(DeviceType.DESKTOP); device.setLastCheckin(Instant.now()); -device.setConfiguration(new DeviceConfiguration()); -device.setHealth(new DeviceHealth()); deviceRepository.save(device); + +// Query all active devices for a tenant +List activeDevices = deviceRepository + .findByTenantIdAndStatus("tenant-abc", "ACTIVE"); ``` -> **Note:** Lombok's `@Data` auto-generates getters, setters, `equals()`, `hashCode()`, and `toString()`. The `lastCheckin` field should be updated on every agent heartbeat to accurately reflect device availability within the Flamingo monitoring stack. \ No newline at end of file +> The `@Indexed` annotation on `tenantId` ensures efficient tenant-scoped queries across large device inventories β€” critical for MSP environments managing multiple clients. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Machine.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Machine.md index 899082efa..8a3c1585b 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Machine.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Machine.md @@ -1,40 +1,37 @@ - -Represents a managed device (machine) in the OpenFrame platform, stored in the MongoDB `machines` collection with full lifecycle, security, and compliance tracking. + +MongoDB document entity representing a managed device (machine) within the OpenFrame platform, storing hardware identity, connectivity state, security posture, and compliance data scoped to a tenant. ## Key Components | Field | Type | Description | -|---|---|---| -| `id` | `String` | MongoDB document ID (`@Id`) | -| `machineId` | `String` | Primary auth identifier, mirrors `OAuthClient` | -| `status` | `DeviceStatus` | Current device status (indexed) | -| `organizationId` | `String` | Tenant scoping key (indexed) | -| `type` | `DeviceType` | Device category β€” workstation, server, etc. (indexed) | -| `osType` / `osVersion` | `String` | OS platform and version details | -| `securityState` | `SecurityState` | Current security posture | -| `complianceState` | `ComplianceState` | Compliance evaluation result | +|-------|------|-------------| +| `id` | `String` | MongoDB primary key (`@Id`) | +| `machineId` | `String` | Unique device identifier, shared with `OAuthClient` for authentication | +| `tenantId` | `String` | Tenant scope (indexed, implements `TenantScoped`) | +| `organizationId` | `String` | Sub-org grouping within a tenant (indexed) | +| `status` | `DeviceStatus` | Current connectivity status (indexed) | +| `type` | `DeviceType` | Device category, e.g. workstation, server (indexed) | +| `securityState` | `SecurityState` | Aggregated security posture | +| `complianceState` | `ComplianceState` | Aggregated compliance posture | | `securityAlerts` | `List` | Active security alerts on the device | -| `complianceRequirements` | `List` | Evaluated compliance rules | -| `registeredAt` | `Instant` | Auto-set on first registration (`@CreatedDate`) | +| `complianceRequirements` | `List` | Compliance rules evaluated against this device | +| `registeredAt` | `Instant` | Auto-set on first insert (`@CreatedDate`) | | `updatedAt` | `Instant` | Auto-updated on every save (`@LastModifiedDate`) | -| `stuckNotifiedAt` | `Instant` | Tracks if a `DEVICE_STUCK` event has been dispatched | +| `stuckNotifiedAt` | `Instant` | Tracks when a `DEVICE_STUCK` event was last emitted | ## Usage Example ```java Machine machine = new Machine(); -machine.setMachineId("mch-abc123"); -machine.setOrganizationId("org-xyz"); +machine.setTenantId("tenant-abc"); +machine.setMachineId("mch-001"); machine.setHostname("workstation-01"); +machine.setDisplayName("Alice's MacBook"); +machine.setOsType("macOS"); +machine.setOsVersion("14.4"); machine.setType(DeviceType.WORKSTATION); -machine.setOsType("WINDOWS"); -machine.setOsVersion("11"); machine.setStatus(DeviceStatus.ONLINE); -machine.setLastSeen(Instant.now()); -machine.setSecurityState(SecurityState.SECURE); -machine.setComplianceState(ComplianceState.COMPLIANT); +machine.setOrganizationId("org-xyz"); machineRepository.save(machine); -``` - -> **Note:** `machineId` is shared with the `OAuthClient` record and serves as the primary authentication identity for agent-to-platform communication. The `stuckNotifiedAt` field remains `null` until a stuck-device event is triggered, preventing duplicate alerts. \ No newline at end of file +``` \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.CoreEvent.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.CoreEvent.md index 3f44653a0..0af8b709e 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.CoreEvent.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.CoreEvent.md @@ -1,38 +1,41 @@ - -MongoDB document model representing a system event within the OpenFrame core service, used to track and persist event lifecycle states. + +Represents a tenant-scoped event document persisted in the MongoDB `events` collection, capturing the full lifecycle of a system event within the OpenFrame platform. ## Key Components | Member | Type | Description | |--------|------|-------------| | `id` | `String` | MongoDB document identifier (`@Id`) | -| `type` | `String` | Categorizes the event (e.g., ticket created, alert triggered) | +| `tenantId` | `String` | Indexed tenant reference (implements `TenantScoped`) | +| `type` | `String` | Event category/name (e.g., `TICKET_CREATED`) | | `payload` | `String` | Serialized event data (typically JSON) | | `timestamp` | `Instant` | UTC time the event was recorded | -| `userId` | `String` | Reference to the user who triggered the event | -| `status` | `EventStatus` | Current lifecycle state of the event | +| `userId` | `String` | Originating user reference | +| `status` | `EventStatus` | Current processing state | | `EventStatus` | `enum` | `CREATED` β†’ `PROCESSING` β†’ `COMPLETED` / `FAILED` | ## Usage Example ```java +// Creating and persisting a new event CoreEvent event = new CoreEvent(); +event.setTenantId("tenant-abc"); event.setType("TICKET_CREATED"); -event.setPayload("{\"ticketId\": \"T-1042\", \"priority\": \"HIGH\"}"); +event.setPayload("{\"ticketId\": \"T-001\", \"priority\": \"HIGH\"}"); event.setTimestamp(Instant.now()); -event.setUserId("user-abc123"); +event.setUserId("user-xyz"); event.setStatus(CoreEvent.EventStatus.CREATED); -// Persist via Spring Data MongoDB repository -eventRepository.save(event); +mongoTemplate.save(event); -// Query events by status -List pending = eventRepository.findByStatus(CoreEvent.EventStatus.PROCESSING); +// Transitioning event status +event.setStatus(CoreEvent.EventStatus.PROCESSING); +mongoTemplate.save(event); ``` ## Notes -- Stored in the `events` MongoDB collection via `@Document(collection = "events")` -- Uses Lombok `@Data` for auto-generated getters, setters, `equals`, `hashCode`, and `toString` -- `payload` is stored as a raw `String`; callers are responsible for serialization/deserialization -- Lifecycle flows: `CREATED` β†’ `PROCESSING` β†’ `COMPLETED` or `FAILED` \ No newline at end of file +- Implements `TenantScoped` to enforce multi-tenant data isolation across all queries +- `tenantId` is indexed for efficient per-tenant event lookups at scale +- `payload` is stored as a raw `String` β€” callers are responsible for serialization/deserialization +- Lombok `@Data` generates all getters, setters, `equals`, `hashCode`, and `toString` \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.Event.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.Event.md index ac9b28aa0..a600d41bc 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.Event.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.Event.md @@ -1,37 +1,36 @@ - -MongoDB document entity representing a system event with metadata including type, payload, timestamp, and user association. + +A MongoDB document model representing a tenant-scoped event record, used to store system or user-triggered events with payload data and metadata. ## Key Components -| Field | Type | Description | -|-------|------|-------------| +| Member | Type | Description | +|--------|------|-------------| | `id` | `String` | MongoDB document identifier (`@Id`) | -| `type` | `String` | Event category or action name | -| `payload` | `String` | Serialized event data | -| `timestamp` | `Instant` | UTC time the event occurred | -| `userId` | `String` | ID of the user who triggered the event | +| `tenantId` | `String` | Indexed tenant reference for multi-tenancy scoping | +| `type` | `String` | Event type/category identifier | +| `payload` | `String` | Serialized event data (typically JSON) | +| `timestamp` | `Instant` | UTC timestamp of when the event occurred | +| `userId` | `String` | Reference to the user who triggered the event | + +Implements `TenantScoped` to enforce multi-tenant data isolation, with `tenantId` indexed for efficient per-tenant queries. ## Usage Example ```java -// Create a new Event using the Lombok builder +// Create a new event using the Lombok builder Event event = Event.builder() + .tenantId("tenant-abc") .type("TICKET_CREATED") - .payload("{\"ticketId\": \"T-1042\", \"priority\": \"high\"}") + .payload("{\"ticketId\": \"T-001\", \"priority\": \"HIGH\"}") .timestamp(Instant.now()) - .userId("user-abc123") + .userId("user-xyz") .build(); -// Save to MongoDB via a repository +// Persist via a MongoRepository eventRepository.save(event); -// Retrieve events by type -List ticketEvents = eventRepository.findByType("TICKET_CREATED"); +// Query all events for a tenant +List tenantEvents = eventRepository.findByTenantId("tenant-abc"); ``` -## Notes - -- Stored in the `events` MongoDB collection via `@Document(collection = "events")` -- `@Data` generates getters, setters, `equals`, `hashCode`, and `toString` -- `@Builder` enables the fluent builder pattern for construction -- `payload` is stored as a raw `String`; consider JSON serialization for structured data \ No newline at end of file +> **Note:** The `tenantId` field is indexed (`@Indexed`) to optimize multi-tenant queries. Always populate this field before persisting to ensure proper data isolation across tenants. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.ExternalApplicationEvent.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.ExternalApplicationEvent.md index 661578b11..da92dd333 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.ExternalApplicationEvent.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.ExternalApplicationEvent.md @@ -1,41 +1,42 @@ - -A MongoDB document model representing events received from external applications, stored in the `external_application_events` collection. + +Represents a MongoDB document model for capturing external application events scoped to a specific tenant, storing event payload, metadata, and contextual information. ## Key Components -| Field | Type | Description | -|-------|------|-------------| -| `id` | `String` | MongoDB document identifier (`@Id`) | -| `type` | `String` | Event type classifier | +| Component | Type | Description | +|-----------|------|-------------| +| `ExternalApplicationEvent` | `@Document` class | Root MongoDB document mapped to the `external_application_events` collection | +| `tenantId` | `@Indexed` field | Indexed tenant identifier enabling efficient multi-tenant queries | +| `type` | `String` | Categorizes the event (e.g., alert, webhook, integration trigger) | | `payload` | `String` | Raw event payload content | -| `timestamp` | `Instant` | UTC timestamp of the event | -| `userId` | `String` | Associated user identifier | -| `metadata` | `EventMetadata` | Nested metadata object | - -### `EventMetadata` (nested class) - -| Field | Type | Description | -|-------|------|-------------| -| `source` | `String` | Origin system or application name | -| `version` | `String` | Event schema or API version | -| `tags` | `Map` | Arbitrary key-value tags for filtering/routing | +| `timestamp` | `Instant` | UTC timestamp of when the event occurred | +| `userId` | `String` | Optional user associated with the event | +| `EventMetadata` | Inner `@Data` class | Nested object holding `source`, `version`, and arbitrary `tags` | ## Usage Example ```java -ExternalApplicationEvent event = new ExternalApplicationEvent(); -event.setType("ticket.created"); -event.setPayload("{\"ticketId\": \"T-123\", \"priority\": \"high\"}"); -event.setTimestamp(Instant.now()); -event.setUserId("user-456"); - +// Build and persist an external application event ExternalApplicationEvent.EventMetadata metadata = new ExternalApplicationEvent.EventMetadata(); -metadata.setSource("connectwise"); -metadata.setVersion("1.0"); -metadata.setTags(Map.of("env", "production", "region", "us-east")); +metadata.setSource("zabbix"); +metadata.setVersion("6.4"); +metadata.setTags(Map.of("env", "production", "priority", "high")); +ExternalApplicationEvent event = new ExternalApplicationEvent(); +event.setTenantId("tenant-abc"); +event.setType("HOST_ALERT"); +event.setPayload("{\"hostId\":\"srv-01\",\"status\":\"DOWN\"}"); +event.setTimestamp(Instant.now()); +event.setUserId("user-123"); event.setMetadata(metadata); + mongoTemplate.save(event); + +// Query events by tenant +List events = mongoTemplate.find( + Query.query(Criteria.where("tenantId").is("tenant-abc")), + ExternalApplicationEvent.class +); ``` -> **Note:** Uses Lombok `@Data` for automatic generation of getters, setters, `equals`, `hashCode`, and `toString` on both the parent and nested `EventMetadata` class. \ No newline at end of file +> The `TenantScoped` interface ensures all queries respect tenant isolation, a core requirement across the Flamingo/OpenFrame multi-tenant architecture. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/installedagents/.InstalledAgent.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/installedagents/.InstalledAgent.md index 5423924f6..4e8022322 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/installedagents/.InstalledAgent.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/installedagents/.InstalledAgent.md @@ -1,36 +1,38 @@ - -Represents a MongoDB document model for tracking agents installed on managed machines within the OpenFrame platform. + +MongoDB document entity representing an installed agent record scoped to a specific tenant, used to track agent deployments across machines. ## Key Components -| Field | Type | Description | -|-------|------|-------------| -| `id` | `String` | MongoDB document identifier (`@Id`) | -| `machineId` | `String` | Reference to the managed machine | -| `agentType` | `String` | Type/category of the installed agent | -| `version` | `String` | Installed agent version string | -| `createdAt` | `String` | Timestamp of agent installation record creation | -| `updatedAt` | `String` | Timestamp of last record update | - -**Annotations:** -- `@Data` β€” Lombok-generated getters, setters, `equals`, `hashCode`, and `toString` -- `@Document(collection = "installed_agents")` β€” Maps to the `installed_agents` MongoDB collection +| Element | Description | +|---|---| +| `@Document("installed_agents")` | Maps to the `installed_agents` MongoDB collection | +| `TenantScoped` | Interface enforcing multi-tenant data isolation | +| `tenantId` | Indexed field for efficient tenant-based queries | +| `machineId` | Identifier of the machine where the agent is installed | +| `agentType` | Classifies the type of installed agent | +| `version` | Tracks the agent version deployed on the machine | ## Usage Example ```java -// Create and persist a new InstalledAgent record +// Creating and persisting a new InstalledAgent record InstalledAgent agent = new InstalledAgent(); -agent.setMachineId("machine-abc-123"); +agent.setTenantId("tenant-123"); +agent.setMachineId("machine-abc"); agent.setAgentType("monitoring"); -agent.setVersion("2.1.0"); +agent.setVersion("1.4.2"); agent.setCreatedAt(Instant.now().toString()); agent.setUpdatedAt(Instant.now().toString()); installedAgentRepository.save(agent); -// Retrieve agents by machine -List agents = installedAgentRepository.findByMachineId("machine-abc-123"); +// Querying by tenant +List tenantAgents = installedAgentRepository + .findByTenantId("tenant-123"); ``` -> **Note:** `createdAt` and `updatedAt` are stored as `String` types. Consider using `Instant` or `LocalDateTime` with a MongoDB converter for timezone-safe timestamp handling in production. \ No newline at end of file +## Notes + +- Uses Lombok `@Data` to auto-generate getters, setters, `equals`, `hashCode`, and `toString` +- `tenantId` is indexed via `@Indexed` to optimize multi-tenant queries at scale +- `createdAt` and `updatedAt` are stored as `String` β€” consider using `Instant` or `Date` if temporal queries are needed \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.GenericContext.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.GenericContext.md index 062db8168..1a9fd58ae 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.GenericContext.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.GenericContext.md @@ -1,37 +1,31 @@ - -A generic notification context implementation that extends `NotificationContext`, carrying an arbitrary string payload for flexible notification data transport. + +A generic, flexible implementation of `NotificationContext` that carries arbitrary notification data using a type identifier and a string-encoded payload. ## Key Components -| Element | Type | Description | -|---|---|---| -| `GenericContext` | Class | Concrete subclass of `NotificationContext` | -| `payload` | `String` | Arbitrary string data attached to the notification | +| Member | Type | Description | +|--------|------|-------------| +| `type` | `String` | Identifies the kind of notification or payload format | +| `payload` | `String` | The raw notification content (e.g., JSON, plain text) | -Lombok annotations handle boilerplate automatically: - -- `@Data` β€” generates getters, setters, `equals`, `hashCode`, and `toString` -- `@SuperBuilder` β€” enables builder pattern with parent class field inclusion -- `@NoArgsConstructor` / `@AllArgsConstructor` β€” generates both constructors -- `@EqualsAndHashCode(callSuper = true)` β€” includes parent fields in equality checks -- `@ToString(callSuper = true)` β€” includes parent fields in string representation +Inherits all fields and behavior from `NotificationContext` via `@EqualsAndHashCode(callSuper = true)` and `@ToString(callSuper = true)`. ## Usage Example ```java -// Builder pattern (includes parent NotificationContext fields) +// Builder pattern (recommended) GenericContext context = GenericContext.builder() - .payload("{\"event\":\"ticket.created\",\"ticketId\":\"123\"}") + .type("ALERT") + .payload("{\"message\": \"Disk usage above 90%\"}") .build(); // All-args constructor -GenericContext context = new GenericContext("{\"event\":\"ticket.updated\"}"); - -// Accessing the payload -String payload = context.getPayload(); +GenericContext context = new GenericContext("ALERT", "{\"message\": \"Disk usage above 90%\"}"); -// Updating the payload -context.setPayload("{\"event\":\"ticket.closed\"}"); +// No-args constructor with setters +GenericContext context = new GenericContext(); +context.setType("TICKET_UPDATE"); +context.setPayload("{\"ticketId\": \"TKT-001\", \"status\": \"resolved\"}"); ``` -> **Tip:** Use `GenericContext` when no strongly-typed context subclass exists for your notification type β€” simply serialize your data to JSON and pass it as the `payload` string. \ No newline at end of file +> **Note:** `payload` is intentionally untyped (`String`) to support any serialization format. Callers are responsible for serializing and deserializing the payload content. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.NotificationContext.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.NotificationContext.md index 2d8c599e1..6068a3a8d 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.NotificationContext.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.NotificationContext.md @@ -1,47 +1,43 @@ - -Abstract base class for notification context payloads, enabling polymorphic JSON deserialization based on a `type` discriminator field. + +Abstract base class for notification context objects, enabling polymorphic JSON deserialization based on a `type` discriminator field. ## Key Components -| Element | Description | -|---|---| -| `type` | String discriminator field used by Jackson to resolve the concrete subclass at deserialization time | -| `@JsonTypeInfo` | Configures polymorphic type handling using `Id.NAME`, reading from the existing `type` property | -| `defaultImpl = GenericContext.class` | Falls back to `GenericContext` when no matching type name is registered | -| `@SuperBuilder` | Enables builder inheritance across subclasses | +- **`NotificationContext`** β€” Abstract base class that all notification context types must extend +- **`getType()`** β€” Abstract method subclasses must implement to return the context's type identifier +- **`@JsonTypeInfo`** β€” Configures Jackson to deserialize concrete subtypes using the `type` property as a discriminator, falling back to `GenericContext` when no matching type is found +- **`@SuperBuilder`** β€” Enables Lombok builder inheritance across subclass hierarchies +- **`@NoArgsConstructor`** β€” Generates a no-arg constructor required for Jackson deserialization ## Usage Example ```java // Define a concrete subclass -@JsonTypeName("ticket") -@Data +@JsonTypeName("ticket_update") @SuperBuilder @NoArgsConstructor -@AllArgsConstructor -public class TicketContext extends NotificationContext { +public class TicketUpdateContext extends NotificationContext { + private String ticketId; - private String summary; + private String status; + + @Override + public String getType() { + return "ticket_update"; + } } -// JSON deserialization β€” Jackson resolves to TicketContext +// Jackson deserializes the correct subtype automatically String json = """ { - "type": "ticket", - "ticketId": "TKT-001", - "summary": "Server is down" + "type": "ticket_update", + "ticketId": "TKT-123", + "status": "resolved" } """; NotificationContext ctx = objectMapper.readValue(json, NotificationContext.class); -// ctx instanceof TicketContext β†’ true - -// Builder usage -TicketContext context = TicketContext.builder() - .type("ticket") - .ticketId("TKT-001") - .summary("Server is down") - .build(); +// ctx is an instance of TicketUpdateContext ``` -> Subclasses must register their type name (e.g. via `@JsonTypeName`) and extend `NotificationContext` to participate in polymorphic deserialization. Unknown types fall back to `GenericContext`. \ No newline at end of file +> Unknown or unregistered `type` values fall back to `GenericContext`, preventing deserialization failures for forward-compatibility. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoOAuth2Authorization.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoOAuth2Authorization.md index a9681de42..5512e74c1 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoOAuth2Authorization.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoOAuth2Authorization.md @@ -1,40 +1,41 @@ - -MongoDB document entity for persisting OAuth2 authorization state, including authorization codes, access tokens, and refresh tokens, with automatic TTL-based expiry cleanup. + +MongoDB document entity for persisting OAuth2 authorization state in a multi-tenant environment, mapped to the `oauth2_authorizations` collection. ## Key Components -| Field / Method | Description | -|---|---| -| `id` | Primary key (`@Id`) for the authorization document | -| `registeredClientId` | Indexed reference to the OAuth2 registered client | -| `principalName` | Indexed name of the authenticated principal | -| `authorizationGrantType` | Grant type used (e.g., `authorization_code`) | -| `ar*` fields | Minimal PKCE snapshot fields (client ID, redirect URI, scopes, state, code challenge) stored as flat strings to avoid MongoDB dotted-key issues | -| `state` | Indexed OAuth2 state parameter | +| Member | Description | +|--------|-------------| +| `id` | Primary document identifier | +| `tenantId` | Indexed field implementing `TenantScoped` for multi-tenant isolation | +| `registeredClientId` / `principalName` | Compound-indexed pair for efficient client/user lookups | +| `arClient*` fields | Minimal PKCE snapshot (client ID, auth URI, redirect URI, scopes, state, additional params) stored as flat strings to avoid dotted-key issues in MongoDB | | `authorizationCode*` | Authorization code value, issuance/expiry timestamps, and metadata | -| `accessToken*` | Access token value, timestamps, type, scopes, and metadata | +| `accessToken*` | Access token value, timestamps, type, space-delimited scopes, and metadata | | `refreshToken*` | Refresh token value, timestamps, and metadata | -| `expiresAt` | TTL index field (`expireAfterSeconds = 0`) β€” MongoDB auto-deletes the document when this timestamp passes | -| `updateExpiresAt()` | Derives `expiresAt` as the maximum non-null expiry across all token types | +| `expiresAt` | TTL index field (`expireAfterSeconds = 0`) β€” set to the latest token expiration for automatic document cleanup | +| `updateExpiresAt()` | Computes `expiresAt` as the maximum non-null value across all three token expiration timestamps | ## Usage Example ```java -MongoOAuth2Authorization authorization = new MongoOAuth2Authorization(); -authorization.setId(UUID.randomUUID().toString()); -authorization.setRegisteredClientId("my-client"); -authorization.setPrincipalName("user@example.com"); -authorization.setAuthorizationGrantType("authorization_code"); - -// Set token details -authorization.setAccessTokenValue(tokenValue); -authorization.setAccessTokenIssuedAt(Instant.now()); -authorization.setAccessTokenExpiresAt(Instant.now().plusSeconds(3600)); - -// Compute TTL β€” must be called after any token expiry field is updated -authorization.updateExpiresAt(); - -mongoOAuth2AuthorizationRepository.save(authorization); +MongoOAuth2Authorization auth = new MongoOAuth2Authorization(); +auth.setId(UUID.randomUUID().toString()); +auth.setTenantId("tenant-42"); +auth.setRegisteredClientId("my-client"); +auth.setPrincipalName("alice"); +auth.setAuthorizationGrantType(AuthorizationGrantType.AUTHORIZATION_CODE.getValue()); + +// Set PKCE snapshot +auth.setArClientId("my-client"); +auth.setArRedirectUri("https://app.example.com/callback"); +auth.setArScopes("openid profile email"); + +// Set token expiry and trigger TTL field update +auth.setAccessTokenExpiresAt(Instant.now().plusSeconds(3600)); +auth.setRefreshTokenExpiresAt(Instant.now().plusSeconds(86400)); +auth.updateExpiresAt(); // expiresAt β†’ refresh token expiry (latest) + +mongoTemplate.save(auth); ``` -> **Note:** Always call `updateExpiresAt()` after modifying any token expiry field to ensure the MongoDB TTL index remains accurate and stale documents are cleaned up automatically. \ No newline at end of file +> **Note:** Call `updateExpiresAt()` after modifying any token expiration field to keep the MongoDB TTL index accurate and ensure stale authorization documents are automatically purged. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoRegisteredClient.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoRegisteredClient.md index 8c8fb4717..5d3983a90 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoRegisteredClient.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoRegisteredClient.md @@ -1,34 +1,35 @@ - -MongoDB document entity representing an OAuth 2.0 registered client, mapped to the `oauth_registered_clients` collection for persisting client credentials and authorization configuration. + +MongoDB document entity representing an OAuth 2.0 registered client, stored in the `oauth_registered_clients` collection with multi-tenant isolation enforced via a compound unique index on `tenantId` and `clientId`. ## Key Components -| Field | Type | Description | -|-------|------|-------------| -| `id` | `String` | Primary document identifier (`@Id`) | -| `clientId` | `String` | Unique OAuth client identifier (`@Indexed(unique = true)`) | -| `clientSecret` | `String` | Hashed client secret | -| `authenticationMethods` | `Set` | Supported authentication methods (e.g., `client_secret_basic`) | -| `grantTypes` | `Set` | Allowed grant types (e.g., `authorization_code`, `refresh_token`) | -| `redirectUris` | `Set` | Permitted redirect URIs post-authorization | -| `scopes` | `Set` | OAuth scopes the client may request | -| `requireProofKey` | `boolean` | Enforces PKCE (Proof Key for Code Exchange) | -| `requireAuthorizationConsent` | `boolean` | Requires explicit user consent screen | -| `accessTokenTtlSeconds` | `long` | Access token time-to-live in seconds | -| `refreshTokenTtlSeconds` | `long` | Refresh token time-to-live in seconds | -| `reuseRefreshTokens` | `boolean` | Whether refresh tokens are reused on rotation | +| Member | Type | Description | +|--------|------|-------------| +| `id` | `String` | MongoDB document `_id` | +| `tenantId` | `String` | Tenant isolation key (from `TenantScoped`) | +| `clientId` | `String` | OAuth 2.0 client identifier | +| `clientSecret` | `String` | Hashed or plain client secret | +| `authenticationMethods` | `Set` | e.g. `client_secret_basic`, `none` | +| `grantTypes` | `Set` | e.g. `authorization_code`, `client_credentials` | +| `redirectUris` | `Set` | Allowed OAuth redirect URIs | +| `scopes` | `Set` | Permitted OAuth scopes | +| `requireProofKey` | `boolean` | Enforces PKCE (`code_challenge`) | +| `requireAuthorizationConsent` | `boolean` | Forces consent screen | +| `accessTokenTtlSeconds` | `long` | Access token lifetime | +| `refreshTokenTtlSeconds` | `long` | Refresh token lifetime | +| `reuseRefreshTokens` | `boolean` | Controls refresh token rotation | ## Usage Example ```java MongoRegisteredClient client = MongoRegisteredClient.builder() - .id(UUID.randomUUID().toString()) - .clientId("openframe-web") + .tenantId("tenant-abc") + .clientId("my-app") .clientSecret("{bcrypt}$2a$10$...") .authenticationMethods(Set.of("client_secret_basic")) .grantTypes(Set.of("authorization_code", "refresh_token")) - .redirectUris(Set.of("https://app.openframe.ai/callback")) - .scopes(Set.of("openid", "profile", "email")) + .redirectUris(Set.of("https://my-app.example.com/callback")) + .scopes(Set.of("openid", "profile", "read:tickets")) .requireProofKey(true) .requireAuthorizationConsent(false) .accessTokenTtlSeconds(3600) @@ -36,7 +37,7 @@ MongoRegisteredClient client = MongoRegisteredClient.builder() .reuseRefreshTokens(false) .build(); -mongoRegisteredClientRepository.save(client); +registeredClientRepository.save(client); ``` -> **Note:** `clientId` is enforced as unique at the database index level, preventing duplicate client registrations across the OpenFrame OAuth authorization server. \ No newline at end of file +> The compound index `tenant_clientId_idx` guarantees that `clientId` values are unique **per tenant**, enabling safe multi-tenant OAuth client registration within a shared MongoDB collection. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthClient.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthClient.md index 418fc38c6..af6b74db8 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthClient.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthClient.md @@ -1,38 +1,35 @@ - -MongoDB document entity representing an OAuth 2.0 client registration, mapped to the `oauth_clients` collection. + +MongoDB document entity representing an OAuth 2.0 client registration, scoped to a specific tenant within the OpenFrame platform. ## Key Components -| Field | Type | Description | -|-------|------|-------------| +| Member | Type | Description | +|--------|------|-------------| | `id` | `String` | MongoDB document identifier (`@Id`) | -| `clientId` | `String` | Public OAuth client identifier | -| `clientSecret` | `String` | Confidential client secret | -| `machineId` | `String` | Associated machine/tenant reference | +| `tenantId` | `String` | Indexed tenant scope for multi-tenancy isolation | +| `clientId` | `String` | OAuth 2.0 client identifier | +| `clientSecret` | `String` | OAuth 2.0 client secret | +| `machineId` | `String` | Associated machine/service identity | | `redirectUris` | `String[]` | Allowed redirect URIs for authorization flows | -| `grantTypes` | `String[]` | Permitted OAuth grant types | -| `scopes` | `String[]` | Access scopes the client may request | +| `grantTypes` | `String[]` | Supported grant types (`authorization_code`, `password`, `client_credentials`, `refresh_token`) | +| `scopes` | `String[]` | Permitted OAuth scopes | | `roles` | `String[]` | Assigned roles (defaults to empty array) | -| `enabled` | `boolean` | Whether the client is active (defaults to `true`) | - -**Supported Grant Types:** `authorization_code`, `password`, `client_credentials`, `refresh_token` +| `enabled` | `boolean` | Client active status (defaults to `true`) | ## Usage Example ```java -// Creating a new machine-to-machine OAuth client OAuthClient client = new OAuthClient(); -client.setClientId("openframe-service"); +client.setTenantId("tenant-abc"); +client.setClientId("my-service-client"); client.setClientSecret("s3cr3t"); -client.setMachineId("machine-abc123"); -client.setGrantTypes(new String[]{"client_credentials"}); +client.setGrantTypes(new String[]{"client_credentials", "refresh_token"}); client.setScopes(new String[]{"read", "write"}); -client.setRoles(new String[]{"ROLE_SERVICE"}); -client.setRedirectUris(new String[]{}); +client.setRedirectUris(new String[]{"https://app.example.com/callback"}); client.setEnabled(true); -// Persisted via Spring Data MongoDB repository +// Persist via repository oAuthClientRepository.save(client); ``` -> **Note:** `clientSecret` should always be stored hashed. The `machineId` field links this client to a specific OpenFrame machine context, enabling per-tenant OAuth isolation. \ No newline at end of file +> The `tenantId` field is indexed for efficient multi-tenant queries. All OAuth clients are stored in the `oauth_clients` MongoDB collection and implement `TenantScoped` to enforce tenant isolation across the OpenFrame platform. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthToken.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthToken.md index 5fb46df11..9aff8380a 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthToken.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthToken.md @@ -1,27 +1,29 @@ - -A MongoDB document entity representing an OAuth 2.0 token record, storing access and refresh tokens alongside their expiry timestamps, client identity, and granted scopes. + +MongoDB document entity representing an OAuth 2.0 token record scoped to a specific tenant, storing access/refresh tokens with expiry metadata. ## Key Components | Field | Type | Description | -|---|---|---| -| `id` | `String` | MongoDB document identifier (`@Id`) | -| `userId` | `String` | Reference to the owning user | -| `accessToken` | `String` | OAuth access token value | -| `refreshToken` | `String` | OAuth refresh token value | -| `accessTokenExpiry` | `Instant` | UTC expiry timestamp for the access token | -| `refreshTokenExpiry` | `Instant` | UTC expiry timestamp for the refresh token | +|-------|------|-------------| +| `id` | `String` | MongoDB document `_id` | +| `tenantId` | `String` | Indexed tenant identifier (multi-tenancy) | +| `userId` | `String` | Associated user identifier | +| `accessToken` | `String` | OAuth 2.0 access token value | +| `refreshToken` | `String` | OAuth 2.0 refresh token value | +| `accessTokenExpiry` | `Instant` | Access token expiration timestamp | +| `refreshTokenExpiry` | `Instant` | Refresh token expiration timestamp | | `clientId` | `String` | OAuth client application identifier | -| `scopes` | `String[]` | Granted permission scopes for this token | +| `scopes` | `String[]` | Granted OAuth permission scopes | -- **Collection:** `oauth_tokens` -- **Annotations:** `@Data` (Lombok) generates getters, setters, `equals`, `hashCode`, and `toString` +- Implements `TenantScoped` for multi-tenant data isolation +- Stored in the `oauth_tokens` MongoDB collection +- `tenantId` is indexed for efficient tenant-based queries ## Usage Example ```java -// Persisting a new OAuth token OAuthToken token = new OAuthToken(); +token.setTenantId("tenant-abc"); token.setUserId("user-123"); token.setClientId("openframe-client"); token.setAccessToken("eyJhbGciOiJSUzI1NiJ9..."); @@ -30,10 +32,5 @@ token.setAccessTokenExpiry(Instant.now().plusSeconds(3600)); token.setRefreshTokenExpiry(Instant.now().plusSeconds(86400)); token.setScopes(new String[]{"read", "write", "admin"}); -mongoTemplate.save(token); - -// Checking token expiry -boolean isExpired = token.getAccessTokenExpiry().isBefore(Instant.now()); -``` - -> **Note:** Token values should be encrypted at rest. Expiry checks should always use `Instant.now()` to ensure timezone-safe comparisons. \ No newline at end of file +oAuthTokenRepository.save(token); +``` \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/organization/.Organization.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/organization/.Organization.md index 2c8694154..616601010 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/organization/.Organization.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/organization/.Organization.md @@ -1,49 +1,43 @@ - -MongoDB document entity representing a company or client organization within the OpenFrame platform, storing business metadata, contract details, and lifecycle status. + +MongoDB document entity representing a company or client organization within a tenant's scope, storing business metadata, contract details, and lifecycle status. ## Key Components | Member | Type | Description | |--------|------|-------------| -| `organizationId` | `String` | Immutable UUID-based unique identifier (indexed, unique) | -| `isDefault` | `Boolean` | Flags the tenant's default organization | -| `status` | `OrganizationStatus` | Lifecycle state: `ACTIVE`, `ARCHIVED`, or `DELETED` | -| `contactInformation` | `ContactInformation` | Nested contacts and address data | +| `organizationId` | `String` | Immutable UUID-based unique identifier assigned at creation | +| `tenantId` | `String` | Tenant isolation field (implements `TenantScoped`) | +| `isDefault` | `Boolean` | Flags the tenant's default organization (defaults to `false`) | +| `status` | `OrganizationStatus` | Lifecycle state: `ACTIVE`, `ARCHIVED`, or `DELETED` (soft delete) | +| `contactInformation` | `ContactInformation` | Nested object holding contacts and addresses | | `monthlyRevenue` | `BigDecimal` | Revenue figure in the organization's currency | | `contractStartDate` / `contractEndDate` | `LocalDate` | Contract validity window | -| `createdAt` / `updatedAt` | `Instant` | Auto-managed audit timestamps via Spring Data | -| `isContractActive()` | `boolean` | Returns `true` if today falls within the contract window | -| `isDeleted()` | `boolean` | Soft-delete check against `OrganizationStatus.DELETED` | -| `isArchived()` | `boolean` | Checks against `OrganizationStatus.ARCHIVED` | + +**Helper Methods:** + +- `isContractActive()` β€” returns `true` if today falls within the contract date range +- `isDeleted()` β€” checks for `DELETED` status (soft delete guard) +- `isArchived()` β€” checks for `ARCHIVED` status ## Usage Example ```java -// Build a new organization Organization org = Organization.builder() + .tenantId("tenant-123") .organizationId(UUID.randomUUID().toString()) .name("Acme Corp") - .category("IT Services") - .numberOfEmployees(150) + .category("Technology") + .numberOfEmployees(250) .websiteUrl("https://acme.example.com") - .monthlyRevenue(new BigDecimal("25000.00")) + .monthlyRevenue(new BigDecimal("15000.00")) .contractStartDate(LocalDate.of(2024, 1, 1)) - .contractEndDate(LocalDate.of(2025, 1, 1)) + .contractEndDate(LocalDate.of(2025, 12, 31)) .isDefault(true) .build(); // Lifecycle checks -if (org.isContractActive()) { - System.out.println("Contract is currently active"); -} - -if (org.isArchived()) { - System.out.println("Organization is hidden but retained for device references"); -} +if (org.isContractActive()) { /* bill the client */ } +if (org.isArchived()) { /* exclude from active queries */ } ``` -## Notes - -- **Soft deletes only** β€” `DELETED` status is preferred over physical removal to preserve references from devices that may reconnect after going offline. -- `ARCHIVED` hides the organization from normal queries while keeping the document intact. -- `organizationId` is treated as immutable; only `id` (MongoDB `_id`) is managed by the database engine. \ No newline at end of file +> **Note on soft deletion:** `ARCHIVED` hides the organization from normal queries while preserving references needed by devices that may reconnect. `DELETED` marks full soft deletion without removing the document from MongoDB. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.Script.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.Script.md new file mode 100644 index 000000000..cd241cd28 --- /dev/null +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.Script.md @@ -0,0 +1,44 @@ + +MongoDB document entity for RMM scripts in the OpenFrame platform, representing reusable scripts that can be executed on managed agents ad-hoc, on a schedule, or as part of automated checks. + +## Key Components + +| Field | Type | Description | +|-------|------|-------------| +| `id` | `String` | MongoDB document ID | +| `tenantId` | `String` | Multi-tenant isolation key (via `TenantScoped`) | +| `name` | `String` | Unique script name per tenant | +| `shell` | `ScriptShell` | Interpreter type (PowerShell, CMD, Bash, Python) | +| `scriptBody` | `String` | Raw script source stored inline | +| `supportedPlatforms` | `List` | Target OS compatibility | +| `defaultTimeoutSeconds` | `Integer` | Agent-enforced execution timeout | +| `defaultArgs` | `List` | Default positional CLI arguments | +| `envVars` | `List` | Environment variables (secret handling pending) | +| `status` | `ScriptStatus` | Lifecycle state, defaults to `ACTIVE` | + +**Indexes:** +- Compound unique index on `(tenantId, name)` β€” enforces name uniqueness per tenant +- Single-field index on `status` β€” supports efficient lifecycle queries + +**Soft-delete:** `DELETED` status preserves document integrity for historic execution record references. + +## Usage Example + +```java +Script script = Script.builder() + .tenantId("tenant-abc") + .name("Disk Cleanup") + .description("Clears temp files on Windows agents") + .shell(ScriptShell.POWERSHELL) + .scriptBody("Remove-Item -Path $env:TEMP\\* -Recurse -Force") + .tag("maintenance") + .supportedPlatforms(List.of(ScriptPlatform.WINDOWS)) + .defaultTimeoutSeconds(120) + .defaultArgs(List.of()) + .envVars(List.of()) + .build(); + +scriptRepository.save(script); +``` + +> **Note:** `envVars` entries flagged as `secret` require encrypted delivery to agents β€” secure transport is tracked as a future follow-up. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptEnvVar.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptEnvVar.md new file mode 100644 index 000000000..3a774f89d --- /dev/null +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptEnvVar.md @@ -0,0 +1,32 @@ + +Represents a single environment variable to be exported on an RMM agent prior to script execution. This class is embedded within the `Script` document and is not persisted as a standalone MongoDB collection. + +## Key Components + +| Field | Type | Description | +|-------|------|-------------| +| `name` | `String` | The environment variable name as exported on the agent | +| `value` | `String` | The variable's value; intended to hold ciphertext when `secret = true`, but currently stored in plaintext | +| `secret` | `boolean` | Marks the variable as sensitive; the service layer currently rejects `secret = true` on write until secure delivery is implemented | + +## Usage Example + +```java +// Plain environment variable +ScriptEnvVar plainVar = ScriptEnvVar.builder() + .name("API_BASE_URL") + .value("https://api.example.com") + .secret(false) + .build(); + +// Secret variable (write currently rejected by service layer) +ScriptEnvVar secretVar = ScriptEnvVar.builder() + .name("API_KEY") + .value("supersecret") + .secret(true) + .build(); +``` + +## Notes + +> **⚠️ Secret management is not yet implemented.** When `secret = true`, the `value` field is intended to store encrypted ciphertext delivered to agents over a secure channel. Until that story lands, the service layer rejects `secret = true` on write to prevent plaintext secrets from being persisted to MongoDB. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptPlatform.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptPlatform.md new file mode 100644 index 000000000..475c82c88 --- /dev/null +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptPlatform.md @@ -0,0 +1,24 @@ + +Enum defining the supported operating-system platforms for RMM script execution targeting. + +## Key Components + +| Value | Description | +|-------|-------------| +| `WINDOWS` | Microsoft Windows environments | +| `LINUX` | Linux-based operating systems | +| `MACOS` | Apple macOS environments | + +## Usage Example + +```java +Script script = new Script(); +script.setPlatform(ScriptPlatform.LINUX); + +// Filter scripts by platform +List" β†’ stripped -// " tag" β†’ escaped to <their> -// "javascript:alert(1)" href β†’ removed +\`\`\`typescript +const x: Array = []; +\`\`\` +`; ``` -> **Note:** This renderer is designed specifically for untrusted LLM output. The sanitization pipeline runs in layers: text pre-processing β†’ `rehype-raw` β†’ `rehypeStripUnsafe` β†’ `rehype-highlight`, ensuring no XSS vectors survive even if one layer is bypassed. Community support is available on the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA). \ No newline at end of file +> **Security model:** `escapeUnknownHtmlTags` handles accidental LLM tag emissions at the text level; `rehypeStripUnsafe` is the authoritative XSS boundary at the HAST level β€” safe even if the text pre-pass is bypassed. \ No newline at end of file diff --git a/openframe-frontend-core/src/components/ui/.square-avatar.md b/openframe-frontend-core/src/components/ui/.square-avatar.md index 058e7d479..e921d363a 100644 --- a/openframe-frontend-core/src/components/ui/.square-avatar.md +++ b/openframe-frontend-core/src/components/ui/.square-avatar.md @@ -1,49 +1,52 @@ - -A flexible avatar component that displays a user image or falls back to initials, supporting both square and round variants with multiple size options. + +A flexible avatar component that renders a user image with an automatic initials-based fallback, supporting multiple sizes and shape variants. ## Key Components ### `SquareAvatarProps` -Extends `React.HTMLAttributes` with: - | Prop | Type | Default | Description | |------|------|---------|-------------| | `src` | `string` | β€” | Image URL | | `alt` | `string` | β€” | Image alt text (also used for initials fallback) | | `size` | `'sm' \| 'md' \| 'lg' \| 'xl'` | `'md'` | Controls dimensions (32/40/48/64px) | -| `fallback` | `string` | β€” | Preferred string for generating initials | +| `fallback` | `string` | β€” | Preferred source string for generating initials | | `variant` | `'square' \| 'round'` | `'square'` | Border radius style | +| `initialsClassName` | `string` | β€” | Tailwind overrides for initials font size/color | ### `SquareAvatar` -A memoized, forwarded-ref component that renders an image with automatic graceful degradation to initials on load error or missing `src`. - -**Fallback behavior:** Uses `getFirstLastInitials()` on `fallback ?? alt`, rendering `'?'` if neither yields initials. The initials layer uses `text-ods-text-primary` to maintain WCAG AA contrast across accent-fill backgrounds (e.g. Mingo cyan, current-user pink). +A memoized, forwarded-ref `div` component. Renders an optimized `next/image` when `src` is provided; falls back to extracted initials via `getFirstLastInitials()`. On image load error, the image is hidden and initials are revealed automatically. ## Usage Example ```typescript -// Image avatar +// Basic image avatar + +// Initials-only fallback (no src) + -// Initials-only avatar (no src) +// Compact avatar with custom initials styling -// With custom className and forwarded ref -const ref = React.useRef(null); +// Accent-fill background (e.g. current user indicator) -``` \ No newline at end of file +``` + +> **Accessibility note:** Initials use `text-ods-text-primary` to maintain WCAG AA contrast on both default (`bg-ods-bg`) and accent-fill backgrounds (`bg-ods-flamingo-pink`, `bg-ods-flamingo-cyan`). \ No newline at end of file diff --git a/openframe-frontend-core/src/components/ui/.tags-input.md b/openframe-frontend-core/src/components/ui/.tags-input.md index 0d2cc4ba2..dec3679c4 100644 --- a/openframe-frontend-core/src/components/ui/.tags-input.md +++ b/openframe-frontend-core/src/components/ui/.tags-input.md @@ -1,51 +1,43 @@ - -A React component that provides an interactive tags input interface, allowing users to add, remove, and manage a collection of string tags with keyboard shortcuts and validation. + +A client-side React component for managing a list of string tags with add/remove functionality, keyboard support, and optional tag limits. ## Key Components -- **TagsInput**: Main component that manages tag state and user interactions -- **handleAddTag**: Adds new tags with duplicate checking and max limit validation -- **handleRemoveTag**: Removes tags from the collection -- **handleKeyPress**: Enables Enter key for adding tags -- **TagsInputProps**: Interface defining component configuration options +**`TagsInput`** β€” Main exported component accepting the following props: + +| Prop | Type | Description | +|------|------|-------------| +| `value` | `string[]` | Controlled array of current tags | +| `onChange` | `(tags: string[]) => void` | Callback fired on tag add/remove | +| `placeholder` | `string` | Input placeholder text | +| `disabled` | `boolean` | Disables all interactions | +| `maxTags` | `number` | Optional upper limit on tag count | +| `inputClassName` | `string` | Extra classes for the text input | +| `badgeClassName` | `string` | Extra classes for tag badges | +| `label` | `string` | Optional label rendered above the input | + +**Internal behavior:** +- Trims whitespace before adding; silently ignores duplicates +- Pressing `Enter` or clicking **Add** adds the current input value +- Input and Add button are auto-disabled when `maxTags` is reached +- Displays a `{current}/{max} tags` counter when `maxTags` is set ## Usage Example ```typescript -import { TagsInput } from "./tags-input" -import { useState } from "react" +import { TagsInput } from "@/components/ui/tags-input" -function MyForm() { - const [tags, setTags] = useState([]) +function TicketForm() { + const [tags, setTags] = React.useState(["network", "urgent"]) return ( ) } - -// With initial tags -function PrefilledTags() { - const [technologies, setTechnologies] = useState([ - "React", "TypeScript", "Node.js" - ]) - - return ( - - ) -} -``` - -The component features duplicate prevention, optional tag limits, keyboard navigation, customizable styling, and accessibility support with proper ARIA labels. \ No newline at end of file +``` \ No newline at end of file diff --git a/openframe-frontend-core/src/components/ui/.textarea.md b/openframe-frontend-core/src/components/ui/.textarea.md index 70e6a7d89..dbafd07f4 100644 --- a/openframe-frontend-core/src/components/ui/.textarea.md +++ b/openframe-frontend-core/src/components/ui/.textarea.md @@ -1,27 +1,31 @@ - -A client-side textarea component supporting labels, validation states, and optional end icons (as decorative spans or interactive buttons), built on top of `FieldWrapper`. + +A client-side textarea component that extends the native HTML `