UNOMI-968: Pluggable persistence IT harness for ES, OS, and other backends

itests/README.md

@@ -32,7 +32,9 @@ and data migration.
   `docker-maven-plugin`).
 - **Maven Failsafe** runs a single entry point — `AllITs` — which aggregates all test
   classes. Each test class extends `BaseIT`, which handles Karaf startup, OSGi service
-  injection, and common test utilities.
+  injection, and common test utilities. Persistence is selected via
+  `unomi.persistence.provider` (Elasticsearch / OpenSearch today; see
+  [Pluggable persistence providers](#pluggable-persistence-providers)).
 
 A full IT run typically takes 20–30 minutes. The Karaf instance is created fresh for each
 run under `itests/target/exam/` with a UUID directory name.
@@ -284,6 +286,98 @@ See the [Maven Failsafe plugin docs](https://maven.apache.org/surefire/maven-fai
 
 ---
 
+## Pluggable persistence providers
+
+Unomi’s runtime persistence is an OSGi capability (`unomi.persistence;provider:=…`).
+The IT harness follows the same idea: `BaseIT` resolves a test-only
+`PersistenceITBackend` instead of hard-coding Elasticsearch / OpenSearch.
+
+### Provider selection
+
+| Property | Role |
+|----------|------|
+| `unomi.persistence.provider` | Preferred provider id (`elasticsearch`, `opensearch`, or an extension id) |
+| `unomi.search.engine` | **Deprecated alias** — used when `unomi.persistence.provider` is unset |
+| `unomi.persistence.it.backend` | Optional FQCN of a `PersistenceITBackend` implementation |
+
+Default CI cells set both `unomi.persistence.provider` and the deprecated
+`unomi.search.engine` so existing `--use-opensearch` / scripts keep working.
+
+### Progress / ETA (local timing cache)
+
+`ProgressSuite` + `ProgressListener` write a best-effort per-test duration cache under
+the `itests` module directory (survives `mvn clean`):
+
+`.test-timing-cache-<provider>.properties`
+
+One file per persistence provider (`elasticsearch`, `opensearch`, `postgresql`, …)
+so ETAs are not mixed across backends. On later runs the listener sums remaining
+historical times and scales them by how fast/slow the current run is vs history
+(clamped). Safe to delete; missing/unwritable cache falls back to in-run averages.
+
+### Built-in backends
+
+| Id | Class | Suite for CI |
+|----|-------|--------------|
+| `elasticsearch` | `ElasticsearchITBackend` | `AllITs` (default) |
+| `opensearch` | `OpenSearchITBackend` | `AllITs` with `-Duse.opensearch=true` |
+
+### Search-only vs core behavioural tests
+
+`AllITs` and `CorePersistenceITs` share the **same** test membership for maximum coverage.
+Tests that need HTTP admin / snapshot / rollover APIs stay in the suite and use
+`Assume` + `PersistenceITCapabilities` so unsupported backends **skip** (reported as
+skipped, not green false-pass):
+
+| Class | Gate |
+|-------|------|
+| `Migrate16xToCurrentVersionIT` | `snapshotRestoreMigration()` |
+| `RolloverIT` | `indexRolloverApi().isPresent()` + `httpAdminApi()`; switch on `LIFECYCLE` / `STATE_MANAGEMENT` |
+| `HealthCheckIT` | Always asserts `karaf` / `unomi` / `persistence`; optional `providerNamedHealthProbe` / `clusterHealthProbe` |
+
+`HealthCheckIT` is not search-only: it runs on every provider.
+
+Elasticsearch / OpenSearch CI continues to use `AllITs`. Portable / non-search cells
+(PostgreSQL, …) should run `CorePersistenceITs` (identical membership, clearer name).
+
+```bash
+mvn clean install -P integration-tests -Dit.test=org.apache.unomi.itests.CorePersistenceITs
+```
+
+Capability flags gate remaining special-cases
+(e.g. `snapshotRestoreMigration`, `flattenedRangeQueryResult`, `indexRolloverApi`)
+so providers skip what they cannot support instead of checking product names.
+
+### Adding a third-party backend
+
+1. Implement `org.apache.unomi.itests.persistence.PersistenceITBackend`
+   (feature options, distribution name for `unomi:setup`, ConfigAdmin PID,
+   capabilities, await-ready / HTTP helpers as needed).
+2. Put the implementation on the Failsafe test classpath.
+3. Register it either:
+   - in `META-INF/services/org.apache.unomi.itests.persistence.PersistenceITBackend`, or
+   - via `-Dunomi.persistence.it.backend=com.example.MyPersistenceITBackend`
+4. Select it with `-Dunomi.persistence.provider=<your-id>`.
+5. Run `CorePersistenceITs` (same membership as `AllITs`). Incomplete SPI
+   implementations may still fail behavioural tests — unsupported HTTP-admin
+   features are skipped via `Assume`, not removed from the suite.
+
+`unomi-itests` publishes a **test-jar** (`mvn -Pintegration-tests -pl itests install`) so external modules can
+depend on `BaseIT`, `CorePersistenceITs`, and `PersistenceITBackend` without forking
+sources. Optionally implement `prepareBeforeUnomiSetup` to patch ConfigAdmin (e.g. JDBC
+DataSource) after `UnomiManagementService` is up and before `unomi:setup`.
+
+Example Failsafe exclude when running `AllITs` against a non-search provider:
+
+```xml
+<excludedGroups>org.apache.unomi.itests.persistence.SearchBackendIT</excludedGroups>
+```
+
+(Note: JUnit 4 category filtering applies to classes Failsafe launches directly;
+prefer `CorePersistenceITs` when using the suite entry point.)
+
+---
+
 ## Debugging Integration Tests
 
 ### Attaching a remote debugger to the test JVM

itests/pom.xml

@@ -28,6 +28,7 @@
     <description>Apache Unomi Context Server integration tests</description>
 
     <properties>
+        <unomi.persistence.provider>elasticsearch</unomi.persistence.provider>
         <unomi.search.engine>elasticsearch</unomi.search.engine>
         <use.opensearch>false</use.opensearch>
         <docker.container.name>itests-opensearch</docker.container.name>
@@ -277,6 +278,33 @@
                     </systemPropertyVariables>
                 </configuration>
             </plugin>
+            <!-- Publish test classes so out-of-tree PersistenceService providers can implement
+                 PersistenceITBackend and reuse CorePersistenceITs / BaseIT (UNOMI-968). -->
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-jar-plugin</artifactId>
+                <executions>
+                    <execution>
+                        <id>test-jar</id>
+                        <goals>
+                            <goal>test-jar</goal>
+                        </goals>
+                        <configuration>
+                            <!-- Include IT classes and test resources (fixtures used by BaseIT.config). -->
+                            <includes>
+                                <include>org/apache/unomi/itests/**</include>
+                                <include>**/*.cfg</include>
+                                <include>**/*.csv</include>
+                                <include>**/*.json</include>
+                                <include>**/*.groovy</include>
+                                <include>migration/**</include>
+                                <include>schemas/**</include>
+                                <include>META-INF/**</include>
+                            </includes>
+                        </configuration>
+                    </execution>
+                </executions>
+            </plugin>
         </plugins>
     </build>
 
@@ -306,6 +334,8 @@
                             </includes>
                             <systemPropertyVariables>
                                 <my.system.property>foo</my.system.property>
+                                <unomi.persistence.provider>elasticsearch</unomi.persistence.provider>
+                                <!-- Deprecated alias; keep for older scripts -->
                                 <unomi.search.engine>elasticsearch</unomi.search.engine>
                                 <elasticsearch.port>${elasticsearch.port}</elasticsearch.port>
                                 <it.karaf.heap>${karaf.heap}</it.karaf.heap>
@@ -432,6 +462,8 @@
                             </includes>
                             <systemPropertyVariables>
                                 <my.system.property>foo</my.system.property>
+                                <unomi.persistence.provider>opensearch</unomi.persistence.provider>
+                                <!-- Deprecated alias; keep for older scripts -->
                                 <unomi.search.engine>opensearch</unomi.search.engine>
                                 <org.apache.unomi.opensearch.addresses>localhost:${opensearch.port}</org.apache.unomi.opensearch.addresses>
                                 <org.ops4j.pax.logging.DefaultServiceLog.level>INFO</org.ops4j.pax.logging.DefaultServiceLog.level>

itests/src/test/java/org/apache/unomi/itests/AllITs.java

@@ -24,7 +24,9 @@
 import org.junit.runners.Suite.SuiteClasses;
 
 /**
- * Defines suite of test classes to run.
+ * Defines suite of test classes to run (Elasticsearch / OpenSearch CI default).
+ * Same membership as {@link CorePersistenceITs}; use capabilities + {@code Assume} for
+ * provider-specific skips rather than maintaining a smaller suite.
  *
  * @author Sergiy Shyrkov
  */