Filter DCB subscriptions server-side - #223
Conversation
Qodana Community for JVM76 new problems were found
View the detailed Qodana reportTo be able to view the detailed Qodana report, you can either:
To get - name: 'Qodana Scan'
uses: JetBrains/[email protected]
with:
upload-result: trueContact Qodana teamContact us at [email protected]
|
There was a problem hiding this comment.
Pull request overview
This PR introduces server-side filtering for Dynamic Consistency Boundary (DCB) subscriptions by making DcbSubscriptionFilter (wrapping a DcbQuery) a first-class SubscriptionFilter, and translating it into a MongoDB change stream $match stage (plus equivalent in-memory matching). It reduces wasted delivery/consumer-side filtering by pushing DCB query semantics down to capable backends while retaining an in-process correctness floor where needed.
Changes:
- Add
DcbSubscriptionFilterand a sharedDcbQuery -> change stream $matchconverter, and route MongoDB (Spring + native) subscription models through it. - Update the DCB DSL (
DcbSubscriptions) to subscribe withDcbSubscriptionFilterwhile keeping post-filtering as a correctness floor. - Remove the overly broad filter-type guard from the DCB catch-up path (keep it only for the stream catch-up path) and expose the
dcbTagsstorage field constant as a shared contract.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| subscription/util/blocking/catchup-subscription/src/main/java/org/occurrent/subscription/blocking/durable/catchup/CatchupSubscriptionModel.java | Moves the “only stream filter supported” guard into the stream path so DCB mode can accept DCB filters. |
| subscription/mongodb/spring/common/src/main/java/org/occurrent/subscription/mongodb/spring/internal/ApplyFilterToChangeStreamOptionsBuilder.java | Adds server-side change stream filtering for DcbSubscriptionFilter in Spring Mongo subscriptions. |
| subscription/mongodb/native/blocking/src/main/java/org/occurrent/subscription/mongodb/nativedriver/blocking/NativeMongoSubscriptionModel.java | Adds server-side change stream filtering for DcbSubscriptionFilter in the native Mongo driver subscription model. |
| subscription/mongodb/common/base/src/main/java/org/occurrent/subscription/mongodb/internal/DcbSubscriptionFilterConverter.java | Implements conversion from DcbQuery to a single change stream $match stage (incl. dcbposition > 0 guard). |
| subscription/mongodb/common/base/src/test/java/org/occurrent/subscription/mongodb/internal/DcbSubscriptionFilterConverterTest.java | Adds unit tests validating the $match stage structure produced for various DcbQuery shapes. |
| subscription/mongodb/common/base/pom.xml | Adds DCB API dependency and test dependencies needed for the new converter tests. |
| subscription/inmemory/src/main/java/org/occurrent/subscription/inmemory/InMemorySubscriptionModel.java | Extends in-memory subscriptions to honor DcbSubscriptionFilter by matching DCB queries in-process. |
| subscription/inmemory/src/main/java/org/occurrent/subscription/inmemory/InMemorySubscription.java | Switches from holding a stream Filter to a general Predicate<CloudEvent> matcher. |
| subscription/inmemory/src/test/java/org/occurrent/subscription/inmemory/InMemorySubscriptionModelDcbFilterTest.java | Adds in-memory tests ensuring DCB filter matching works (tags, types, and position guard). |
| subscription/inmemory/pom.xml | Adds DCB API dependency required for in-memory DCB filtering. |
| subscription/core/src/main/java/org/occurrent/subscription/DcbSubscriptionFilter.java | Introduces DcbSubscriptionFilter as a SubscriptionFilter implementation wrapping a DcbQuery. |
| subscription/core/pom.xml | Adds DCB API dependency to expose DcbSubscriptionFilter in subscription core. |
| framework/spring-boot-starter-mongodb/src/test/java/org/occurrent/springboot/mongo/blocking/DcbServerSideSubscriptionFilteringMongoTest.java | Adds end-to-end Spring Boot test proving server-side DCB change stream filtering works. |
| eventstore/mongodb/spring/blocking/src/main/java/org/occurrent/eventstore/mongodb/spring/blocking/SpringMongoEventStore.java | Removes duplicated dcbTags field constant and uses the mapper’s shared constant. |
| eventstore/mongodb/common/src/main/java/org/occurrent/eventstore/mongodb/internal/OccurrentCloudEventMongoDocumentMapper.java | Promotes DCB_TAGS_INDEX_FIELD to a public constant as a storage contract shared by writer and subscriptions. |
| dsl/dcb-dsl/blocking/src/main/kotlin/org/occurrent/dsl/dcb/blocking/DcbSubscriptions.kt | Updates Kotlin DCB DSL to subscribe using DcbSubscriptionFilter instead of subscribing with Filter.all(). |
| dsl/dcb-dsl/blocking/src/main/java/org/occurrent/dsl/dcb/blocking/DcbSubscriptions.java | Updates Java DCB DSL to subscribe using DcbSubscriptionFilter instead of subscribing with Filter.all(). |
| doc/architecture/decisions/0023-server-side-filtering-for-dcb-subscriptions.md | Adds ADR documenting the design decision and tradeoffs for server-side DCB subscription filtering. |
| changelog.md | Documents the new server-side filtering behavior and the new public constant contract. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
3a1e398 to
0fffed8
Compare
0ed1b59 to
c02696c
Compare
0fffed8 to
30f0efd
Compare
f55c26c to
fdc1470
Compare
16c732c to
91fc0f0
Compare
DcbSubscriptions subscribed with Filter.all() and post-filtered every delivered
CloudEvent in process, so a DCB read model that cares about a few types or a tag
boundary still received the whole DCB stream. Add DcbSubscriptionFilter(DcbQuery)
as a first-class SubscriptionFilter and translate it to a MongoDB change stream
$match so DCB events are filtered server-side.
- DcbSubscriptionFilter (subscription-core), the DCB counterpart to
OccurrentSubscriptionFilter.
- DcbSubscriptionFilterConverter (subscription-mongodb-base) builds one {$match}
Document (fullDocument-prefixed: dcbposition>0 always, types $in, excluded
types $nin, tags $all on the dcbTags array, items $or), used by both the Spring
and native change-stream models.
- InMemorySubscriptionModel honors DcbSubscriptionFilter in process; the
subscription now holds a Predicate<CloudEvent> instead of a stream Filter.
- CatchupSubscriptionModel's filter-type guard moves into the stream arm so the
DCB arm passes the filter through to the inner model.
- DcbSubscriptions (Java and Kotlin) passes DcbSubscriptionFilter(query) and
keeps the in-process check as a correctness floor for backends that do not
filter.
- Promote OccurrentCloudEventMongoDocumentMapper.DCB_TAGS_INDEX_FIELD to public
so the event store that writes the dcbTags array and the subscription that
matches against it share one definition.
See ADR 0023.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
fdc1470 to
d0a71df
Compare
91fc0f0 to
22b6240
Compare
|
Landed via the consolidated #234. |
DCB subscriptions filtered every event in process.
DcbSubscriptionssubscribed withFilter.all()and threw away most events in the consumer thread, so a read model that cares about a few event types or one tag boundary still received the whole DCB stream. Under load that is a lot of wasted delivery.This makes DCB filtering server-side.
DcbSubscriptionFilter, wrapping aDcbQuery, is now a first-classSubscriptionFilteralongside the streamOccurrentSubscriptionFilter. One converter turns aDcbQueryinto a single change stream$match: types to$in, excluded types to$nin, tags to$allon the indexeddcbTagsarray, items OR-ed, anddcbposition > 0always so only DCB events arrive. The Spring and native MongoDB models both use it. The in-memory model matches the same query in process.DcbSubscriptionsnow subscribes with the filter and keeps its in-process check only as a correctness floor, since it wraps an arbitrarySubscribableand cannot assume the backend filters. A capable backend reduces the volume, the floor guarantees correctness everywhere.CatchupSubscriptionModelhad one upfront guard that rejected every filter except the stream one. That guard belongs to the stream catch-up path, which converts the filter into an OccurrentFilter, so it now lives only there. The DCB path passes the filter straight to the inner model.dcbTagsfield name was a private constant duplicated in the event store. It is now the publicOccurrentCloudEventMongoDocumentMapper.DCB_TAGS_INDEX_FIELD, the one storage contract the writer and the subscription reader share.On the open question of whether to split the
Filterclass itself: no.SubscriptionFilteris already the common seam, so the clean split is streamFilterfor stream subscriptions andDcbQueryfor DCB subscriptions, each an implementation over shared Mongo plumbing. There is no shared structure today that a common filter base would capture without inventing it. ADR 0023 records this.Verification
rtk mvn -pl subscription/mongodb/common/base -am -Dtest=DcbSubscriptionFilterConverterTest test: Tests run: 8, Failures: 0, Errors: 0.rtk mvn -pl subscription/inmemory -Dtest=InMemorySubscriptionModelDcbFilterTest testplus the existing in-memory suite: green.rtk mvn -pl framework/spring-boot-starter-mongodb -Dtest=DcbServerSideSubscriptionFilteringMongoTest test: Tests run: 4, Failures: 0, Errors: 0. This subscribes the model directly, not throughDcbSubscriptions, so a non-matching delivered event would mean the server-side$matchis wrong.CatchupSubscriptionModelTest,DcbCatchupSubscriptionModelTest,DcbCatchupSubscriptionModelMongoTest(23 total), plus the starterDcbCatchupSubscriptionAutoConfigurationMongoTestandOccurrentMongoAutoConfigurationCombinedModeTest: green.rtk mvn -pl dsl/dcb-dsl/blocking test: green, including the DCB DSL routing through the new filter.