Implementing ZooKeeper persistent watcher into discovery service. - #728
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
💤 Files with no reviewable changes (1)
📝 WalkthroughWalkthroughDiscoveryService now combines persistent ZooKeeper watch events with periodic full rescans, maintains separate raw and transformed caches, returns isolated discovery data, and warns when the legacy advertised-instance API is used. The kazoo requirement now points to a Git repository. ChangesDiscovery synchronization and cache
Dependency sourcing
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant ZooKeeper
participant DiscoveryService
participant MainLoop
participant DiscoveryCache
ZooKeeper->>DiscoveryService: Ready notification
DiscoveryService->>ZooKeeper: Install persistent recursive watch
ZooKeeper-->>DiscoveryService: Node event and JSON data
DiscoveryService->>MainLoop: Forward event
MainLoop->>DiscoveryService: Update _advertised_raw
DiscoveryService->>DiscoveryCache: Apply normalized advertised state
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
asab/api/discovery.py (1)
46-58:⚠️ Potential issue | 🟡 MinorSilence the unused PubSub argument.
Ruff flags
msgas unused in_on_tick600. Rename to_msg(or_) to keep the signature but avoid lint noise.🛠️ Proposed fix
- def _on_tick600(self, msg): + def _on_tick600(self, _msg):
🤖 Fix all issues with AI agents
In `@asab/api/discovery.py`:
- Line 86: Rename the unused local variable stat to _stat in the unpacking
assignments from self.ZooKeeperContainer.ZooKeeper.Client.get(...) to satisfy
lint; specifically change occurrences like "data, stat =
self.ZooKeeperContainer.ZooKeeper.Client.get(...)" to "data, _stat =
self.ZooKeeperContainer.ZooKeeper.Client.get(...)" for both places (the one at
the top-level discovery logic and the later occurrence around the same
Client.get call).
- Around line 236-293: In _apply_advertised_raw, the loop over web entries
extracts ip and port via ip = i[0]; port = i[1] but only catches KeyError, so
malformed entries can raise IndexError or TypeError and crash the method; update
the error handling around the ip/port extraction in the for i in web block to
either validate i is a sequence with at least two elements before indexing or
catch IndexError and TypeError (e.g., except (IndexError, TypeError, KeyError):)
and log the unexpected format using the existing L.error message, then continue;
this change targets the ip/port extraction lines and the except clause in
_apply_advertised_raw.
- Around line 72-77: The handler _on_change_threadsafe should guard against
WatchedEvent with path=None and use the KazooState enum; change the initial
check to compare event.state to KazooState.CONNECTED and return early if not
matched, and before slicing event.path using BasePath ensure event.path is not
None (or fall back to an empty string or skip calling _on_change), then call
App.TaskService.schedule_threadsafe(self._on_change(event.path[len(self.BasePath)
+ 1:], event.type)) only when event.path is present to avoid the TypeError.
- Around line 60-69: _on_zk_ready currently registers a persistent watch every
time a CONNECTED transition occurs, causing duplicate events; add a one-time
flag (e.g., self._zk_watch_registered) on the instance to guard the
ZooKeeper.Client.add_watch call so the watch is only registered once, set the
flag immediately after successful registration, and ensure you still call
self.App.TaskService.schedule(self._rescan_advertised_instances()) as before;
also rename the unused msg parameter (e.g., to _msg or remove it) to reflect it
is unused and avoid lint warnings; locate these changes in the _on_zk_ready
method and reference ZooKeeperContainer, BasePath, _on_change_threadsafe, and
_rescan_advertised_instances when applying the fix.
🧹 Nitpick comments (1)
asab/api/discovery.py (1)
221-231: Avoid swallowing unexpected errors in rescan.The broad
except Exceptioncan mask programming errors and leave stale cache state. Consider re‑raising unexpected exceptions after logging, or narrowing the handler.🛠️ Proposed fix (re‑raise after logging)
- except Exception: - L.exception("Error when scanning advertised instances") - return + except Exception: + L.exception("Error when scanning advertised instances") + raise
There was a problem hiding this comment.
Pull request overview
This PR migrates ASAB's Discovery service from traditional ZooKeeper watches to persistent watches (introduced in ZooKeeper 3.6). This reduces load on ZooKeeper by establishing a single persistent watch instead of repeatedly setting watches on each read operation.
Changes:
- Switched from per-operation watches to a single persistent recursive watch on the base path
- Optimized rescan frequency from 5 minutes to 10 minutes due to improved real-time change detection
- Deprecated
get_advertised_instances()method in favor ofdiscover()
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| setup.py | Updated kazoo dependency to use vendored version with persistent watch support |
| asab/api/discovery.py | Implemented persistent watch mechanism, refactored cache update logic, and deprecated old method |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| 'aiohttp>=3.8.3,<4', | ||
| 'fastjsonschema>=2.16.2,<3', | ||
| 'kazoo>=2.9.0,<3', | ||
| 'git+https://github.com/TeskaLabs/kazoo.git', |
There was a problem hiding this comment.
Installing directly from a git repository URL without specifying a commit hash, tag, or branch makes builds non-reproducible and potentially unstable. Consider pinning to a specific commit SHA or tag (e.g., 'git+https://github.com/TeskaLabs/kazoo.git@') to ensure consistent builds across environments.
| 'git+https://github.com/TeskaLabs/kazoo.git', | |
| 'git+https://github.com/TeskaLabs/kazoo.git@master', |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@asab/api/discovery.py`:
- Around line 55-57: The handler _on_tick600 currently defines an unused
parameter named msg; rename it to _msg to indicate it is intentionally unused by
updating the function signature def _on_tick600(self, msg): → def
_on_tick600(self, _msg): in asab.api.discovery so linters understand the
argument is unused; leave the body (self.App.TaskService.schedule(...))
unchanged and update any internal references if the parameter is later used.
In `@setup.py`:
- Line 88: Replace the unpinned VCS requirement
'git+https://github.com/TeskaLabs/kazoo.git' with a PEP 508 direct reference
pinned to a specific tag or commit (for example: 'kazoo @
git+https://github.com/TeskaLabs/kazoo.git@<TAG_OR_COMMIT>#egg=kazoo'); update
the dependency string in setup.py accordingly so the installer pulls the fixed
commit/tag instead of the floating branch.
🧹 Nitpick comments (1)
asab/api/discovery.py (1)
223-233: Don’t swallow task cancellation in rescan.Catching
Exceptionhere can maskasyncio.CancelledError. Re-raise cancellations explicitly.🔧 Suggested change
- except Exception: + except asyncio.CancelledError: + raise + except Exception: L.exception("Error when scanning advertised instances") return
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (3)
asab/api/discovery.py (2)
60-71:⚠️ Potential issue | 🟠 MajorPersistent watch registered on every reconnect leads to duplicate events.
The
_on_zk_readyhandler fires on everyCONNECTEDstate transition (including reconnects). Each invocation registers another persistent watch viaadd_watch(). Without a guard, this accumulates duplicate watchers that deliver duplicate events.Add a one-time flag to prevent re-registration:
Proposed fix
def __init__(self, app, zkc, service_name="asab.DiscoveryService") -> None: ... self._cache_lock = asyncio.Lock() self._ready_event = asyncio.Event() + self._zk_watch_installed = False self.App.PubSub.subscribe("Application.tick/600!", self._on_tick600) ... def _on_zk_ready(self, msg, zkcontainer): if zkcontainer != self.ZooKeeperContainer: return self.App.TaskService.schedule(self._rescan_advertised_instances()) - zkcontainer.ZooKeeper.Client.add_watch( - self.BasePath, - self._on_change_zookeeper_thread, - kazoo.protocol.states.AddWatchMode.PERSISTENT_RECURSIVE - ) + if not self._zk_watch_installed: + zkcontainer.ZooKeeper.Client.add_watch( + self.BasePath, + self._on_change_zookeeper_thread, + kazoo.protocol.states.AddWatchMode.PERSISTENT_RECURSIVE + ) + self._zk_watch_installed = True🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@asab/api/discovery.py` around lines 60 - 71, The _on_zk_ready handler currently calls zkcontainer.ZooKeeper.Client.add_watch every time it sees the ZooKeeperContainer (including reconnects), causing duplicate persistent watches; add a one-time guard flag (e.g., self._zk_watch_installed) on the class so that in _on_zk_ready you only call add_watch and schedule the rescan when the flag is false, then set the flag to True after successful registration; reference the symbols _on_zk_ready, ZooKeeperContainer, self.App.TaskService.schedule(self._rescan_advertised_instances()), self.BasePath, self._on_change_zookeeper_thread and add_watch to locate and change the code.
74-82:⚠️ Potential issue | 🟠 MajorCompare
event.stateagainstKazooStateenum, not a string literal.Line 75 compares
event.stateto the string'CONNECTED', but Kazoo'sWatchedEvent.stateis aKazooStateenum. This comparison may silently fail depending on how the enum's__eq__is implemented. Use the proper enum for robustness:+import kazoo.protocol.states + def _on_change_zookeeper_thread(self, event): - if event.state != 'CONNECTED': + if event.state != kazoo.protocol.states.KazooState.CONNECTED: returnNote:
kazoo.protocol.statesis already imported forAddWatchMode, so just use the full path or add an alias.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@asab/api/discovery.py` around lines 74 - 82, The check in _on_change_zookeeper_thread currently compares event.state to the string 'CONNECTED'; change it to compare against the KazooState enum (e.g. KazooState.CONNECTED) to ensure a reliable enum comparison. Update the code in the _on_change_zookeeper_thread function to use kazoo.protocol.states.KazooState.CONNECTED (or add an import alias for KazooState) instead of the string literal, leaving the rest of the method (including the event.path checks and the call to self.App.TaskService.schedule_threadsafe(self._on_change(...))) unchanged.setup.py (1)
88-88:⚠️ Potential issue | 🟠 MajorPin the Git dependency to a specific commit or tag for reproducible builds.
The unpinned VCS requirement can silently pull breaking changes and makes builds non-deterministic. While the PR notes this is temporary until the upstream Kazoo PR is merged, pinning to a specific commit ensures stability:
- 'kazoo @ git+https://github.com/TeskaLabs/kazoo.git', + 'kazoo @ git+https://github.com/TeskaLabs/kazoo.git@<commit-sha>',#!/bin/bash # Get the latest commit SHA from the TeskaLabs/kazoo fork to suggest pinning curl -s https://api.github.com/repos/TeskaLabs/kazoo/commits/master | jq -r '.sha'🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@setup.py` at line 88, Replace the unpinned VCS dependency string 'kazoo @ git+https://github.com/TeskaLabs/kazoo.git' with a pinned reference to a specific commit SHA or tag so installs are reproducible; update the dependency entry in setup.py to include the commit or tag suffix (e.g., append @<commit-or-tag>) and keep the package name identifier so pip can resolve it, choosing the latest stable commit from the TeskaLabs/kazoo fork until the upstream PR is merged.
🧹 Nitpick comments (1)
asab/api/discovery.py (1)
234-238: Consider narrowing the exception catch.The broad
except Exceptionat line 236 (flagged by Ruff BLE001) catches all exceptions including programming errors. While theCancelledErrorre-raise is correct, consider catching more specific exceptions (e.g.,kazoo.exceptions.KazooException,json.JSONDecodeError) to avoid masking unexpected bugs.That said, for a background rescan task, defensive broad catching with logging may be acceptable to prevent service crashes.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@asab/api/discovery.py` around lines 234 - 238, Replace the broad except Exception in the background scan that currently logs "Error when scanning advertised instances" with narrow exception handlers: catch kazoo.exceptions.KazooException and json.JSONDecodeError (and optionally ValueError/KeyError if parsing dicts) as specific exceptions and log them via L.exception including the exception info; keep the asyncio.CancelledError re-raise as-is; if you still want a defensive fallback, add a final generic except Exception as e that logs "Unexpected error when scanning advertised instances" with exc_info but does not swallow critical errors silently. Use the unique symbols L.exception and the scan coroutine (the block that logs "Error when scanning advertised instances") to locate and update the handlers.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@asab/api/discovery.py`:
- Around line 240-249: Race window: callers (_rescan_advertised_instances and
_on_change) release _cache_lock then call _apply_advertised_raw which
re-acquires the lock, allowing concurrent mutation of _advertised_raw; fix by
having _apply_advertised_raw operate on a snapshot passed in by callers. Change
_apply_advertised_raw to accept an advertised snapshot argument (e.g.
advertised_snapshot) and have callers (_rescan_advertised_instances and
_on_change) create a shallow/deep copy of _advertised_raw while holding
_cache_lock and pass that copy to _apply_advertised_raw; remove or avoid
re-acquiring _cache_lock inside _apply_advertised_raw so it processes the stable
snapshot without race.
- Around line 85-110: The _on_change coroutine currently calls the blocking
kazoo Client.get() directly (in _on_change) which will block the asyncio loop;
wrap the blocking call in ProactorService.execute (same pattern used in
_iter_zk_items) so ZooKeeper.Client.get(self.BasePath + '/' + item) runs in the
threadpool and returns its result to the async function, preserving the existing
exception handling for SessionExpiredError, ConnectionLoss and NoNodeError; also
replace string literal event_type checks in _on_change with
kazoo.protocol.states.EventType (e.g., EventType.CREATED, EventType.CHANGED,
EventType.DELETED) to make the comparisons consistent.
---
Duplicate comments:
In `@asab/api/discovery.py`:
- Around line 60-71: The _on_zk_ready handler currently calls
zkcontainer.ZooKeeper.Client.add_watch every time it sees the ZooKeeperContainer
(including reconnects), causing duplicate persistent watches; add a one-time
guard flag (e.g., self._zk_watch_installed) on the class so that in _on_zk_ready
you only call add_watch and schedule the rescan when the flag is false, then set
the flag to True after successful registration; reference the symbols
_on_zk_ready, ZooKeeperContainer,
self.App.TaskService.schedule(self._rescan_advertised_instances()),
self.BasePath, self._on_change_zookeeper_thread and add_watch to locate and
change the code.
- Around line 74-82: The check in _on_change_zookeeper_thread currently compares
event.state to the string 'CONNECTED'; change it to compare against the
KazooState enum (e.g. KazooState.CONNECTED) to ensure a reliable enum
comparison. Update the code in the _on_change_zookeeper_thread function to use
kazoo.protocol.states.KazooState.CONNECTED (or add an import alias for
KazooState) instead of the string literal, leaving the rest of the method
(including the event.path checks and the call to
self.App.TaskService.schedule_threadsafe(self._on_change(...))) unchanged.
In `@setup.py`:
- Line 88: Replace the unpinned VCS dependency string 'kazoo @
git+https://github.com/TeskaLabs/kazoo.git' with a pinned reference to a
specific commit SHA or tag so installs are reproducible; update the dependency
entry in setup.py to include the commit or tag suffix (e.g., append
@<commit-or-tag>) and keep the package name identifier so pip can resolve it,
choosing the latest stable commit from the TeskaLabs/kazoo fork until the
upstream PR is merged.
---
Nitpick comments:
In `@asab/api/discovery.py`:
- Around line 234-238: Replace the broad except Exception in the background scan
that currently logs "Error when scanning advertised instances" with narrow
exception handlers: catch kazoo.exceptions.KazooException and
json.JSONDecodeError (and optionally ValueError/KeyError if parsing dicts) as
specific exceptions and log them via L.exception including the exception info;
keep the asyncio.CancelledError re-raise as-is; if you still want a defensive
fallback, add a final generic except Exception as e that logs "Unexpected error
when scanning advertised instances" with exc_info but does not swallow critical
errors silently. Use the unique symbols L.exception and the scan coroutine (the
block that logs "Error when scanning advertised instances") to locate and update
the handlers.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: c9c9e630-0ae5-4375-a162-50f92c06a98c
📒 Files selected for processing (2)
asab/api/discovery.pysetup.py
|
@coderabbitai review |
✅ Action performedReview finished.
|
Co-authored-by: Cursor <[email protected]>
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
asab/api/discovery.py (1)
78-100: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winUnhandled ZooKeeper exceptions in
_on_change_zookeeper_threadcan silently drop updates.Line 96 calls
self.ZooKeeperContainer.ZooKeeper.Client.get(event.path)directly with no exception handling. This runs on the Kazoo callback thread, not the asyncio loop. If the node was deleted between the watch firing and this call (a realistic race for rapid create/delete),NoNodeErroris raised; a connection hiccup can also raiseConnectionLoss/SessionExpiredError. An unhandled exception here means_on_changeis never scheduled for that event, so_advertised_rawsilently misses the update until the next periodic rescan (now every 10 minutes per theApplication.tick/600!change), leaving stale discovery data for an extended window.Suggested fix
# Handle the change event in the thread-safe manner in the main event loop thread if event.type == 'CREATED' or event.type == 'CHANGED': # We are on the zookeeper thread, so we can directly get the data - data, _ = self.ZooKeeperContainer.ZooKeeper.Client.get(event.path) + try: + data, _ = self.ZooKeeperContainer.ZooKeeper.Client.get(event.path) + except kazoo.exceptions.NoNodeError: + # Node was deleted before we could read it; treat as a deletion + event = event._replace(type='DELETED') + data = None + except (kazoo.exceptions.SessionExpiredError, kazoo.exceptions.ConnectionLoss): + L.warning("Connection to ZooKeeper lost while handling change for '{}'".format(event.path)) + return else: data = None🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@asab/api/discovery.py` around lines 78 - 100, Update _on_change_zookeeper_thread to handle exceptions from Client.get(event.path) on the ZooKeeper callback thread, including node deletion and transient connection/session failures, so an exception cannot prevent scheduling _on_change. Preserve the existing data=None behavior for non-created/changed events and ensure each watch event still reaches the main event loop with an appropriate payload.
♻️ Duplicate comments (1)
asab/api/discovery.py (1)
63-75: 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy liftPersistent watch is re-installed on every reconnect, contradicting the comment's own claim.
The comment states persistent watches "survive reconnect of the same session" and says to "re-install only when needed," but the code unconditionally calls
add_watch()on everyZooKeeperContainer.state/CONNECTED!event, which fires on every (re)connection, not just the first. This was previously flagged as a major concern requesting an install-once guard; no such guard (e.g., a_zk_watch_installedflag) exists in__init__(Lines 39-51). If the underlying session survives the reconnect, this re-registers a redundant watch each time; if watcher identity isn't perfectly deduplicated by the vendored Kazoo fork, this can accumulate duplicate registrations over the life of a long-running service.Separately, the unused
msgparameter (Line 63) was also flagged for renaming to_msgin the earlier review but remains unchanged.Suggested fix
self._cache_lock = asyncio.Lock() self._rescan_lock = asyncio.Lock() self._ready_event = asyncio.Event() self._rescan_requested = False + self._zk_watch_installed = False @@ - def _on_zk_ready(self, msg, zkcontainer): + def _on_zk_ready(self, _msg, zkcontainer): if zkcontainer != self.ZooKeeperContainer: return self.App.TaskService.schedule(self._rescan_advertised_instances()) # Persistent watches survive reconnect of the same session, but are # cleared on session loss. Re-install only when needed. - zkcontainer.ZooKeeper.Client.add_watch( - self.BasePath, - self._on_change_zookeeper_thread, - kazoo.protocol.states.AddWatchMode.PERSISTENT_RECURSIVE - ) + if not self._zk_watch_installed: + zkcontainer.ZooKeeper.Client.add_watch( + self.BasePath, + self._on_change_zookeeper_thread, + kazoo.protocol.states.AddWatchMode.PERSISTENT_RECURSIVE + ) + self._zk_watch_installed = TruePlease confirm with the vendored Kazoo fork (TeskaLabs, pending upstream PR
#715) whether repeatedadd_watch()calls with the same bound-method watcher on an already-watched path are safely deduplicated across reconnects of the same session.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@asab/api/discovery.py` around lines 63 - 75, Update _on_zk_ready to install the persistent watch only when it is not already active, using an instance guard initialized in __init__; reset that guard when the ZooKeeper session is lost so the watch is reinstalled for a new session. Rename the unused msg parameter to _msg, and preserve the existing container check and rescan scheduling.
🧹 Nitpick comments (2)
test/test_discovery/test_on_change.py (1)
25-51: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMutable class-level
MOCK_DATArisks test-order-dependent leakage.
MOCK_DATAis a class attribute, andsetUp()passesself.MOCK_DATA(the same shared dict object) directly toMockZooKeeperContainerwithout copying._set_zk_node/_delete_zk_nodethen mutate this shared dict in place.test_on_change_changedsets the node to port 9000 and never resets/deletes it, so that mutated state persists into subsequent test methods'setUp/rescan. Current tests happen to pass because each explicitly overwrites the checked item before asserting, but this is fragile for future tests and is flagged by Ruff's RUF012 (mutable default class attribute).Suggested fix
def setUp(self): super().setUp() - self.MockedZKC = MockZooKeeperContainer(mock_data=self.MOCK_DATA) + self.MockedZKC = MockZooKeeperContainer(mock_data=copy.deepcopy(self.MOCK_DATA)) self.DiscoveryService = DiscoveryService(self.App, zkc=self.MockedZKC) self.App.Loop.run_until_complete(self.DiscoveryService._rescan_advertised_instances())Note this requires
_set_zk_node/_delete_zk_nodeto mutateself.MockedZKC's copy rather thanself.MOCK_DATAdirectly — verify againstMockZooKeeperContainer's stored attribute name inbaseclass.py.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/test_discovery/test_on_change.py` around lines 25 - 51, Make the test fixture data isolated per test: replace the mutable class-level MOCK_DATA usage with a fresh deep copy in setUp when constructing MockZooKeeperContainer, and update _set_zk_node and _delete_zk_node to mutate the container’s stored mock-data attribute from baseclass.py rather than self.MOCK_DATA. Preserve the existing rescan setup behavior and verify the correct MockZooKeeperContainer attribute name before changing the helpers.Source: Linters/SAST tools
asab/api/discovery.py (1)
103-121: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueStale comment: no ZooKeeper read happens in
_on_changeanymore.The comment "Read ZooKeeper outside the cache lock to avoid blocking other updates" (Line 105) is now inaccurate — the ZK read was moved to
_on_change_zookeeper_thread(Line 96); this function only deserializes already-fetcheddata. Leaving the stale comment could mislead future readers about where the blocking I/O occurs.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@asab/api/discovery.py` around lines 103 - 121, The comment in _on_change is stale because this function only deserializes the provided data; remove or replace it with wording describing the current operation, while keeping the ZooKeeper read explanation in _on_change_zookeeper_thread where the I/O occurs.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@asab/api/discovery.py`:
- Around line 78-100: Update _on_change_zookeeper_thread to handle exceptions
from Client.get(event.path) on the ZooKeeper callback thread, including node
deletion and transient connection/session failures, so an exception cannot
prevent scheduling _on_change. Preserve the existing data=None behavior for
non-created/changed events and ensure each watch event still reaches the main
event loop with an appropriate payload.
---
Duplicate comments:
In `@asab/api/discovery.py`:
- Around line 63-75: Update _on_zk_ready to install the persistent watch only
when it is not already active, using an instance guard initialized in __init__;
reset that guard when the ZooKeeper session is lost so the watch is reinstalled
for a new session. Rename the unused msg parameter to _msg, and preserve the
existing container check and rescan scheduling.
---
Nitpick comments:
In `@asab/api/discovery.py`:
- Around line 103-121: The comment in _on_change is stale because this function
only deserializes the provided data; remove or replace it with wording
describing the current operation, while keeping the ZooKeeper read explanation
in _on_change_zookeeper_thread where the I/O occurs.
In `@test/test_discovery/test_on_change.py`:
- Around line 25-51: Make the test fixture data isolated per test: replace the
mutable class-level MOCK_DATA usage with a fresh deep copy in setUp when
constructing MockZooKeeperContainer, and update _set_zk_node and _delete_zk_node
to mutate the container’s stored mock-data attribute from baseclass.py rather
than self.MOCK_DATA. Preserve the existing rescan setup behavior and verify the
correct MockZooKeeperContainer attribute name before changing the helpers.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 78ca6581-c1bd-4983-bd2e-05b2bdd6e6d3
📒 Files selected for processing (2)
asab/api/discovery.pytest/test_discovery/test_on_change.py
This PR introduces a significant change in how ASAB reacts to changes in Apache Zookeeper.
It leverages "persistent watches" - introduces in Zookeeper in version 3.6 and being introduced in Kazoo.
This PR particularly introduces this change into a Discovery service; the result is much more lighter (and more logical) pressure on the Apache Zookeeper.
There are other place in ASAB that can switch to this change
IMPORTANT: We need to use vendored version of Kazoo from https://github.com/TeskaLabs/kazoo till python-zk/kazoo#715 is merged.
Summary by CodeRabbit
locate(),discover(), anddiscover_raw()return types/semantics and strengthened copy guarantees.get_advertised_instances()with a deprecation warning.session()auth typing.kazooto be installed from its Git source.