populate task name cache from worker inspect#32
Conversation
Current Aviator status
This PR was merged using Aviator.
See the real-time status of this PR on the
Aviator webapp.
Use the Aviator Chrome Extension
to see the status of your PR within GitHub.
|
There was a problem hiding this comment.
Code Review
This pull request introduces thread-safe task name management in EventWatcher by implementing a lock and refactoring name resolution. WorkerWatcher is updated to populate this cache during its periodic updates, and the project version is bumped to 0.1.11. Feedback suggests optimizing EventWatcher.on_event to avoid redundant lock acquisitions when the task name is already present in the event data.
| if "name" in event: | ||
| self._task_names_by_uuid.pop(uuid, None) | ||
| self._task_names_by_uuid[uuid] = event["name"] | ||
| while len(self._task_names_by_uuid) > self._TASK_NAMES_CACHE_LIMIT: | ||
| self._task_names_by_uuid.popitem(last=False) | ||
| self.record_task_name(uuid, event["name"]) | ||
|
|
||
| task_name = self._task_names_by_uuid.get(uuid, "(UNKNOWN)") | ||
| task_name = self._resolve_task_name(uuid) |
There was a problem hiding this comment.
The current implementation acquires the _task_names_by_uuid_lock twice when a task name is present in the event: once during record_task_name and again during _resolve_task_name. This can be optimized by using the name directly from the event when available, avoiding the second lock acquisition and dictionary lookup.
| if "name" in event: | |
| self._task_names_by_uuid.pop(uuid, None) | |
| self._task_names_by_uuid[uuid] = event["name"] | |
| while len(self._task_names_by_uuid) > self._TASK_NAMES_CACHE_LIMIT: | |
| self._task_names_by_uuid.popitem(last=False) | |
| self.record_task_name(uuid, event["name"]) | |
| task_name = self._task_names_by_uuid.get(uuid, "(UNKNOWN)") | |
| task_name = self._resolve_task_name(uuid) | |
| if "name" in event: | |
| task_name = event["name"] | |
| self.record_task_name(uuid, task_name) | |
| else: | |
| task_name = self._resolve_task_name(uuid) |
c9cdc7d to
f46f738
Compare
f46f738 to
cd078cf
Compare
keeps the event-driven runtime histogram from bucketing tasks as (UNKNOWN) after celerymon restarts, since the worker watcher already has the names from inspect.active