Add API endpoint and UI for deadline callback log visibility#66610
Add API endpoint and UI for deadline callback log visibility#66610seanghaeli wants to merge 6 commits into
Conversation
2b1e618 to
9102967
Compare
e303ead to
f6103b3
Compare
7f042b1 to
b300616
Compare
|
I like the interface but I would like to have @bbovenzi opinion on this one |
e8a1b93 to
528ce64
Compare
368fb19 to
8ce33e6
Compare
|
Why is the callback logs a modal, rather than a tab? (I'm not saying it's wrong, I would just like to know your thinking and what made you chose this over another option) Additionally we should likely use the existing log component so that it gets the same filtering, formatting, and grouping, not just displayed as plain text. |
|
@ashb I thought about making the logs appear in a new tab next to Details where the rest of the logs are, but this page only appears when a deadline is missed, so grouping it up with the “Deadline Missed” section makes more spatial sense in my opinion. |
6ab1317 to
1c8fc49
Compare
|
@ashb updated to use |
ddaab67 to
7567838
Compare
SameerMesiah97
left a comment
There was a problem hiding this comment.
I have not inspected the diff itself, but I saw your dev list post requesting feedback on potential UI additions.
Overall, I think you've covered most of what users would reasonably expect to see for DeadlineAlerts in the UI. One possible enhancement would be surfacing a few additional pieces of metadata:
- Configured interval value
- Interval source (
timedeltavsVariableInterval) - Callback type (for example,
SlackWebhookNotifier)
Users can always inspect the DAG definition for this information, but exposing it directly in the UI would make deadline investgation more self-service. For e.g., if critical SLA is missed and a DeadlineAlert fires, an operator or stakeholder could immediately understand what deadline was configured and what notification mechanism was invoked without needing to inspect the DAG code.
Add a callback-logs API endpoint and UI viewer so users can see the logs produced during async (triggerer) and executor deadline callback execution, matching the task-log UX. Includes the callback log reader, the CallbackLogViewer component, and triggerer-side callback log capture/upload.
ab9c123 to
299fe32
Compare
|
Thanks @SameerMesiah97, that's good feedback. I'll leave it out of this PR so we're making less assumptions about what the user wants to see, but will keep this in mind for a follow-up |
Yes. Personally, it’s up to you (and the AWS team). Those were just my thoughts when I saw your dev list post. |
|
Can you update the screenshots to reflect the last changes? |
Refreshed deadline_status.png (run-page Deadline section: Missed badge, Callback Logs button, Expected/Actual) and callback_logs_modal.png (the structured Callback Logs modal with the line-numbered log viewer + source details) to match the latest UI on apache#66610. Per @vincbeck's request.
|
@vincbeck Screenshots in the PR description are up to date now |
…anceState Address review feedback on the deadline callback-log viewer: - Replace the hand-rolled stateColorMap + Badge with the shared StateBadge component (maps state to colorPalette and adds the state icon), matching the usage in Run/Details. - Type the callbackState prop as TaskInstanceState instead of a bare string; cast dl.callback_state at the two call sites since the generated API schema types it as string.
…ructuredLog) The prettier import-sort is case-sensitive (uppercase before lowercase), so StateBadge must precede renderStructuredLog. Fixes the ts-compile-lint-ui static check.
Summary
Make deadline callback logs visible in the Airflow UI. When a deadline is missed and a callback fires, users can now view the callback's execution logs directly from the web interface.
Changes
Backend:
CallbackLogReaderutility (airflow-core/src/airflow/utils/log/callback_log_reader.py) — reads callback logs from remote or local storage using the sameRemoteLogIOinfrastructure as task logsGET /ui/dags/{dag_id}/dagRuns/{dag_run_id}/callbacks/{callback_id}/logson the deadlines UI routerDeadlineResponsedatamodel — addedcallback_idandcallback_statefieldsFrontend:
CallbackLogViewer.tsxcomponent — fetches and displays callback execution logs in a modal dialogMotivation
Kaxil requested this: "Some way to know if a deadline didn't fire and it failed: something on UI would be great, like Task log. If deadlines are visible, it would be even better experience than previous SLAs."
Dependencies
executor_callbacks/{dag_id}/{run_id}/{callback_id}Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.6) following the guidelines
End-to-End Verification
Verified in breeze with full Airflow stack (scheduler + dag-processor + triggerer + API server + LocalExecutor). A test DAG with a
SyncCallbackdeadline was parsed, unpaused, scheduled, and the deadline missed naturally. The callback executed throughsupervise_callback()and produced real structured logs.Dag Run Page — Deadline Status with Callback
Callback Logs Modal
Verified:
test_callback_logs.py)test_callback_log_reader.py)os.path.commonpathcontainment check