Skip to content

fix(review): fix durationSeconds null bug, async misuse, param ordering, and Octokit type cast#24

Merged
adamhenson merged 8 commits into
mainfrom
fix/code-review
Jun 21, 2026
Merged

fix(review): fix durationSeconds null bug, async misuse, param ordering, and Octokit type cast#24
adamhenson merged 8 commits into
mainfrom
fix/code-review

Conversation

@adamhenson

Copy link
Copy Markdown
Contributor

Summary

  • run.ts: durationSeconds now returns null instead of 0 when timestamps are unavailable — fixes misleading "0s duration" being sent to the ingest API for runs with unknown timing
  • workflow-run.ts: removed spurious async from parseAgentTokensZip (uses only unzipSync + JSON.parse, both synchronous); fixed alphabetical param ordering in resolveTrigger, parseAgentTokensArtifact, parseAgentStdioLog; added missing inline JSDoc on parseAgentTokensArtifact/parseAgentStdioLog params
  • comment.ts: replaced @octokit/core Octokit import + runtime type casts with a local type Octokit = ReturnType<typeof github.getOctokit> alias; fixed alphabetical param ordering and added missing inline JSDoc in findExistingComment/upsertComment

Test plan

  • npm run type-check passes (verified locally)
  • npm run lint passes with no warnings (verified locally)
  • All 112 tests pass (npm test) (verified locally)
  • Confirm durationSeconds: null reaches the API when run timestamps are absent (workflow_run mode with no timestamps)

…ng, and Octokit type cast

- run.ts: durationSeconds returns null (not 0) when timestamps are unavailable,
  fixing misleading "0s duration" in API payloads for runs with unknown timing
- workflow-run.ts: remove async from parseAgentTokensZip — it uses only unzipSync
  and JSON.parse, both synchronous; the async wrapper added unnecessary overhead
- workflow-run.ts: fix alphabetical param ordering in resolveTrigger,
  parseAgentTokensArtifact, and parseAgentStdioLog; add missing inline JSDoc
  comments on parseAgentTokensArtifact and parseAgentStdioLog params
- comment.ts: replace @octokit/core Octokit import + runtime type casts with a
  local type alias from @actions/github getOctokit, eliminating unsafe assertions
- comment.ts: fix alphabetical param ordering in findExistingComment and
  upsertComment; add missing inline JSDoc on findExistingComment params
@github-actions

github-actions Bot commented Jun 21, 2026

Copy link
Copy Markdown

⚡ AgentMeter

# Workflow Model Status Cost Duration
1 Agent: Code Review claude-sonnet-4-5 $0.32 5m
2 Agent: Code Review (Codex) gpt-5.4-mini $0.71 3m
3 AgentMeter — Inline Test claude-sonnet-4-5 $0.01 11s
4 Agent: Code Review claude-sonnet-4-5 $0.30 5m
5 Agent: Code Review (Codex) gpt-5.4-mini $0.74 2m
Total $2.50
All 7 runs
# Workflow Model Status Cost Duration
1 Agent: Code Review claude-sonnet-4-5 $0.32 5m
2 Agent: Code Review (Codex) gpt-5.4-mini $0.71 3m
3 AgentMeter — Inline Test claude-sonnet-4-5 $0.01 11s
4 Agent: Code Review claude-sonnet-4-5 $0.30 5m
5 Agent: Code Review (Codex) gpt-5.4-mini $0.74 2m
6 AgentMeter — Inline Test claude-sonnet-4-5 $0.01 11s
7 Agent: Code Review claude-sonnet-4-5 $0.41 6m
Total $2.50
Token breakdown
Type Tokens Cost
Input 50 $0.0002
Output 5,179 $0.08
Cache writes 49,803 $0.19
Cache reads 178,979 $0.05

Model: claude-sonnet-4-5 · 7 turns · 78% cache hit rate

View in AgentMeter →

@github-actions

Copy link
Copy Markdown

Findings:

  • src/workflow-run.ts:204-219: checkConclusionJobCompleted() fails closed on any GitHub API error after two attempts, and run.ts then returns before submitRun(). In practice, a transient actions:read outage or a token-scoping issue will suppress ingestion entirely, even though the action already has a githubRunId dedupe key and could still record the run. This is a production data-loss path. Consider degrading more gracefully, or at least making the skip explicit and observable.

  • src/workflow-run.ts:305-313: The code trusts pull_requests[0] without validating head_sha, while the fallback branch lookup does SHA validation. If GitHub returns multiple associated PRs or a stale first entry for a reused branch, the run/comment can be attributed to the wrong PR. I’d apply the same SHA-match logic here before accepting the first PR entry.

Overall the implementation is well structured, and the test coverage around token extraction, comment updates, and workflow-run resolution is a good sign. The main remaining risk is correctness under GitHub API edge cases, especially in workflow_run mode.

…A in fast path

checkConclusionJobCompleted now fails open (returns true) when both API attempts
fail, rather than silently dropping the ingest. The server deduplicates by
githubRunId so proceeding on an uncertain gate is safer than data loss.

resolveTrigger now validates pull_requests entries against head_sha before
accepting the first entry, matching the same SHA-validation logic already used in
the branch-lookup fallback. On a SHA mismatch it falls through to the branch
lookup rather than trusting a potentially stale entry.

Tests: updated fail-closed → fail-open expectation; added two new tests covering
SHA match success and SHA-mismatch fallthrough in the pull_requests fast path.
@github-actions

Copy link
Copy Markdown

Two correctness issues stood out:

  1. workflow_run status is not actually auto-resolved
    In src/run.ts:70-75, resolveWorkflowRun() is called with rawConclusion: inputs.status, so the companion workflow still depends on the caller passing status correctly. In this mode the action can silently record a failed/timed-out run as success if the input is omitted or miswired, because action.yml defaults status to success. If workflow_run_id is intended to handle status normalization automatically, this should read from github.event.workflow_run.conclusion (or github.context.payload.workflow_run.conclusion) instead of the input.

  2. Missing timestamps are still sent as empty strings
    In src/run.ts:90-97 and src/run.ts:190-191, when the agent run timestamps cannot be resolved, the code emits '' for startedAt / completedAt rather than omitting them or using null. That can produce invalid ISO-8601 fields in the ingest payload and makes downstream validation brittle. The comment says duration will be omitted, but the payload still carries malformed timestamps. I’d either leave both fields unset/nullable in this case or skip ingesting timestamp fields entirely when either side is unavailable.

Everything else looked broadly solid, especially the PR/issue resolution and artifact parsing logic.

…l for missing timestamps

workflow_run status: rawConclusion in workflow_run mode now reads from
github.context.payload.workflow_run.conclusion instead of inputs.status.
The action.yml default is 'success', which would silently misattribute
failed/timed-out runs when callers omit the status input. Falls back to
inputs.status for backward compatibility when the payload field is absent.

Timestamps: resolvedStartedAt / resolvedCompletedAt now default to null
(not '') when unavailable in workflow_run mode. IngestPayload.startedAt
and completedAt are updated to string | null. durationSeconds computation
handles null timestamps directly rather than relying on new Date('') → NaN.
This prevents invalid ISO-8601 empty strings from reaching the ingest API.
@github-actions

Copy link
Copy Markdown

One regression to watch:

  • src/workflow-run.ts:213-220 changes the conclusion-job gate from fail-closed to fail-open. If listJobsForWorkflowRun is transiently unavailable or the token is under-scoped, the action will now continue into fetchRun/fetchAgentTokens and submit an incomplete run. That can create duplicates or partial records unless the ingest API is guaranteed to upsert strictly by githubRunId. I’d keep this closed on API failure unless that backend contract is explicit.

One additional risk:

  • src/run.ts:96-97 and src/types.ts:109-112 now allow startedAt/completedAt to be null and durationSeconds to be null. That looks intentional, but make sure the ingest endpoint and dashboard already accept nulls here. If either still expects strings/number-only, workflow_run mode will start failing for the runs where GitHub timestamps are unavailable.

Otherwise the PR looks directionally good, especially the SHA validation for PR attribution and the comment-upsert typing cleanup.

…rom payload

Conclusion gate: revert fail-open (previous commit) back to fail-closed.
Without a guaranteed backend upsert-by-githubRunId contract, failing open
risks duplicate or partial records on every gh-aw firing (~5x per run).
Upgraded the terminal failure from core.warning to core.notice so it
surfaces in the job summary and is not silently swallowed.

Timestamps: change IngestPayload.startedAt/completedAt from string|null to
optional string (?:). Payload construction now uses spread to omit the
fields entirely when null, rather than sending explicit null/empty values.
This is universally safe across API implementations that may not accept null.
@github-actions

Copy link
Copy Markdown

Findings

  • [src/comment.ts](/home/runner/work/agentmeter-action/agentmeter-action/src/comment.ts#L122) keeps appending the full run history into the <details> block with no cap. On long-lived PRs this will eventually exceed GitHub’s comment size limit, and updateComment will start failing, so the action stops updating the PR comment. Add a hard cap or prune older rows from the hidden history.
  • [src/workflow-run.ts](/home/runner/work/agentmeter-action/agentmeter-action/src/workflow-run.ts#L196) treats any non-completed read of the conclusion job as a permanent skip. If GitHub’s jobs API is briefly behind the event stream, the terminal firing can be misread as “not yet completed” and the ingest is dropped forever. This should retry or re-check before skipping.
  • [src/run.ts](/home/runner/work/agentmeter-action/agentmeter-action/src/run.ts#L127) sets isApproximate from baseTokens, so if agent_output is present and a user also passes explicit token counts, exact manual counts can still be labeled approximate. That leaks into the dashboard/comment and makes authoritative data look uncertain.

Overall the structure is solid, but I’d fix those before merging.

…ith explicit overrides

comment.ts: add MAX_STORED_RUNS=20 cap on allRuns to prevent the comment
from growing unboundedly and eventually exceeding GitHub's 65 536-byte limit.
Collapsible summary label switches to 'Last 20 runs' when the cap is hit.

workflow-run.ts: refactor checkConclusionJobCompleted to re-check once after
a 3s delay when the conclusion job reads in_progress. A single stale jobs-API
read should not permanently drop an ingest — the workflow_run event can arrive
before the API reflects the terminal job state. Tests use vi.useFakeTimers()
to avoid real delays; new test covers the lag→completed path.

run.ts: isApproximate is now always false when the caller provides any explicit
token override. Previously the flag could leak from regex-extracted baseTokens
even when authoritative manual counts were supplied, making exact data appear
uncertain in the dashboard and comment.
@github-actions

Copy link
Copy Markdown

Findings:

  1. Long-lived PR comments will lose history after the 20-run cap kicks in. In src/comment.ts:127 the summary label changes to Last 20 runs, but parseExistingRuns() in src/comment.ts:279-285 still only recognizes <summary>All N runs</summary>. Once a comment is capped, the next update falls back to the visible 5-row table and drops the older hidden rows. That regresses the new size-limit protection into history loss on subsequent runs. Update the parser to accept both labels, or keep the collapsible label stable.

  2. Mixed explicit/extracted token payloads are now marked as exact even when they are still partially approximate. In src/run.ts:128-140, isApproximate is forced to false whenever any explicit token override is present. But the code still fills missing fields from workflowRunTokens or extractedTokens, which can come from regex/artifact parsing and remain approximate. This will mislabel run data in the API/comment whenever a caller overrides only one or two token fields. Preserve the source approximation flag unless all token fields are explicitly provided.

…erride check

comment.ts: parseExistingRuns regex now matches both 'All N runs' and
'Last N runs' summary labels, so the collapsible history is preserved
across updates after the 20-run cap is hit. Without this, capped comments
fell back to the 5-row table and dropped hidden history on the next update.

run.ts: isApproximate is now only forced to false when all four token
fields are explicitly provided (hasAllExplicit). A partial override — e.g.
only input_tokens — leaves the remaining fields sourced from extracted or
artifact data that may be approximate, so the flag from baseTokens is
correctly preserved in that case.
@github-actions

Copy link
Copy Markdown

Two production issues stood out:

  1. workflow_run gating can still duplicate-ingest when the jobs API lags. In src/workflow-run.ts, a missing conclusion job is treated as “not a gh-aw workflow” and proceeds immediately. If GitHub’s jobs list is temporarily stale, a real gh-aw run can look like not_found on the first poll, which defeats the duplicate-prevention gate. The retry/recheck logic only covers in_progress, not this case. Consider rechecking not_found once before proceeding, or only treating it as non-gh-aw after a second confirmatory read.

  2. Trigger resolution is too narrow for companion workflows. In src/workflow-run.ts, the PR lookup fallback only runs for issue_comment, pull_request, and pull_request_review_comment. That means separate-workflow setups triggered by workflow_dispatch, workflow_call, or push that still correspond to a PR will end up with triggerNumber = null, and src/run.ts will skip posting the comment. If this action is meant to support “any agent that runs in a separate workflow,” the fallback should either run for more event types or accept an explicit comment target in workflow-run mode.

Everything else looks reasonably solid, especially the token parsing and comment upsert path.

…kflow

A transiently stale jobs API could return an empty list for a real gh-aw
run, causing the not_found path to proceed without gating and ingest on
all ~5 workflow_run firings. The fix applies the same 3s re-check already
used for in_progress: if not_found is confirmed on the second read we
treat it as non-gh-aw and proceed; if the job appears (completed) we gate
normally. Skipping finding 2 (trigger resolution for workflow_dispatch/push)
as that is a feature request already covered by the explicit trigger_number
input.
@github-actions

Copy link
Copy Markdown
  1. [src/run.ts](/home/runner/work/agentmeter-action/agentmeter-action/src/run.ts#L187) still submits runs when status normalizes to skip. That means an inline job that is actually skipped (status: skipped, e.g. from steps.agent.outcome) will be ingested and possibly commented on, even though [src/workflow-run.ts](/home/runner/work/agentmeter-action/agentmeter-action/src/workflow-run.ts#L124) treats skipped conclusions as “do not ingest”. Add the same early return in inline mode before submitRun().

  2. [src/run.ts](/home/runner/work/agentmeter-action/agentmeter-action/src/run.ts#L141) and [src/token-extractor.ts](/home/runner/work/agentmeter-action/agentmeter-action/src/token-extractor.ts#L321) zero-fill unspecified token fields whenever any explicit override is present. If a caller passes only input_tokens and there is no artifact/extracted fallback, the action sends outputTokens=0, cacheReadTokens=0, and cacheWriteTokens=0, which silently undercounts usage and cost. Missing fields should stay undefined or be resolved from a real fallback source.

  3. [src/ingest.ts](/home/runner/work/agentmeter-action/agentmeter-action/src/ingest.ts#L50) trusts any 2xx JSON response shape, and [src/run.ts](/home/runner/work/agentmeter-action/agentmeter-action/src/run.ts#L210) immediately uses result.id, result.totalCostCents, and result.dashboardUrl. A malformed 200 response from the API would therefore produce bad outputs and comment links instead of failing cleanly. Add minimal schema validation before treating the ingest as successful.

Overall the action wiring looks solid, but these three cases can misreport or mis-ingest production runs.

…sponse shape

run.ts: add early return when normalizeConclusion(inputs.status) === 'skip'
before submitRun(). workflow-run mode already gates on skip (via resolveWorkflowRun),
but inline mode had no equivalent guard — a caller passing status='skipped'
(e.g. steps.agent.outcome) would ingest a run record unnecessarily. Also
hoists normalizedStatus to a const so the two call sites no longer recompute.

ingest.ts: add isValidIngestResponse type guard before returning the parsed
JSON. A malformed 200 response (wrong shape, unexpected envelope) now returns
null + warning instead of passing through undefined fields to outputs and the
PR comment link.
@github-actions

Copy link
Copy Markdown

Findings:

  • src/run.ts ignores the documented pr_number input when posting comments. The comment path only runs when resolvedTriggerNumber is set, so manual/scheduled workflows that set pr_number but have no trigger number will never comment. If pr_number is meant to target the PR for inline runs, this is a functional gap. Consider falling back to inputs.prNumber for comment targeting.

  • src/run.ts src/run.ts workflow_run overrides are not actually respected for the displayed trigger ref. resolvedTriggerRef and resolvedTriggerType from the GitHub API always win over the explicit trigger_number/trigger_event inputs, so a stale or incorrect API lookup cannot be corrected by the documented inputs. That can mislabel comments and ingested runs.

  • src/comment.ts zero-turn runs are dropped from the token breakdown metadata because turns is checked truthily (turns ? ...). If a run legitimately has 0 turns, the comment omits that information. Use an explicit null check instead.

Otherwise the implementation looks solid, and the error handling around API failures is generally sensible.

@adamhenson adamhenson merged commit 5aa20ef into main Jun 21, 2026
6 checks passed
@adamhenson adamhenson deleted the fix/code-review branch June 21, 2026 23:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant