Skip to content

Commit b0c9501

Browse files
authored
feat: add GitHub Action usage metrics for telemetry (#475)
## Description This PR implements the necessary infrastructure in the \`run-gemini-cli\` GitHub Action to support the Gemini CLI usage metrics (P0 requirements). It aligns with the core CLI telemetry changes introduced in [google-gemini/gemini-cli#21129](google-gemini/gemini-cli#21129). ### Changes Made: * **Strongly-Typed Telemetry Tracking:** Replaced the generic \`github_item_number\` with three specific inputs: \`github_pr_number\`, \`github_issue_number\`, and \`github_custom_tracking_id\`. This ensures that Pull Requests, Issues, and custom batch IDs are logged to distinct fields, providing maximum accuracy for downstream analytics without overlapping ID spaces. * **Installation Telemetry:** Added \`GH_WORKFLOW_NAME\`, \`GH_PR_NUMBER\`, \`GH_ISSUE_NUMBER\`, and \`GH_CUSTOM_TRACKING_ID\` to the \`Install Gemini CLI\` step to ensure extension installations are properly tracked. * **OTEL Collector Enhancement:** Updated the \`scripts/collector-gcp.yaml.template\` to include specific resource attributes for \`github.pr.number\`, \`github.issue.number\`, and \`github.custom_tracking.id\`. * **Example Workflow Updates:** Updated all example workflows (Review, Triage, Assistant, Scheduled Triage) to explicitly pass the correct IDs to the action, improving clarity and ensuring accurate telemetry. ### Example Telemetry Scenarios Because the CLI appends these context fields to the \`baseMetadata\` of all events (API Request, Response, Error), downstream analytics can perfectly distinguish and count unique issues vs. PRs based on the specific fields. #### Scenario A: Automated PR Review When PR 42 triggers the \`gemini-review\` workflow, the CLI logs: ```json { \"event_name\": \"api_request\", \"event_metadata\": [ [ { \"gemini_cli_key\": 130, \"value\": \"gemini-review\" }, { \"gemini_cli_key\": 172, \"value\": \"pull_request\" }, { \"gemini_cli_key\": 173, \"value\": \"42\" } // GH_PR_NUMBER ] ] } ``` *Analysts can count \`DISTINCT gh_pr_number WHERE gh_workflow_name LIKE '%review%'\` to satisfy the P0 PR metric.* #### Scenario B: Automated Issue Triage When Issue 88 triggers the \`gemini-triage\` workflow, the CLI logs: ```json { \"event_name\": \"api_request\", \"event_metadata\": [ [ { \"gemini_cli_key\": 130, \"value\": \"gemini-triage\" }, { \"gemini_cli_key\": 172, \"value\": \"issues\" }, { \"gemini_cli_key\": 174, \"value\": \"88\" } // GH_ISSUE_NUMBER ] ] } ``` *Analysts can count \`DISTINCT gh_issue_number WHERE gh_workflow_name LIKE '%triage%' AND gh_event_name = 'issues'\` to satisfy the P0 Automated Issue metric.* #### Scenario C: Scheduled Batch Triage When a cron job runs and triages 3 issues (IDs 101, 102, 103) in a single CLI invocation, the CLI logs: ```json { \"event_name\": \"api_request\", \"event_metadata\": [ [ { \"gemini_cli_key\": 130, \"value\": \"gemini-scheduled-triage\" }, { \"gemini_cli_key\": 172, \"value\": \"schedule\" }, { \"gemini_cli_key\": 175, \"value\": \"101,102,103\" } // GH_CUSTOM_TRACKING_ID ] ] } ``` *Analysts can split the \`gh_custom_tracking_id\` string by commas to count exactly 3 unique issues processed during a scheduled invocation.* ## Related Issues * Fixes P0 metrics requirements for GitHub Action usage. * Depends on [google-gemini/gemini-cli#21129](google-gemini/gemini-cli#21129)
1 parent d87bfa2 commit b0c9501

7 files changed

Lines changed: 42 additions & 0 deletions

File tree

action.yml

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,14 @@ inputs:
9191
description: 'The GitHub workflow name, used for telemetry purposes.'
9292
required: false
9393
default: '${{ github.workflow }}'
94+
github_pr_number:
95+
description: 'The Pull Request number the CLI is operating on. Defaults to the event payload.'
96+
required: false
97+
default: '${{ github.event.pull_request.number }}'
98+
github_issue_number:
99+
description: 'The Issue number (or comma-separated list of issue numbers) the CLI is operating on. Defaults to the event payload.'
100+
required: false
101+
default: '${{ github.event.issue.number }}'
94102

95103
outputs:
96104
summary:
@@ -231,6 +239,10 @@ runs:
231239
GEMINI_CLI_VERSION: '${{ inputs.gemini_cli_version }}'
232240
EXTENSIONS: '${{ inputs.extensions }}'
233241
USE_PNPM: '${{ inputs.use_pnpm }}'
242+
SURFACE: 'GitHub'
243+
GH_WORKFLOW_NAME: '${{ steps.sanitize_workflow_name.outputs.gh_workflow_name }}'
244+
GH_PR_NUMBER: '${{ inputs.github_pr_number }}'
245+
GH_ISSUE_NUMBER: '${{ inputs.github_issue_number }}'
234246
shell: 'bash'
235247
run: |-
236248
set -euo pipefail
@@ -414,6 +426,8 @@ runs:
414426
PROMPT: '${{ inputs.prompt }}'
415427
GEMINI_MODEL: '${{ inputs.gemini_model }}'
416428
GH_WORKFLOW_NAME: '${{ steps.sanitize_workflow_name.outputs.gh_workflow_name }}'
429+
GH_PR_NUMBER: '${{ inputs.github_pr_number }}'
430+
GH_ISSUE_NUMBER: '${{ inputs.github_issue_number }}'
417431

418432
- name: 'Upload Gemini CLI outputs'
419433
if: |-
@@ -440,6 +454,8 @@ runs:
440454
sed -e "s#OTLP_GOOGLE_CLOUD_PROJECT#${OTLP_GOOGLE_CLOUD_PROJECT}#g" \
441455
-e "s#GITHUB_REPOSITORY_PLACEHOLDER#${GITHUB_REPOSITORY}#g" \
442456
-e "s#GITHUB_RUN_ID_PLACEHOLDER#${GITHUB_RUN_ID}#g" \
457+
-e "s#GITHUB_PR_NUMBER_PLACEHOLDER#${GITHUB_PR_NUMBER}#g" \
458+
-e "s#GITHUB_ISSUE_NUMBER_PLACEHOLDER#${GITHUB_ISSUE_NUMBER}#g" \
443459
"${GITHUB_ACTION_PATH}/scripts/collector-gcp.yaml.template" > ".gemini/collector-gcp.yaml"
444460
445461
# Ensure credentials file has the right permissions
@@ -490,6 +506,8 @@ runs:
490506
GITHUB_ACTION_PATH: '${{ github.action_path }}'
491507
GITHUB_REPOSITORY: '${{ github.repository }}'
492508
GITHUB_RUN_ID: '${{ github.run_id }}'
509+
GITHUB_PR_NUMBER: '${{ inputs.github_pr_number }}'
510+
GITHUB_ISSUE_NUMBER: '${{ inputs.github_issue_number }}'
493511

494512
branding:
495513
icon: 'terminal'

examples/workflows/gemini-assistant/gemini-invoke.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,10 @@ jobs:
6666
use_vertex_ai: '${{ vars.GOOGLE_GENAI_USE_VERTEXAI }}'
6767
upload_artifacts: '${{ vars.UPLOAD_ARTIFACTS }}'
6868
workflow_name: 'gemini-invoke'
69+
# Assistant workflows can be triggered by comments on either Issues or PRs.
70+
# We explicitly map both fields so the CLI can correctly categorize the interaction.
71+
github_pr_number: '${{ github.event.pull_request.number }}'
72+
github_issue_number: '${{ github.event.issue.number }}'
6973
settings: |-
7074
{
7175
"model": {

examples/workflows/gemini-assistant/gemini-plan-execute.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,10 @@ jobs:
6868
use_vertex_ai: '${{ vars.GOOGLE_GENAI_USE_VERTEXAI }}'
6969
upload_artifacts: '${{ vars.UPLOAD_ARTIFACTS }}'
7070
workflow_name: 'gemini-plan-execute'
71+
# Assistant workflows can be triggered by comments on either Issues or PRs.
72+
# We explicitly map both fields so the CLI can correctly categorize the interaction.
73+
github_pr_number: '${{ github.event.pull_request.number }}'
74+
github_issue_number: '${{ github.event.issue.number }}'
7175
settings: |-
7276
{
7377
"model": {

examples/workflows/issue-triage/gemini-scheduled-triage.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,9 @@ jobs:
8282
echo '📝 Setting output for GitHub Actions...'
8383
echo "issues_to_triage=${ISSUES}" >> "${GITHUB_OUTPUT}"
8484
85+
ISSUE_NUMBERS="$(echo "${ISSUES}" | jq -r '.[].number | tostring' | paste -sd, -)"
86+
echo "issue_numbers=${ISSUE_NUMBERS}" >> "${GITHUB_OUTPUT}"
87+
8588
ISSUE_COUNT="$(echo "${ISSUES}" | jq 'length')"
8689
echo "✅ Found ${ISSUE_COUNT} issue(s) to triage! 🎯"
8790
@@ -109,6 +112,9 @@ jobs:
109112
use_vertex_ai: '${{ vars.GOOGLE_GENAI_USE_VERTEXAI }}'
110113
upload_artifacts: '${{ vars.UPLOAD_ARTIFACTS }}'
111114
workflow_name: 'gemini-scheduled-triage'
115+
# Overriding default telemetry inputs because scheduled workflows lack an event payload
116+
# We pass the dynamically generated list of batch issues to the issue number field
117+
github_issue_number: '${{ steps.find_issues.outputs.issue_numbers }}'
112118
settings: |-
113119
{
114120
"model": {

examples/workflows/issue-triage/gemini-triage.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,8 @@ jobs:
7979
use_vertex_ai: '${{ vars.GOOGLE_GENAI_USE_VERTEXAI }}'
8080
upload_artifacts: '${{ vars.UPLOAD_ARTIFACTS }}'
8181
workflow_name: 'gemini-triage'
82+
# Explicitly set the issue number to handle `issue_comment` triggers where the context might be ambiguous
83+
github_issue_number: '${{ github.event.issue.number }}'
8284
settings: |-
8385
{
8486
"model": {

examples/workflows/pr-review/gemini-review.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,8 @@ jobs:
6666
use_vertex_ai: '${{ vars.GOOGLE_GENAI_USE_VERTEXAI }}'
6767
upload_artifacts: '${{ vars.UPLOAD_ARTIFACTS }}'
6868
workflow_name: 'gemini-review'
69+
# Explicitly set the PR number to handle `issue_comment` triggers (which GitHub treats as issues, not PRs)
70+
github_pr_number: '${{ github.event.pull_request.number }}'
6971
settings: |-
7072
{
7173
"model": {

scripts/collector-gcp.yaml.template

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,12 @@ processors:
1212
- key: 'github.run_id'
1313
value: 'GITHUB_RUN_ID_PLACEHOLDER'
1414
action: 'upsert'
15+
- key: 'github.pr.number'
16+
value: 'GITHUB_PR_NUMBER_PLACEHOLDER'
17+
action: 'upsert'
18+
- key: 'github.issue.number'
19+
value: 'GITHUB_ISSUE_NUMBER_PLACEHOLDER'
20+
action: 'upsert'
1521
batch:
1622
send_batch_size: 100
1723
timeout: '10s'

0 commit comments

Comments
 (0)