Skip to content

Handle unauthorized org membership check gracefully#253

Open
arnavk23 wants to merge 4 commits intoJuliaWeb:masterfrom
arnavk23:fix-membership-test
Open

Handle unauthorized org membership check gracefully#253
arnavk23 wants to merge 4 commits intoJuliaWeb:masterfrom
arnavk23:fix-membership-test

Conversation

@arnavk23
Copy link
Copy Markdown

Summary

Replace the conditional skip for org membership tests with a try-catch that gracefully handles unauthorized access (when the token lacks org permissions), skipping the test instead of erroring.

Testing

  • julia --project -e 'using Pkg; Pkg.test(; test_args=["read_only_api_tests"])' passes with tokens that have org access.
  • With tokens lacking org access, the test is skipped with an informative message instead of failing.

@codecov
Copy link
Copy Markdown

codecov Bot commented Jan 16, 2026

Codecov Report

❌ Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 57.86%. Comparing base (97e94e4) to head (ba0afb2).

Files with missing lines Patch % Lines
src/owners/owners.jl 0.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #253      +/-   ##
==========================================
- Coverage   58.45%   57.86%   -0.59%     
==========================================
  Files          37       37              
  Lines         982      985       +3     
==========================================
- Hits          574      570       -4     
- Misses        408      415       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@DilumAluthge
Copy link
Copy Markdown
Member

Thanks for splitting this out!

I think we might have to give this some thought.

So, if I understand correctly, there are three possible cases:

  1. Case 1: The token is the GITHUB_TOKEN provided automatically by GitHub Actions.
  2. Case 2: The token is a personal access token (PAT) for a user, and the user is a member of the JuliaWeb org.
  3. Case 3: The token is a PAT for a user, and the user is not a member of the JuliaWeb org.

We already detect Case 1 - the result is stored in the is_gha_token::Bool variable.

@arnavk23 Can you write some logic to detect case 2 vs case 3, without using try-catch? You'll want to use the "List organizations for the authenticated user" API. I don't know if that API is currently supported by GitHub.jl, so it might be helpful if you could add it.

Once we have the ability to distinguish between case 2 and case 3, I think we should then use the following logic:

  1. First, check if we are running in CI or not.
  2. If we are running in CI:
    • If we are in case 3, throw an error. We should never hit case 3 in the CI on this repo.
    • If we are in case 2, then run the the check_membership() test.
      • If we are in case 1 or case 3, skip the check_membership() test (with a @test_skip).
  3. If we are not running in CI:
    • If we are in case 2, then run the the check_membership() test.
    • If we are in case 1 or case 3, skip the check_membership() test (with a @test_skip).

Here, when I say "the check_membership() test", I'm referring to this test:

@test GitHub.check_membership(julweb, testuser; auth = auth)

Does that make sense? This way, instead of using try-catch (which might mask an error that we actually want to surface), we can more specifically run or skip the test based on the specific circumstances.

@arnavk23 arnavk23 marked this pull request as draft January 18, 2026 21:43
@arnavk23 arnavk23 marked this pull request as ready for review January 25, 2026 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants