From c061b635b95311187112a2603e8aa848d0ed0cd4 Mon Sep 17 00:00:00 2001 From: jeff Date: Wed, 22 Apr 2026 16:20:08 -0500 Subject: [PATCH 1/6] adding verbose mode, readme info for environment, and cleanup Signed-off-by: jeff --- deactivate-reactivate/README.md | 47 +++++++++- deactivate-reactivate/main.py | 115 ++++++++++++++++++++++--- deactivate-reactivate/requirements.txt | 4 + 3 files changed, 153 insertions(+), 13 deletions(-) diff --git a/deactivate-reactivate/README.md b/deactivate-reactivate/README.md index dab80aa..7bf32e6 100644 --- a/deactivate-reactivate/README.md +++ b/deactivate-reactivate/README.md @@ -29,7 +29,9 @@ The script uses the Snyk API with **retry behavior for rate limits**: HTTP **429 pip install -r requirements.txt ``` -4. Run `main.py` with the options below. +4. **Set the API region** if your Snyk organization is not on the default instance. Use `SNYK_ENVIRONMENT` and/or `--environment` so API v1, REST, and the OAuth token endpoint all match your tenant (see the **Region / environment** section below). If you use `snyk config environment` for the CLI, use the same value here. + +5. Run `main.py` with the options below. ## Snyk for Government (FedRAMP / `SNYK-GOV-01`) @@ -49,9 +51,39 @@ python3 main.py --orgs your-org-slug Optional: override URLs with `--api-url`, `--rest-api-url`, or `--oauth-token-url`, or the `SNYK_API_URL`, `SNYK_REST_API_URL`, and `SNYK_OAUTH_TOKEN_URL` environment variables. For security, URLs must use **HTTPS** and hostnames under **`*.snyk.io`** or **`*.snykgov.io`**. If your tenant uses another hostname, set `SNYK_ALLOW_UNVERIFIED_API_URL=1` (HTTPS is still required). -## Region / environment (`--environment`) +## Region / environment + +Snyk hosts data in [regional API endpoints](https://docs.snyk.io/snyk-data-and-governance/regional-hosting-and-data-residency). This script must use the same region as the org you are managing, or you will not see the right organizations and projects (or calls may fail). + +**Ways to set the region (pick one or combine; CLI flags override the env var where both apply):** + +- **`export SNYK_ENVIRONMENT=...`** before running the script. If unset, the default is **`SNYK-US-01`**. +- **`--environment NAME`** on the command line, for example `python3 main.py --environment SNYK-EU-01 --orgs my-org`. + +Each preset below sets the **API v1 base**, **REST API base**, and **OAuth2 token** URL together (aligned with `snyk config environment`): -Use the same region names as `snyk config environment` (for example `SNYK-US-02`, `SNYK-EU-01`, `SNYK-GOV-01`). This sets default API and OAuth token URLs. You can override individual URLs as needed. +| Name | Notes | +| --- | --- | +| `SNYK-US-01` | Default when `SNYK_ENVIRONMENT` is not set. | +| `SNYK-US-02` | United States (e.g. `api.us.snyk.io`) | +| `SNYK-EU-01` | Europe | +| `SNYK-AU-01` | Australia | +| `SNYK-GOV-01` | Snyk for Government (FedRAMP); also see the **Snyk for Government** section above. | + +**Examples:** + +```bash +# EU org — env var +export SNYK_ENVIRONMENT=SNYK-EU-01 +python3 main.py --orgs my-org +``` + +```bash +# US-02 org — flag only +python3 main.py --environment SNYK-US-02 --orgs my-org +``` + +**Advanced:** Override individual bases without changing the rest of the preset: `--api-url`, `--rest-api-url`, and `--oauth-token-url`, or `SNYK_API_URL`, `SNYK_REST_API_URL`, and `SNYK_OAUTH_TOKEN_URL`. Use the same HTTPS and hostname rules as in the **Snyk for Government** section (including `SNYK_ALLOW_UNVERIFIED_API_URL` for nonstandard hosts). ## Selecting organizations (`--orgs`) @@ -107,6 +139,15 @@ Example: python3 main.py --orgs my-org --rate-limit-attempts 12 ``` +## Dry run (`--dry-run`) and verbose (`-v` / `--verbose`) + +- **`--dry-run`** — Lists orgs and projects that would be cycled; does **not** call deactivate or activate. Use to confirm the org, region, and (if any) origin filter before a real run. +- **`-v` / `--verbose`** — Prints the resolved **environment and API base URLs**, every org the token can see (id and slug), and for each project the **project id**, **monitored** flag, and each API **ok** result. Use when a run “succeeds” but you are unsure anything happened, or the Snyk UI is confusing. + +**If the script exits cleanly but the UI “doesn’t change”:** a full cycle **deactivates and then reactivates** every selected project, so the **normal end state** is still **active** / monitored in the UI—only the **webhooks / integration** are refreshed, which may not look like a visible status flip. If you see a **warning** that no projects were processed, check **region** (`--environment`), **org** slug or UUID, and **`--origins`** (try omitting the origin filter once). If the script prints that some org names were not found, the token or region may not match that org. + +**`RequestsDependencyWarning` (urllib3 / chardet / charset_normalizer):** That message comes from the `requests` library on import. It often appears when **`chardet` 6.x** is installed: `requests` only accepts an older `chardet` (see the `chardet` line in `requirements.txt`). Reinstall with `pip install -U -r requirements.txt` (ideally in a venv) so dependencies match. The script still runs; the warning is from `requests` / your environment, not this repo’s code. + # Full examples **Commercial (API token):** diff --git a/deactivate-reactivate/main.py b/deactivate-reactivate/main.py index ad7a01d..fdf27ee 100644 --- a/deactivate-reactivate/main.py +++ b/deactivate-reactivate/main.py @@ -462,6 +462,24 @@ def main() -> None: "Also set via SNYK_OAUTH_TOKEN_URL." ), ) + parser.add_argument( + "--dry-run", + action="store_true", + help=( + "List organizations and projects that would be cycled; do not call " + "deactivate or activate. Still requires valid credentials to list data." + ), + ) + parser.add_argument( + "-v", + "--verbose", + action="store_true", + help=( + "Print environment/API endpoints, which organizations the token can see, " + "and per-project API details (id, monitored flag, result). Use when results " + "in the Snyk UI are unclear or something seems skipped." + ), + ) args = parser.parse_args() allowed_origins = _parse_allowed_origins(args) @@ -525,24 +543,44 @@ def main() -> None: print(f"💥 {TOKEN_ERROR_HINT}") raise SystemExit(1) - if not input_orgs: + if args.verbose: print( - "Input the org slug(s) or org id(s) (UUID) for which you would like " - "to reactivate projects to generate webhooks." + f"\n[verbose] environment={env_name!r} api_v1={urls['api_url']!r} " + f"rest={urls['rest_api_url']!r} oauth={urls['oauth_token_url']!r}\n" + f"[verbose] this token can see {len(user_orgs)} organization(s):" ) + for o in user_orgs: + print(f" - id={o.id} slug={o.slug!r} name={o.name!r}") + print() + + if not input_orgs: + prompt = ( + "to preview which projects would be cycled (deactivate + reactivate)" + if args.dry_run + else "to reactivate projects to generate webhooks" + ) + print(f"Input the org slug(s) or org id(s) (UUID) for which you would like {prompt}.") input_orgs = input().split() + total_projects_cycled = 0 for curr_org in user_orgs: matched = [t for t in input_orgs if _org_matches_token(curr_org, t)] if not matched: continue for t in matched: input_orgs.remove(t) - print( - "Processing" - + """ \033[1;32m"{}" """.format(curr_org.name) - + "\u001b[0morganization" - ) + if args.dry_run: + print( + "\n\u001b[1;33m[DRY-RUN]\u001b[0m No API changes will be made for organization " + f'\033[1;32m"{curr_org.name}"\u001b[0m (projects listed below are what ' + "would be deactivated, then reactivated).\n" + ) + else: + print( + "Processing" + + """ \033[1;32m"{}" """.format(curr_org.name) + + "\u001b[0morganization" + ) projects = [ p @@ -550,10 +588,38 @@ def main() -> None: if _should_process_project(p.origin, allowed_origins) ] + if args.verbose: + print( + f"[verbose] matched org: id={curr_org.id} slug={curr_org.slug!r} " + f'name="{curr_org.name}" — {len(projects)} project(s) after origin filter\n' + ) + + if not args.dry_run and not projects: + print( + "\n\u001b[1;33mWARNING\u001b[0m: No projects to process in this org after " + "applying the origin filter (if any). No deactivate/activate was called for " + f"organization \"{curr_org.name}\" (id {curr_org.id}). " + "If that is unexpected, check \u001b[1m--origins\u001b[0m, " + "\u001b[1m--environment\u001b[0m (region), and that the org has imported projects.\n" + ) + if args.dry_run and projects: + print( + f" \u001b[90m{len(projects)} project(s) would be cycled in this org." + + "\u001b[0m" + ) + elif args.dry_run: + print(" \u001b[90mNo projects match the origin filter (if any) in this org.\u001b[0m") + for curr_project in projects: curr_project_details = ( f"Origin: {curr_project.origin}, Type: {curr_project.type}" ) + if args.dry_run: + print( + f" [DRY-RUN] would deactivate: \u001b[1;33m{curr_project.name}\u001b[0m " + f"(\u001b[34m{curr_project_details}\u001b[0m)" + ) + continue action = "Deactivating" spinner = yaspin( text=f"{action}\033[1;33m {curr_project.name}", color="yellow" @@ -563,7 +629,14 @@ def main() -> None: ) spinner.start() try: - curr_project.deactivate() + if args.verbose: + print( + f" [verbose] POST deactivate project id={curr_project.id} " + f"isMonitored={curr_project.isMonitored!r} origin={curr_project.origin!r}" + ) + ok = curr_project.deactivate() + if args.verbose: + print(f" [verbose] deactivate response ok={ok!r}") spinner.ok("🆗 ") except Exception as err: spinner.fail("💥 ") @@ -573,6 +646,12 @@ def main() -> None: curr_project_details = ( f"Origin: {curr_project.origin}, Type: {curr_project.type}" ) + if args.dry_run: + print( + f" [DRY-RUN] would reactivate: \u001b[1;32m{curr_project.name}\u001b[0m " + f"(\u001b[34m{curr_project_details}\u001b[0m)" + ) + continue action = "Activating" spinner = yaspin( text=f"{action}\033[1;32m {curr_project.name}", color="yellow" @@ -582,7 +661,16 @@ def main() -> None: ) spinner.start() try: - curr_project.activate() + if args.verbose: + print( + f" [verbose] POST activate project id={curr_project.id} " + f"isMonitored={curr_project.isMonitored!r} origin={curr_project.origin!r}" + ) + ok = curr_project.activate() + if args.verbose: + print(f" [verbose] activate response ok={ok!r}") + if ok: + total_projects_cycled += 1 spinner.ok("🆗 ") except Exception as err: spinner.fail("💥 ") @@ -597,6 +685,13 @@ def main() -> None: ) ) + if not args.dry_run and not input_orgs and total_projects_cycled == 0: + # All requested org names were matched, but no activate returned success (0 projects, or errors). + print( + "\n\u001b[1;33mNote\u001b[0m: no successful \u001b[1mactivate\u001b[0m in this run. If you " + "expected projects, re-check region (\u001b[1m--environment\u001b[0m), org slug/UUID, " + "and \u001b[1m--origins\u001b[0m; use \u001b[1m--verbose\u001b[0m to show API details.\n" + ) if __name__ == "__main__": main() diff --git a/deactivate-reactivate/requirements.txt b/deactivate-reactivate/requirements.txt index 6a530c6..bd2ebcb 100644 --- a/deactivate-reactivate/requirements.txt +++ b/deactivate-reactivate/requirements.txt @@ -1,3 +1,7 @@ pysnyk==0.9.19 yaspin==3.0.2 urllib3==2.6.3 +requests>=2.28.0 +# requests imports chardet first if present; chardet 6.x fails requests' version check +chardet>=3.0.2,<6.0.0 +charset-normalizer>=2.0.0,<4.0.0 From 100a28aea29b3c34d1990ea4f59703219d5c0f8b Mon Sep 17 00:00:00 2001 From: jeff Date: Thu, 23 Apr 2026 10:17:03 -0500 Subject: [PATCH 2/6] adding activate-only flag Signed-off-by: jeff --- deactivate-reactivate/README.md | 22 +++- deactivate-reactivate/main.py | 172 ++++++++++++++++++++++---------- 2 files changed, 142 insertions(+), 52 deletions(-) diff --git a/deactivate-reactivate/README.md b/deactivate-reactivate/README.md index 7bf32e6..cdf9486 100644 --- a/deactivate-reactivate/README.md +++ b/deactivate-reactivate/README.md @@ -2,6 +2,8 @@ The purpose of this script is to run through Snyk organizations, deactivate every selected project, then activate it again. That cycle helps rebuild webhooks for SCM integrations. +Alternatively, **`--activate-inactive-only`** skips deactivate and **only calls activate** on projects that are already inactive (not monitored) in Snyk—see **Activating only inactive projects** below. + The script uses the Snyk API with **retry behavior for rate limits**: HTTP **429** responses are retried using the **`Retry-After`** header (seconds or HTTP-date), and transient **5xx** responses use exponential backoff via the `pysnyk` client. # Requirements @@ -129,6 +131,24 @@ python3 main.py --orgs my-org --origin gitlab Common `origin` values include `github`, `github-enterprise`, `gitlab`, `bitbucket-cloud`, and `azure-repos` (exact strings depend on how projects were imported in Snyk). +## Activating only inactive projects (`--activate-inactive-only`) + +By default the script **deactivates then reactivates** every matching project. With **`--activate-inactive-only`**, it **never** deactivates: it only lists and activates projects whose Snyk API field **`isMonitored`** is **false** (inactive / not monitored in the UI). Already-active projects are left unchanged. + +The same **`--origin` / `--origins`** filters apply: only inactive projects whose origin passes the filter are activated. + +Examples: + +```bash +# Preview which inactive projects would be activated +python3 main.py --orgs my-org --activate-inactive-only --dry-run + +# Activate only inactive GitHub projects +python3 main.py --orgs my-org --activate-inactive-only --origin github +``` + +In `main.py`, inactive projects for an org are resolved by the helper **`inactive_projects_in_org`** (origin filter + `not project.isMonitored`). + ## Rate limit retries (`--rate-limit-attempts`) - **`--rate-limit-attempts N`** — maximum number of **consecutive HTTP 429** responses to retry **per API request** before failing (default: **8**). Waits honor `Retry-After` when present. @@ -141,7 +161,7 @@ python3 main.py --orgs my-org --rate-limit-attempts 12 ## Dry run (`--dry-run`) and verbose (`-v` / `--verbose`) -- **`--dry-run`** — Lists orgs and projects that would be cycled; does **not** call deactivate or activate. Use to confirm the org, region, and (if any) origin filter before a real run. +- **`--dry-run`** — Lists orgs and projects that would be changed; does **not** call deactivate or activate. Without **`--activate-inactive-only`**, that means the full deactivate-then-reactivate cycle. With **`--activate-inactive-only`**, only inactive projects that would be activated are listed. Use to confirm the org, region, and (if any) origin filter before a real run. - **`-v` / `--verbose`** — Prints the resolved **environment and API base URLs**, every org the token can see (id and slug), and for each project the **project id**, **monitored** flag, and each API **ok** result. Use when a run “succeeds” but you are unsure anything happened, or the Snyk UI is confusing. **If the script exits cleanly but the UI “doesn’t change”:** a full cycle **deactivates and then reactivates** every selected project, so the **normal end state** is still **active** / monitored in the UI—only the **webhooks / integration** are refreshed, which may not look like a visible status flip. If you see a **warning** that no projects were processed, check **region** (`--environment`), **org** slug or UUID, and **`--origins`** (try omitting the origin filter once). If the script prints that some org names were not found, the token or region may not match that org. diff --git a/deactivate-reactivate/main.py b/deactivate-reactivate/main.py index fdf27ee..72445d9 100644 --- a/deactivate-reactivate/main.py +++ b/deactivate-reactivate/main.py @@ -5,7 +5,7 @@ import time import urllib.parse from datetime import datetime, timezone -from typing import Any, Dict, Optional, Set +from typing import Any, Dict, List, Optional, Set import requests import snyk @@ -341,6 +341,21 @@ def _should_process_project(origin: str, allowed: Optional[Set[str]]) -> bool: return origin.casefold() in allowed +def inactive_projects_in_org( + org: Any, + allowed_origins: Optional[Set[str]], +) -> List[Any]: + """ + Projects in the organization that match the origin filter and are inactive + (Snyk isMonitored is False: not monitored / deactivated in the UI). + """ + return [ + p + for p in org.projects.all() + if _should_process_project(p.origin, allowed_origins) and not p.isMonitored + ] + + def _org_matches_token(org: Any, token: str) -> bool: """True if token is this org's id (UUID) or slug (case-insensitive).""" t = token.strip() @@ -390,7 +405,8 @@ def _resolve_urls(args: argparse.Namespace) -> Dict[str, str]: def main() -> None: parser = argparse.ArgumentParser( description=( - "Deactivate and reactivate Snyk projects to rebuild SCM webhooks. " + "Deactivate and reactivate Snyk projects to rebuild SCM webhooks, " + "or with --activate-inactive-only only activate projects that are already inactive. " "Optionally limit to projects with specific Snyk project origins " "(SCM / import source, e.g. github, gitlab)." ) @@ -462,6 +478,15 @@ def main() -> None: "Also set via SNYK_OAUTH_TOKEN_URL." ), ) + parser.add_argument( + "--activate-inactive-only", + action="store_true", + help=( + "Only activate projects that are currently inactive (not monitored). " + "Skips deactivate entirely; does not touch already-active projects. " + "Same origin filters as without this flag." + ), + ) parser.add_argument( "--dry-run", action="store_true", @@ -554,11 +579,18 @@ def main() -> None: print() if not input_orgs: - prompt = ( - "to preview which projects would be cycled (deactivate + reactivate)" - if args.dry_run - else "to reactivate projects to generate webhooks" - ) + if args.activate_inactive_only: + prompt = ( + "to preview which inactive projects would be activated" + if args.dry_run + else "to activate inactive (not monitored) projects only" + ) + else: + prompt = ( + "to preview which projects would be cycled (deactivate + reactivate)" + if args.dry_run + else "to reactivate projects to generate webhooks" + ) print(f"Input the org slug(s) or org id(s) (UUID) for which you would like {prompt}.") input_orgs = input().split() @@ -570,10 +602,18 @@ def main() -> None: for t in matched: input_orgs.remove(t) if args.dry_run: + if args.activate_inactive_only: + dry_intro = ( + "projects listed below are inactive (not monitored) and would be " + "activated only (no deactivate)." + ) + else: + dry_intro = ( + "projects listed below are what would be deactivated, then reactivated." + ) print( "\n\u001b[1;33m[DRY-RUN]\u001b[0m No API changes will be made for organization " - f'\033[1;32m"{curr_org.name}"\u001b[0m (projects listed below are what ' - "would be deactivated, then reactivated).\n" + f'\033[1;32m"{curr_org.name}"\u001b[0m ({dry_intro})\n' ) else: print( @@ -582,73 +622,103 @@ def main() -> None: + "\u001b[0morganization" ) - projects = [ - p - for p in curr_org.projects.all() - if _should_process_project(p.origin, allowed_origins) - ] + if args.activate_inactive_only: + projects = inactive_projects_in_org(curr_org, allowed_origins) + else: + projects = [ + p + for p in curr_org.projects.all() + if _should_process_project(p.origin, allowed_origins) + ] if args.verbose: + filt = ( + "inactive + origin filters" + if args.activate_inactive_only + else "origin filter" + ) print( f"[verbose] matched org: id={curr_org.id} slug={curr_org.slug!r} " - f'name="{curr_org.name}" — {len(projects)} project(s) after origin filter\n' + f'name="{curr_org.name}" — {len(projects)} project(s) after {filt}\n' ) if not args.dry_run and not projects: + if args.activate_inactive_only: + warn_detail = ( + "No inactive (not monitored) projects matched the origin filter (if any). " + "No activate was called" + ) + else: + warn_detail = ( + "No projects to process in this org after applying the origin filter (if any). " + "No deactivate/activate was called" + ) print( - "\n\u001b[1;33mWARNING\u001b[0m: No projects to process in this org after " - "applying the origin filter (if any). No deactivate/activate was called for " + f"\n\u001b[1;33mWARNING\u001b[0m: {warn_detail} for " f"organization \"{curr_org.name}\" (id {curr_org.id}). " "If that is unexpected, check \u001b[1m--origins\u001b[0m, " "\u001b[1m--environment\u001b[0m (region), and that the org has imported projects.\n" ) if args.dry_run and projects: - print( - f" \u001b[90m{len(projects)} project(s) would be cycled in this org." - + "\u001b[0m" - ) + if args.activate_inactive_only: + summary = ( + f"{len(projects)} inactive project(s) would be activated in this org." + ) + else: + summary = f"{len(projects)} project(s) would be cycled in this org." + print(f" \u001b[90m{summary}\u001b[0m") elif args.dry_run: - print(" \u001b[90mNo projects match the origin filter (if any) in this org.\u001b[0m") + if args.activate_inactive_only: + msg = "No inactive projects match the origin filter (if any) in this org." + else: + msg = "No projects match the origin filter (if any) in this org." + print(f" \u001b[90m{msg}\u001b[0m") - for curr_project in projects: - curr_project_details = ( - f"Origin: {curr_project.origin}, Type: {curr_project.type}" - ) - if args.dry_run: - print( - f" [DRY-RUN] would deactivate: \u001b[1;33m{curr_project.name}\u001b[0m " - f"(\u001b[34m{curr_project_details}\u001b[0m)" + if not args.activate_inactive_only: + for curr_project in projects: + curr_project_details = ( + f"Origin: {curr_project.origin}, Type: {curr_project.type}" ) - continue - action = "Deactivating" - spinner = yaspin( - text=f"{action}\033[1;33m {curr_project.name}", color="yellow" - ) - spinner.write( - f"\u001b[0m Processing project: \u001b[34m{curr_project_details}\u001b[0m, Status Below👇" - ) - spinner.start() - try: - if args.verbose: + if args.dry_run: print( - f" [verbose] POST deactivate project id={curr_project.id} " - f"isMonitored={curr_project.isMonitored!r} origin={curr_project.origin!r}" + f" [DRY-RUN] would deactivate: \u001b[1;33m{curr_project.name}\u001b[0m " + f"(\u001b[34m{curr_project_details}\u001b[0m)" ) - ok = curr_project.deactivate() - if args.verbose: - print(f" [verbose] deactivate response ok={ok!r}") - spinner.ok("🆗 ") - except Exception as err: - spinner.fail("💥 ") - spinner.write(f"\u001b[31m {err}\u001b[0m") + continue + action = "Deactivating" + spinner = yaspin( + text=f"{action}\033[1;33m {curr_project.name}", color="yellow" + ) + spinner.write( + f"\u001b[0m Processing project: \u001b[34m{curr_project_details}\u001b[0m, Status Below👇" + ) + spinner.start() + try: + if args.verbose: + print( + f" [verbose] POST deactivate project id={curr_project.id} " + f"isMonitored={curr_project.isMonitored!r} origin={curr_project.origin!r}" + ) + ok = curr_project.deactivate() + if args.verbose: + print(f" [verbose] deactivate response ok={ok!r}") + spinner.ok("🆗 ") + except Exception as err: + spinner.fail("💥 ") + spinner.write(f"\u001b[31m {err}\u001b[0m") for curr_project in projects: curr_project_details = ( f"Origin: {curr_project.origin}, Type: {curr_project.type}" ) if args.dry_run: + label = ( + "would activate (inactive)" + if args.activate_inactive_only + else "would reactivate" + ) print( - f" [DRY-RUN] would reactivate: \u001b[1;32m{curr_project.name}\u001b[0m " + f" [DRY-RUN] {label}: \u001b[1;32m{curr_project.name}\u001b[0m " f"(\u001b[34m{curr_project_details}\u001b[0m)" ) continue From 004c6497c62d6a9134611839136a78d838bfafb5 Mon Sep 17 00:00:00 2001 From: jeff Date: Thu, 23 Apr 2026 10:47:45 -0500 Subject: [PATCH 3/6] added the origin for github-cloud-app Signed-off-by: jeff --- deactivate-reactivate/README.md | 12 ++++++++++-- deactivate-reactivate/main.py | 8 +++++--- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/deactivate-reactivate/README.md b/deactivate-reactivate/README.md index cdf9486..16d3e4d 100644 --- a/deactivate-reactivate/README.md +++ b/deactivate-reactivate/README.md @@ -104,7 +104,7 @@ python3 main.py --orgs 70158a6b-3a6d-4bff-aace-9699582f5950 **Either form works**—pick one style, or mix them. Both apply the same filter (and are merged if you use both). -Snyk stores an **`origin`** on each project (SCM / import source, for example `github`, `gitlab`). By default the script processes **all** projects in the chosen orgs. +Snyk stores an **`origin`** on each project (SCM / import source). By default the script processes **all** projects in the chosen orgs—**no** origin mapping is applied: whatever the API returns (for example `github-cloud-app` for many GitHub App–imported repos) is what gets listed and, on a real run, cycled or activated. To limit work to specific origins: @@ -127,9 +127,17 @@ python3 main.py --orgs my-org --origins github github-enterprise # Only GitLab python3 main.py --orgs my-org --origin gitlab + +# Only projects whose API origin is github-cloud-app (GitHub App imports) +python3 main.py --orgs my-org --origin github-cloud-app + +# Inactive GitHub App imports only +python3 main.py --orgs my-org --activate-inactive-only --origin github-cloud-app ``` -Common `origin` values include `github`, `github-enterprise`, `gitlab`, `bitbucket-cloud`, and `azure-repos` (exact strings depend on how projects were imported in Snyk). +Common `origin` values include `github`, **`github-cloud-app`** (GitHub via the Snyk GitHub App—often what you see for Cloud imports), `github-enterprise`, `gitlab`, `bitbucket-cloud`, and `azure-repos`. The exact string depends on how the project was imported. **To filter with `--origin` / `--origins`, use the same value the API uses** (run once with `--dry-run` and copy the `Origin:` text, for example `Origin: github-cloud-app`). + +**Note:** `github` and `github-cloud-app` are different filters. If your dry-run shows `github-cloud-app`, use `--origin github-cloud-app` (or omit the origin flags to process those projects with everything else). ## Activating only inactive projects (`--activate-inactive-only`) diff --git a/deactivate-reactivate/main.py b/deactivate-reactivate/main.py index 72445d9..88a1093 100644 --- a/deactivate-reactivate/main.py +++ b/deactivate-reactivate/main.py @@ -408,7 +408,7 @@ def main() -> None: "Deactivate and reactivate Snyk projects to rebuild SCM webhooks, " "or with --activate-inactive-only only activate projects that are already inactive. " "Optionally limit to projects with specific Snyk project origins " - "(SCM / import source, e.g. github, gitlab)." + "(SCM / import source, e.g. github, github-cloud-app, gitlab)." ) ) parser.add_argument( @@ -427,8 +427,10 @@ def main() -> None: metavar="ORIGIN", help=( "Only process projects with this Snyk project origin (repeat for " - "several). Examples: github, github-enterprise, gitlab, " - "bitbucket-cloud, azure-repos. Omit to include all origins." + "several). Examples: github, github-cloud-app (GitHub App imports), " + "github-enterprise, gitlab, bitbucket-cloud, azure-repos. " + "Use the exact string the API returns (see dry-run). " + "Omit to include all origins." ), ) parser.add_argument( From ffca863be556900eb8c10cee4f708221969f5c26 Mon Sep 17 00:00:00 2001 From: jeff Date: Thu, 23 Apr 2026 12:51:56 -0500 Subject: [PATCH 4/6] added support for parallel workers for largdr orgs Signed-off-by: jeff --- deactivate-reactivate/main.py | 369 +++++++++++++++++++++++++--------- 1 file changed, 277 insertions(+), 92 deletions(-) diff --git a/deactivate-reactivate/main.py b/deactivate-reactivate/main.py index 88a1093..023b831 100644 --- a/deactivate-reactivate/main.py +++ b/deactivate-reactivate/main.py @@ -1,11 +1,13 @@ import argparse +import concurrent.futures import email.utils import os import sys +import threading import time import urllib.parse from datetime import datetime, timezone -from typing import Any, Dict, List, Optional, Set +from typing import Any, Dict, List, Optional, Set, Tuple import requests import snyk @@ -364,6 +366,155 @@ def _org_matches_token(org: Any, token: str) -> bool: return org.slug.casefold() == t.casefold() +def _create_snyk_client( + args: argparse.Namespace, + urls: Dict[str, str], + *, + interactive_token: Optional[str] = None, +) -> Any: + """ + Build a Snyk client from CLI args, resolved URLs, and environment. + ``interactive_token`` is only used when no OAuth/API env credentials are set. + """ + env_name = urls["environment"] + client_kw = { + "url": urls["api_url"], + "rest_api_url": urls["rest_api_url"], + "rate_limit_max_attempts": args.rate_limit_attempts, + } + oauth_client_id = os.environ.get("SNYK_OAUTH_CLIENT_ID") + oauth_client_secret = os.environ.get("SNYK_OAUTH_CLIENT_SECRET") + oauth_access = os.environ.get("SNYK_OAUTH_TOKEN") + api_key = os.environ.get("SNYK_TOKEN") + + if oauth_client_id and oauth_client_secret: + return OAuthRateLimitAwareSnykClient( + "", + use_bearer=True, + oauth_client_id=oauth_client_id, + oauth_client_secret=oauth_client_secret, + oauth_token_url=urls["oauth_token_url"], + **client_kw, + ) + if oauth_access: + return OAuthRateLimitAwareSnykClient( + oauth_access, + use_bearer=True, + **client_kw, + ) + if api_key: + if env_name == "SNYK-GOV-01": + print( + "Snyk for Government (FedRAMP) does not support static API tokens; " + "use OAuth2 service account credentials (SNYK_OAUTH_CLIENT_ID and " + "SNYK_OAUTH_CLIENT_SECRET) or a short-lived SNYK_OAUTH_TOKEN.", + file=sys.stderr, + ) + raise SystemExit(1) + return RateLimitAwareSnykClient(api_key, **client_kw) + if env_name == "SNYK-GOV-01": + raise SystemExit( + "FedRAMP / Snyk for Government requires OAuth2. Set " + "SNYK_OAUTH_CLIENT_ID and SNYK_OAUTH_CLIENT_SECRET (recommended), or " + "SNYK_OAUTH_TOKEN for a short-lived access token. " + "Static SNYK_TOKEN is not allowed." + ) + if interactive_token is None: + raise SystemExit( + "No Snyk credentials in environment (SNYK_TOKEN or OAuth variables)." + ) + return RateLimitAwareSnykClient(interactive_token.strip(), **client_kw) + + +def _post_project_state_change( + client: Any, org_id: str, project_id: str, verb: str +) -> bool: + """POST deactivate or activate (matches pysnyk Project.activate/deactivate paths).""" + path = f"org/{org_id}/project/{project_id}/{verb}" + return bool(client.post(path, {})) + + +class _ThreadLocalClientHolder: + """One Snyk client per worker thread (OAuth refresh and requests are not shared).""" + + def __init__(self, args: argparse.Namespace, urls: Dict[str, str]) -> None: + self._args = args + self._urls = urls + self._local = threading.local() + + def get(self) -> Any: + client = getattr(self._local, "client", None) + if client is None: + self._local.client = _create_snyk_client(self._args, self._urls) + return self._local.client + + +def _project_work_row(p: Any) -> Tuple[str, str, str, str, str, bool]: + return ( + p.organization.id, + p.id, + p.name, + p.origin, + p.type, + p.isMonitored, + ) + + +def _run_parallel_project_posts( + holder: _ThreadLocalClientHolder, + rows: List[Tuple[str, str, str, str, str, bool]], + verb: str, + *, + workers: int, + verbose: bool, + log_lock: threading.Lock, + count_truthy_ok: bool, +) -> Tuple[int, List[str]]: + """ + Run deactivate or activate for many projects in parallel. + Returns (success_count, error_messages). + + When ``count_truthy_ok`` is True (activate), increments success for truthy POST results, + matching ``if ok: total_projects_cycled += 1`` in the sequential path. + When False (deactivate), any completed POST without an exception counts as success + (the sequential path does not treat a falsy body as failure). + """ + successes = 0 + errors: List[str] = [] + + def _one(row: Tuple[str, str, str, str, str, bool]) -> Tuple[bool, Optional[str]]: + org_id, project_id, name, origin, typ, is_monitored = row + try: + c = holder.get() + if verbose: + with log_lock: + print( + f" [verbose] POST {verb} project id={project_id} " + f"isMonitored={is_monitored!r} origin={origin!r}" + ) + ok = _post_project_state_change(c, org_id, project_id, verb) + if verbose: + with log_lock: + print(f" [verbose] {verb} response ok={ok!r}") + if count_truthy_ok: + return ok, None + return True, None + except Exception as err: + return False, f"{name} ({verb}): {err}" + + with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as pool: + futures = [pool.submit(_one, row) for row in rows] + for fut in concurrent.futures.as_completed(futures): + ok, err = fut.result() + if err: + with log_lock: + print(f"\u001b[31m {err}\u001b[0m") + errors.append(err) + elif ok: + successes += 1 + return successes, errors + + def _resolve_urls(args: argparse.Namespace) -> Dict[str, str]: """API v1 base, REST base, and OAuth2 token URL for this run.""" env_name = ( @@ -507,61 +658,50 @@ def main() -> None: "in the Snyk UI are unclear or something seems skipped." ), ) + parser.add_argument( + "--workers", + type=int, + default=1, + metavar="N", + help=( + "Parallel HTTP workers for deactivate/activate (default: 1, fully sequential). " + "Try 8–16 for large orgs; higher values increase throughput but may trigger " + "more HTTP 429 responses (retries still apply). When N>1, credentials must come " + "from the environment (SNYK_TOKEN or OAuth variables), not an interactive prompt." + ), + ) args = parser.parse_args() + if args.workers < 1: + raise SystemExit("--workers must be >= 1") + if args.workers > 64: + raise SystemExit("--workers cannot exceed 64") + allowed_origins = _parse_allowed_origins(args) input_orgs = list(args.orgs) if args.orgs else [] urls = _resolve_urls(args) env_name = urls["environment"] - client_kw = { - "url": urls["api_url"], - "rest_api_url": urls["rest_api_url"], - "rate_limit_max_attempts": args.rate_limit_attempts, - } - oauth_client_id = os.environ.get("SNYK_OAUTH_CLIENT_ID") oauth_client_secret = os.environ.get("SNYK_OAUTH_CLIENT_SECRET") oauth_access = os.environ.get("SNYK_OAUTH_TOKEN") api_key = os.environ.get("SNYK_TOKEN") - - if oauth_client_id and oauth_client_secret: - client = OAuthRateLimitAwareSnykClient( - "", - use_bearer=True, - oauth_client_id=oauth_client_id, - oauth_client_secret=oauth_client_secret, - oauth_token_url=urls["oauth_token_url"], - **client_kw, - ) - elif oauth_access: - client = OAuthRateLimitAwareSnykClient( - oauth_access, - use_bearer=True, - **client_kw, + has_env_creds = bool( + (oauth_client_id and oauth_client_secret) or oauth_access or api_key + ) + if args.workers > 1 and not has_env_creds: + raise SystemExit( + "--workers > 1 requires SNYK_TOKEN or OAuth credentials in the environment; " + "interactive API token entry is not supported with parallel workers." ) - elif api_key: - if env_name == "SNYK-GOV-01": - print( - "Snyk for Government (FedRAMP) does not support static API tokens; " - "use OAuth2 service account credentials (SNYK_OAUTH_CLIENT_ID and " - "SNYK_OAUTH_CLIENT_SECRET) or a short-lived SNYK_OAUTH_TOKEN.", - file=sys.stderr, - ) - raise SystemExit(1) - client = RateLimitAwareSnykClient(api_key, **client_kw) - else: - if env_name == "SNYK-GOV-01": - raise SystemExit( - "FedRAMP / Snyk for Government requires OAuth2. Set " - "SNYK_OAUTH_CLIENT_ID and SNYK_OAUTH_CLIENT_SECRET (recommended), or " - "SNYK_OAUTH_TOKEN for a short-lived access token. " - "Static SNYK_TOKEN is not allowed." - ) + + interactive_token: Optional[str] = None + if not has_env_creds: print("Enter your Snyk API Token") - snyk_token = input() - client = RateLimitAwareSnykClient(snyk_token, **client_kw) + interactive_token = input() + + client = _create_snyk_client(args, urls, interactive_token=interactive_token) user_orgs = [] try: @@ -676,20 +816,102 @@ def main() -> None: msg = "No projects match the origin filter (if any) in this org." print(f" \u001b[90m{msg}\u001b[0m") + log_lock = threading.Lock() + parallel_holder: Optional[_ThreadLocalClientHolder] = None + if not args.dry_run and args.workers > 1 and projects: + parallel_holder = _ThreadLocalClientHolder(args, urls) + mode_hint = ( + "activate only" + if args.activate_inactive_only + else "deactivate and activate" + ) + print( + f" Using \033[1;33m{args.workers}\u001b[0m concurrent worker(s) for " + f"{mode_hint} ({len(projects)} project(s))." + ) + if not args.activate_inactive_only: - for curr_project in projects: - curr_project_details = ( - f"Origin: {curr_project.origin}, Type: {curr_project.type}" - ) - if args.dry_run: + if args.dry_run: + for curr_project in projects: + curr_project_details = ( + f"Origin: {curr_project.origin}, Type: {curr_project.type}" + ) print( f" [DRY-RUN] would deactivate: \u001b[1;33m{curr_project.name}\u001b[0m " f"(\u001b[34m{curr_project_details}\u001b[0m)" ) - continue - action = "Deactivating" + elif args.workers > 1 and parallel_holder is not None: + rows = [_project_work_row(p) for p in projects] + _run_parallel_project_posts( + parallel_holder, + rows, + "deactivate", + workers=args.workers, + verbose=args.verbose, + log_lock=log_lock, + count_truthy_ok=False, + ) + else: + for curr_project in projects: + curr_project_details = ( + f"Origin: {curr_project.origin}, Type: {curr_project.type}" + ) + action = "Deactivating" + spinner = yaspin( + text=f"{action}\033[1;33m {curr_project.name}", color="yellow" + ) + spinner.write( + f"\u001b[0m Processing project: \u001b[34m{curr_project_details}\u001b[0m, Status Below👇" + ) + spinner.start() + try: + if args.verbose: + print( + f" [verbose] POST deactivate project id={curr_project.id} " + f"isMonitored={curr_project.isMonitored!r} origin={curr_project.origin!r}" + ) + ok = curr_project.deactivate() + if args.verbose: + print(f" [verbose] deactivate response ok={ok!r}") + spinner.ok("🆗 ") + except Exception as err: + spinner.fail("💥 ") + spinner.write(f"\u001b[31m {err}\u001b[0m") + + if args.dry_run: + for curr_project in projects: + curr_project_details = ( + f"Origin: {curr_project.origin}, Type: {curr_project.type}" + ) + label = ( + "would activate (inactive)" + if args.activate_inactive_only + else "would reactivate" + ) + print( + f" [DRY-RUN] {label}: \u001b[1;32m{curr_project.name}\u001b[0m " + f"(\u001b[34m{curr_project_details}\u001b[0m)" + ) + elif args.workers > 1 and parallel_holder is not None: + rows = [_project_work_row(p) for p in projects] + n_ok, _ = _run_parallel_project_posts( + parallel_holder, + rows, + "activate", + workers=args.workers, + verbose=args.verbose, + log_lock=log_lock, + count_truthy_ok=True, + ) + total_projects_cycled += n_ok + else: + for curr_project in projects: + curr_project_details = ( + f"Origin: {curr_project.origin}, Type: {curr_project.type}" + ) + action = "Activating" spinner = yaspin( - text=f"{action}\033[1;33m {curr_project.name}", color="yellow" + text=f"{action}\033[1;32m {curr_project.name}", color="yellow" ) spinner.write( f"\u001b[0m Processing project: \u001b[34m{curr_project_details}\u001b[0m, Status Below👇" @@ -698,56 +920,19 @@ def main() -> None: try: if args.verbose: print( - f" [verbose] POST deactivate project id={curr_project.id} " + f" [verbose] POST activate project id={curr_project.id} " f"isMonitored={curr_project.isMonitored!r} origin={curr_project.origin!r}" ) - ok = curr_project.deactivate() + ok = curr_project.activate() if args.verbose: - print(f" [verbose] deactivate response ok={ok!r}") + print(f" [verbose] activate response ok={ok!r}") + if ok: + total_projects_cycled += 1 spinner.ok("🆗 ") except Exception as err: spinner.fail("💥 ") spinner.write(f"\u001b[31m {err}\u001b[0m") - for curr_project in projects: - curr_project_details = ( - f"Origin: {curr_project.origin}, Type: {curr_project.type}" - ) - if args.dry_run: - label = ( - "would activate (inactive)" - if args.activate_inactive_only - else "would reactivate" - ) - print( - f" [DRY-RUN] {label}: \u001b[1;32m{curr_project.name}\u001b[0m " - f"(\u001b[34m{curr_project_details}\u001b[0m)" - ) - continue - action = "Activating" - spinner = yaspin( - text=f"{action}\033[1;32m {curr_project.name}", color="yellow" - ) - spinner.write( - f"\u001b[0m Processing project: \u001b[34m{curr_project_details}\u001b[0m, Status Below👇" - ) - spinner.start() - try: - if args.verbose: - print( - f" [verbose] POST activate project id={curr_project.id} " - f"isMonitored={curr_project.isMonitored!r} origin={curr_project.origin!r}" - ) - ok = curr_project.activate() - if args.verbose: - print(f" [verbose] activate response ok={ok!r}") - if ok: - total_projects_cycled += 1 - spinner.ok("🆗 ") - except Exception as err: - spinner.fail("💥 ") - spinner.write(f"\u001b[31m {err}\u001b[0m") - if input_orgs: print( "\033[1;32m{}\u001b[0m are organizations which do not exist or you " From 93f3cc3b7c68c1b22b52266c37eea2db2d56b4b1 Mon Sep 17 00:00:00 2001 From: jeff Date: Thu, 23 Apr 2026 12:55:04 -0500 Subject: [PATCH 5/6] readme cleanup Signed-off-by: jeff --- deactivate-reactivate/README.md | 247 ++++++++++++++++---------------- 1 file changed, 127 insertions(+), 120 deletions(-) diff --git a/deactivate-reactivate/README.md b/deactivate-reactivate/README.md index 16d3e4d..0f10773 100644 --- a/deactivate-reactivate/README.md +++ b/deactivate-reactivate/README.md @@ -1,98 +1,98 @@ # deactivate-reactivate-all-projects -The purpose of this script is to run through Snyk organizations, deactivate every selected project, then activate it again. That cycle helps rebuild webhooks for SCM integrations. +Cycle Snyk projects **deactivate → activate** so SCM **webhooks** can be rebuilt. You pick one or more organizations; the script walks the projects you care about and calls the Snyk API for each. -Alternatively, **`--activate-inactive-only`** skips deactivate and **only calls activate** on projects that are already inactive (not monitored) in Snyk—see **Activating only inactive projects** below. +**Optional:** **`--activate-inactive-only`** skips deactivate and only **activates** projects that are already inactive (not monitored)—useful when you only want to turn monitoring back on. -The script uses the Snyk API with **retry behavior for rate limits**: HTTP **429** responses are retried using the **`Retry-After`** header (seconds or HTTP-date), and transient **5xx** responses use exponential backoff via the `pysnyk` client. +**Reliability:** HTTP **429** (rate limit) responses are retried using **`Retry-After`**; transient **5xx** errors use backoff from the underlying `pysnyk` client. -# Requirements +--- -- Python 3.9+ recommended -- Dependencies in `requirements.txt` (`pysnyk`, `yaspin`, and their transitive packages) +## Quick start (most customers) -# Usage +1. **Clone** this repo and install: -1. Clone this repository locally. - -2. **Authenticate.** The script picks credentials in this order (first match wins): + ```bash + pip install -r requirements.txt + ``` - | Priority | Variables | Notes | - | --- | --- | --- | - | 1 | `SNYK_OAUTH_CLIENT_ID` and `SNYK_OAUTH_CLIENT_SECRET` | OAuth 2.0 **client credentials** (recommended for automation). Short-lived access tokens are obtained from the OAuth token endpoint and refreshed automatically before expiry. | - | 2 | `SNYK_OAUTH_TOKEN` | A short-lived OAuth **access token** (same usage as the Snyk CLI). You must refresh it yourself when it expires; use client credentials instead if you need long runs. | - | 3 | `SNYK_TOKEN` | Classic API token (`Authorization: token …`). **Not available in Snyk for Government (FedRAMP)**; see below. | +2. **Set a token** (same idea as the Snyk CLI—use a token that can manage the target orgs): - If none of these are set and you are **not** targeting Gov, the script prompts for `SNYK_TOKEN` interactively. + ```bash + export SNYK_TOKEN="your-snyk-api-token" + ``` -3. Install dependencies: +3. **Preview**, then run: ```bash - pip install -r requirements.txt + # See what would change (no API writes) + python3 main.py --orgs your-org-slug --dry-run + + # Then run for real + python3 main.py --orgs your-org-slug ``` -4. **Set the API region** if your Snyk organization is not on the default instance. Use `SNYK_ENVIRONMENT` and/or `--environment` so API v1, REST, and the OAuth token endpoint all match your tenant (see the **Region / environment** section below). If you use `snyk config environment` for the CLI, use the same value here. +If your Snyk data is **not** on the default US instance, set region first (see **Region** below)—wrong region usually means “wrong orgs or empty project list.” -5. Run `main.py` with the options below. +**Large orgs (many thousands of projects):** add **`--workers 12`** (or 8–16) so deactivate/activate calls run in parallel. Requires `SNYK_TOKEN` (or OAuth env vars) in the environment—not an interactive prompt. -## Snyk for Government (FedRAMP / `SNYK-GOV-01`) +--- -[Snyk for Government (US)](https://docs.snyk.io/snyk-data-and-governance/snyk-for-government-us) does **not** allow static API tokens. You must use **OAuth 2.0**—typically a service account with the **client credentials** grant. Access tokens are used like API keys but with `Authorization: Bearer` semantics and a short TTL; this script refreshes them when you use `SNYK_OAUTH_CLIENT_ID` / `SNYK_OAUTH_CLIENT_SECRET`. +## What to pass (cheat sheet) -1. Create an OAuth 2.0 service account (UI or API) and store **client ID** and **client secret** securely. -2. Point the script at the Gov API endpoints using **`--environment SNYK-GOV-01`** (or `export SNYK_ENVIRONMENT=SNYK-GOV-01`). That selects `https://api.snykgov.io` for API v1, REST, and `/oauth2/token`, consistent with [regional URL documentation](https://docs.snyk.io/snyk-data-and-governance/regional-hosting-and-data-residency). -3. Export credentials and run: +| You want to… | Use | +| --- | --- | +| Pick org(s) | **`--orgs slug-or-uuid`** (repeat or space-separate multiple). Omit to be prompted. | +| Preview only | **`--dry-run`** | +| Speed up huge runs | **`--workers N`** (try **8–16**; max **64**) | +| Limit to GitHub / GitLab / etc. | **`--origin github`** (repeat) or **`--origins github gitlab`** | +| Only turn on inactive projects | **`--activate-inactive-only`** | +| EU / AU / second US shard | **`--environment SNYK-EU-01`** (etc.) or **`export SNYK_ENVIRONMENT=...`** | +| More 429 retries per request | **`--rate-limit-attempts 12`** (default **8**) | +| Debug API / org visibility | **`-v`** / **`--verbose`** | -```bash -export SNYK_ENVIRONMENT=SNYK-GOV-01 -export SNYK_OAUTH_CLIENT_ID="your-client-id" -export SNYK_OAUTH_CLIENT_SECRET="your-client-secret" -pip install -r requirements.txt -python3 main.py --orgs your-org-slug -``` +Use **`--dry-run`** until the org names, region, and optional origin filter look right. -Optional: override URLs with `--api-url`, `--rest-api-url`, or `--oauth-token-url`, or the `SNYK_API_URL`, `SNYK_REST_API_URL`, and `SNYK_OAUTH_TOKEN_URL` environment variables. For security, URLs must use **HTTPS** and hostnames under **`*.snyk.io`** or **`*.snykgov.io`**. If your tenant uses another hostname, set `SNYK_ALLOW_UNVERIFIED_API_URL=1` (HTTPS is still required). +--- -## Region / environment +## Authentication -Snyk hosts data in [regional API endpoints](https://docs.snyk.io/snyk-data-and-governance/regional-hosting-and-data-residency). This script must use the same region as the org you are managing, or you will not see the right organizations and projects (or calls may fail). +The script uses the **first** of these that is set: -**Ways to set the region (pick one or combine; CLI flags override the env var where both apply):** +| Order | Environment variables | When to use | +| --- | --- | --- | +| 1 | **`SNYK_OAUTH_CLIENT_ID`** + **`SNYK_OAUTH_CLIENT_SECRET`** | Automation / long runs; tokens refresh automatically. | +| 2 | **`SNYK_OAUTH_TOKEN`** | Short-lived access token (you refresh it yourself). | +| 3 | **`SNYK_TOKEN`** | Classic Snyk API token. | -- **`export SNYK_ENVIRONMENT=...`** before running the script. If unset, the default is **`SNYK-US-01`**. -- **`--environment NAME`** on the command line, for example `python3 main.py --environment SNYK-EU-01 --orgs my-org`. +If nothing is set and you are **not** on Snyk for Government, the script may **prompt** for `SNYK_TOKEN`. **Parallel mode** (`--workers` > 1) **always** needs credentials in the environment (no prompt). -Each preset below sets the **API v1 base**, **REST API base**, and **OAuth2 token** URL together (aligned with `snyk config environment`): +**Snyk for Government (FedRAMP):** static **`SNYK_TOKEN` is not supported**—use OAuth client credentials and **`--environment SNYK-GOV-01`**. Short instructions: **[Snyk for Government](#optional-snyk-for-government-fedramp)** at the end of this file. -| Name | Notes | -| --- | --- | -| `SNYK-US-01` | Default when `SNYK_ENVIRONMENT` is not set. | -| `SNYK-US-02` | United States (e.g. `api.us.snyk.io`) | -| `SNYK-EU-01` | Europe | -| `SNYK-AU-01` | Australia | -| `SNYK-GOV-01` | Snyk for Government (FedRAMP); also see the **Snyk for Government** section above. | +--- -**Examples:** +## Region / environment -```bash -# EU org — env var -export SNYK_ENVIRONMENT=SNYK-EU-01 -python3 main.py --orgs my-org -``` +Snyk is hosted in [regions](https://docs.snyk.io/snyk-data-and-governance/regional-hosting-and-data-residency). The script must match **your** org’s region or listing will look wrong. -```bash -# US-02 org — flag only -python3 main.py --environment SNYK-US-02 --orgs my-org -``` +- **Default:** **`SNYK-US-01`** if you set nothing (same idea as `snyk config environment`). +- **Set once:** `export SNYK_ENVIRONMENT=SNYK-EU-01` **or** `python3 main.py --environment SNYK-EU-01 --orgs my-org`. -**Advanced:** Override individual bases without changing the rest of the preset: `--api-url`, `--rest-api-url`, and `--oauth-token-url`, or `SNYK_API_URL`, `SNYK_REST_API_URL`, and `SNYK_OAUTH_TOKEN_URL`. Use the same HTTPS and hostname rules as in the **Snyk for Government** section (including `SNYK_ALLOW_UNVERIFIED_API_URL` for nonstandard hosts). +| `SNYK_ENVIRONMENT` / `--environment` | Typical use | +| --- | --- | +| `SNYK-US-01` | Default US | +| `SNYK-US-02` | US (`api.us.snyk.io`) | +| `SNYK-EU-01` | Europe | +| `SNYK-AU-01` | Australia | +| `SNYK-GOV-01` | US Government / FedRAMP (OAuth only—see appendix) | + +**Advanced:** Override URLs with **`--api-url`**, **`--rest-api-url`**, **`--oauth-token-url`** or **`SNYK_API_URL`**, **`SNYK_REST_API_URL`**, **`SNYK_OAUTH_TOKEN_URL`**. URLs must be **HTTPS**; by default hostnames must be under **`*.snyk.io`** or **`*.snykgov.io`**. Other hosts: set **`SNYK_ALLOW_UNVERIFIED_API_URL=1`** (HTTPS still required). -## Selecting organizations (`--orgs`) +--- -- Pass one or more **organization slugs** and/or **organization IDs** (UUID from the Snyk UI or API). Slug matching is **case-insensitive**; the ID must match exactly. -- Omit `--orgs` to be prompted for orgs at runtime. +## Organizations (`--orgs`) -Examples: +Pass **slug** and/or **org id (UUID)**. Slug match is case-insensitive; UUID must match exactly. ```bash python3 main.py --orgs my-org-slug @@ -100,104 +100,111 @@ python3 main.py --orgs org-one org-two python3 main.py --orgs 70158a6b-3a6d-4bff-aace-9699582f5950 ``` -## Filtering by project origin (`--origin` / `--origins`) +--- -**Either form works**—pick one style, or mix them. Both apply the same filter (and are merged if you use both). +## Filtering by SCM origin (`--origin` / `--origins`) -Snyk stores an **`origin`** on each project (SCM / import source). By default the script processes **all** projects in the chosen orgs—**no** origin mapping is applied: whatever the API returns (for example `github-cloud-app` for many GitHub App–imported repos) is what gets listed and, on a real run, cycled or activated. +By default **all** project origins in the org are processed. To limit to certain import sources, use either style (you can mix them): -To limit work to specific origins: +- **`--origin github --origin gitlab`** +- **`--origins github gitlab`** -- **`--origin ORIGIN`** — pass once per value (repeat the flag): `--origin github --origin gitlab`. -- **`--origins ORIGIN [ORIGIN ...]`** — pass several values in one go: `--origins github gitlab` (same effect as two `--origin` flags). - -Matching is **case-insensitive**. If you do not pass `--origin` or `--origins`, every project origin is included. - -Examples: +Values are **case-insensitive** and must match what the API returns. Many GitHub App imports show as **`github-cloud-app`**, not `github`—run **`--dry-run`** once and copy the **`Origin:`** line if you are unsure. ```bash -# Only GitHub.com projects -python3 main.py --orgs my-org --origin github - -# GitHub.com and GitHub Enterprise -python3 main.py --orgs my-org --origin github --origin github-enterprise +python3 main.py --orgs my-org --origin github-cloud-app +python3 main.py --orgs my-org --origins github gitlab +``` -# Equivalent using --origins -python3 main.py --orgs my-org --origins github github-enterprise +--- -# Only GitLab -python3 main.py --orgs my-org --origin gitlab +## Activate only inactive projects (`--activate-inactive-only`) -# Only projects whose API origin is github-cloud-app (GitHub App imports) -python3 main.py --orgs my-org --origin github-cloud-app +Does **not** deactivate. Only **activates** projects that are already inactive (`isMonitored` false). Same **`--origin` / `--origins`** filters apply. -# Inactive GitHub App imports only -python3 main.py --orgs my-org --activate-inactive-only --origin github-cloud-app +```bash +python3 main.py --orgs my-org --activate-inactive-only --dry-run +python3 main.py --orgs my-org --activate-inactive-only --origin github ``` -Common `origin` values include `github`, **`github-cloud-app`** (GitHub via the Snyk GitHub App—often what you see for Cloud imports), `github-enterprise`, `gitlab`, `bitbucket-cloud`, and `azure-repos`. The exact string depends on how the project was imported. **To filter with `--origin` / `--origins`, use the same value the API uses** (run once with `--dry-run` and copy the `Origin:` text, for example `Origin: github-cloud-app`). - -**Note:** `github` and `github-cloud-app` are different filters. If your dry-run shows `github-cloud-app`, use `--origin github-cloud-app` (or omit the origin flags to process those projects with everything else). +--- -## Activating only inactive projects (`--activate-inactive-only`) +## Parallel workers (`--workers`) -By default the script **deactivates then reactivates** every matching project. With **`--activate-inactive-only`**, it **never** deactivates: it only lists and activates projects whose Snyk API field **`isMonitored`** is **false** (inactive / not monitored in the UI). Already-active projects are left unchanged. +Each project needs **two** API calls for a full cycle (deactivate, then activate). On large tenants that is mostly **waiting on the network**, so **`--workers N`** runs up to **N** of those calls at the same time (default **1** = same as always). -The same **`--origin` / `--origins`** filters apply: only inactive projects whose origin passes the filter are activated. - -Examples: +- Try **8–16** first; raise if 429s stay rare, lower if runs stall on rate limits. +- Max **64**. +- **`--workers` > 1:** set **`SNYK_TOKEN`** or OAuth variables in the environment (no interactive token). +- With **N > 1**, spinners are replaced by a one-line note per org. ```bash -# Preview which inactive projects would be activated -python3 main.py --orgs my-org --activate-inactive-only --dry-run - -# Activate only inactive GitHub projects -python3 main.py --orgs my-org --activate-inactive-only --origin github +export SNYK_TOKEN="your-api-token" +python3 main.py --orgs my-org --workers 12 ``` -In `main.py`, inactive projects for an org are resolved by the helper **`inactive_projects_in_org`** (origin filter + `not project.isMonitored`). - -## Rate limit retries (`--rate-limit-attempts`) +--- -- **`--rate-limit-attempts N`** — maximum number of **consecutive HTTP 429** responses to retry **per API request** before failing (default: **8**). Waits honor `Retry-After` when present. +## Rate limits (`--rate-limit-attempts`) -Example: +Max **consecutive HTTP 429** retries **per request** before failing (default **8**). Waits follow **`Retry-After`** when the API sends it. ```bash python3 main.py --orgs my-org --rate-limit-attempts 12 ``` -## Dry run (`--dry-run`) and verbose (`-v` / `--verbose`) +--- + +## Dry run and verbose + +- **`--dry-run`** — List what would happen; **no** deactivate/activate. +- **`-v` / `--verbose`** — Print API bases, orgs visible to the token, and per-project ids/results. + +**UI looks unchanged after a full cycle:** projects end **active** again; the goal is often **webhooks / integration**, not a lasting “off” state. If nothing processed, check **region**, **org slug/UUID**, and try without **`--origin`** once. + +**`RequestsDependencyWarning`:** Usually a **`chardet`** version mismatch in your environment. Use **`pip install -U -r requirements.txt`** in a venv; see `requirements.txt` for the pinned range. -- **`--dry-run`** — Lists orgs and projects that would be changed; does **not** call deactivate or activate. Without **`--activate-inactive-only`**, that means the full deactivate-then-reactivate cycle. With **`--activate-inactive-only`**, only inactive projects that would be activated are listed. Use to confirm the org, region, and (if any) origin filter before a real run. -- **`-v` / `--verbose`** — Prints the resolved **environment and API base URLs**, every org the token can see (id and slug), and for each project the **project id**, **monitored** flag, and each API **ok** result. Use when a run “succeeds” but you are unsure anything happened, or the Snyk UI is confusing. +--- -**If the script exits cleanly but the UI “doesn’t change”:** a full cycle **deactivates and then reactivates** every selected project, so the **normal end state** is still **active** / monitored in the UI—only the **webhooks / integration** are refreshed, which may not look like a visible status flip. If you see a **warning** that no projects were processed, check **region** (`--environment`), **org** slug or UUID, and **`--origins`** (try omitting the origin filter once). If the script prints that some org names were not found, the token or region may not match that org. +## Requirements -**`RequestsDependencyWarning` (urllib3 / chardet / charset_normalizer):** That message comes from the `requests` library on import. It often appears when **`chardet` 6.x** is installed: `requests` only accepts an older `chardet` (see the `chardet` line in `requirements.txt`). Reinstall with `pip install -U -r requirements.txt` (ideally in a venv) so dependencies match. The script still runs; the warning is from `requests` / your environment, not this repo’s code. +- Python **3.9+** recommended +- **`pip install -r requirements.txt`** -# Full examples +--- -**Commercial (API token):** +## Copy-paste examples + +**API token, commercial (typical):** ```bash export SNYK_TOKEN="your-api-token" pip install -r requirements.txt -python3 main.py \ - --orgs my-org-slug \ - --origin github --origin github-enterprise \ - --rate-limit-attempts 8 +python3 main.py --orgs my-org-slug --dry-run +python3 main.py --orgs my-org-slug --workers 12 ``` -Same run using the other flag: `python3 main.py --orgs my-org-slug --origins github github-enterprise --rate-limit-attempts 8`. - -**OAuth client credentials (Enterprise / Gov-compatible):** +**OAuth client credentials (good for automation):** ```bash export SNYK_OAUTH_CLIENT_ID="…" export SNYK_OAUTH_CLIENT_SECRET="…" -# For Gov: -# export SNYK_ENVIRONMENT=SNYK-GOV-01 pip install -r requirements.txt -python3 main.py --orgs my-org-slug -``` \ No newline at end of file +python3 main.py --orgs my-org-slug --workers 12 +``` + +--- + +## Optional: Snyk for Government (FedRAMP) + +[Snyk for Government](https://docs.snyk.io/snyk-data-and-governance/snyk-for-government-us) does **not** allow static API tokens. Use **OAuth 2.0** (service account **client credentials**). This script refreshes access tokens when **`SNYK_OAUTH_CLIENT_ID`** and **`SNYK_OAUTH_CLIENT_SECRET`** are set. + +```bash +export SNYK_ENVIRONMENT=SNYK-GOV-01 +export SNYK_OAUTH_CLIENT_ID="your-client-id" +export SNYK_OAUTH_CLIENT_SECRET="your-client-secret" +pip install -r requirements.txt +python3 main.py --orgs your-org-slug +``` + +Same **HTTPS / hostname** rules and optional **`SNYK_ALLOW_UNVERIFIED_API_URL=1`** as in **Region / environment** above. From 9f6a08c198de5aa7241defa0b4318c6a1ef65f01 Mon Sep 17 00:00:00 2001 From: jeff Date: Thu, 23 Apr 2026 14:36:46 -0500 Subject: [PATCH 6/6] added workers for parallel runs Signed-off-by: jeff --- deactivate-reactivate/.vscode/settings.json | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 deactivate-reactivate/.vscode/settings.json diff --git a/deactivate-reactivate/.vscode/settings.json b/deactivate-reactivate/.vscode/settings.json new file mode 100644 index 0000000..d41ebe4 --- /dev/null +++ b/deactivate-reactivate/.vscode/settings.json @@ -0,0 +1,5 @@ +{ + "python.languageServer": "Pylance", + "snyk.advanced.organization": "70158a6b-3a6d-4bff-aace-9699582f5950", + "snyk.advanced.autoSelectOrganization": true +} \ No newline at end of file