checkout-data-sync

• What is this?
• Prerequisites
• Usage
  • Running the Docker Image
    • Build
    • Run
• Examples
• Sync Behaviour
  • Applications
  • Payment Integrations
• Scopes

What is this?

A Dockerized CLI application which migrates commercetools Checkout resources between projects — for example when moving between cloud providers or regions.

The following resource types are supported, synced in this order:

• Applications
• Payment Integrations

Applications are synced first because Payment Integrations reference them by ID. The tool builds a sourceAppID → targetAppID map during the application phase and uses it to rewrite references before touching payment integrations.

Prerequisites

• Docker (to run the image) or Go 1.21+ (to build from source)

• A commercetools API client with Checkout permissions on both source and target projects

• A config.yml file — copy the example and fill in your credentials:

  cp config.example.yml config.yml

  source:
    project_key: "source-project-key"
    client_id: "sourceClientId"
    client_secret: "sourceClientSecret"
    # OAuth token endpoint (Composable Commerce), not the Checkout API host.
    auth_url: "https://auth.europe-west1.gcp.commercetools.com/oauth/token"
    checkout_api_url: "https://checkout.europe-west1.gcp.commercetools.com"
    # Optional: space-separated OAuth scopes. Omit to use the API client's default scopes.
    scopes: "manage_project"
  
  target:
    project_key: "target-project-key"
    client_id: "targetClientId"
    client_secret: "targetClientSecret"
    auth_url: "https://auth.eu-central-1.aws.commercetools.com/oauth/token"
    checkout_api_url: "https://checkout.eu-central-1.aws.commercetools.com"
    scopes: ""
  
  # Map source connector deployment UUIDs to target deployment UUIDs.
  # Required for every Payment Integration that has a connectorDeployment.
  # Connector deployment IDs are environment-specific and will not resolve
  # across cloud providers without an explicit mapping.
  deployment_mapping:
    "source-deployment-uuid": "target-deployment-uuid"

  Note: auth_url and checkout_api_url must not have a trailing slash.

• The following fields are required to be set on resources that will be synced:

  Resource	Required Fields
  Application	key
  Payment Integration	key (derived from source ID + name if absent — see Sync Behaviour)

Usage

usage: checkout-data-sync
 -c, --config <path>   Path to config file. When omitted, configuration
                       is read from environment variables (see below).
 -f, --full            Execute the migration. Omit to perform a dry-run
                       instead (shows what would be created or updated,
                       without making any changes).

By default the tool runs in dry-run mode: it compares source and target resources and prints a plan without writing anything. Pass -f to execute the migration.

Configuring via environment variables

When -c is not provided, the tool reads configuration from the environment. Each YAML field maps to an env var prefixed with SOURCE_ or TARGET_:

YAML field	Env var (per project)	Required
project_key	SOURCE_PROJECT_KEY / TARGET_PROJECT_KEY	yes
client_id	SOURCE_CLIENT_ID / TARGET_CLIENT_ID	yes
client_secret	SOURCE_CLIENT_SECRET / TARGET_CLIENT_SECRET	yes
auth_url	SOURCE_AUTH_URL / TARGET_AUTH_URL	yes
checkout_api_url	SOURCE_CHECKOUT_API_URL / TARGET_CHECKOUT_API_URL	yes
api_url	SOURCE_API_URL / TARGET_API_URL	no
scopes	SOURCE_SCOPES / TARGET_SCOPES	no

DEPLOYMENT_MAPPING is read as a JSON object, e.g.:

export DEPLOYMENT_MAPPING='{"source-deployment-uuid":"target-deployment-uuid"}'

If any required variable is missing, the tool exits with an error.

Running the Docker Image

Build

docker build -t checkout-data-sync .

Run

Mount your config.yml at /app/config.yml via a volume — credentials must never be baked into the image.

docker run --rm \
  -v $(pwd)/config.yml:/app/config.yml \
  checkout-data-sync

Examples

With a config file

• Dry-run (default), mounting config.yml into the container:

  docker run --rm -v $(pwd)/config.yml:/app/config.yml checkout-data-sync

  Example output:

  === DRY RUN — pass -f to execute ===
  
  --- Applications ---
  Found 2 application(s) in source project "source-project-key"
  
    [CREATE] application "my-application"
    [SKIP]   application "existing-app" — already up to date
  
  Applications: 1 to create, 0 to update, 1 skipped/errors
  
  --- Payment Integrations ---
  Found 3 payment integration(s) in source project "source-project-key"
  
    [CREATE] payment integration "credit-card-via-adyen"
    [UPDATE] payment integration "paypal" (target id: abc123, version: 2)
             • action: setStatus
    [SKIP]   payment integration "apple-pay" — already up to date
  
  Payment integrations: 1 to create, 1 to update, 1 skipped/errors
  

• Execute the migration:

  docker run --rm -v $(pwd)/config.yml:/app/config.yml checkout-data-sync -f

• Use a config file at a custom path:

  docker run --rm \
    -v $(pwd)/staging.yml:/app/staging.yml \
    checkout-data-sync -c /app/staging.yml -f

With environment variables

Copy .env.example to .env, fill in the values, then pass it to Docker via --env-file. Omit -c so the binary reads from the environment.

Note: with --env-file, Docker does not strip surrounding quotes — write DEPLOYMENT_MAPPING={"src":"tgt"} (no surrounding '…'). When source-ing the file in a shell, the surrounding single quotes shown in .env.example are correct.

• Dry-run:

  docker run --rm --env-file .env checkout-data-sync

• Execute the migration:

  docker run --rm --env-file .env checkout-data-sync -f

• Pass variables individually instead of --env-file:

  docker run --rm \
    -e SOURCE_PROJECT_KEY=src-project \
    -e SOURCE_CLIENT_ID=... \
    -e SOURCE_CLIENT_SECRET=... \
    -e SOURCE_AUTH_URL=https://auth.europe-west1.gcp.commercetools.com/oauth/token \
    -e SOURCE_CHECKOUT_API_URL=https://checkout.europe-west1.gcp.commercetools.com \
    -e TARGET_PROJECT_KEY=tgt-project \
    -e TARGET_CLIENT_ID=... \
    -e TARGET_CLIENT_SECRET=... \
    -e TARGET_AUTH_URL=https://auth.eu-central-1.aws.commercetools.com/oauth/token \
    -e TARGET_CHECKOUT_API_URL=https://checkout.eu-central-1.aws.commercetools.com \
    -e DEPLOYMENT_MAPPING='{"source-deployment-uuid":"target-deployment-uuid"}' \
    checkout-data-sync -f

Without Docker

go build -o checkout-data-sync .

# config-file mode
./checkout-data-sync -c config.yml        # dry-run
./checkout-data-sync -c config.yml -f     # execute

# env-var mode (requires SOURCE_*, TARGET_* exported in the shell)
set -a; source .env; set +a
./checkout-data-sync                      # dry-run
./checkout-data-sync -f                   # execute

Sync Behaviour

Applications

Applications are matched between source and target by their key.

Condition	Action
Key not found in target	CREATE
Key found, no fields differ	SKIP
Key found, fields differ	UPDATE

The following update actions are used: setName, setStatus, setDescription, setApplicationLogo, setCountries, setAllowedOrigins, setPaymentsConfiguration, setDiscountsConfiguration.

Agreements within an application are synced by name:

Condition	Action
Agreement name not in target	addAgreement
Agreement name not in source	removeAgreement
Agreement exists but differs	setAgreementName / setAgreementType / setAgreementStatus / setAgreementText

Payment Integrations

Payment integrations are matched against target resources using a two-step lookup:

1. Exact key match — looks up the source key in the target index.
2. Name fallback — if no key match, looks up by name. When a name match is found with a different key, the target key is considered stale and a setKey action is prepended to the update. This handles key renames in the source project.

Note: If multiple target payment integrations share the same name, the name index entry is removed to prevent ambiguous matching. A missing key match in that case results in a new resource being created.

Condition	Action
No match by key or name	CREATE
Key match, no fields differ	SKIP
Key match, fields differ	UPDATE
Name match, key differs	UPDATE with setKey prepended

The following update actions are used: setKey, setName, setStatus, setComponentType, setPredicate, setDisplayInfo, setSortingInfo, setAutomatedReversalConfiguration, setConnectorDeployment.

Key derivation for keyless integrations

When a payment integration has no key, one is derived from the combination of its source ID and name: {sanitised-id}-{sanitised-name}. Using the source ID as a prefix guarantees that two integrations sharing the same display name receive distinct keys.

Connector deployment mapping

Connector deployment IDs are environment-specific. Any payment integration that has a connectorDeployment must have its source deployment UUID listed in deployment_mapping. Without a mapping entry the integration is skipped with a clear error:

payment integration "credit-card-via-adyen": connectorDeployment "src-uuid" has no entry
in deployment_mapping — add it under deployment_mapping in config.yml and retry

Error handling

Errors on individual resources are non-fatal. The tool logs each failure, continues processing the remaining resources, and exits with a non-zero status when any error occurred. This means a single bad resource never blocks the rest of the migration.

Sample response

=== MIGRATING resources (source → target) ===

--- Applications ---
Found 1 application(s) in source project "hello-johnson-test"

applications   0% |                                        | (0/1, 0 res/hr) [0s:0s]  [CREATE] application "sample-johnson"
           → created (id: 7936a42c-8d88-4ba9-b135-d9ed1b01438d)
                                                                                    
Applications: 1 to create, 0 to update, 0 skipped/errors

--- Payment Integrations ---
Found 2 payment integration(s) in source project "hello-johnson-test"

payment-integrations   0% |                                        | (0/2, 0 res/hr) [0s:0s]  WARN: payment integration "9d99323c-5f8c-43f0-af0a-a56543a0c04b" has no key; derived key "9d99323c-5f8c-43f0-af0a-a56543a0c04b-credit-card"
  [CREATE] payment integration "9d99323c-5f8c-43f0-af0a-a56543a0c04b-credit-card"
           → created (id: cab9fbf3-84bf-4be7-8689-51b80274956e)
                                                                                            
Payment integrations: 1 to create, 0 to update, 1 skipped/errors

ERROR during payment-integration sync: payment integration errors:
payment integration "helo-jay": cannot resolve target app ID for source app "ff3947a3-f9a6-429f-9614-7d875ccc9411"

=== source HTTP metrics ===
  4 requests (3.7 req/s over 1.08s)
  2xx=3  4xx=1  5xx=0  other=0  net-errors=0
  latency p50=94ms  p95=170ms  p99=170ms

=== target HTTP metrics ===
  5 requests (5.2 req/s over 970ms)
  2xx=5  4xx=0  5xx=0  other=0  net-errors=0
  latency p50=127ms  p95=457ms  p99=457ms
Error: sync completed with errors (see output above)
sync completed with errors (see output above)
exit status 1

Scopes

For least-privilege access, use the following scopes instead of manage_project:

Operation	Source scope	Target scope
Read applications	view_checkout_applications:{projectKey}	—
Write applications	—	manage_checkout_applications:{projectKey}
Read payment integrations	view_checkout_payment_integrations:{projectKey}	—
Write payment integrations	—	manage_checkout_payment_integrations:{projectKey}

---

Maintained by: ogwurujohnson