feat: add write-only attributes for secret fields (Terraform >= 1.11) [ON HOLD]#329
feat: add write-only attributes for secret fields (Terraform >= 1.11) [ON HOLD]#329Aaron ("AJ") Steers (aaronsteers) wants to merge 3 commits intomainfrom
Conversation
Add x-speakeasy-terraform-write-only: true to the Speakeasy overlay for all airbyte_secret fields. This generates WriteOnly: true in the Terraform schema, preventing secrets from being persisted to plan or state artifacts. Also add documentation for ephemeral variable usage with Terraform >= 1.11 and OpenTofu >= 1.11. Closes #328 Co-Authored-By: AJ Steers <[email protected]>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Thanks for opening this pull request!Your contribution is appreciated. Here are some helpful tips and resources. 💡 Show Tips and TricksTerraform Example Commands
📚 Show Repo GuidanceAbout This RepositoryThis repository uses Speakeasy to generate the Terraform provider from the Airbyte OpenAPI specification. The CI will automatically build the provider, validate code generation, and run acceptance tests across Terraform versions 1.0-1.4. Note: This is a generated codebase. Direct modifications to generated files are not accepted - changes must be made to the upstream OpenAPI spec. |
Co-Authored-By: AJ Steers <[email protected]>
Add a global DEFAULT_ENTITY_VERSION constant (set to 1) and per-resource ENTITY_VERSION_OVERRIDES dict to the spec generation script. This sets x-speakeasy-entity-version on all typed connector resource schemas, which causes Speakeasy to: - Set Version: 1 on each resource's Terraform schema - Bootstrap boilerplate state upgrader files at internal/stateupgraders/ This is required to support the v0 -> v1 state migration when adding write-only attributes to secret fields. Without a version bump, the Terraform framework's same-version passthrough does not nullify write-only values in existing state, causing upgrade failures for users with secrets already stored in state from prior provider versions. Refs: #328 Co-Authored-By: AJ Steers <[email protected]>
|
Note: We have decided to deprecate the typed source and destination resources, in favor of the generic airbyte_source and airbyte_destination resources - along with the validation helper airbyte_connector_configuration. With typed resources phasing out (target removal in 1.1), the write-only annotation for those resources has no long-term value. Putting this PR on hold for now. (Devin, please update the PR description with a header to this affect.) |
Summary
Adds
x-speakeasy-terraform-write-only: trueto the Speakeasy overlay for all connector fields marked withairbyte_secret: true. After regeneration, this will generateWriteOnly: truein the Terraform schema, preventing secrets from being persisted to plan or state files entirely.Also adds README documentation covering write-only attributes and ephemeral variable usage for Terraform >= 1.11 / OpenTofu >= 1.11.
This is an overlay + docs + generation-script change. The generated provider code will not reflect write-only attributes until the next Speakeasy regeneration run (
poe generate-full).Closes #328
See also: #330 (follow-on for generic
airbyte_source/airbyte_destinationresources)Changes
Overlay (
overlays/terraform_speakeasy.yaml): Addsx-speakeasy-terraform-write-only: trueto allairbyte_secretfields via the same JSONPath used forx-speakeasy-param-sensitive.Entity version bump (
scripts/generate_terraform_spec.py): AddsDEFAULT_ENTITY_VERSION = 1(bumped from implicit 0) andx-speakeasy-entity-versionto all typed resource schema templates. This causes Speakeasy to setVersion: 1on each resource and bootstrap boilerplate state upgrader files atinternal/stateupgraders/. Individual resources can override the default viaENTITY_VERSION_OVERRIDES. This is required to prevent a breaking upgrade path — see details below.README docs: Section on write-only attributes and ephemeral variable usage.
Test project (
test-projects/v1-tf-latest-test/main.tf): Added "Approach 4" demonstrating ephemeral variables with write-only secret fields.Breaking change mitigation (upgrade path)
During manual testing, we discovered that adding
WriteOnly: trueto fields that previously stored values in state is a breaking change without a schema version bump. When Terraform reads existing state (schema v0) with non-null secret values and the new provider declares those fields as write-only (still schema v0), the framework's same-version passthrough does not nullify write-only values, and Terraform core rejects the state:The fix: bumping the schema version to 1 triggers the explicit state upgrader code path, where the framework does nullify write-only attributes after the upgrader runs. This was validated with a proof-of-concept (apply with RC6 binary → plan with patched v1 binary → no error).
Important: After the next
poe generate-full, Speakeasy will bootstrap boilerplate state upgrader files. These upgraders need to be implemented to pass through the prior state correctly (the framework handles write-only nullification automatically after the upgrader runs, so the upgraders primarily just need to forward state as-is).Manual test results (prior to entity version changes)
Tested with Terraform 1.12.1 + CI binary:
terraform validate— passedterraform plan— showstoken = (sensitive, write-only attribute)terraform apply— source created successfully"token": null(secret not persisted)terraform plan(post-apply) — "No changes" (no false drift)terraform destroy— clean teardownReview & Testing Checklist for Human
This PR is on hold. If/when it is revisited (likely for generic resources per #330), the following should be verified:
airbyte_source/airbyte_destinationresources instead of (or in addition to) typed resources, given the deprecation plan.internal/stateupgraders/) return empty state and need implementation. Apost-generatescript approach was proposed but not yet built.Notes