fix(apigee): retry "the resource is locked by another operation" - #18649
Open
alec-pinson wants to merge 1 commit into
Open
fix(apigee): retry "the resource is locked by another operation"#18649alec-pinson wants to merge 1 commit into
alec-pinson wants to merge 1 commit into
Conversation
Apigee serialises long-running operations per organization. Any mutation issued while another operation is in flight against the same org is rejected immediately with HTTP 400 "the resource is locked by another operation" rather than being queued. google_apigee_instance already retries this via transport_tpg.IsApigeeRetryableError (GoogleCloudPlatform#10979), but every other org-scoped asynchronous Apigee resource can hit the same error and does not. This is easy to reproduce with a for_each that creates two of them concurrently, e.g. two google_apigee_endpoint_attachment resources in one apply: the first succeeds, the second fails outright. Apply the existing predicate to the remaining org-scoped async resources: EndpointAttachment, InstanceAttachment, EnvgroupAttachment, Environment, Envgroup and NatAddress. Reported for endpoint attachments in hashicorp/terraform-provider-google#12369 and for instance attachments in hashicorp/terraform-provider-google#11445 and hashicorp/terraform-provider-google#10084.
|
Googlers: For automatic test runs see go/terraform-auto-test-runs. @c2thorn, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look. You can help make sure that review is quick by doing a self-review and by running impacted tests locally. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Apigee serialises long-running operations per organization. A mutation issued while another operation is in flight against the same org is rejected immediately with HTTP 400
the resource is locked by another operationrather than being queued.google_apigee_instancealready retries this viatransport_tpg.IsApigeeRetryableError(added for hashicorp/terraform-provider-google#10979), but every other org-scoped asynchronous Apigee resource can hit the same error and does not.This is easy to reproduce with a
for_eachthat creates two of them concurrently in a single apply — the first succeeds, the second fails outright:There is no practical config-side workaround: Terraform builds dependency edges at the config-node level rather than per instance, so neither a
time_sleepstagger nor a child-module-per-item stagger serialises a dynamicfor_each— every instance still starts after the longest sleep. That leaves-parallelism=1for the whole apply, which also serialises the 30–45 minutegoogle_apigee_instancecreates.This PR applies the existing predicate to the remaining org-scoped async Apigee resources:
EndpointAttachment,InstanceAttachment,EnvgroupAttachment,Environment,EnvgroupandNatAddress.Note that
InstanceAttachmentalready carriesmutex: apigeeInstanceAttachments. That mutex only covers a single provider process and does not prevent collisions against the same org from a different resource type, so the predicate is complementary rather than redundant.Contributes to hashicorp/terraform-provider-google#12369, hashicorp/terraform-provider-google#11445 and hashicorp/terraform-provider-google#10084.