Describe the bug
When applying entitlements, customers report 429 Too Many Requests api responses starting at 20-50 entitlements.
The entitlements are synced and ready, but this is a general scaleability issue in our provider.
I think entitlements can easily grow in numbers for bigger user projects so we should make sure to handle them efficiently.

Tested with Provider Version
v1.2.1

Time-box: 3d

Cheap workaround

• Adjust Poll interval - use different Default

To Reproduce
Steps to reproduce the behavior:

1. Apply Entitlement CR (around 20)
2. Run controller locally with debug flag
3. See logs for entitlements (e.g. HTTP Request "method": "GET", "path": "/entitlements/v1/assignments"), that many entitlements should then probably lead to 429 Too Many Requests.

Expected behavior
Our provider should be able to handle many entitlements at once without running into rate limits since customers might have a huge number of entitlements.

Additional context
For every entitlement, for every sync (default 1min) for every service name we make one call to the btl cli to get all entitlements in the subaccount:

	response, _, err := c.btp.EntitlementsServiceClient.
		GetDirectoryAssignments(ctx).
		SubaccountGUID(cr.Spec.ForProvider.SubaccountGuid).
		AssignedServiceName(cr.Spec.ForProvider.ServiceName).
		Execute()
	if err != nil {
		return nil, err
	}

https://github.com/SAP/crossplane-provider-btp/blob/main/internal/clients/entitlement/entitlement.go#L36

If we have multiple entitlements with the same service names we would make this call repeatedly.
This call can be changed to:

• return all entitlements for all subaccount and directories in the global account
• return all entitlements for all service names
  We could also make it without the specification of the service name and would get the entitlements for all service names.
  Then we would only need to do one network call for all entitlements and reuse this for this sync round for all entitlment crs we have. Same would go for the subaccount specification here.
  The network request might get larger but we reduce the overall number of requests (maybe it is possible to specify multiple service names/subaccounts in one request even, then we could have the same network request size as we have now but without multiple requests.

Steps

• reproduce the situation that 20 entitlement crs produce 429. By this we can analyse what entitlement pattern lead to this (maybe all in one subaccount or multiple, maybe all one service plan or multiple, maybe there is no pattern)
• based on this decide on how to improve this by changing the btp cli call and reuse the answer for others, maybe a cache

Acceptance critera

• put followup work into new issues
• bring to dev sync - add to community guide