Conversation
… distinctId causing excessive CDN requests
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
| getmyinode(), | ||
| getlastmod() | ||
| ])); | ||
| ?: md5($sdkSettings->getApiKey()); |
There was a problem hiding this comment.
The API key is not the distinct ID: doing this would lead to every client for this SDK key appearing as a single 'user' for our usage counting (assuming we started counting server usage in the future). But it would also affect split functionality (if user has not overridden the stickiness property). I suggest keeping this as it was - is there a reason we need to change the distinctId?
There was a problem hiding this comment.
(I know you mentioned in the description something about changing the cache key, which makes sense - this should not include the distinctId. But we can change cache key to use sdk key (& maybe buid? i.e. the cdn path part only) without changing the distinctId value.)
There was a problem hiding this comment.
Removed distinctId from CDN GET query params so the Guzzle cache key is stable across containers/pods — fixes CDN cache thrashing without touching the distinctId value
| PropertyType::getDistinctId()->getName() => | ||
| (string)$properties[PropertyType::getDistinctId()->getName()], |
There was a problem hiding this comment.
I don't know php at all, but is this removing distinctId from the query params (?distinctId=...)? Or just from the cache key? We should continue to send the parameter... just not use it as the cache key.
There was a problem hiding this comment.
Reverted. you're right, we were removing it from the request entirely which was wrong. Fixed: distinct_id is still sent as a query param to the CDN, we now override getCacheKey in CdnCacheStrategy to strip it from the URI before computing the cache key hash. So the CDN gets the param, but the local Guzzle cache key stays stable across instances.
| $this->_flagSetter->setExperiments(); | ||
|
|
||
| $hasChanges = ($this->_lastConfigurations == null || $this->_lastConfigurations->equals($result)); | ||
| $hasChanges = ($this->_lastConfigurations == null || !$this->_lastConfigurations->equals($result)); |
| // Strip distinct_id from the URI before computing the cache key so that | ||
| // per-instance identity doesn't cause cache misses across containers/pods. | ||
| // The param is still sent in the actual HTTP request. | ||
| $uri = Uri::withoutQueryValue($request->getUri(), PropertyType::getDistinctId()->getName()); |
There was a problem hiding this comment.
Are all the other URL query parameters going to be stable? I think simplest/ideal would just be Uri::path($request->getUri())` - if that's even a thing in php!
There was a problem hiding this comment.
simplified to $request->getUri()->withQuery('') which strips all query params. The other params (sdkVersion, platformVersion, etc.) are stable per deployment so no cache correctness concern, but keying on path only is cleaner 👍
markawm
left a comment
There was a problem hiding this comment.
👍 (at least, as much as I understand php!)
|
@ankurv02 |
let me cover this here before we push for release - #55 |
This PR have two major bugs reported by ChargePoint (SECO-5292 and SECO-5553) that were causing broken monitoring metrics and massive CDN traffic spikes.
Bug 1: AppliedFromLocalStorage status never fired
The Problem:
Our docs promise that the ConfigurationFetchedHandler will receive an AppliedFromLocalStorage status whenever the SDK serves a config from the local cache. In reality, it was always sending AppliedFromNetwork.
Why it happened:
The PHP SDK uses a Guzzle caching library to save CDN responses to the disk. This library sits entirely in the background. When it intercepted a request and served it from the local disk, our core SDK code just saw a standard HTTP 200 response. It had no clue the data came from the cache, so it defaulted to saying it came from the network.
The Fix:
The Guzzle caching library actually tags its cache hits with an X-Kevinrob-Cache: HIT header. We now check for this header and bubble that flag up the response chain to Core::fetch(). Now, the SDK can accurately fire AppliedFromLocalStorage on cache hits and AppliedFromNetwork on real network hits.
Bug 2: Bad cache keys causing ~1M CDN requests/hour
The Problem:
ChargePoint noticed that setting setConfigFetchIntervalInSeconds(300) (5 minutes) wasn't doing anything. They were still hammering the CDN on every single execution—totaling around 1 million requests per hour.
Why it happened:
The SDK builds a CDN query URL using a distinctId to identify the server node. If a user didn't explicitly set the ROLLOUT_INSTANCE_ID environment variable, the SDK tried to generate a fallback ID using the PHP script's file inode and last-modified timestamp.
In a Docker/Kubernetes setup, these file properties constantly change across different containers and pods. Because the distinctId was unstable, the CDN URL was different on almost every request. Since Guzzle uses the exact URL as its cache key, the cache missed every single time and forced a real network call.
The Fix:
Swapped out the unstable file-based fallback for a hash of the API key: md5($apiKey). The API key is static, uniquely identifies the app, and stays perfectly stable across containers, pods, and deployments. (Note: If a customer relies on unique instance tracking, ROLLOUT_INSTANCE_ID still takes priority over the fallback).
What changed under the hood
Psr7ResponseWrapper: Now sniffs out the X-Kevinrob-Cache: HIT header.
HttpResponseInterface / AbstractHttpResponse: Added an isFromCache() method (defaults to false so nothing breaks upstream).
ConfigurationFetchResult: Acts as the carrier to pass the cache boolean up to the core engine.
ConfigurationFetcherBase / ConfigurationFetcher: Forwards the cache flag specifically for CDN responses.
Core::fetch(): Evaluates the cache flag to fire the correct FetcherStatus.
ServerProperties: Switched the distinctId fallback logic to use md5($apiKey).
Tests: Added two new unit tests and updated the E2E handler to assert against both cache statuses.