HTTP 400 error code for HTTP proxy workaround with LDClientConfigBuilder_ServiceEndpoints_RelayProxyBaseURL

[TouHouPolice]

Is this a support request?
Yes

Describe the bug
I'm trying to use LDClientConfigBuilder_ServiceEndpoints_RelayProxyBaseURL walk-around mentioned in #394 to set HTTP proxy.
To reproduce

const std::string kSelectedProxy = "http://xx.xx.xx.xxx:443";

LDClientConfigBuilder builder = LDClientConfigBuilder_New(SDKKey);

std::cout<<"Using Proxy: "<<kSelectedProxy<<std::endl;
LDClientConfigBuilder_ServiceEndpoints_RelayProxyBaseURL(builder, kSelectedProxy.c_str());
......
std::cout<<"Starting client..."<<std::endl;
if (LDClientSDK_Start(client, kMaxWait, &initialized_successfully))
......

Expected behavior
SDK connects to LD endpoints through proxy.

Logs
Logs in streaming mode:

Using Proxy: http://xx.xx.xx.xxx:443
Starting client...
[LaunchDarkly] sse-client: backing off in (1) seconds due to HTTP status 400 (Bad Request)
[LaunchDarkly] sse-client: backing off in (3) seconds due to HTTP status 400 (Bad Request)
[LaunchDarkly] sse-client: backing off in (5) seconds due to HTTP status 400 (Bad Request)
Connection Timeout.

Logs in polling mode:

Using Proxy: http://xx.xx.xx.xxx:443
Starting client...
Connection Timeout.

SDK version
3.5.0

Language version, developer tools
C++17

OS/platform
Apple M3 Pro, 14.3

Additional context
The proxy servers are working without issue for other testings. I've also verified them using curl -x http://my-proxy:port https://www.google.com

[cwaldren-ld]

Hi @TouHouPolice , the client determines whether to use TLS or not based on the scheme of the URL.

For example, if you give it http://host:port, it will use a plaintext HTTP client. If you give it https://host:port, then it will use a TLS client.

I see you are using port 443. If the proxy only accepts TLS connections, then you will need to change the scheme to https://.

Can you verify if the proxy is an http or https proxy?

[TouHouPolice]

Hi @cwaldren-ld , I believe the proxy I'm testing against is http proxy.
If I change it to https, this is what I'm getting:

Using Proxy: https://xx.xx.xx.xxx:443
Starting client...
[LaunchDarkly] sse-client: backing off in (1) seconds due to wrong version number (SSL routines) [asio.ssl:167772427]
[LaunchDarkly] sse-client: backing off in (3) seconds due to wrong version number (SSL routines) [asio.ssl:167772427]
[LaunchDarkly] sse-client: backing off in (5) seconds due to wrong version number (SSL routines) [asio.ssl:167772427]

Additionally, I've actually tried your deprecated SDK's LDConfigSetProxyURI, and it can indeed connect to my proxy without issue.

[cwaldren-ld]

Hi @TouHouPolice ,

Thanks for testing that out. The reason I'm asking is because it seems confusing for the proxy to serve HTTP traffic on port 443. That is usually reserved for HTTPS traffic. It's unclear to me if this proxy is transparent or explicit (needs HTTP CONNECT tunneling.) Currently the SDK does not support explicit proxy.

Can you please try the following:

1. Connecting to http://xx.xx.xx.xxx:80 using the SDK, also test with curl.
2. Connect to ``https://xx.xx.xx.xxx:443`, with curl.

For the curl commands, please use the --verbose flag so we can see exactly how the proxy responds to the request.

Paste the output here, redact any sensitive information. If you aren't comfortable doing that, you can contact LaunchDarkly support (link to this Github issue) and I can review it internally.

[TouHouPolice]

Hi @cwaldren-ld , though the proxy server is not setup by myself, our document did label it as transparent.
Here are the test results:
Using SDK to connect to http://xx.xx.xx.xxx:80

Using Proxy: http://xx.xx.xx.xxx:80
Starting client...
[LaunchDarkly] sse-client: backing off in (1) seconds due to Connection refused [system:61 at /Users/singng/Git/boost/include/boost/asio/detail/reactive_socket_connect_op.hpp:98:37 in function 'do_complete']
[LaunchDarkly] sse-client: backing off in (3) seconds due to Connection refused [system:61 at /Users/singng/Git/boost/include/boost/asio/detail/reactive_socket_connect_op.hpp:98:37 in function 'do_complete']
[LaunchDarkly] sse-client: backing off in (5) seconds due to Connection refused [system:61 at /Users/singng/Git/boost/include/boost/asio/detail/reactive_socket_connect_op.hpp:98:37 in function 'do_complete']
Connection Timeout.

curl to http://xx.xx.xx.xxx:80:

% curl -x  http://xx.xx.xx.xxx:80 http://www.example.com --verbose
*   Trying xx.xx.xx.xxx:80...
* connect to xx.xx.xx.xxx port 80 from yy.yyy.yyy.yy port 64653 failed: Connection refused
* Failed to connect to xx.xx.xx.xxx port 80 after 12 ms: Couldn't connect to server
* Closing connection
curl: (7) Failed to connect to xx.xx.xx.xxx port 80 after 12 ms: Couldn't connect to server

curl to http://xx.xx.xx.xxx:443:

% curl -x  http://xx.xx.xx.xxx:443 http://www.example.com --verbose
*   Trying xx.xx.xx.xxx:443...
* Connected to xx.xx.xx.xxx (xx.xx.xx.xxx) port 443
> GET http://www.example.com/ HTTP/1.1
> Host: www.example.com
> User-Agent: curl/8.7.1
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
* Request completely sent off
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: max-age=604800
< Content-Type: text/html; charset=UTF-8
< Date: Wed, 23 Oct 2024 20:12:26 GMT
< ETag: "3147526947+gzip"
< Expires: Wed, 30 Oct 2024 20:12:26 GMT
< Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
< Server: ECAcc (sac/254E)
< Vary: Accept-Encoding
< X-Cache: HIT
< Content-Length: 1256
< Age: 524410
< Warning: 113 ecs-0668322f (squid/3.5.20) This cache hit is still fresh and more than 1 day old
< X-Cache: HIT from ecs-0668322f
< X-Cache-Lookup: HIT from ecs-0668322f:443
< Via: 1.1 ecs-0668322f (squid/3.5.20)
< Connection: keep-alive
< 
<!doctype html>
<html>
...
</html>
* Connection #0 to host xx.xx.xx.xxx left intact

curl to https://xx.xx.xx.xxx:443:

% curl -x  https://xx.xx.xx.xxx:443 http://www.example.com --verbose 
*   Trying xx.xx.xx.xxx:443...
* Connected to xx.xx.xx.xxx (xx.xx.xx.xxx) port 443
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /opt/anaconda3/ssl/cacert.pem
*  CApath: none
* OpenSSL/3.0.14: error:0A00010B:SSL routines::wrong version number
* Closing connection
curl: (35) OpenSSL/3.0.14: error:0A00010B:SSL routines::wrong version number

[cwaldren-ld]

Ok great, thanks for testing. It looks like it is Squid proxy.

I believe I misunderstood your issue. Unfortunately I don't believe the workaround I provided in the other link will work.

If the proxy is truly transparent and the target machine (running the LaunchDarkly SDK) has a Squid SSL root certificate installed on it, then it should be able to intercept HTTPS requests from the SDK without modifying any SDK configuration.

In this case, the SDK will think that it is communicating with LaunchDarkly but Squid will act as a man-in-the-middle.

It sounds like this might not be the case for your setup, especially since you are given a proxy IP address to use, but please correct me if I'm wrong.

Assuming this is true, I will file a feature request for properly supporting HTTPS proxy. It would help if you could email me at cwaldren@launchdarkly.com from your company email address so I could associate it with the request.

[TouHouPolice]

@cwaldren-ld Can I just confirm with you about the setup you were talking about?
I've had some success testing this workaround with Charles using Map Remote(Reverse Proxy) to map request explicitly to remote LD Endpoints.
Is this what you thought we were doing? I believe this workaround will definitely won't work for general public end-users who use proxies that we don't have any control over?
I'm not entirely sure about the talks around Transparent Proxy, because:

• LD SDK seems to ignore system proxy settings.
• If I configure RelayProxyBaseURL to my proxy server, then the outbound request does not seem to contain any information about LD endpoints, making it impossible to reach your server without configuring my proxy explicitly. This is not what I understood as a transparency proxy scenario.

Can you clarify the scenarios where this walkaround could be used? Thanks!

[cwaldren-ld]

Yes, you have the idea.

It definitely will not work for the public as a general solution, you are correct. It would only work if you can map the specific request paths to the correct LD endpoints, which means this would only work if the SDK is running in a network that you control.

This is why it's called RelayProxyBaseURL: because the LaunchDarkly Relay Proxy has handlers for each of the paths that the SDK can request, and serves the correct response.

What I'm talking about with the transparent proxy, is a scenario where the proxy software is totally invisible to the application running on the host (because the proxy is running at the network layer.) Think a corporate or school network, or at the airport.

The application makes a request to https://example.com, the proxy is a man-in-the-middle and forwards the data back and forth. The application has no need to configure the proxy specifically.

For your use-case of operating in networks you don't control, where people need to configure explicit proxy, the SDK needs to add support for this feature (similar to how you can set HTTP_PROXY or HTTPS_PROXY when using curl.)

In this case, the SDK might be configured to use a proxy, but other applications running on the host might not be.