Websocket shim is not getting injected to the response if accept encoding header is present

[pkvprakash]

Websocket shim code is not getting injected if Accept-Encoding header is present with the request.

Response when Accept-Encoding header is present with the request:

Response when Accept-Encoding header is not present in the request:

[ojarjur]

My guess is that the response body is being sent back compressed (due to the Accept-Encoding header being set to gzip, deflate, br).

The code in the proxy agent that injects the shim code does not support decompressing/recompressing it, so compressed responses just get sent back as-is.

That is something that we are unlikely to support directly as the added complexity of decompressing/recompressing the output is probably not worth the added benefit.

It would be better to instead disable the compressed responses from your backend server; either directly (if your server supports that), or via a proxy that drops this header when the Accept header is text/html.

However, the best option would be to just support websocket directly instead of requiring the shim code injection.

For the case of the Inverting Proxy server, that could be implemented here, by converting the shim logic from Javascript to Go.

I'd be open to any pull requests that added that.

If you were to use the AppEngine version of the proxy that would not help you (since AppEngine does not support websocket), but from your screenshots it looks like you are not doing that.

[anavin-pub]

Hi @ojarjur,

This is the approach I was planning to go ahead with. Please let me know if you have any suggestions:

For an incoming HTTP request from the client, we check if the headers suffice to open a websocket connection (between the client and the proxy) if yes, we open it.

When the agent polls and receives the upgrade request, we open a websocket connection between agent and backend too.

For further communication of the websocket frames,

• client <> proxy communication happens via the open websocket connection.
• proxy <> agent happens via polling by the agent. I'm not sure how this data can be represented in the agent yet. Currently ForwardedRequest has a field Contents of type *http.Request which works for HTTP but might not work for websockets. Will check a few options.
• agent <> backend communication happens via the open websocket connection.

Thanks!

[ojarjur]

Hi @ojarjur,

This is the approach I was planning to go ahead with. Please let me know if you have any suggestions:

For an incoming HTTP request from the client, we check if the headers suffice to open a websocket connection (between the client and the proxy) if yes, we open it.

When the agent polls and receives the upgrade request, we open a websocket connection between agent and backend too.

For further communication of the websocket frames,

• client <> proxy communication happens via the open websocket connection.
• proxy <> agent happens via polling by the agent. I'm not sure how this data can be represented in the agent yet. Currently ForwardedRequest has a field Contents of type *http.Request which works for HTTP but might not work for websockets. Will check a few options.
• agent <> backend communication happens via the open websocket connection.

Thanks!

That's sort of what the existing shim logic in the agent already does.

I.E. your proposal is roughly the same as what I described above when I said "that could be implemented here, by converting the shim logic from Javascript to Go".

More specifically:

1. Inside the proxy's ServerHTTP method, the websocket upgrade (HTTP) request is upgraded into a websocket connection.
2. The proxy's ServerHTTP method then converts websocket messages in both directions into HTTP requests using the same protocol used by the websocket shim.
3. The agent's existing polling fetches those HTTP requests from the proxy.
4. The agent's existing logic for handling the websocket shim requests take care of converting these back into a websocket connection to the backend.

Only the first two steps are actually new work. The agent code is left completely unchanged from what it is today.

[anavin-pub]

Thanks, I had not fully understood your earlier comment.

Here is an attempt to add websocket support in server.go: #151

Please let me know if any changes are required.

[ojarjur]

Thanks for taking the time and putting in the effort to add that @anavin-pub!

I really appreciate the work you put in here.

We do need a contributor license agreement first before we can review your change, though.

This site should walk you through everything that is needed: https://cla.developers.google.com/

Thanks again for your efforts here; we really appreciate the time you've taken to make the codebase better.

[anavin-pub]

Thanks @ojarjur!

Trying to figure out the CLA part internally, let me get back once I have the approval.

[anavin-pub]

Hi @ojarjur
The CLA check has passed! Could you please have a look at the changes?

[ojarjur]

@anavin-pub Thank you for all of your work here.

I'll try to get a review done by the end of this week.

[anavin-pub]

Hey @ojarjur, did you get a chance to have a look at this?

[anavin-pub]

Hi @ojarjur
just checking if you got a chance to review?