feat: polymorphic helper for request body streaming

[tank-bohr]

Hey guys. Let me explain the problem I faced with. Below is the typical code for the sreaming body request

defmodule MintDialyzerRepro do
  @default_chunk_size 16_384

  def run(method, %URI{scheme: scheme} = uri, headers, body, opts)
      when scheme in ~w[http https] do
    scheme = String.to_existing_atom(scheme)
    port = uri.port || URI.default_port(uri.scheme)
    path = URI.to_string(%URI{uri | scheme: nil, host: nil, authority: nil})

    with {:ok, conn} <- Mint.HTTP.connect(scheme, uri.host, port, opts) do
      stream(conn, method, path, headers, body)
    end
  end

  defp stream(conn, method, path, headers, body) do
    with {:ok, conn, ref} <- Mint.HTTP.request(conn, method, path, headers, :stream) do
      stream_body(conn, ref, body)
    end
  end

  defp stream_body(conn, ref, "") do
    Mint.HTTP.stream_request_body(conn, ref, :eof)
  end

  defp stream_body(conn, ref, body) do
    chunk_size = min(chunk_size(conn, ref), byte_size(body))
    <<chunk::binary-size(chunk_size), rest::binary>> = body

    with {:ok, conn} <- Mint.HTTP.stream_request_body(conn, ref, chunk) do
      stream_body(conn, ref, rest)
    end
  end

  defp chunk_size(conn, ref) do
    case Mint.HTTP.protocol(conn) do
      :http1 ->
        @default_chunk_size

      :http2 ->
        chunk_size_http2(conn, ref)
    end
  end

  defp chunk_size_http2(conn, ref) do
    min(
      Mint.HTTP2.get_window_size(conn, :connection),
      Mint.HTTP2.get_window_size(conn, {:request, ref})
    )
  end
end

This code generates dialyzer warnings

Total errors: 2, Skipped: 0, Unnecessary Skips: 0
done in 0m1.78s
lib/mint_dialyzer_repro.ex:46:34:call_without_opaque
Function call without opaqueness type mismatch.

Call does not have expected opaque term of type Mint.HTTP2.t() in the 1st position.

Mint.HTTP2.get_window_size(_conn :: Mint.HTTP.t(), :connection)

________________________________________________________________________________
lib/mint_dialyzer_repro.ex:47:34:call_without_opaque
Function call without opaqueness type mismatch.

Call does not have expected opaque term of type Mint.HTTP2.t() in the 1st position.

Mint.HTTP2.get_window_size(_conn :: Mint.HTTP.t(), {:request, reference()})

________________________________________________________________________________
done (warnings were emitted)
Halting VM with exit status 2

It was discussed in the #380 and was left with a dirty workaround.
And people are using the workaround. And on the OTP-28 such trick in the k8s library leads to the infinite hanging during the PLT building phase.

But it's a legit issue. And it's totally worth to be fixed.

Opaqueness

Since the caller have to know the exact type for streaming, the Mint.HTTP.t() cannot be opaque. And the caller cannot know the exact type in advance either, because it's not their decision, but the server's decision.

Missing API for streaming

Another consequence of this code and dialyzer warning is the request body streaming API is not sufficient for using it without knowing the underlying protocol. But it's kinda designed to be such. To close the gap the current PR is introducing the missing piece - an ability to get the chunk size for the streaming.

[tank-bohr]

@ericmj Could you please take a look

[whatyouhide]

What if you make

@type Mint.HTTP.t() :: Mint.HTTP1.t() | Mint.HTTP2.t()

so that the "top-level" type is not opaque, but the inner types are?

[tank-bohr]

What if you make

@type Mint.HTTP.t() :: Mint.HTTP1.t() | Mint.HTTP2.t()

so that the "top-level" type is not opaque, but the inner types are?

That's exactly what my PR does. It un-opaques (exposes) the Mint.HTTP.t(). This will help to solve the dialyzer issue in the existing code (like k8s). Because existing code has to know about internals.

And get_send_window callback potentially can allow to opaque the Mint.HTTP.t() type as well. But it requires changing the callers code. But it's helpful anyhow. It bring the missing piece to the ubiquities API of Mint.HTTP

[tank-bohr]

the original example with the get_send_window will look like

defmodule MintDialyzerRepro do
  def run(method, %URI{scheme: scheme} = uri, headers, body, opts)
      when scheme in ~w[http https] do
    scheme = String.to_existing_atom(scheme)
    port = uri.port || URI.default_port(uri.scheme)
    path = URI.to_string(%URI{uri | scheme: nil, host: nil, authority: nil})

    with {:ok, conn} <- Mint.HTTP.connect(scheme, uri.host, port, opts) do
      stream(conn, method, path, headers, body)
    end
  end

  defp stream(conn, method, path, headers, body) do
    with {:ok, conn, ref} <- Mint.HTTP.request(conn, method, path, headers, :stream) do
      stream_body(conn, ref, body)
    end
  end

  defp stream_body(conn, ref, "") do
    Mint.HTTP.stream_request_body(conn, ref, :eof)
  end

  defp stream_body(conn, ref, body) do
    chunk_size = Mint.HTTP.next_body_chunk_size(conn, ref)
    <<chunk::binary-size(chunk_size), rest::binary>> = body

    with {:ok, conn} <- Mint.HTTP.stream_request_body(conn, ref, chunk) do
      stream_body(conn, ref, rest)
    end
  end
end

As you see no need for using the HTTP2 module directly. No need to detect protocol. The Mint.HTTP module dispatches to the correct implementation itself. With this change we can keep Mint.HTTP.t() opaque. But I still recommend to unopaque it because of the existing code