YARP returns 502 without waiting for a response when upstream closes the stream without reading to end.

[fknshwow]

Not entirely sure if this is expected behaviour but I only seem to have this problem when using YARP as a proxy. So this is in the context of a file upload via a POST request and I have YARP configured with no frills, just a straight pass through.

My API, for a file upload (Any response/cancellation prior to reading the stream will do but for this example we will look at file size validation), prior to reading the stream will look at the request content length then decide if it will continue with the request or return a 400 bad request. What I have found is if my API returns a 400, it closes the stream and YARP will immediately respond with a 502 without looking at the response from my API meaning my 400 is never passed back through to my client.

YARP logs a message such as:
One or more errors occurred. (Unable to write data to the transport connection: Broken pipe.) (Sent 577536 request content bytes, but Content-Length promised 12526142.) Unable to write data to the transport connection: Broken pipe. Broken pipe Sent 577536 request content bytes, but Content-Length promised 12526142.

Here is a sample controller method which will result in a 502 from YARP. For my testing I am sending a file that is around 12mb to invoke my validation response.

[HttpPost]
[Route("csv/upload")]
public async Task<ActionResult> CreateRecordsViaCsv()
{
    const int maxFileSizeInMegaBytes = 10;
    const int maxContentLength = maxFileSizeInMegaBytes * 1024 * 1024; // 10MB limit

    // Validate content size before reading the request stream
    if (Request.ContentLength > maxContentLength)
    {
        return BadRequest("File too big");
    }

    var insertedCount = await _recordService.CreateBatchCsvAsync(Request.Body);

    return Ok(new RecordBatchCreateDto(insertedCount));
}

And here is a sample with a workaround/fix.

I am able to get around this by completely reading the stream to end prior to doing any validations, I am doing this by reading into another stream which I then pass to my service for processing, I might later stream to a blob store instead of keeping the file in memory.

[HttpPost]
[Route("csv/upload")]
public async Task<ActionResult> CreateRecordsViaCsv()
{
    const int maxFileSizeInMegaBytes = 10;
    const int maxContentLength = maxFileSizeInMegaBytes * 1024 * 1024; // 10MB limit

    // Read the stream first to ensure the request body is fully consumed before any validation
    // This prevents yarp from returning 502 when validation fails
    using var memoryStream = new MemoryStream();
    await Request.Body.CopyToAsync(memoryStream);
    
    // Validate after reading the request stream to ensure proper error responses are sent to client
    if (memoryStream.Length > maxContentLength)
    {
        return BadRequest("File too big");
    }

    memoryStream.Position = 0;
    var insertedCount = await _recordService.CreateBatchCsvAsync(memoryStream);

    return Ok(new RecordBatchCreateDto(insertedCount));
}

Now, I know there are improvements in my API that I can and intend to make, but is YARP expected to behave in this way?

EDIT: I should add, this is producible in a Kubernetes environment where other gateways are involved. On my local I cannot get it to reproduce

[MihaZupan]

You've hit #1501 / #2731, I'd subscribe to updates there.

Possible workarounds include using HTTP/2 between the proxy and backend, or enabling Expect: 100-Continue on outgoing requests. See #2731 (comment)

[fknshwow]

Thanks, I will take a look