Microsoft.Owin.Testing.OwinClientHandler: Sometimes the offload task never runs, resulting in hanging tests.

[kalbert312]

My team is utilizing this package@4.2.3 to run an in-memory test server for our E2E tests. While debugging tests, I've observed that most of the time, tests that utilize the server hang and the requests are never received by the web controllers. Debugging led me to OwinClientHandler where I saw:

            state.OwinContext.Request.Body = body;
            CancellationTokenRegistration registration = cancellationToken.Register(state.Abort);

            // Async offload, don't let the test code block the caller.
            Task offload = Task.Factory.StartNew(async () =>
                {
                    try
                    {
                        await _next(state.Environment);
                        state.CompleteResponse();
                    }
                    catch (Exception ex)
                    {
                        state.Abort(ex);
                    }
                    finally
                    {
                        registration.Dispose();
                        state.Dispose();
                    }
                });

            return await state.ResponseTask;

When the test hangs, the breakpoint set at await _next(state.Environment); is never hit. When it does not hang, the breakpoint is hit, and then subsequently the web controller breakpoint is hit. I'm not familiar with Task.Factory.StartNew but it seems like there might be a bug here based on some quick google searches. There's mentions that StartNew does not understand async delegates, and that a call to Unwrap() is necessary, or use Task.Run instead. https://sergeyteplyakov.github.io/Blog/async/2019/05/21/The-Dangers-of-Task.Factory.StartNew.html

[kalbert312]

This doc has a StartNew example for an async delegate that shows TaskCreateOptions.DenyChildAttach used as the creation option, but Task.Factory appears to use TaskCreateOptions.None if the argument is not specified on StartNew.

When it hangs, my breakpoint at await state.ResponseTask, shows the offload task status as "WaitingToRun"

[kalbert312]

More details:

• .NET Framework 4.7.2
• MSTest 4.0.1
• It only happens when the debugger is attached. Running the test normally works 100% of the time.

I reproed the issue with a barebones case that mocks the same pattern OwinClientHandler is using to resolve a response:

        [TestMethod]
        public async Task TempTest()
        {
            var otherTask = new TaskCompletionSource<string>();

            // no parameters except delegate = doesn't run most of the time
#pragma warning disable VSTHRD105
            var t = Task.Factory.StartNew(async () =>
#pragma warning restore VSTHRD105
            {
                await Task.Delay(100);

#pragma warning disable CS4014 // Because this call is not awaited, execution of the current method continues before the call is completed
#pragma warning disable VSTHRD105
                Task.Factory.StartNew(() => otherTask.TrySetResult("Result"));
#pragma warning restore VSTHRD105
#pragma warning restore CS4014 // Because this call is not awaited, execution of the current method continues before the call is completed
            });

            var result = await otherTask.Task.ConfigureAwait(false); // ConfigureAwait(false) must be here, or it hangs.
            TestContext.WriteLine("Result: " + result);
        }

This completes 100% in debug mode:

        [TestMethod]
        public async Task TempTest()
        {
            var otherTask = new TaskCompletionSource<string>();

#pragma warning disable VSTHRD105
            var t = Task.Factory.StartNew(async () =>
#pragma warning restore VSTHRD105
            {
                await Task.Delay(100);

#pragma warning disable CS4014 // Because this call is not awaited, execution of the current method continues before the call is completed
#pragma warning disable VSTHRD105
                Task.Factory.StartNew(() => otherTask.TrySetResult("Result"));
#pragma warning restore VSTHRD105
#pragma warning restore CS4014 // Because this call is not awaited, execution of the current method continues before the call is completed
            }, CancellationToken.None, TaskCreationOptions.DenyChildAttach, TaskScheduler.Default); // parameters must be set to these, or the task may not run.

            var result = await otherTask.Task.ConfigureAwait(false); // ConfigureAwait(false) must be here, or it hangs.
            TestContext.WriteLine("Result: " + result);
        }