Quick and dirty unsafe queue API

[djspiewak]

Starting point for an unsafe queue API as part of std. Opening WIP PR to start the discussion.

For background… @armanbilge and I have been discussing this at length for a few months, prompted by some work that @kamilkloch was doing on high performance websocket handling. Specifically, we were looking at how to reduce the overhead associated with Dispatcher.sequential down to a minimum. In Kamil's case, there is a need for something much more bespoke than what could go into Cats Effect itself, but the line of reasoning was interesting nonetheless.

The point was made that most practical use of Dispatcher – particularly Dispatcher.sequential – is composed with Queue#offer. In other words, most practical code isn't actually running general effects, it's just putting stuff onto an asynchronous queue. While in theory the general variant of that problem still requires Dispatcher, there are special cases which can be handled with less overhead. An example of one is actually the work queue inside of Dispatcher itself, which allows for enqueue from impure code and dequeue from pure code. This is precisely the sort of machinery that most people are looking for, and they use Dispatcher + Queue to get it. We can special-case this to remove a whole layer of overhead in what is generally a fairly hot path.

The way to special case here is to make the observation that our existing async queues already allow for impure offer-ish implementations. Or more specifically, our unbounded queue allows for an impure offer (since it will never block), and our bounded queue allows for an impure tryOffer (since the queue might be full). This PR reifies this observation in a pair of traits: cats.effect.std.unsafe.BoundedQueueSink and cats.effect.std.unsafe.UnboundedQueueSink, and then implements those traits (trivially) in the existing implementations, exposing them in a pair of new constructors: Queue.unsafeBounded and Queue.unsafeUnbounded. These constructors return real Queues which can be used as normal, but with the added ability to use the appropriate offer variant impurely.

This btw is a fun theoretical aside, since there's something modestly profound here. Cats Effect safely abstracts over bounded and unbounded queues because fiber blocking is cheap, so we're allowed to have a single offer signature (A => F[Unit]) which works for both, despite the fact that it might block in the bounded case. We got this idea from Java's standard library, which plays the same trick but with threads, but blocking threads is expensive and so arguably Java's abstraction is a bit leaky. Rather than repeat that mistake with our own impure queues, we simply don't allow for unsafeOffer: A => Unit on a bounded queue, which means we differentiate bounded and unbounded queues at the type level in this API (note that, unintuitively, an unbounded queue is a special case of a bounded queue in this paradigm).

Anyway, I'm not super thrilled with the Queue[F, A] with unsafe.BoundedQueueSink[F, A] (and similar with unbounded) paradigm, so it might be saner to build an unsafe.BoundedQueue and unsafe.UnboundedQueue trait which simply names this type, particularly since F is polyvariant. I am fond of the unsafe package prefix though, and I think it works reasonably well.

TODO

• Become happy with the API
• Write more than zero tests
• Write more than zero lines of docs

[durban]

I like this idea. I've left a small comment about the scaladoc.

---

Another fun thing we could do is a Deferred with a def unsafeComplete(a: A): Boolean. (I'm not saying to do it in this PR, it's just a similar idea.)

[armanbilge]

Another fun thing we could do is a Deferred with a def unsafeComplete(a: A): Boolean. (I'm not saying to do it in this PR, it's just a similar idea.)

There have been multiple attempts by you and me at similar ideas 😅

• Deferred with different effects for source and sink #1583
• Introduce GenDeferred #2810

I think unlike offering to a Queue, completing a Deferred may wake up more than one fiber. In that case you really want to be on the runtime first, see #2810 (comment).

[durban]

There have been multiple attempts by you and me at similar ideas 😅

Yep 😄

In that case you really want to be on the runtime first

Yeah, ok, that's fair. (Although apparently I have trouble letting go of this idea...)