Add concurrency_limit middleware

[zdenek-crha]

Motivation

The tower::limit::ConcurrencyLimit middleware does not work properly when used with services that return response with streaming body. The middleware considers concurrency only from call to service to response future resolution.

But response with streaming body can't be considered finished until body has been consumed.

Solution

Add concurrency_limit module with ConcurrencyLimit middleware implementation that holds on the semaphore permit until response and its body has been consumed.

It uses original middleware from tower crate and tower-http::metrics::InFlightRequests middleware as inspiration.

Questions/Considerations

As far as module organization goes, it would be nicer to add this as tower-http::limit::concurrency, similar to the module layout in tower crate. But the tower-http::limit module is fully dedicated to RequestBody middleware now and moving it into sub-module would probably be breaking change.

I have considered leaving RequestBody middleware where it is and just add tower-http::limit::concurrency sub-module, but it felt confusing. Adding another top level module felt as cleanest alternative I had.