I needed to write a simple task specific HTTP reverse-proxy. I spiked out the initial version in Go, but we're a Ruby shop, so I then attempted to translate it into Ruby with async-http. I ran into enough issues that we had to stick with the Go version. Here are the things I wasn't able to figure out.
Graceful shutdown of Async::HTTP::Server – The only way to stop the server seems to be calling stop on its task. I needed something closer to Server.Shutdown in Go's net/http package. The server should stop accepting new connections immediately, and then should wait a configurable amount of time for existing connections to finish. Something like this:
task.with_timeout(10) do
server.shutdown
end
I didn't have to deal with long-running connections like WebSockets, so this API would be good enough for me. Go provides func (srv *Server) RegisterOnShutdown(f func()) for that use-case (f gets called by Shutdown) but I'm not sure that's totally necessary. I don't understand the use-case well enough.
Read timeouts – I needed to be able to make sure nobody could DOS the proxy opening up a ton of connections and not writing any data to them, eventually using up all available file descriptors. I tried setting Async::HTTP::Endpoint#timeout which does in fact time out the request in that situation, but that would raise an exception that I wasn't able to catch, and (I believe) the exception was raised all the way up through Server#run which meant that once the first request timed out, the task accepting new connections was dead. I wasn't able to find a place where I could rescue that exception usefully.
I attempted to subclass Server to get this working, but based on the way that the server ping-pongs control to the Endpoint and back again during accept, there didn't seem to be any method I could override to make this work.
Having more control over where tasks get canceled – This is probably more for async than async/http. If you call Task#stop on another Task, I believe it'll stop the next time that task yields back to the scheduler (for IO, etc.). It would be nice to have more control of when the task stops – e.g. to be able to clean up some in-flight work. In Go, the context API gives you this control. Goroutines can get canceled wherever you read from the Done channel. I'm not sure that the context API is the best one to copy (I found it pretty confusing at first), but it does give you this capability.
In the graceful shutdown example above, I'm not actually guaranteed that "stop accepting new connections" work will happen before the timeout hits, though it's admittedly very unlikely that it wouldn't be done within 10 seconds.
Hopefully this is helpful. I'm in awe of the amount of work that's gone into making the Ruby interpreter and ecosystem work with Async. I never thought we could get something quite so nice given that the language wasn't originally designed with this execution model in mind. Thank you!
I needed to write a simple task specific HTTP reverse-proxy. I spiked out the initial version in Go, but we're a Ruby shop, so I then attempted to translate it into Ruby with async-http. I ran into enough issues that we had to stick with the Go version. Here are the things I wasn't able to figure out.
Graceful shutdown of
Async::HTTP::Server– The only way to stop the server seems to be callingstopon its task. I needed something closer toServer.Shutdownin Go'snet/httppackage. The server should stop accepting new connections immediately, and then should wait a configurable amount of time for existing connections to finish. Something like this:I didn't have to deal with long-running connections like WebSockets, so this API would be good enough for me. Go provides
func (srv *Server) RegisterOnShutdown(f func())for that use-case (fgets called byShutdown) but I'm not sure that's totally necessary. I don't understand the use-case well enough.Read timeouts – I needed to be able to make sure nobody could DOS the proxy opening up a ton of connections and not writing any data to them, eventually using up all available file descriptors. I tried setting
Async::HTTP::Endpoint#timeoutwhich does in fact time out the request in that situation, but that would raise an exception that I wasn't able to catch, and (I believe) the exception was raised all the way up throughServer#runwhich meant that once the first request timed out, the task accepting new connections was dead. I wasn't able to find a place where I could rescue that exception usefully.I attempted to subclass
Serverto get this working, but based on the way that the server ping-pongs control to theEndpointand back again during accept, there didn't seem to be any method I could override to make this work.Having more control over where tasks get canceled – This is probably more for
asyncthanasync/http. If you callTask#stopon another Task, I believe it'll stop the next time that task yields back to the scheduler (for IO, etc.). It would be nice to have more control of when the task stops – e.g. to be able to clean up some in-flight work. In Go, the context API gives you this control. Goroutines can get canceled wherever you read from theDonechannel. I'm not sure that the context API is the best one to copy (I found it pretty confusing at first), but it does give you this capability.In the graceful shutdown example above, I'm not actually guaranteed that "stop accepting new connections" work will happen before the timeout hits, though it's admittedly very unlikely that it wouldn't be done within 10 seconds.
Hopefully this is helpful. I'm in awe of the amount of work that's gone into making the Ruby interpreter and ecosystem work with Async. I never thought we could get something quite so nice given that the language wasn't originally designed with this execution model in mind. Thank you!