Gracefully restart running application

[kinnalru]

Hi. I investigate possibility to gracefully restart currently running rails application from inside running instance. Main aim is to restart application after serving certain number of requests (ex. After 4000 requests) or after hitting memory limit.

In Passenger or Puma this is possible from middleware through passenger-config dettach-process $pid or something like this.

I see some methods is async::containers like reload or other but I can't understand how to use them from inside middleware. Is it sufficient to "kill self by INT signal" ?

[ioquatix]

Yes, but that will terminate all current requests handled by that process... try it out and let me know how you get on?

[kinnalru]

Yes, but that will terminate all current requests handled by that process... try it out and let me know how you get on?

I make some experimets... Test server through falcon serve and falcon host
... killing self from inside application (from middleware) kills current request immidiate ...
... falcon supervisor restart stops application I don't understand why ...

But killing through signal HUP whole server works as expected but some requests are broken. Example (2 forks):

1. starting rails server (without DB just api) by RAILS_ENV=production bundle exec falcon host

# falcon.rb

load :rack, :supervisor

rack 'prober' do
  scheme 'http'
  protocol { Async::HTTP::Protocol::HTTP1 }

  count 2

  endpoint do
    Async::HTTP::Endpoint.for(scheme, '0.0.0.0', port: 3000, protocol: protocol)
  end

end

supervisor

2. requesting with Yandex Tank 10 rps.
3. killing with single kill -HUP $PID
4. ... aplication reloads...
5. see broken requests :(

[ioquatix]

Okay let me take a look.

[trevorturk]

I believe this is a related issue: #71, and also maybe #22

[ioquatix]

Sorry, I did not follow up on this, I will take a look today.

[ioquatix]

I won't get a chance to look at this but I've put it on my planning board.

[trevorturk]

No worries, it's a nice-to-have, but I appreciate your attention in any case!

[kinnalru]

Sometimes when I go back I see this in Service::Supervisor:

	# Restart the process group that the supervisor belongs to.
	def do_restart(message)
		# Tell the parent of this process group to spin up a new process group/container.
		# Wait for that to start accepting new connections.
		# Stop accepting connections.
		# Wait for existing connnections to drain.
		# Terminate this process group.
		signal = message[:signal] || :INT
		
		Process.kill(signal, Process.ppid)
	end

...but there is no such feature in Service::Application :(