Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stats Server: unix socket removed (vacuum) in Zerg-mode on 3rd initialization of vassal.ini #489

Open
dblyon opened this issue Jan 13, 2021 · 0 comments

Comments

@dblyon
Copy link

dblyon commented Jan 13, 2021

I'm running uWSGI in Zerg-mode (please see config and log output below). The idea being that after starting the zerg-server, I can repeatedly call "uwsgi vassal.ini" which will load a new master and workers, which will replace the currently running master and workers. This works nicely.
The issue: But once I call this command for the 3rd time (in total), the socket for the stats is removed (vacuumed) unexpectedly, which means I can't use uwsgitop any longer. It doens't help to issue the "uwsgi vassal.ini" command again.

uWSGI version 2.0.19.1

zerg.ini

[uwsgi]
master = true
zergpool = zerg.sock:app.sock
safe-pidfile = pid_zergpool.pid
daemonize = log_zerg.log
master-fifo = zerg.fifo
auto-procname = true
procname-prefix = agotool_zerg_

vassal.ini

[uwsgi]
master-fifo = new.fifo
master-fifo = running.fifo
master-fifo = sleeping.fifo

if-exists = sleeping.fifo
  hook-accepting1-once = writefifo:sleeping.fifo Q
endif =
if-exists = running.fifo
  hook-accepting1-once = writefifo:running.fifo 2p
endif =
hook-accepting1-once = writefifo:new.fifo 1

;; attach to zerg
socket = app.sock
zerg = zerg.sock
zerg-fallback = true

master = true
strict = true
enable-threads = false
vacuum = false
single-interpreter = true
die-on-term = true
need-app = true
lazy-apps = false

;; aGOtool settings
wsgi-file = runserver.py
callable = app
stats = stats_uwsgi.sock
memory-report = true
auto-procname = true ; give processes names (master and worker)
procname-prefix = agotool_vassal_

;; Logging
daemonize = log_vassal.log
safe-pidfile = pid_vassal.pid

;; Buffer size
buffer-size = 32768

;; When to restart workers
max-requests = 80000
max-worker-lifetime = 604800

;; Dynamic worker scaling
cheaper-algo = busyness
processes = 4
cheaper = 2
cheaper-initial = 2
cheaper-overload = 60
cheaper-step = 2
cheaper-busyness-multiplier = 60
cheaper-busyness-min = 20
cheaper-busyness-max = 70
cheaper-busyness-backlog-alert = 16
cheaper-busyness-backlog-step = 2
harakiri = 600

uwsgi output/log

*** Starting uWSGI 2.0.19.1 (64bit) on [Wed Jan 13 15:18:25 2021] ***
compiled with version: 7.5.0 on 15 October 2020 10:51:33
os: Linux-5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020
nodename: phobos
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 256
current working directory: ...somepath/agotool/app
detected binary path: ..somepath/bin/uwsgi
your processes number limit is 4126214
your memory page size is 4096 bytes
 *** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
[busyness] settings: min=20%, max=70%, overload=60, multiplier=60, respawn penalty=2
[busyness] backlog alert is set to 16 request(s), step is 2
[busyness] backlog non-zero alert is set to 60 second(s)
uwsgi zerg socket 1 attached to UNIX address app.sock fd 4
zerg sockets attached
Python version: 3.8.6 | packaged by conda-forge | (default, Oct  7 2020, 19:08:05)  [GCC 7.5.0]
Python main interpreter initialized at 0x5575faac6cd0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 507960 bytes (496 KB) for 4 cores
*** Operational MODE: preforking ***
spawned uWSGI master process (pid: 761070)
spawned uWSGI worker 1 (pid: 761074, cores: 1)
writing pidfile to pid_vassal.pid
spawned uWSGI worker 2 (pid: 761075, cores: 1)
writing pidfile to pid_vassal.pid
writing pidfile to pid_vassal.pid
*** Stats server enabled on stats_uwsgi.sock fd: 16 ***
loading version  1
loading version  2
WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x5575faac6cd0 pid: 761075 (default app)
WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x5575faac6cd0 pid: 761074 (default app)
running "writefifo:sleeping.fifo Q" (accepting1-once)...
open("sleeping.fifo"): No such device or address [core/hooks.c line 134]
running "writefifo:running.fifo 2p" (accepting1-once)...
open("running.fifo"): No such device or address [core/hooks.c line 134]
running "writefifo:new.fifo 1" (accepting1-once)...
Wed Jan 13 15:18:28 2021 - active master fifo is now running.fifo
*** Starting uWSGI 2.0.19.1 (64bit) on [Wed Jan 13 15:18:46 2021] ***
compiled with version: 7.5.0 on 15 October 2020 10:51:33
os: Linux-5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020
nodename: phobos
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 256
current working directory: ...somepath/agotool/app
detected binary path: ..somepath/bin/uwsgi
your processes number limit is 4126214
your memory page size is 4096 bytes
 *** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
[busyness] settings: min=20%, max=70%, overload=60, multiplier=60, respawn penalty=2
[busyness] backlog alert is set to 16 request(s), step is 2
[busyness] backlog non-zero alert is set to 60 second(s)
uwsgi zerg socket 1 attached to UNIX address app.sock fd 4
zerg sockets attached
Python version: 3.8.6 | packaged by conda-forge | (default, Oct  7 2020, 19:08:05)  [GCC 7.5.0]
Python main interpreter initialized at 0x55f9b72a9cc0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 507960 bytes (496 KB) for 4 cores
*** Operational MODE: preforking ***
spawned uWSGI master process (pid: 768640)
spawned uWSGI worker 1 (pid: 768647, cores: 1)
writing pidfile to pid_vassal.pid
spawned uWSGI worker 2 (pid: 768648, cores: 1)
writing pidfile to pid_vassal.pid
writing pidfile to pid_vassal.pid
*** Stats server enabled on stats_uwsgi.sock fd: 16 ***
loading version  3
loading version  4
WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x55f9b72a9cc0 pid: 768648 (default app)
WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x55f9b72a9cc0 pid: 768647 (default app)
running "writefifo:sleeping.fifo Q" (accepting1-once)...
open("sleeping.fifo"): No such device or address [core/hooks.c line 134]
running "writefifo:running.fifo 2p" (accepting1-once)...
running "writefifo:new.fifo 1" (accepting1-once)...
Wed Jan 13 15:18:49 2021 - active master fifo is now sleeping.fifo
Wed Jan 13 15:18:49 2021 - *** PAUSE (press start to resume, if you do not have a joypad send SIGTSTP) ***
Wed Jan 13 15:18:49 2021 - active master fifo is now running.fifo
Wed Jan 13 15:18:49 2021 - *** worker 2 suspended ***
Wed Jan 13 15:18:49 2021 - *** worker 1 suspended ***
*** Starting uWSGI 2.0.19.1 (64bit) on [Wed Jan 13 15:18:57 2021] ***
compiled with version: 7.5.0 on 15 October 2020 10:51:33
os: Linux-5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020
nodename: phobos
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 256
current working directory: ...somepath/agotool/app
detected binary path: ..somepath/bin/uwsgi
your processes number limit is 4126214
your memory page size is 4096 bytes
 *** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
[busyness] settings: min=20%, max=70%, overload=60, multiplier=60, respawn penalty=2
[busyness] backlog alert is set to 16 request(s), step is 2
[busyness] backlog non-zero alert is set to 60 second(s)
uwsgi zerg socket 1 attached to UNIX address app.sock fd 4
zerg sockets attached
Python version: 3.8.6 | packaged by conda-forge | (default, Oct  7 2020, 19:08:05)  [GCC 7.5.0]
Python main interpreter initialized at 0x557e61f7ecc0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 507960 bytes (496 KB) for 4 cores
*** Operational MODE: preforking ***
spawned uWSGI master process (pid: 772605)
spawned uWSGI worker 1 (pid: 772606, cores: 1)
writing pidfile to pid_vassal.pid
spawned uWSGI worker 2 (pid: 772607, cores: 1)
writing pidfile to pid_vassal.pid
writing pidfile to pid_vassal.pid
*** Stats server enabled on stats_uwsgi.sock fd: 16 ***
loading version  5
loading version  6
WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x557e61f7ecc0 pid: 772606 (default app)
running "writefifo:sleeping.fifo Q" (accepting1-once)...
running "writefifo:running.fifo 2p" (accepting1-once)...
SIGINT/SIGQUIT received...killing workers...
running "writefifo:new.fifo 1" (accepting1-once)...
Wed Jan 13 15:19:00 2021 - active master fifo is now sleeping.fifo
Wed Jan 13 15:19:00 2021 - *** PAUSE (press start to resume, if you do not have a joypad send SIGTSTP) ***
Wed Jan 13 15:19:00 2021 - active master fifo is now running.fifo
Wed Jan 13 15:19:00 2021 - *** worker 1 suspended ***
Wed Jan 13 15:19:00 2021 - *** worker 2 suspended ***
WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x557e61f7ecc0 pid: 772607 (default app)
worker 1 buried after 1 seconds
worker 2 buried after 1 seconds
goodbye to uWSGI.
VACUUM: safe pidfile removed.
VACUUM WARNING: unix socket app.sock changed inode. Skip removal
VACUUM: unix socket stats_uwsgi.sock (stats) removed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant