zopefoundation / zdaemon Goto Github PK

It would be nice to provide this as a cmdline option.

This is something I'd really love to do because it would provide a lot more flexibility when using zdaemon from within a python script (as is my case =)).

Has it ever been considered to allow using zdaemon as a lib you can import instead of just as a CLI tool and would that be acceptable?

These three should be equivalent

zdaemon kill 12
zdaemon kill USR2
zdaemon kill SIGUSR2

Why should there be two separate commands for logfile reopening?

Try this:

$ cat conf
<runner>
program sleep 100
</runner>
$ zdaemon -C conf start
$ zdaemon -C conf kill; zdaemon -C stop; zdaemon -C status

I once got

kill(2289, 12)
signal SIGTERM sent to process 2289
daemon process not running
daemon manager running; daemon process not running

and then a second or so later

$ zdaemon -C conf status
program running; pid=2370

PROBLEM REPORT

What I did:

Run the tests for PyPy3 on GHA.

What I expect to happen:

green tests

What actually happened:

one test fails, see https://github.com/zopefoundation/zdaemon/actions/runs/7738159321/job/21098442355

What version of Python and Zope/Addons I am using:

• PyPy 3.9
• The difference in the dependencies are:
  • cffi==1.15.1 -> 1.16.0
  • hpy==0.0.4 -> 0.9.0

Any ideas what is happening here?

As far as I can see, zdaemon never reopens the files configured in the section, so you cannot rotate them externally (e.g. with logrotate) without restarting zdaemon.

I intend to change the logreopen and reopen_transcript commands to also call ZConfig.components.logger.loghandler.reopenFiles().

Two days ago I had a bit of a problem with a disk full. Today my entire web server hung and stopped processing requests.

Long story short, when zdaemon's Transcript thread gets an IOError while writing to a log file, it just dies. zdaemon itself and the child program it is managing remain running. The child's stdout/stderr are now pointing to a pipe that is still open at both ends, but now no process ever reads from it. Things seem to run fine for a couple of days, then the kernel's pipe buffer becomes full and the child blocks in write(). While holding the logging mutex. Fun for the whole family, conveniently delayed from the root cause you think you already fixed.

During my analysis of #32 I hit a behavior of zdctl start for daemon off mode which I consider a bug but which might be a feature:
In daemon off mode, zdctl start starts a zdrun process (which in turn starts the "program process") but waits for the zdrun process to terminate. This will only happen when the "program process" terminates (more or less regularly) or the zdrun process gets terminated by a signal.
This makes zdctl start behave very much like zdctl fg; the difference: the intermediate zdrun process may restart the "program process" should this terminate irregularly.
In daemon on mode, zdctl start returns immediately after the zdrun process is started; the zdrun and "program" processes can then be controlled via subsequent zdctl commands.

I expected the zdctl start behavior independent of "daemon" mode, because "daemon mode" essentially controls whether the effective zdrun process should be a true daemon (with its own process group, detached from the control terminal, input and output streams redirected to /dev/null, ...).

I found, however, that zdctl start in daemon off mode is more like zdctl fg than zdctl start (in `daemon on' mode).

I was interested in daemon off mode to be able to debug the zdrun process. In demon on mode, the zdrun process has its input and output streams (used by the debugger) redirected to /dev/null, making debugging extremely difficult. I found that the "transcript" feature (controlled via the -t option) allows at least the use of debugging output.

I have a patch to make the zdctl start behavior independent of the "daemon" mode. However, this would break cases depending on the current behavior.

It is no longer used neither in tox.ini nor in travis.yml.

Could you please consider to utilize unittest.mock instead of mock, as a fallback at least?

Right now, if I'm not mistaken, we can only run one process at a time. Are there any plans to add support for multiple processes? Perhaps to view status/start/stop you could just supply the pid as an arg instead of the full process bash code in quotes.

The test suite takes over 60 seconds.

Two tests are especially slow:

• nonzero_exit_on_program_failure (zdaemon.tests.tests) (33.362 s)
• src/zdaemon/README.rst (10.386 s)

Some other tests are longer than 1 second:

• DAEMON_MANAGER_MODE_leak (zdaemon.tests.tests) (2.109 s)
• allow_duplicate_arguments (zdaemon.tests.tests) (2.327 s)
• dont_hang_when_program_doesnt_start (zdaemon.tests.tests) (2.138 s)
• make_sure_non_daemon_mode_doesnt_hang_when_program_exits (zdaemon.tests.tests) (1.237 s)
• test_start_test_program (zdaemon.tests.tests) (6.328 s)
• test_start_timeout (zdaemon.tests.tests) (3.311 s)
• test_stop_timeout (zdaemon.tests.tests) (3.550 s)

The rest are fast.

Firstly, thanks for zdaemon - it has proved to be an awesome tool! 👍

I just wanted to let you know that it was mildly surprising that zdaemon warns about some existing programs, but not all:

> cat example.conf
<runner>
  program ./script.sh foo
</runner>

> zdaemon -C example.conf -d start
. 
daemon process restarted, pid=17438

> zdaemon -C example.conf -d status
program running; pid=17862

tap tap...

> cat example.conf
<runner>
  program ./script.sh
</runner>

> zdaemon -C example.conf -d status
program running; pid=17862

tap tap...

> cat example.conf                  

<runner>
  program ./script.sh wibble
</runner>

> zdaemon -C example.conf -d status 

WARNING! zdrun is managing a different program!
our program   = ['./script.sh', 'blas']
daemon's args = ['./script.sh', 'wibble']
program running; pid=17862

The line was pretty easy to find:

if args[:len(program)] != program:
    print("WARNING! zdrun is managing a different program!")

I was just wondering if there is a reason for only warning about programs with the number length of args?

It was removed in b95d4c9 because it was undocumented and untested.

The alternative is zdaemon kill 12, which requires me to know the number of the SIGUSR2 signal on my system and is just ugh.

My recent commit 95e5714 broke two tests on PyPy, although Travis sees only one of the failures:

======================================================================
FAIL: test_start_timeout (zdaemon.tests.tests)
Doctest: zdaemon.tests.tests.test_start_timeout
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/python/pypy-2.5.0/lib-python/2.7/doctest.py", line 2226, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for zdaemon.tests.tests.test_start_timeout
  File "/home/travis/build/zopefoundation/zdaemon/src/zdaemon/tests/tests.py", line 307, in test_start_timeout
----------------------------------------------------------------------
File "/home/travis/build/zopefoundation/zdaemon/src/zdaemon/tests/tests.py", line 327, in zdaemon.tests.tests.test_start_timeout
Failed example:
    system("./zdaemon -Cconf start")
Expected:
    <BLANKLINE>
    Program took too long to start
    Failed: 1
Got:
    <BLANKLINE>
    daemon process started, pid=NNN
----------------------------------------------------------------------

After tests are run, two processes are running:

$ ps uxa | grep zdaemon
menesis  13797  0.0  0.2 123868  8848 ?        Ssl  16:43   0:00 /home/menesis/src/zope/zdaemon/python/bin/python ./zdaemon -S /home/menesis/src/zope/zdaemon/src/zdaemon/schema.xml -C conf tail -f data
menesis  13805  0.0  0.2 118208  7800 ?        Ssl  16:43   0:00 /home/menesis/src/zope/zdaemon/python/bin/python /home/menesis/src/zope/zdaemon/src/zdaemon/zdrun.py -d -b 10 -s /tmp/tmpfBlZIn/testsock -x 0,2 -z /home/menesis/src/zope/zdaemon/src/zdaemon/tests /tmp/tmpfBlZIn/donothing.sh

They should be stopped in test cleanup.

Hello.
I have a configuration file looking like this:

<runner>
    daemon on
    program easymonitor
    socket-name _zdaemon.sock
    transcript myapp.log
</runner>

I would like "socket-name" and "transcript" paths to be relative compared to where the configuration file lives.
As zdaemon is right now, in case a relative path is specified, os.getcwd() is assumed.
That means that I'm forced to specify an absolute path which of course is valid on my machine but may change somewhere else (production, staging. etc.).

I think there should a way to override this behavior, either by providing some kind of templating systems (e.g. "transcript {{HERE}}/myapp.log") or by assuming os.path.abspath(os.path.dirname(file)) in case of a relative path.

Thoughts?