systemctl restart ndeploy_watcher fails about autom8n

[AnoopAlias]

After running /opt/nDeploy/scripts/attempt_autofix.sh

you are still seeing watcher in ps aux?
or if you run manually

systemctl restart ndeploy_watcher.service

does it still show failed ? and what is the status of the process in ps aux |grep watcher

[ivangrynenko]

Once I run attempt_autofix.sh, which ends up with this error:

Attempting to restart ndeploy_watcher daemon:
Job for ndeploy_watcher.service failed because the control process exited with error code. See "systemctl status ndeploy_watcher.service" and "journalctl -xe" for details.
Redirecting to /bin/systemctl restart ndeploy_watcher.service

Job for ndeploy_watcher.service failed because the control process exited with error code. See "systemctl status ndeploy_watcher.service" and "journalctl -xe" for details.

Further commands and output are listed in the order of execution

ps waux | grep watcher

root 410498 0.0 0.0 433748 12216 ? Sl 09:35 0:00 python /opt/nDeploy/scripts/watcher.py start

systemctl restart ndeploy_watcher.service

Job for ndeploy_watcher.service failed because the control process exited with error code. See "systemctl status ndeploy_watcher.service" and "journalctl -xe" for details.

systemctl status ndeploy_watcher.service

● ndeploy_watcher.service - nDeploy Watcher
Loaded: loaded (/usr/lib/systemd/system/ndeploy_watcher.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2015-12-22 12:51:03 AEDT; 28s ago
Process: 457892 ExecStart=/opt/nDeploy/scripts/watcher.py start (code=exited, status=1/FAILURE)

Dec 22 12:51:03 systemd[1]: Starting nDeploy Watcher...
Dec 22 12:51:03 systemd[1]: ndeploy_watcher.service: control process exited, code=exited status=1
Dec 22 12:51:03 systemd[1]: Failed to start nDeploy Watcher.
Dec 22 12:51:03 systemd[1]: Unit ndeploy_watcher.service entered failed state.

Dec 22 12:51:03 systemd[1]: ndeploy_watcher.service failed.

ps waux | grep watcher

root 410498 0.0 0.0 433748 12216 ? Sl 09:35 0:00 python /opt/nDeploy/scripts/watcher.py start

[AnoopAlias]

Can you try

systemctl stop ndeploy_watcher
systemctl status ndeploy_watcher.service
systemctl start ndeploy_watcher
systemctl status ndeploy_watcher.service

[ivangrynenko]

Here is the screenshot with command outputs:
https://www.evernote.com/l/AKGGvR3iuoRIY7LLNK6L5UMVUaomhmIeRdY

Additionally:
ps waux | grep watcher
root 2951 0.0 0.0 112648 956 pts/0 S+ 11:47 0:00 grep --color=auto watcher
root 617000 0.0 0.0 329968 10040 ? Sl Dec22 0:00 python /opt/nDeploy/scripts/watcher.py restart

[AnoopAlias]

Can you confirm if the watcher process start and stop are indeed showing a non-zero exit status. I was unable to find any issues on a plain centos7 server

root@centos7 [~]# systemctl status ndeploy_watcher
● ndeploy_watcher.service - nDeploy Watcher
Loaded: loaded (/usr/lib/systemd/system/ndeploy_watcher.service; enabled; vendor preset: disabled)
Active: active (exited) since Tue 2015-12-22 23:18:57 EST; 34min ago
Process: 12393 ExecStop=/opt/nDeploy/scripts/watcher.py stop (code=exited, status=0/SUCCESS)
Process: 12400 ExecStart=/opt/nDeploy/scripts/watcher.py start (code=exited, status=0/SUCCESS)
Main PID: 12408 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/ndeploy_watcher.service
└─12409 python /opt/nDeploy/scripts/watcher.py start

Dec 22 23:18:57 centos7.sysally.net systemd[1]: Starting nDeploy Watcher...
Dec 22 23:18:57 centos7.sysally.net systemd[1]: Started nDeploy Watcher.
root@centos7 []# /opt/nDeploy/scripts/watcher.py stop
root@centos7 []# echo $?
0
root@centos7 []# /opt/nDeploy/scripts/watcher.py start
root@centos7 []# echo $?
0

[ivangrynenko]

My output is the same, except of no CGroup (I guess because it is not running).

The server where it fails is CentOS 7 / cPanel / CloudLinux
which may be the reason it is different to the plain CentOS 7 install.

Any other ideas?

[mdpuma]

Check watcher.log

[ivangrynenko]

watcher.log file is empty

[AnoopAlias]

It shouldnt be empty

root@centos7 [~]# cat /opt/nDeploy/watcher.log
2015-12-23 01:49:14.613774
<open file '/opt/nDeploy/conf/jobs.yml', mode 'r' at 0x1c70f60>
job3
job2
job1

will be there in the file in a normal start .

Can you manually kill the watcher process and remove the watcher.pid file and start watcher using systemctl .See if the error persist and see watcher log is populated with the above status.

[ivangrynenko]

Worked.

Normally starting a process complains about existing .pid file. Maybe you could add this check into ndeploy_backends.service file, if .pid file exists, fail.

[AnoopAlias]

Does systemctl restart ndeploy_backends also work fine now?

Check that also .

Also the systemd service was indeed failing in the status when the pid file exist. Dont think there is need for an additional check.

[ivangrynenko]

Yes, systemctl restart ndeploy_backends works now. I do reckon you need to add a check for the pid file, as it seem to be the problem for the failure.

[vladislavkononenko]

Absolutely the same situation, Centos 7 + just installed Cpanel + nDeploy - "Job for ndeploy_watcher.service failed because the control process exited with error code".

(ndeploy_watcher, NOT ndeploy_backends!)

/opt/nDeploy/watcher.pid exists, but no running process with this PID.
Situation resolved by removing /opt/nDeploy/watcher.pid.
Please fix such incorrect behaviour, if it's possible. Or at least add cleaning of PID file to attempt_autofix.sh.

[AnoopAlias]

Just added removal of the pid file as a step in autofix.sh script.