I'm trying to figure out how to upgrade my Wazuh managers. I tried the simple approach: change the image tag, apply, pray.
rm: cannot remove '/var/ossec/queue/db/.template.db': No such file or directory
Identified Wazuh configuration files to mount...
'/wazuh-config-mount/etc/authd.pass' -> '/var/ossec/data/etc/authd.pass'
'/wazuh-config-mount/etc/ossec.conf' -> '/var/ossec/data/etc/ossec.conf'
'/wazuh-config-mount/etc/rules/local_rules.xml' -> '/var/ossec/data/etc/rules/local_rules.xml'
'/wazuh-config-mount/etc/shared/default/agent.conf' -> '/var/ossec/data/etc/shared/default/agent.conf'
Performing Wazuh API port and credentials setup
### Wazuh API Configuration ###
Using 55000 port.
Adding password for user ndev-wazuh-manager.
Configuration changed.
Restarting API.
### [Configuration changed] ###
sed: cannot rename /etc/filebeat/sedJyZzQl: Device or resource busy
*** Running /etc/my_init.d/00_regen_ssh_host_keys.sh...
*** Running /etc/my_init.d/10_syslog-ng.init...
Sep 13 20:27:19 wazuh-manager-3-9-0 syslog-ng[60]: syslog-ng starting up; version='3.13.2'
*** Booting runit daemon...
*** Runit started as PID 67
WAZUH-API is already running.
WazuhAPI 2019-09-13 20:21:43 ndev-wazuh-manager: [::ffff:10.6.21.70] GET /agents/?limit=500&offset=0 - 200 - error: '0'.
WazuhAPI 2019-09-13 20:21:44 ndev-wazuh-manager: [::ffff:100.102.185.128] GET /version? - 200 - error: '0'.
WazuhAPI 2019-09-13 20:21:45 ndev-wazuh-manager: [::ffff:10.6.21.70] GET /cluster/status? - 200 - error: '0'.
WazuhAPI 2019-09-13 20:21:45 ndev-wazuh-manager: [::ffff:100.114.66.128] GET /cluster/node? - 200 - error: '0'.
WazuhAPI 2019-09-13 20:21:45 ndev-wazuh-manager: [::ffff:100.100.234.128] GET /version? - 200 - error: '0'.
WazuhAPI 2019-09-13 20:21:46 ndev-wazuh-manager: [::ffff:100.100.26.64] GET /agents/summary? - 200 - error: '0'.
WazuhAPI 2019-09-13 20:21:46 ndev-wazuh-manager: [::ffff:100.100.234.128] GET /rules/pci? - 200 - error: '0'.
WazuhAPI 2019-09-13 20:21:46 ndev-wazuh-manager: [::ffff:100.123.247.192] GET /rules/gdpr? - 200 - error: '0'.
WazuhAPI 2019-09-13 20:24:52 : ERROR: Wazuh manager v3.9.5 found. Wazuh manager v3.6.x expected. Exiting.
WazuhAPI 2019-09-13 20:24:53 : ERROR: Wazuh manager v3.9.5 found. Wazuh manager v3.6.x expected. Exiting.
Sep 13 20:27:19 wazuh-manager-3-9-0 cron[73]: (CRON) INFO (pidfile fd = 3)
Sep 13 20:27:19 wazuh-manager-3-9-0 cron[73]: (CRON) INFO (Running @reboot jobs)
2019-09-13T20:27:19.460Z INFO instance/beat.go:611 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2019-09-13T20:27:19.465Z INFO instance/beat.go:618 Beat UUID: ecf3180a-9430-4918-a210-6729a6b1a191
2019-09-13T20:27:19.465Z INFO [beat] instance/beat.go:931 Beat info {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "ecf3180a-9430-4918-a210-6729a6b1a191"}}}
2019-09-13T20:27:19.465Z INFO [beat] instance/beat.go:940 Build info {"system_info": {"build": {"commit": "0ffbeab5a52fa93586e4178becf1252e6a837028", "libbeat": "6.8.2", "time": "2019-07-24T14:24:45.000Z", "version": "6.8.2"}}}
2019-09-13T20:27:19.465Z INFO [beat] instance/beat.go:943 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":16,"version":"go1.10.8"}}}
2019-09-13T20:27:19.466Z INFO [beat] instance/beat.go:947 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-09-13T15:48:26Z","containerized":true,"name":"wazuh-manager-3-9-0","ip":["127.0.0.1/8","::1/128","100.107.198.212/32","fe80::10ac:caff:fed6:59e7/64"],"kernel_version":"4.15.0-1044-aws","mac":["12:ac:ca:d6:59:e7"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"18.04.3 LTS (Bionic Beaver)","major":18,"minor":4,"patch":3,"codename":"bionic"},"timezone":"UTC","timezone_offset_sec":0}}}
2019-09-13T20:27:19.466Z INFO [beat] instance/beat.go:976 Process info {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 102, "ppid": 97, "seccomp": {"mode":"disabled","no_new_privs":false}, "start_time": "2019-09-13T20:27:18.919Z"}}}
2019-09-13T20:27:19.466Z INFO instance/beat.go:280 Setup Beat: filebeat; Version: 6.8.2
2019-09-13T20:27:19.466Z INFO [publisher] pipeline/module.go:110 Beat name: wazuh-manager-3-9-0
Config OK
tail: cannot open '/var/log/filebeat/filebeat' for reading: No such file or directory
tail: no files remaining
2019/09/13 20:27:19 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:27:19 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
ossec-analysisd: Configuration error. Exiting
2019/09/13 20:26:48 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:26:48 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:26:52 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:26:52 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:27:02 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:27:02 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:27:12 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:27:12 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:27:19 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:27:19 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
WazuhAPI 2019-09-13 20:27:19 : ERROR: Wazuh manager v3.9.5 found. Wazuh manager v3.6.x expected. Exiting.
2019-09-13T20:27:20.565Z INFO instance/beat.go:611 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2019-09-13T20:27:20.565Z INFO instance/beat.go:618 Beat UUID: ecf3180a-9430-4918-a210-6729a6b1a191
2019-09-13T20:27:20.565Z INFO [beat] instance/beat.go:931 Beat info {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "ecf3180a-9430-4918-a210-6729a6b1a191"}}}
2019-09-13T20:27:20.565Z INFO [beat] instance/beat.go:940 Build info {"system_info": {"build": {"commit": "0ffbeab5a52fa93586e4178becf1252e6a837028", "libbeat": "6.8.2", "time": "2019-07-24T14:24:45.000Z", "version": "6.8.2"}}}
2019-09-13T20:27:20.565Z INFO [beat] instance/beat.go:943 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":16,"version":"go1.10.8"}}}
2019-09-13T20:27:20.566Z INFO [beat] instance/beat.go:947 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-09-13T15:48:26Z","containerized":true,"name":"wazuh-manager-3-9-0","ip":["127.0.0.1/8","::1/128","100.107.198.212/32","fe80::10ac:caff:fed6:59e7/64"],"kernel_version":"4.15.0-1044-aws","mac":["12:ac:ca:d6:59:e7"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"18.04.3 LTS (Bionic Beaver)","major":18,"minor":4,"patch":3,"codename":"bionic"},"timezone":"UTC","timezone_offset_sec":0}}}
2019-09-13T20:27:20.566Z INFO [beat] instance/beat.go:976 Process info {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 159, "ppid": 158, "seccomp": {"mode":"disabled","no_new_privs":false}, "start_time": "2019-09-13T20:27:20.099Z"}}}
2019-09-13T20:27:20.566Z INFO instance/beat.go:280 Setup Beat: filebeat; Version: 6.8.2
2019-09-13T20:27:20.567Z INFO [publisher] pipeline/module.go:110 Beat name: wazuh-manager-3-9-0
Config OK
/usr/share/filebeat/bin/filebeat-god already running.
2019-09-13T20:27:19.554Z INFO [publisher] pipeline/module.go:110 Beat name: wazuh-manager-3-9-0
2019-09-13T20:27:19.554Z INFO instance/beat.go:402 filebeat start running.
2019-09-13T20:27:19.554Z INFO registrar/registrar.go:97 No registry file found under: /var/lib/filebeat/registry. Creating a new registry file.
2019-09-13T20:27:19.560Z INFO registrar/registrar.go:134 Loading registrar data from /var/lib/filebeat/registry
2019-09-13T20:27:19.560Z INFO registrar/registrar.go:141 States Loaded from registrar: 0
2019-09-13T20:27:19.560Z WARN beater/filebeat.go:367 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2019-09-13T20:27:19.560Z INFO crawler/crawler.go:72 Loading Inputs: 1
2019-09-13T20:27:19.561Z INFO log/input.go:148 Configured paths: [/var/ossec/logs/alerts/alerts.json]
2019-09-13T20:27:19.561Z INFO input/input.go:114 Starting input of type: log; ID: 13571056894027297000
2019-09-13T20:27:19.561Z INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1
2019/09/13 20:28:22 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:28:22 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:28:32 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:28:32 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:28:42 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:28:42 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:28:52 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:28:52 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:28:58 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:28:58 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:29:02 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:29:02 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:29:08 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:29:08 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:29:12 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:29:12 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:29:18 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:29:18 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:29:22 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:29:22 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:29:32 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:29:32 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
2019/09/13 20:29:42 ossec-analysisd: ERROR: Invalid decoder name: 'syscheck_integrity_changed_2nd'.
2019/09/13 20:29:42 ossec-analysisd: CRITICAL: (1220): Error loading the rules: 'ruleset/rules/0015-ossec_rules.xml'.
What is the correct upgrade path for a Wazuh manager 3.6.1 deployed in Kubernetes?