yorokobicom / agent Goto Github PK
View Code? Open in Web Editor NEWYorokobi Agent
Home Page: https://www.yorokobi.com
License: ISC License
agent's People
Contributors
Stargazers
Watchers
agent's Issues
Remove password request after enter license
Authentication is performed entering only the license.
Remove asking for Yorokobi account password here which is not needed and causes confusion to users.
Enter your license Key below.
License key: xxxxxx
Now enter your account password.
Password:
Please wait, attempting to authenticate your license...
Success. Your license key worked fine :-)
Show more accurate failure message when `yorokobi backup` and `yorokobi unregister` fails
These two commands fails if the agent is in the middle of a backup, however the error message is not stating why. It should shows why.
For instance,
For yorokobi unregister, three different message types:
Agent was successfully unregisteredAgent isn't registered yetAgent can't be unregistered in the middle of a backup
Should register command be removed in favor of setup?
It looks to me the yorokobi register now is deprecated in favor of the yorokobi setup instead.
Should it be removed then?
Make use of logger and log all actions
The logger is implemented but for now it isn't logging any kind of information. The entire code should be reviewed to log useful information.
socket.gaierror: [Errno -2] Name does not resolve
When I try to setup the agent and enter the license key I get the following error.
Enter your license Key below.
License key: sdsd
Please wait, attempting to authenticate your license...
Traceback (most recent call last):
File "/usr/local/bin/yorokobi", line 11, in <module>
load_entry_point('yorokobi==0.1.0', 'console_scripts', 'yorokobi')()
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/yorokobi-0.1.0-py3.7.egg/yorokobi/cli.py", line 148, in setup_agent
File "/usr/local/lib/python3.7/site-packages/yorokobi-0.1.0-py3.7.egg/yorokobi/agent.py", line 417, in configure_agent
File "/usr/local/lib/python3.7/site-packages/yorokobi-0.1.0-py3.7.egg/yorokobi/license.py", line 76, in identify_agent
File "/usr/local/lib/python3.7/site-packages/yorokobi-0.1.0-py3.7.egg/yorokobi/license.py", line 35, in register_agent
socket.gaierror: [Errno -2] Name does not resolve
Agent not always sending backups to Yorokobi
The agent that is not always sending the backups to Yorokobi.
These can be reproduced using the databases from the test server.
When the agent is configured using postgres database it works perfectly. The result gzipped file containing the dump is less than 10 kb.
However, when the agent is configured using shakespeare database it gets stuck here without any trace. The compressed file here is around 5 mb.
Could it be related that tarfile.open default is buffsize=10240 bytes ?
bufsize specifies the blocksize and defaults to 20 * 512 bytes. Use this variant in combination with e.g. sys.stdin, a socket file object or a tape device. However, such a TarFile object is limited in that it does not allow random access, see Examples.
From: https://docs.python.org/3/library/tarfile.html#tarfile.open
Yorokobi daemon not starting
The yorokobi daemon is not starting when installing the app via snap.
martin@martin-VirtualBox:~/Documents$ snap install yorokobi --edge
martin@martin-VirtualBox:~/Documents$ sudo snap start yorokobi.daemon
[sudo] password for martin:
Started.
See the journalctl log below.
ene 12 17:29:19 martin-VirtualBox sudo[5469]: martin : TTY=pts/4 ; PWD=/home/martin/Documents ; USER=root ; COMMAND=/usr/bin/snap start yorokobi.daemon
ene 12 17:29:19 martin-VirtualBox sudo[5469]: pam_unix(sudo:session): session opened for user root by (uid=0)
ene 12 17:29:19 martin-VirtualBox systemd[1]: Started Service for snap application yorokobi.daemon.
-- Subject: Unit snap.yorokobi.daemon.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit snap.yorokobi.daemon.service has finished starting up.
--
-- The start-up result is done.
ene 12 17:29:19 martin-VirtualBox sudo[5469]: pam_unix(sudo:session): session closed for user root
ene 12 17:29:20 martin-VirtualBox audit[5712]: AVC apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5712/mounts" pid=5712 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:20 martin-VirtualBox kernel: kauditd_printk_skb: 2 callbacks suppressed
ene 12 17:29:20 martin-VirtualBox kernel: audit: type=1400 audit(1547324960.520:28499): apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5712/mounts" pid=5712 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:20 martin-VirtualBox audit[5712]: AVC apparmor="DENIED" operation="create" profile="snap.yorokobi.daemon" pid=5712 comm="python3" family="inet6" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
ene 12 17:29:20 martin-VirtualBox kernel: audit: type=1400 audit(1547324960.604:28500): apparmor="DENIED" operation="create" profile="snap.yorokobi.daemon" pid=5712 comm="python3" family="inet6" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
ene 12 17:29:20 martin-VirtualBox audit[5712]: AVC apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5712/mounts" pid=5712 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:20 martin-VirtualBox kernel: audit: type=1400 audit(1547324960.836:28501): apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5712/mounts" pid=5712 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:20 martin-VirtualBox audit[5712]: SECCOMP auid=4294967295 uid=0 gid=0 ses=4294967295 pid=5712 comm="python3" exe="/snap/yorokobi/65/usr/bin/python3.5" sig=0 arch=c000003e syscall=41 compat=0 ip=0x7f5574fd65a7 code=0x50000
ene 12 17:29:20 martin-VirtualBox yorokobi.daemon[5712]: Operation not permitted (src/tcp_address.cpp:208)
ene 12 17:29:20 martin-VirtualBox kernel: audit: type=1326 audit(1547324960.940:28502): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=5712 comm="python3" exe="/snap/yorokobi/65/usr/bin/python3.5" sig=0 arch=c000003e syscall=41 compat=0 ip=0x7f5574fd65a7 code=0x50000
ene 12 17:29:21 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Main process exited, code=dumped, status=6/ABRT
ene 12 17:29:21 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Unit entered failed state.
ene 12 17:29:21 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Failed with result 'core-dump'.
ene 12 17:29:21 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Service hold-off time over, scheduling restart.
ene 12 17:29:21 martin-VirtualBox systemd[1]: Stopped Service for snap application yorokobi.daemon.
-- Subject: Unit snap.yorokobi.daemon.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit snap.yorokobi.daemon.service has finished shutting down.
ene 12 17:29:21 martin-VirtualBox systemd[1]: Started Service for snap application yorokobi.daemon.
-- Subject: Unit snap.yorokobi.daemon.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit snap.yorokobi.daemon.service has finished starting up.
--
-- The start-up result is done.
ene 12 17:29:21 martin-VirtualBox audit[5731]: AVC apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5731/mounts" pid=5731 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:21 martin-VirtualBox kernel: audit: type=1400 audit(1547324961.888:28503): apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5731/mounts" pid=5731 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:21 martin-VirtualBox audit[5731]: AVC apparmor="DENIED" operation="create" profile="snap.yorokobi.daemon" pid=5731 comm="python3" family="inet6" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
ene 12 17:29:21 martin-VirtualBox kernel: audit: type=1400 audit(1547324961.968:28504): apparmor="DENIED" operation="create" profile="snap.yorokobi.daemon" pid=5731 comm="python3" family="inet6" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
ene 12 17:29:22 martin-VirtualBox audit[5731]: AVC apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5731/mounts" pid=5731 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:22 martin-VirtualBox kernel: audit: type=1400 audit(1547324962.256:28505): apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5731/mounts" pid=5731 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:22 martin-VirtualBox audit[5731]: SECCOMP auid=4294967295 uid=0 gid=0 ses=4294967295 pid=5731 comm="python3" exe="/snap/yorokobi/65/usr/bin/python3.5" sig=0 arch=c000003e syscall=41 compat=0 ip=0x7f833692a5a7 code=0x50000
ene 12 17:29:22 martin-VirtualBox yorokobi.daemon[5731]: Operation not permitted (src/tcp_address.cpp:208)
ene 12 17:29:22 martin-VirtualBox kernel: audit: type=1326 audit(1547324962.316:28506): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=5731 comm="python3" exe="/snap/yorokobi/65/usr/bin/python3.5" sig=0 arch=c000003e syscall=41 compat=0 ip=0x7f833692a5a7 code=0x50000
ene 12 17:29:22 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Main process exited, code=dumped, status=6/ABRT
ene 12 17:29:22 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Unit entered failed state.
ene 12 17:29:22 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Failed with result 'core-dump'.
ene 12 17:29:22 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Service hold-off time over, scheduling restart.
ene 12 17:29:22 martin-VirtualBox systemd[1]: Stopped Service for snap application yorokobi.daemon.
-- Subject: Unit snap.yorokobi.daemon.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit snap.yorokobi.daemon.service has finished shutting down.
ene 12 17:29:22 martin-VirtualBox systemd[1]: Started Service for snap application yorokobi.daemon.
-- Subject: Unit snap.yorokobi.daemon.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit snap.yorokobi.daemon.service has finished starting up.
--
-- The start-up result is done.
ene 12 17:29:23 martin-VirtualBox audit[5749]: AVC apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5749/mounts" pid=5749 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:23 martin-VirtualBox kernel: audit: type=1400 audit(1547324963.316:28507): apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5749/mounts" pid=5749 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:23 martin-VirtualBox audit[5749]: AVC apparmor="DENIED" operation="create" profile="snap.yorokobi.daemon" pid=5749 comm="python3" family="inet6" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
ene 12 17:29:23 martin-VirtualBox kernel: audit: type=1400 audit(1547324963.404:28508): apparmor="DENIED" operation="create" profile="snap.yorokobi.daemon" pid=5749 comm="python3" family="inet6" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
ene 12 17:29:23 martin-VirtualBox audit[5749]: AVC apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5749/mounts" pid=5749 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:23 martin-VirtualBox audit[5749]: SECCOMP auid=4294967295 uid=0 gid=0 ses=4294967295 pid=5749 comm="python3" exe="/snap/yorokobi/65/usr/bin/python3.5" sig=0 arch=c000003e syscall=41 compat=0 ip=0x7f749106c5a7 code=0x50000
ene 12 17:29:23 martin-VirtualBox yorokobi.daemon[5749]: Operation not permitted (src/tcp_address.cpp:208)
ene 12 17:29:23 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Main process exited, code=dumped, status=6/ABRT
ene 12 17:29:23 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Unit entered failed state.
ene 12 17:29:23 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Failed with result 'core-dump'.
ene 12 17:29:23 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Service hold-off time over, scheduling restart.
ene 12 17:29:23 martin-VirtualBox systemd[1]: Stopped Service for snap application yorokobi.daemon.
-- Subject: Unit snap.yorokobi.daemon.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit snap.yorokobi.daemon.service has finished shutting down.
ene 12 17:29:23 martin-VirtualBox systemd[1]: Started Service for snap application yorokobi.daemon.
-- Subject: Unit snap.yorokobi.daemon.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit snap.yorokobi.daemon.service has finished starting up.
--
-- The start-up result is done.
ene 12 17:29:24 martin-VirtualBox audit[5768]: AVC apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5768/mounts" pid=5768 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:24 martin-VirtualBox audit[5768]: AVC apparmor="DENIED" operation="create" profile="snap.yorokobi.daemon" pid=5768 comm="python3" family="inet6" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
ene 12 17:29:25 martin-VirtualBox audit[5768]: AVC apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5768/mounts" pid=5768 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:25 martin-VirtualBox audit[5768]: SECCOMP auid=4294967295 uid=0 gid=0 ses=4294967295 pid=5768 comm="python3" exe="/snap/yorokobi/65/usr/bin/python3.5" sig=0 arch=c000003e syscall=41 compat=0 ip=0x7f3abef875a7 code=0x50000
ene 12 17:29:25 martin-VirtualBox yorokobi.daemon[5768]: Operation not permitted (src/tcp_address.cpp:208)
ene 12 17:29:25 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Main process exited, code=dumped, status=6/ABRT
ene 12 17:29:25 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Unit entered failed state.
ene 12 17:29:25 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Failed with result 'core-dump'.
ene 12 17:29:25 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Service hold-off time over, scheduling restart.
ene 12 17:29:25 martin-VirtualBox systemd[1]: Stopped Service for snap application yorokobi.daemon.
-- Subject: Unit snap.yorokobi.daemon.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit snap.yorokobi.daemon.service has finished shutting down.
ene 12 17:29:25 martin-VirtualBox systemd[1]: Started Service for snap application yorokobi.daemon.
-- Subject: Unit snap.yorokobi.daemon.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit snap.yorokobi.daemon.service has finished starting up.
--
-- The start-up result is done.
ene 12 17:29:26 martin-VirtualBox audit[5786]: AVC apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5786/mounts" pid=5786 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:26 martin-VirtualBox kernel: kauditd_printk_skb: 6 callbacks suppressed
ene 12 17:29:26 martin-VirtualBox kernel: audit: type=1400 audit(1547324966.372:28515): apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5786/mounts" pid=5786 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:26 martin-VirtualBox audit[5786]: AVC apparmor="DENIED" operation="create" profile="snap.yorokobi.daemon" pid=5786 comm="python3" family="inet6" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
ene 12 17:29:26 martin-VirtualBox kernel: audit: type=1400 audit(1547324966.484:28516): apparmor="DENIED" operation="create" profile="snap.yorokobi.daemon" pid=5786 comm="python3" family="inet6" sock_type="stream" protocol=0 requested_mask="create" denied_mask="create"
ene 12 17:29:26 martin-VirtualBox audit[5786]: AVC apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5786/mounts" pid=5786 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:26 martin-VirtualBox kernel: audit: type=1400 audit(1547324966.768:28517): apparmor="DENIED" operation="open" profile="snap.yorokobi.daemon" name="/proc/5786/mounts" pid=5786 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
ene 12 17:29:26 martin-VirtualBox audit[5786]: SECCOMP auid=4294967295 uid=0 gid=0 ses=4294967295 pid=5786 comm="python3" exe="/snap/yorokobi/65/usr/bin/python3.5" sig=0 arch=c000003e syscall=41 compat=0 ip=0x7fe914b0b5a7 code=0x50000
ene 12 17:29:26 martin-VirtualBox yorokobi.daemon[5786]: Operation not permitted (src/tcp_address.cpp:208)
ene 12 17:29:26 martin-VirtualBox kernel: audit: type=1326 audit(1547324966.828:28518): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=5786 comm="python3" exe="/snap/yorokobi/65/usr/bin/python3.5" sig=0 arch=c000003e syscall=41 compat=0 ip=0x7fe914b0b5a7 code=0x50000
ene 12 17:29:26 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Main process exited, code=dumped, status=6/ABRT
ene 12 17:29:26 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Unit entered failed state.
ene 12 17:29:26 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Failed with result 'core-dump'.
ene 12 17:29:27 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Service hold-off time over, scheduling restart.
ene 12 17:29:27 martin-VirtualBox systemd[1]: Stopped Service for snap application yorokobi.daemon.
-- Subject: Unit snap.yorokobi.daemon.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit snap.yorokobi.daemon.service has finished shutting down.
ene 12 17:29:27 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Start request repeated too quickly.
ene 12 17:29:27 martin-VirtualBox systemd[1]: Failed to start Service for snap application yorokobi.daemon.
-- Subject: Unit snap.yorokobi.daemon.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit snap.yorokobi.daemon.service has failed.
--
-- The result is failed.
ene 12 17:29:27 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Unit entered failed state.
ene 12 17:29:27 martin-VirtualBox systemd[1]: snap.yorokobi.daemon.service: Failed with result 'start-limit-hit'.
Yorokobi ASCII logo is broken
The Yorokobi ASCII logo is appearing broken when running yorokobi setup.
$ docker exec -it yorokobi yorokobi setup
__ __
/\ \ /\ \ __
__ __ ___ _ __ ___ \ \ \/'\ ___ \ \ \____ /\_/\ \/\ \ / __`\ /\`'__\ / __`\ \ \ , < / __`\ \ \ '__`\ \/\ \ \ \_\ \ /\ \L\ \ \ \/ /\ \L\ \ \ \ \`\ /\ \L\ \ \ \ \L\ \ \ \ \/`____ \ \ \____/ \ \_\ \ \____/ \ \_\ \_\ \____/ \ \_,__/ \ \_ `/___/> \ \/___/ \/_/ \/___/ \/_/\/_/ \/___/ \/___/ \/_/
/\___/
\/__/
Welcome to Yorokobi. For more information please visit www.yorokobi.com.
Start agent service on boot
Set up the daemon to automatically start on boot after the package is installed.
Backup sending a new backup request instead of mark it as complete
I noticed the agent is sending a new backup instead of mark it as complete.
The endpoint to mark the backup as complete should be like this, replacing :id with the backup id.
POST https://api.yorokobi.com/v1/backups/:id/complete
See the log below.
2019-01-20 14:51:01,961 - yorokobi - INFO - Agent has successfull started
2019-01-20 14:53:16,858 - yorokobi - INFO - The agent received a {'type': <Request.BACKUP_NOW: 4>} request
2019-01-20 14:53:16,858 - yorokobi - INFO - A manual backup is requested
2019-01-20 14:53:16,858 - yorokobi - INFO - Sending a backup request to Yorokobi server
2019-01-20 14:53:16,859 - yorokobi - INFO - Request data is {'agent_id': 'Ya5aacF8MvyeQn7982CT', 'hostname': 'martin-VirtualBox', 'ip_address': '127.0.1.1'}
2019-01-20 14:53:17,976 - yorokobi - INFO - Response status is 200
2019-01-20 14:53:17,980 - yorokobi - INFO - Response data is:
2019-01-20 14:53:17,980 - yorokobi - INFO - {'id': 'jXWovfV1TOBLlY2gxO_', 'hostname': 'backup.yorokobi.com', 'port': 56626, 'token': 'jXWovfV1TOBLlY2gxO_', 'state': 'initial'}
2019-01-20 14:53:17,980 - yorokobi - INFO - Got a positive response from server; backup was accepted
2019-01-20 14:53:17,980 - yorokobi - INFO - Reading response to retrieve the backup ID, the remofile port and the remofile token
2019-01-20 14:53:17,980 - yorokobi - INFO - Backup ID is jXWovfV1TOBLlY2gxO_
2019-01-20 14:53:17,981 - yorokobi - INFO - Remofile port is 56626
2019-01-20 14:53:17,981 - yorokobi - INFO - Remofile token is jXWovfV1TOBLlY2gxO_
2019-01-20 14:53:17,981 - yorokobi - INFO - Starting a backup in external thread
2019-01-20 14:53:17,982 - yorokobi - INFO - Starting backup now
2019-01-20 14:53:17,982 - yorokobi - INFO - The backup request was accepted
2019-01-20 14:53:17,982 - yorokobi - INFO - Remofile port: 56626
2019-01-20 14:53:17,991 - yorokobi - INFO - Remofile token: jXWovfV1TOBLlY2gxO_
2019-01-20 14:53:17,993 - yorokobi - INFO - ['postgres']
2019-01-20 14:53:18,471 - yorokobi - INFO - Sending the tarball
2019-01-20 14:53:18,472 - yorokobi - INFO - Sending '/tmp/tmpkw7n84_y/2019.01.20.14.01.18.tar.gz' to backup.yorokobi.com using Remofile protocol with port 56626 and token 'jXWovfV1TOBLlY2gxO_'...
2019-01-20 14:53:30,333 - yorokobi - INFO - Backup successfully finished
2019-01-20 14:53:30,338 - yorokobi - INFO - The ongoing backup is detected as terminated; terminating the backup
2019-01-20 14:53:30,338 - yorokobi - INFO - Sending a terminate backup request to Yorokobi server
2019-01-20 14:53:30,338 - yorokobi - INFO - Request data is {'agent_id': 'Ya5aacF8MvyeQn7982CT', 'hostname': 'martin-VirtualBox', 'ip_address': '127.0.1.1'}
2019-01-20 14:53:31,129 - yorokobi - INFO - Response status is 200
2019-01-20 14:53:31,130 - yorokobi - INFO - Response data is:
2019-01-20 14:53:31,131 - yorokobi - INFO - {'id': 'uwFMdjaAekjlYjTmTldH', 'hostname': 'backup.yorokobi.com', 'port': 39657, 'token': 'uwFMdjaAekjlYjTmTldH', 'state': 'initial'}
Add Yorokobi welcome screen
Implement the Yorokobi ASCII art and welcome text.
root@server:/# yorokobi
__ __
/\ \ /\ \ __
__ __ ___ _ __ ___ \ \ \/'\ ___ \ \ \____ /\_\
/\ \/\ \ / __`\ /\`'__\ / __`\ \ \ , < / __`\ \ \ '__`\ \/\ \
\ \ \_\ \ /\ \L\ \\ \ \/ /\ \L\ \ \ \ \\`\ /\ \L\ \ \ \ \L\ \ \ \ \
\/`____ \ \ \____/ \ \_\ \ \____/ \ \_\ \_\\ \____/ \ \_,__/ \ \_\
`/___/> \ \/___/ \/_/ \/___/ \/_/\/_/ \/___/ \/___/ \/_/
/\___/
\/__/
Welcome to Yorokobi. For more information please visit www.yorokobi.com.
Enter your License Key below:
License Key: SKSICNALASCMASKCKASMCJDFVJFJFJV
Support Python 3.5 in remofile
TLDR; Add this commit to remofile and release it in PyPi.
Snapcraft python plugin uses python 3.5 to build the snap and this is failing the process because remofile requires python 3.7 and so the build can not find a version of remofile.
To find a solution to this I forked the remofile and added support for python 3.5: See: https://github.com/yorokobicom/remofile/commit/e6e0408dee2767f49574cb02d3450da3c29ffdea
Then, I added this repo as a 'part' of the snapcraft recipe to build it there. See: https://github.com/yorokobicom/agent/blob/master/snap/snapcraft.yaml#L21-L28
However this is causing a conflict and failing the build in s390x, ppc64el, arm64 and armhf architectures. It does compile normally in amd64 and i386. See error at the end.
I'd like to try if installing remofile via Pip (adding it in the python-packages list in the snapcraft.yml) instead of trying to build it as a part would fix this problem.
/build/yorokobi/parts/yorokobi/install/usr/bin/python3 setup.py --no-user-cfg install --single-version-externally-managed --user --record install.txt
Failed to stage: Parts 'remofile' and 'yorokobi' have the following files, but with different contents:
lib/python3.5/site-packages/zmq/backend/cython/_device.cpython-35m-s390x-linux-gnu.so
lib/python3.5/site-packages/zmq/backend/cython/_poll.cpython-35m-s390x-linux-gnu.so
lib/python3.5/site-packages/zmq/backend/cython/_version.cpython-35m-s390x-linux-gnu.so
lib/python3.5/site-packages/zmq/backend/cython/constants.cpython-35m-s390x-linux-gnu.so
lib/python3.5/site-packages/zmq/backend/cython/context.cpython-35m-s390x-linux-gnu.so
lib/python3.5/site-packages/zmq/backend/cython/error.cpython-35m-s390x-linux-gnu.so
lib/python3.5/site-packages/zmq/backend/cython/message.cpython-35m-s390x-linux-gnu.so
lib/python3.5/site-packages/zmq/backend/cython/socket.cpython-35m-s390x-linux-gnu.so
lib/python3.5/site-packages/zmq/backend/cython/utils.cpython-35m-s390x-linux-gnu.so
lib/python3.5/site-packages/zmq/devices/monitoredqueue.cpython-35m-s390x-linux-gnu.so
lib/python3.5/site-packages/zmq/utils/compiler.json
lib/python3.5/site-packages/zmq/utils/config.json
Snapcraft offers some capabilities to solve this by use of the following keywords:
- `filesets`
- `stage`
- `snap`
- `organize`
To learn more about these part keywords, run `snapcraft help plugins`.
Build failed
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/lpbuildd/target/build_snap.py", line 231, in run
self.build()
File "/usr/lib/python2.7/dist-packages/lpbuildd/target/build_snap.py", line 220, in build
env=env)
File "/usr/lib/python2.7/dist-packages/lpbuildd/target/build_snap.py", line 75, in run_build_command
return self.backend.run(args, env=full_env, **kwargs)
File "/usr/lib/python2.7/dist-packages/lpbuildd/target/lxd.py", line 460, in run
subprocess.check_call(cmd, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['lxc', 'exec', 'lp-xenial-s390x', '--env', 'LANG=C.UTF-8', '--env', 'SHELL=/bin/sh', '--env', 'SNAPCRAFT_BUILD_INFO=1', '--env', 'SNAPCRAFT_IMAGE_INFO={"build_url": "https://launchpad.net/~build.snapcraft.io/+snap/9bc23507e335fb54c4484719e01a72a0-xenial/+build/430299"}', '--env', 'SNAPCRAFT_BUILD_ENVIRONMENT=host', '--env', 'http_proxy=http://10.10.10.1:8222/', '--env', 'https_proxy=http://10.10.10.1:8222/', '--env', 'GIT_PROXY_COMMAND=/usr/local/bin/snap-git-proxy', '--', '/bin/sh', '-c', 'cd /build/yorokobi && linux64 snapcraft']' returned non-zero exit status 2
Revoking proxy token...
RUN: /usr/share/launchpad-buildd/slavebin/in-target scan-for-processes --backend=lxd --series=xenial --arch=s390x SNAPBUILD-430299
Scanning for processes to kill in build SNAPBUILD-430299
You can see the list of all builds and logs: https://build.snapcraft.io/user/yorokobicom/agent
I sent you an invite. You should be able to see all the builds there.
snap fails when calling pg_dump
When triggering a backup using:
$ yorokobi backup
The backup is not complete as the snap fails when trying triggering pg_dump around here: https://github.com/yorokobicom/agent/blob/master/yorokobi/backup.py#L46-L50
This is the error recorded yorokobi logs.
2019-02-28 18:23:47,720 - yorokobi - INFO - End Of File (EOF). Exception style platform.
<pexpect.pty_spawn.spawn object at 0x7f5b56c12da0>
command: /snap/yorokobi/209/usr/bin/pg_dump
args: ['/snap/yorokobi/209/usr/bin/pg_dump', '-Fc', '--host=localhost', '--port=5432', '--username=postgres', '--password', 'shakespeare']
buffer (last 100 chars): b''
before (last 100 chars): b'pg_dump line 22.\r\nBEGIN failed--compilation aborted at /snap/yorokobi/209/usr/bin/pg_dump line 22.\r\n'
after: <class 'pexpect.exceptions.EOF'>
match: None
match_index: None
exitstatus: 2
flag_eof: True
pid: 5341
child_fd: 20
closed: False
timeout: None
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
0: re.compile(b'Password')
Update config and log file paths
The current config and log files are saved at ~/yorokobi.conf and ~/yorokobi.log.
Make sure they are saved at /etc/yorokobi.conf and /var/log/yorokobi.log.
Backup not working
When I try to perform a backup running yorokobi-backup I get the following error.
martin@martin-VirtualBox:~/Repos/yorokobi-agent$ yorokobi-backup
The agent doesn't appear running; ensure the agent is started.
And the daemon returns the following error.
backup 112BxTr_vEPBgNNWYbnb is starting
backup with port 50712 and token 112BxTr_vEPBgNNWYbnb
backup-now: accepted
None
Exception in thread Thread-2:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.6/dist-packages/yorokobi-0.1.0-py3.6.egg/yorokobi/backup.py", line 63, in run
self.dump_databases(temporary_dir)
File "/usr/local/lib/python3.6/dist-packages/yorokobi-0.1.0-py3.6.egg/yorokobi/backup.py", line 98, in dump_databases
for database in selected_dbs:
TypeError: 'NoneType' object is not iterable
about to terminate the backup
200
{'id': 'FMFSndIMJjhvQlXAwDhm', 'hostname': 'backup.yorokobi.com', 'port': 50228, 'token': 'FMFSndIMJjhvQlXAwDhm', 'state': 'initial'}
{"id":"FMFSndIMJjhvQlXAwDhm","hostname":"backup.yorokobi.com","port":50228,"token":"FMFSndIMJjhvQlXAwDhm","state":"initial"}
Check for backup.state before sending backup
I just implemented a new waiting state in backups when remofile is effectively started. This state means that is ready and waiting to receive the backup.
Important: This requires remofile start command to return finished (see email I sent you regarding this) otherwise It won't work as it will keep waiting for this signal to change the state.
You can do a GET request to this endpoint including the backup id:
GET https://api.yorokobi.com/v1/backups/:id
It will return a JSON like this
{
"id": "ucQBK48NGZpurvM6b5Vy",
"hostname": "127.0.0.1",
"port": 85571,
"token": "ucQBK48NGZpurvM6b5Vy",
"state": "waiting"
}
You should perform a request to the backup before sending and if its state is not waiting you should try again after n seconds and abort after some attempts.
snap.yorokobi.daemon.service is failing to start
In the last edge release of the yorokobi snap, trying to start the service is returning an error.
sudo snap install yorokobi --edge
sudo snap start yorokobi.daemon
This is returning (in journalctl -xe):
feb 25 16:29:53 martin-VirtualBox yorokobi.daemon[26889]: Operation not permitted (src/ip_resolver.cpp:520)
Start yorokobi after installation
After installing the agent, set up daemon to start a boot and start the daemon execute yorokobi command to begin with authentication and configuration.
Implement unregister agent
How do you want that ?
yorokobi unregister
or
yorokobi-unregister
or else ?
Parse array of errors
Parse the array of errors given by the API and show it one per line.
Currently it displaying errors like this:
Enter your license Key below.
License key: sdsd
Please wait, attempting to authenticate your license...
The License key given does not exist in our records.
error message returned is: { "errors": [{ "type": "access_denied", "title": "HTTP Basic: Access denied." }] }
Retry? [Y/n]
Parse the array of errors and show the title one per line like this.
Enter your license Key below.
License key: sdsd
Please wait, attempting to authenticate your license...
The License key given does not exist in our records.
HTTP Basic: Access denied.
Retry? [Y/n]
Make sure this applies to all API requests in the agent.
Implement logs command
Following your email suggestion implement a yorokobi logs command.
Start daemon automatically after install
Start yorokobi daemon automatically after installation so users don't need to.
Missing tests
Write missing tests.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.