Coder Social home page Coder Social logo

lucascbeyeler / zmbackup Goto Github PK

View Code? Open in Web Editor NEW
143.0 31.0 73.0 422 KB

A reliable software written in Shell Script to help you in your daily task to backup and restore mails and accounts from Zimbra Open Source Email Platform. Project in deprecation process. Please help me with Waddles.

License: GNU General Public License v3.0

Shell 100.00%
zimbra backup backup-solution zmbkpose zimbra-server backup-routine restore mailbox

zmbackup's People

Contributors

ananiasfilho avatar baconeusk avatar cavassin avatar coudot avatar lucascbeyeler avatar vdeville avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zmbackup's Issues

Error RestoreonAccount

The script for restore not found the archive in /backup/sessions.txt
My backup directory is in /opt/zimbra/backup
The message say SUCCESS, but isn't really.

_ $ zmbackup -r -ro full-20170417111450 [email protected] [email protected]
grep: /backup/sessions.txt: Arquivo ou diretório não encontrado
Restore mail process started at - Seg Abr 17 11:29:21 -03 2017

Restoring Session - [email protected]
Session [email protected] restored with SUCCESS

Restore mail process completed at - Seg Abr 17 11:29:21 -03 2017 _

CRITICAL - Restore backups hangs during the execution

Reported by: kharinva@****.com

When you try to execute the restore, the process hangs and nothing happens.

Motive: the functions create_temp was not called by none of the restore activities, and all these functions open the file $TEMPACCOUNT to know which account should be restored.

backup zimbra server configuration

ISSUE TYPE: Question

ENVIRONMENT VERSION: Zimbra

SUMMARY: Zimbra backup server configuration

Hi,

I am looking for procedure how to backup zimbra server configuration. Do you have ideas how to backup server configuration?

incremental backup issue

Hello, first of all thanks for the script.
My zimbra version - Release 8.7.11_GA_1854.RHEL7_64_20170531151956 RHEL7_64 FOSS edition.

I ran into strange thing - with total mailbox size 32gb increment backup lasts more than 3 days.

SESSION: inc-20170811020001 started on Fri Aug 11 02:00:03 EEST 2017
SESSION: inc-20170811020001 completed in Mon Aug 14 15:42:11 EEST 2017

At the same time full backup lasts half an hour.
Is it normal?

I can't restore backup file to mailbox

Hi,

I backup my mailboxes via zmbackup. My zmbackup version is 1.1.6

My backup folder is /mnt/zimbra1/.

But, when I want restore backup to mailbox I get error. It has [email protected] file in /mnt/zimbra1/full-20170809020001

When I use this command: zmbackup -r -m full-20170809020001 [email protected]
Answer: No backup found. Impossible to restore

What am I doing wrong?

parallel: Error: Command (--no-notice) starts with '-'. Is this a wrong option?

Hi,
I'm sorry but we're on ZCS-8.7.11 and I've some problems with this backup script.

zmbackup --full
parallel: Warning: YOU ARE USING --tollef. IF THINGS ARE ACTING WEIRD USE --gnu.
parallel: Warning: --tollef is obsolete and will be retired 20140222.
parallel: Warning: See: http://lists.gnu.org/archive/html/parallel/2013-02/msg00018.html
Backup session full-20171127111316 started on Mon Nov 27 11:13:16 CET 2017
parallel: Error: Command (--no-notice) starts with '-'. Is this a wrong option?
Backup session full-20171127111316 finished on Mon Nov 27 11:13:17 CET 2017
ls: cannot access /opt/zimbra/backup/full-20171127111316/*.ldiff: No such file or directory

What am I doing wrong?

My parallel version is:
parallel --version
WARNING: YOU ARE USING --tollef. IF THINGS ARE ACTING WEIRD USE --gnu.
GNU parallel 20130522

Thanks!!
Andrea

PID file

Hi,

It is great that you develop zimbra backup system. I have several recommendations:

  1. Add PID file creation while script is running
  2. Could you add option to transfer older files to remote location via FTP?

Store backups in AWS Glacier

Zmbackup only stores the files in a folder locally. The script should have a option for you import the backups directly to AWS Glacier, if the client wants.

New Feature: A parameter to check the status of Zmbackup in the server

ISSUE TYPE
  • Feature Idea
ENVIRONMENT VERSION
  • Zmbackup Version: Any
  • Zimbra Version: Any
  • Linux Distribution & Version: Any
SUMMARY

Create a new parameter for Zmbackup to check the status of all the files and services necessary for it work, like LDAP, Zmmailbox, backup directory access, SQL/TXT file access or if exist, and so goes on.

ldap_bkp not implemented

Hi lucas,
when i try to backup ldap i get an error message that ldap_bkp in line 516 is not implemented.
It looks like that the function is missing in the script.

Incremantal backup issue

@lucascbeyeler

Incremental backup take backup of any new user is created in zimbra. It does not take the backup of modified data of current user present at the time of Full backup.
Command executed zmbackup -i
Please help me with the solution.

Release Candidate Checklist

ISSUE TYPE
  • Checklist
ENVIRONMENT VERSION
  • Zmbackup Version: 1.2.0 Release Candidate
  • Zimbra Version: 8.7.11
  • Linux Distribution & Version: Ubuntu & CentOS
SUMMARY

Bellow there are all the bugs found during the tests and that will be needed to fix before the next release.

  • Zmbackup is deleting lock file if you run a second time - should keep until the first time is running;
  • install.sh don't remove the zmbackup.cron if --remove is used;
  • Zmbackup -r is not working if you don't inform the restore type;
  • The message "preserve backup storage" is jumping a line;
  • Asked by saschadd: include all the backup routines inside zmbackup.cron;
  • Incremental backup should be doing a full backup instead when there is no full backup;
  • Migration should only run if the user change the option inside zmbackup.conf;
  • Database is locked for some reason;
  • Incremental is not running Full backup at first;
  • Mailbox don't count correctly how much accounts are backed up when try to send the e-mail;
  • Reported by Germán I. Paul: Zmbackup does not execute the backup routine inside CentOS;
  • Migrate should run only if Zmbackup is not running;
  • Zmbackup wasn't removing all the files after a routine;
  • Migration is broken;
  • Zmbackup computing wrong size in SQLITE3.

max parallel processes

Hi,

it seems that MAX_PARALLEL_PROCESS works wrong. If I set this property to 2 it starts 4 parallel processes.

Documentation Checklist

ISSUE TYPE
  • Checklist
ENVIRONMENT VERSION
  • Zmbackup Version: 1.2.0 Release Candidate
  • Zimbra Version: 8.7.11
  • Linux Distribution & Version: Ubuntu & CentOS
SUMMARY

Bellow there are the pages need to be wrote as part of the documentation before the release of Zmbackup 1.2.0

  • Introduction;
  • Installation;
  • Inside Zmbackup;
  • Backup;
  • Restore;
  • Trash management;
  • Listing data;
  • Configuration file.

Support to Zimbra 8.8

ISSUE TYPE
  • Feature Idea
ENVIRONMENT VERSION
  • Zmbackup Version: 1.2 RC
  • Zimbra Version: 8.8
  • Linux Distribution & Version: Any Distribution
SUMMARY

We need to validate the script in the new release of Zimbra if it is working with no problem.

Log level to turn Zmbackup less verbose, if needed

ISSUE TYPE
  • Feature Idea
ENVIRONMENT VERSION
  • Zmbackup Version:1.2.0
  • Zimbra Version: Any
  • Linux Distribution & Version: Any
SUMMARY

Create log level for each message to turn Zmbackup less verbose if needed

Mailbox dump not working

I setup the new 1.2 version.
zmbackup -f backup the ldap (ldiff) but creating an empty tgz mailbox dump. It does not wrote any error log, just not working. Any help about this.?
Every login credentials is valid.

edit cron job

Hi,

It seems that house keep - zmbhousekeep could not be started from cron, bebause in /etc/cron.d/zmbackup.cron house keep script is named as zmhousekeep. Could you replace zmhousekeep with zmbhousekeep

Network backup

Some peoples find annoying the fact that you can only backup if the script is running in the same server that should be backed up. Modify the script to don't have this dependency.

Zmbackup -d crash if SQLITE3 options is enabled

ISSUE TYPE
  • Bug Report
ENVIRONMENT VERSION
  • Zmbackup Version: 1.2.0
  • Zimbra Version: any
  • Linux Distribution & Version: any
SUMMARY

If SQLITE3 is active, if you try to remove an backup a lot of warn messages is showed

STEPS TO REPRODUCE
zmbackup -d full-20180407132949
ACTUAL RESULTS
Removing session full-20180407132949 - please wait.
Backup session full-20180407132949 removed.
du: cannot access `/opt/zimbra/backup/full-20180407132949': No such file or directory
ls: cannot access /opt/zimbra/backup/full-20180407132949/*.ldiff: No such file or directory
/usr/local/lib/zmbackup/bash/NotifyAction.sh: line 51: $MESSAGE: ambiguous redirect
/usr/local/lib/zmbackup/bash/NotifyAction.sh: line 52: $MESSAGE: ambiguous redirect
/usr/local/lib/zmbackup/bash/NotifyAction.sh: line 53: $MESSAGE: ambiguous redirect
/usr/local/lib/zmbackup/bash/NotifyAction.sh: line 54: $MESSAGE: ambiguous redirect
/usr/local/lib/zmbackup/bash/NotifyAction.sh: line 55: $MESSAGE: ambiguous redirect
/usr/local/lib/zmbackup/bash/NotifyAction.sh: line 56: $MESSAGE: ambiguous redirect
/usr/local/lib/zmbackup/bash/NotifyAction.sh: line 57: $MESSAGE: ambiguous redirect
/usr/local/lib/zmbackup/bash/NotifyAction.sh: line 58: $MESSAGE: ambiguous redirect
/usr/local/lib/zmbackup/bash/NotifyAction.sh: line 59: $MESSAGE: ambiguous redirect
/usr/local/lib/zmbackup/bash/NotifyAction.sh: line 60: $MESSAGE: ambiguous redirect
/usr/local/lib/zmbackup/bash/NotifyAction.sh: line 61: $MESSAGE: ambiguous redirect

Zmbackup doesn't work in SQLITE3 mode

ISSUE TYPE
  • Bug Report
ENVIRONMENT VERSION
  • Zmbackup Version: 1.2.0 RC2
  • Zimbra Version: 8.8.6
  • Linux Distribution & Version: Ubuntu 16.04
SUMMARY

Zmbackup restore doesn't work in SQLITE3 mode. When you try to execute the following message appears:

zimbra@zimbra:~$ zmbackup -r -m full-20180326211421 [email protected]

Restore mail process with session full-20180326211421 started at Mon Mar 26 21:30:36 -03 2018

Error during the restore process for account [email protected]. Error message below:
[email protected]: /bin/bash: /opt/zimbra/backup/full-20180326211421/[email protected]: No such file or directory
Restore mail process with session full-20180326211421 completed at Mon Mar 26 21:30:37 -03 2018
du: cannot access '/opt/zimbra/backup/full-20180326211421|2018-03-26T21:14:27.050983701|2018-03-26T21:16:20.465508349|8.0M|Full': No such file or directory
ls: cannot access '/opt/zimbra/backup/full-20180326211421|2018-03-26T21:14:27.050983701|2018-03-26T21:16:20.465508349|8.0M|Full/*.ldiff': No such file or directory

Better backup rotation

Zmbackup only exclude the backup you ask to remove, not rotate them. Zmbkpose had the option, but never works. You should implement a new way to rotate the backups with the same options present in zmbkpose, plus the one implemented in the client ViaSoft.

New Feature: Zmbackup installer inside Ansible Galaxy

ISSUE TYPE
  • Feature Idea
ENVIRONMENT VERSION
  • Zmbackup Version: Any
  • Zimbra Version: Any
  • Linux Distribution & Version: Any
SUMMARY

Create an Ansible script to install Zmbackup and host inside Ansible Galaxy for anyone who wants to install.

Zimbra incremental backup doens't work.

I am not sure Incremental backups not working

 zmbackup -i
Running the Incremental Backup - Please wait...
WARN: [email protected] already has backup today. Nothing to do.
Nothing to do..

Tried with account name as well : It's not working.

** Thanks for your great effort **

Recreate all the tests for Travis CI

ISSUE TYPE
  • Bug Report
ENVIRONMENT VERSION
  • Zmbackup Version: 1.2.0 RC 2
  • Zimbra Version: any
  • Linux Distribution & Version: any
SUMMARY

Recreate all the tests for Travis CI validate if Zmbackup script is okay (installer and the main code)

New error codes

ISSUE TYPE
  • Enhancement
ENVIRONMENT VERSION
  • Zmbackup Version: 1.2.0
  • Zimbra Version: Any
  • Linux Distribution & Version: Any
SUMMARY

Change all the error codes for Zmbackup because today they are a mess.

Support to CentOS 6

ISSUE TYPE
  • Bug Report
ENVIRONMENT VERSION
  • Zmbackup Version: 1.2 RC
  • Zimbra Version: 8.8
  • Linux Distribution & Version: CentOS 6
SUMMARY

There are a lot of bugs for CentOS 6, mostly that the script doesn't work very well for the OS. I need to test the script and fix all the bugs that is occuring in this version of the OS before the next release.

Better config file error handle

ISSUE TYPE
  • Enhancement
ENVIRONMENT VERSION
  • Zmbackup Version: 1.2.0
  • Zimbra Version: Any
  • Linux Distribution & Version: Any
SUMMARY

A better way to handle the error messages for the file zmbackup.conf having not all the variables informed.

Mail report stopped working

ISSUE TYPE
  • Bug Report
ENVIRONMENT VERSION
  • Zmbackup Version: 1.2.0
  • Zimbra Version: any
  • Linux Distribution & Version: any
SUMMARY

Mail report wasn't being send after the end of the backup routine.

zmbackup single mail account restore issue

ISSUE TYPE
  • Bug Report
  • Feature Idea
  • Documentation Report
  • Question
  • Checklist

Question

ENVIRONMENT VERSION
  • Zmbackup Version: 1.2.0, release candidate 2
  • Zimbra Version: 8.8.6
  • Linux Distribution & Version: CentOS 6.9
SUMMARY

I am doing a test with Zimbra OSE 8.8.6 installed on CentOS 6.9. It is configured properly and working fine. I have installed Zmbackup (version 1.2.0, release candidate 2) after downloading from Github for mail account backup. It is also doing it's backup work properly without any failure.

Problem is arising when I am trying to restore any account from the full backup. Whenever I am trying to restore any ldap information it is doing properly(perhaps as no error message is there) but when I am trying to restore the mail account( with "-m" option) it is giving error message as follows -


[zimbra@mail2 ~]$ zmbackup -r -ldp full-20180217170655 [email protected]
Restore LDAP process with session full-20180217170655 started at Tue Feb 27 13:12:53 IST 2018
Restore LDAP process with session full-20180217170655 completed at Tue Feb 27 13:12:54 IST 2018


[zimbra@mail2 ~]$ zmbackup -r -m full-20180217170655 [email protected]
Restore mail process with session full-20180217170655 started at Tue Feb 27 13:14:15 IST 2018

Error during the restore process for account [email protected]. Error message below:
[email protected]: /bin/bash: http: command not found
Restore mail process with session full-20180217170655 completed at Tue Feb 27 13:14:15 IST 2018


Please note, the server I am restoring to is a different server(mail2.sailrmd.co.in) from the source server(mail1.sailrmd.co.in).

STEPS TO REPRODUCE
ACTUAL RESULTS

Wget not installing and Storage not being removed

ISSUE TYPE
  • Bug Report
ENVIRONMENT VERSION
  • Zmbackup Version: 1.2.0 RC2
  • Zimbra Version: 8.8.6
  • Linux Distribution & Version: CentOS 6
SUMMARY

During the install process, the wget is ignored by the process install and the storage is not removed during the uninstall.

Use a database to store sessions.txt rather than a file

Today, zmbackup use the file session.txt as a way to store the backup's history. This file is susceptible to failures and zmbackup only use this file because zmbkpose used before the fork. Now, that this application is restructured, is a good time to study a better way to store this log.

Two good candidates are MariaDB and PostgreSQL.

Process to restore the zimbra backup

Dear Sir,

I have successfull taken eh full backup of zimbra using zmbackup -f command.
The new zimbra machine is ready and trying to restore the backup using zmbackup -r. But the error comes "Error! Incorrect options" Please let me know the option that i need to include to restore the full backup.

Zmbackup need to find a way to restore after a broken backup

If something kills the process, or you stop the process in the middle of the execution, when you start zmbackup again, it will create a new backup session for the rest of the accounts. The correct is the script complete the previous backup, rather than create a new one and stay with two different folders for the same process.

New feature: backingup only active accounts

ISSUE TYPE

  • Feature Idea
    ENVIRONMENT VERSION

  • Zmbackup Version: 1.2.0 RC2

  • Zimbra Version: any

  • Linux Distribution & Version: any
    SUMMARY
    At the moment, the command zmbackup -f realize a full backup routine to all accounts. The new feature this a option to backup only active accounts.

I can't run backups | DNS SRV: out of memory?

Hey Lucas,

I'm trying do a backup, but when I run the command, I have this issue, DNS SRV: out of memory?.

[zimbra@webmail ~]$ zmbackup -f user@domain
Running the Full Backup - Please wait...
DNS SRV: out of memory?
Backup finished!
[zimbra@webmail ~]$ zmbackup -v
zmbackup version: 1.1.4

How can I fix it? Do I need to do any different configuration when I run the wizard?

P.S.: I did the installation process without change any option.

Thanks.

mail filter backup

Hi,

You could add users mail filter backup. I wrote simple script to do it

cat /backups/zmigrate/filters-export.sh
#!/bin/bash
set -x
clear
echo “Retrieve zimbra user name…”
USERS=cat /backups/zmigrate/emails.txt;
for ACCOUNT in $USERS; do
NAME=echo $ACCOUNT;
filter=su – zimbra -c “zmprov ga $NAME zimbraMailSieveScript” > /tmp/$NAME
sed -i -e “1d” /tmp/$NAME
sed ‘s/zimbraMailSieveScript: //g’ /tmp/$NAME > /backups/zmigrate/filters/$NAME;
rm /tmp/$NAME
echo “Export filter for $NAME…”
done
echo “All filter has been export successfully”

And script for import filters:

#!/bin/bash
set -x
for file in /backups/zmigrate/filters/*
do
StrFilter=cat “$file”
Acc=$file
su – zimbra -c “zmprov ma $(echo $file | grep -E -o “\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+.[A-Za-z]{2,6}\b”) zimbraMailSieveScript ‘$StrFilter'”
echo “Process filter $Acc”
done
echo “All filter has been import successfully”

Backup and restore to a specific domain

ISSUE TYPE
Feature Idea

ENVIRONMENT VERSION
Zmbackup Version: Any
Zimbra Version: Any
Linux Distribution & Version: Any

SUMMARY
New feature to backup and restore to a specific domain only.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.