Hi,

We use csync2 together with lsyncd to synchronize 80G of data across three webservers. Ever since we dist-upgraded from Debian Jessie to Debian Stretch (and went from csync2 1.34 to 2.0), we've been experiencing issues with temporary files being synced:

Fri Aug 2 16:30:54 2019 Normal: Processing syncing list:
web01:/srv/production.scholieren.com/storage/app/public/images/avatars/222781_628213156475615611jpg_500x_100.jpg
Fri Aug 2 16:31:25 2019 Normal: Finished events list = 0
Fri Aug 2 16:31:26 2019 Normal: Processing syncing list:
web01:/srv/production.scholieren.com/storage/app/public/images/avatars/.222781_628213156475615611jpg_500x_100.jpg.Ps0JDy
web01:/srv/production.scholieren.com/storage/app/public/images/avatars/.222781_628213156475615611jpg_500x_100.jpg.h9ZPzU
web01:/srv/production.scholieren.com/storage/app/public/images/avatars/.222781_628213156475615611jpg_500x_100.jpg.r2jZvg
web01:/srv/production.scholieren.com/storage/app/public/images/avatars/.222781_628213156475615611jpg_500x_100.jpg.xsOFUC
web01:/srv/production.scholieren.com/storage/app/public/images/avatars/.222781_628213156475615611jpg_500x_100.jpg.12ZvjZ
web01:/srv/production.scholieren.com/storage/app/public/images/avatars/.222781_628213156475615611jpg_500x_100.jpg.RILpIl
Fri Aug 2 16:31:58 2019 Normal: Finished events list = 0

We could exclude dot-files, but that would also exclude .htaccess for example. We've tried setting the tempdir-variabele, but this led to all sorts of problems (all remote files being truncated to 512 bytes was the funniest one).

Why is this suddenly the case? Maybe we've missed a change in the way the flags work? Is there an upgrade guide for 2.0? Please advise.

Jon

Analyzing the csync2 parser for extending the auto command I stumbled over a problem with an ambiguity between unquoted strings and keywords. For example if a host is called "auto", the statement 'host auto' will fail requiring quotes because auto is a keyword.

If you want to give it a try I have modified the flex and bison files in the parser-path branch of my fork. I can also file a pull request.

https://github.com/ggs67/csync2/tree/parser-patch

Commit comment of modifications:

1. Disambiguation of strings

  Modified lexer with non-exclusive start state STRTOKEN for strings without quotes.
  This resolves the conflict where strings matching keywords will fail if not quoted
  (ex. 'host auto;' required 'host "auto";')

2. Tokenized auto-methods

  Modified auto-methods from separate C-parsing code to include those by separate tokens in the
  lexer.

  This change was done to cleanup parsing, becoming more readable and in prepartation to extend the
  auto command.

When trying to use csync2 on Alpine Linux (either with the Alpine csync2 community package or with the compiled current csync2 version here from Github), the tool stops with
"Assertion failed: i == len-1 (checktxt.c: csync_genchecktxt: 94)".

I fixed this by changing the string format in two lines in "checktxt.c":

--- checktxt.c.orig
+++ checktxt.c
@@ -49,7 +49,7 @@
        xxprintf("v1");

        if ( !S_ISLNK(st->st_mode) && !S_ISDIR(st->st_mode) )
-               xxprintf(":mtime=%Ld", ign_mtime ? (long long)0 : (long long)st->st_mtime);
+               xxprintf(":mtime=%lld", ign_mtime ? (long long)0 : (long long)st->st_mtime);

        if ( !csync_ignore_mod )
                xxprintf(":mode=%d", (int)st->st_mode);
@@ -61,7 +61,7 @@
                xxprintf(":gid=%d", (int)st->st_gid);

        if ( S_ISREG(st->st_mode) )
-               xxprintf(":type=reg:size=%Ld", (long long)st->st_size);
+               xxprintf(":type=reg:size=%lld", (long long)st->st_size);

        if ( S_ISDIR(st->st_mode) )
                xxprintf(":type=dir");

The format specifiers listed in "http://www.cplusplus.com/reference/cstdio/printf/" were used for this.

Segmentation faults in a later phase of execution of "csync2 -xv" could be fixed in "update.c" accordingly:

--- update.c.orig
+++ update.c
@@ -469,7 +469,7 @@

 skip_action:
        if ( !S_ISLNK(st.st_mode) ) {
-               conn_printf("SETIME %s %s %Ld\n",
+               conn_printf("SETIME %s %s %lld\n",
                                url_encode(key), url_encode(filename),
                                (long long)st.st_mtime);
                last_conn_status = read_conn_status(filename, peername);

csync2 works for me on Alpine Linux with the changes shown above applied. Note that I neither checked on other platforms nor did I investigate on the root cause.

it would be cool to be able to split csync2.cfg into peaces (e.g. per-group) to be able to write something like:

nossl * *;

//other global params

include_dir /etc/csync2.d/*.cfg;

and manage /etc/csync2.d/*.cfg configuration files instead of single/monolithic;

Any chance of a new release to pick up the latest fixes for version 2? Especially to include 999f153 which is the main problem we see from time to time.

Documentation says: "Csync2 automatically ignores all groups which do not contain the local hostname in the host list. This way you can use one big Csync2 configuration file for the entire cluster.", but in fact csync2 reads all the groups, and requires all the keys on all hosts.

This makes stated use case insecure: if all the hosts knows all the keys, then whats the point of having different keys for different groups?

Its even impossible to make zero-sized stubs for missing keys: they are not accepted either.

Detailed setup:

Shared config file for all hosts:

group somegroup01
{
    host srv01 (srv02);
    key /etc/csync2/somegroup01.key;
   ...
}

group somegroup02
{
    host srv01 (srv03);
    key /etc/csync2/somegroup02.key;
   ...
}

Syncing from srv01 to srv03:

$ csync2 -xv -G somegroup02
Connecting to host srv03 (SSL) ...
Connect to 10.0.0.3:30865 (srv03).
Config command failed.
ERROR: Connection to remote host `srv03' failed.
Host stays in dirty state. Try again later...

Logs on srv03 are following:

Mar 14 19:59:42 srv03 csync2[2225785]: Config error: Can't read keyfile /etc/csync2/somegroup01.key.

So host srv03 requires key for somegroup01 group, despite not being listed there.

Please adapt the code so that the directory mtime is not lost when replicating.

Hi @lge ,

I meet an environment,

On node1,

# ip a
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:e8:7f:24 brd ff:ff:ff:ff:ff:ff
    inet 10.10.10.121/24 brd 10.10.10.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet 10.10.10.191/24 brd 10.10.10.255 scope global secondary eth1
       valid_lft forever preferred_lft forever

# hostname
node1

# cat /etc/hosts
10.10.10.191 node1
10.10.10.192 node2
10.10.10.121 vnode1
10.10.10.122 vnode2

On node2,

# ip a
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:39:4a:c2 brd ff:ff:ff:ff:ff:ff
    inet 10.10.10.122/24 brd 10.10.10.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet 10.10.10.192/24 brd 10.10.10.255 scope global secondary eth1
       valid_lft forever preferred_lft forever

# hostname
node2

# cat /etc/hosts
10.10.10.191 node1
10.10.10.192 node2
10.10.10.121 vnode1
10.10.10.122 vnode2

Problem:

When verify_peername, node1 raise Identification failed!, since node2's address get from getaddrinfo is 10.10.10.192, while "peername" get from getpeername is 10.10.10.122(although node2 using 10.10.10.192 to connect with node1), so here is the difference to make verify_peername failed

Any suggestions about this?

Thank you!

Can csync2 detect if I have moved files or a directory structure?

I want to be able to move files locally without them being deleted then recreated remotely.

I'd like any locally moved files to be moved remotely also.

As sysadmin, I am trying to set up my environments as secure as possible. csync2 uses SSL, which is already a good start, however the current implementation does not allow for a few features I would like to set to meet our security standards. I would like to:

• have the documentation updated to use keys longer than 1024 bits (it works with 4096).
• be able to configure the file names of the certificate and key in the configuration file
• check key file and ssl key permissions on application start (e.g. mode 0600, owned by the user running it or mode 0640, owned by root and readably by a group the user running the application if != root)
• use passwords on the ssl key
• restrict the protocol to TLSv1.2 or higher
• restrict the allowed ciphers
• use a certificate tied to the actual host name instead of sharing the self-signed certificate across hosts
• validate the CA key chain (I can use a certificate signed by a CA instead of self signed certs, but as the CA certificate is not checked, this is quite irrelevant)
• use perfect forward secrecy

I'm trying to debug an issue we're seeing in a production environment: a csync2 group has several actions defined, in a specific order:

• action 1 sets the correct ownership of the file via chown;
• action 2 runs setfacl to ensure correct filesystem ACLs are in place;
• action 3 runs a custom script via sudo -u ... to do some processing on the file;

After running some sync's with -v -v it seems that csync2 is reading the actions in the reverse order, and thus executing them in reverse order, which fails because the script is run as a user who doesn't yet own the file.

Is this intended behaviour, or have we discovered some kind of bug? Is there an explicit way to specify the order the actions should be performed?

I looked quickly in action.c and there's nothing obvious there to my eyes, about why it would be reversing the order, but I also don't really know enough about how the config parser works to know that it isn't being parsed backwards.

Lastly - if the order is not deterministic currently, would you be open to adding the ability to make it deterministic?
I'm happy to try and help with a patch for this, but C really isn't my day job, so it's quite possible that even a patch which "works" will need to be fixed to make it usable/safe/what have you...

The exclude pattern is global or directory-specific (it seems to me); however a subtree-specific exclude pattern seems quite useful.

One example is firewalld creating ".old" files I do not want to sync (while in other directories I might want them).
So I would have to list each directory (even those not existing yet) where I do not want to sync ".old" files.

I did not find the option to automatically create parent directories. It would be very convenient to do for inital run (-TI) and indeed always.

Use case. Add a new host to the group and initialize it as csync2 -TI -U -v. I would like a kind of push of all files to occur even if not the entire (parent) directory structure is created on the remote side.

There is a condition which causes files in the destination server to be owned by root instead of the correct user. It has been discussed on the mailing list and one hypothesis is here:

http://lists.linbit.com/pipermail/csync2/2017-September/000080.html

This is a pretty serious bug for us since we run a WebDAV server cluster with csync2 and lots of small file operations to servers across the cluster at the same time.

I had included /etc/firewalld in the configuration, and when syncing via csync -xv, the remote node logged Updated /etc/firewalld/services/cluster.xml from XXX.
But in fact on that host the directory /etc/firewalld/services was missing, so there was no cluster.xml after sync.
Only after I had created the directory manually, csync was actually successful.
There should not be a mis-leading syslog message.

Version being used was from SLES15 SP2 (csync2-2.0+git.1461714863a4-4.3.1.x86_64).

I am a bit new to csync but I need help in troubleshooting an issue of csync not syncing files between cluster nodes. We have just one group mentioned in /etc/csync2/csync2.cfg with entries for host, key and include directives. csync2 is running in inet mode. Every minute this command is triggered:

csync2 -D /var/lib/csync2 -N node1 -xv
And my xinetd has this config for csync2:

service csync2
{
        flags           = REUSE
        socket_type     = stream
        wait            = no
        user            = root
        group           = root
        server          = /usr/sbin/csync2
        server_args     = -i -N node1
        port            = 30865
        type            = UNLISTED
        disable         = no
}

Some questions I have:

• I do not see csync2 running as a service (nothing like /etc/systemd/system/csync2.conf), how how and when does csync detects file changes ? Does this happen because of periodically running the command 'csync2 -D /var/lib/csync2 -N node1 -xv' ?
• Is there a way to enable debug logs on what csync is doing ? even when I run with -v command it simply gives this kind of output, I want to know more.

Connection closed.
Finished with 0 errors.

What i do wrong?

I am not experienced with csync (other than building it off the shelf i.e. what is there in git repo here) but I want to know how do I build it with SSL libraries. Is there is any example how to build for SSL enabled csync2. Right now I get this error after enabling SSL.

csync2 -D /var/lib/csync2 -N host1 -xv
Connecting to host host22 (SSL) ...
Bound to 10.0.0.1:0 as host1.
Connect to 10.0.0.2:30865 (host2).
ERROR: Config request SSL but this csync2 is built without SSL support.
ERROR: Connection to remote host `host2' failed.

At least on ubuntu 20.04 with gnutls 3.6.13-2ubuntu1.6 SSL connections fail with GNUTLS_E_CERTIFICATE_REQUIRED error in syslog even when everything is otherwise correctly configured. Adding nossl allows for sync to work. I straced the process to see that it does indeed read the certificates and otherwise works as it should, however something must have been updated in gnutls so that it throws a message that csync2 cannot handle.

easily reproducible with ie. following config

group replicated
{
host host1;
host host2;
key /etc/csync2.d/csync2_clusterkey.key;
include /opt/replicated;

    action
    {
            pattern /opt/replicated;
            exec "/usr/bin/systemctl restart nginx";
            do-local;
    }

    backup-directory /opt/replicated-backup;
    backup-generations 3;

    auto none;

}

csync2 2.0 do not work with command csync2 -vx -C xxx
csync2 -vx -C xxx

centos 7.2 iptables close or open
when use csync2 -vx
all works with node name centos20 //centos30

//////////////////////
#/etc/csync2/csync2.cfg
nossl * ;
group centos
{
host centos10;
host centos20;
host centos30;
key /etc/csync2/csync2.key;
include /home/data/wwwroot/www.antbay.com;
exclude *~ .;
auto younger;
}
////////////////////

/etc/csync2/csync2_centos10.cfg
is same above also other node

but when use
csync2 -vx -C centos10
Connecting to host centos30 (PLAIN) ...
Connect to 192.168.1.30:30865 (centos30).
Config command failed.
ERROR: Connection to remote host `centos30' failed.
Host stays in dirty state. Try again later...

can you resolve it?

/etc/xinetd.d/csync2

default: on

description: csync2

service csync2
////////////////////////
{
disable = no
flags = REUSE IPv4
socket_type = stream
wait = no
user = root
group = root
server = /usr/sbin/csync2
server_args = -i -l
#log_on_failure += USERID
# only_from = 192.168.199.3 192.168.199.4
}

////////////////////

i add IPv4 on flags if not it listening on ipv6 no ipv4

use rpm or compile all the same problem

Docs said https://github.com/LINBIT/csync2/blob/master/doc/csync2.adoc#conflict-detection:

The younger, older, bigger and smaller methods let the remote side win the conflict if the file has been removed on the local side.

Maybe add some parameter just for auto resolve for deletetion? auto-delete (yes|no)?

refs:

1. http://lists.linbit.com/pipermail/csync2/2009-August/000571.html
2. #8

Hi,

I'm having this issue on my Fedora 31 :

/usr/sbin/csync2 -xrv

Connecting to host XXX (SSL) ...
Updating /etc/sysconfig/garb on XXX ...
*** stack smashing detected ***: terminated
Aborted (core dumped)

Any ideas?

I have configured 1 way sync. Second host to receive only updates from node1 and I don't want to delete the files on node2 when the files from node1 being deleted.

I am newbei and kindly apologize for considering this as issue.
Please let me know in case of any option to stop deleting files from node2.

Thanks in advance

The title says it all ;-) I am actually asking because I am looking forward for more options in the auto-conflic-resolution part (eg. "auto" statement).

It would be great to have an additional mode specifying the winning node directly (ex.: "auto host host1") specifying which host will win (this may of course require an additional run (possibly automatic) in multi-host configs.

Also every "auto" statement should allow for a file pattern (just like exclude) applying it to only those files. (if not present changes the global setting, thus not breaking any existing config)

Building rpm from supplied SPEC file fails due to missing doc file.
RPM build errors:
File not found: /root/rpmbuild/BUILDROOT/csync2-2.0-1.x86_64/usr/share/doc/csync2/README
File not found: /root/rpmbuild/BUILDROOT/csync2-2.0-1.x86_64/usr/share/doc/csync2/AUTHORS

The files exist, but the names contain the 'adoc' extension.
Adding '.adoc' to the file names in the spec file fixes the issue.

$ diff -u SPECS/csync2.spec.old SPECS/csync2.spec
--- SPECS/csync2.spec.old       2019-11-17 18:57:37.342151054 -0500
+++ SPECS/csync2.spec   2019-11-17 18:54:17.517245168 -0500
@@ -82,8 +82,8 @@
 %doc %{_mandir}/man1/csync2.1.gz
 %doc %{_docdir}/csync2/csync2.adoc
 %doc %{_docdir}/csync2/ChangeLog
-%doc %{_docdir}/csync2/README
-%doc %{_docdir}/csync2/AUTHORS
+%doc %{_docdir}/csync2/README.adoc
+%doc %{_docdir}/csync2/AUTHORS.adoc
 %config(noreplace) %{_sysconfdir}/xinetd.d/csync2
 %config(noreplace) %{_sysconfdir}/csync2.cfg

Thanks

System:
Csync 2-2.0
Xinetd
Ubuntu 16.04

http://lists.linbit.com/pipermail/csync2/2016-September/000051.html

We had the same issue recently and it took some time for us to debug this.

That would be (x)inetd mixing stderr of csync2 into stdout,
thereby "corrupting" the csync2 protocol.

add "-l" to the inetd invocation.  as in
/etc/inetd.conf:
-csync2          stream  tcp     nowait  root    /usr/sbin/csync2 csync2 -i
+csync2          stream  tcp     nowait  root    /usr/sbin/csync2 csync2 -i -l

xinetd: server_args = -i -l

This -l fixed everything, I'm writing this issue to acknowledge that 👍
Also the -l parameter is not described in the help. So I can't explain what it does exactly

Peer did provide a wrong SSL X509 cetrificate.

Hi, I'm trying to sync three nodes of a web cluster with lots of files/directories. There are many instances of the same kind of directory tree (lots of similar vhosts) where we want to exclude certain common directories eg. cache directories.

I've tried using exclude patterns eg:
files/cache
files/cache/*
/files/cache/
/files/cache/

But these directories aren't being excluded. Sample full paths would be something like:
/srv/www/vhosts/<domain-name>/<site-name>/html/application/files/cache/...
where and change between each vhost directory.

Is this possible with csync2?

Hi,
I was thinking, in case of 5 node setup, when 2 nodes become unavailable what happens?

Do they sync with the changes after they are connected? Do we keep history for this?

Thank you.

i have small doubt i wants to clarify it, can csync2 replicate 1TB of data in bidirectional all the time i mean whenever changes happen on node1 or on node2 data has to be replicated to and fro, just like mirroring and what are the limitations of csync2 what is max size of a file it can replicate.

csync2 can replicate huge files like 5Gb iso images.

Thanks & Regards

We have a setup with two web servers, using csync2 to propagated user-uploaded files between them - nothing groundbreaking here. We have a single "action" in the config, which sets ownership of the synced file.

One thing I guess is less common, is that we use Linux filesystem ACLs. The directories being synced have the required default ACL permissions and mask, and files created on each host 'normally' (i.e. the initial uploaded file, or a test file created using e.g. touch) correctly inherit the correct ACL permissions and mask, and the file is usable.

For some reason, files copied by csync2 are created on the destination host with the correct inherited filesystem ACL permissions, but an empty ACL mask.

I'll try to setup a minimally reproducible setup for this tomorrow.

What is the undocumented (hoping I was not blind) "compare" statement in the config file for ?

Some background info:

After I did run out of time in my attempt to submit some changes for csync2 I decided to restart from scratch, resetting my fork and started cleaning up the parser/scanner before implementing new features.

For this I did switch from yacc to requiring bison (i.e. also running bison in native mode, not yacc compatible mode). This step is required by some changes I introduce but seemed also trivial as the original file uses the non-posix %expect statement. However my clean-up of the parser also includes changing the one step recursions to two steps. This removes the requirement for empty rules which has two positive effects : it disallows useless (and probably bogus) empty statements and in the same way removes the shift/reduce errors and thus does not require the %expect anymore

In my progress through the parser I am now on the "compare " and I think there is a bug in the parser or I alternatively I do not understand the use of the command. The comp_list rule parses to "incl_list TK_STRING" , while I guess this should be comp_list ? This seems top be confirmed by the code as there is no way for the code to distinguish between include patterns from the compare statement or a previous include statement.

Orgiginal rule:

comp_list:
        /* empty */
|       incl_list TK_STRING
                { add_patt(2, on_cygwin_lowercase($2)); }
;

I guess it should be comp_list and I will change to:

comp_list: comp
         | comp_list comp
         ;
         
comp     : TK_STRING
              { add_patt(2, on_cygwin_lowercase($1)); }
         ;

Still would be interested to know the use case of this statement.

update.c: In function ‘csync_diff’:
update.c:915:9: warning: ‘fclose’ called on pointer returned from a mismatched allocation function [-Wmismatched-dealloc]
915 | fclose(p);
| ^~~~~~~~~
update.c:909:13: note: returned from ‘popen’
909 | p = popen(buffer, "w");
| ^~~~~~~~~~~~~~~~~~
update.c:915:9: warning: ‘fclose’ called on pointer returned from a mismatched allocation function [-Wmismatched-dealloc]
915 | fclose(p);
| ^~~~~~~~~
update.c:909:13: note: returned from ‘popen’
909 | p = popen(buffer, "w");
| ^~~~~~~~~~~~~~~~~~
mv -f .deps/update.Tpo .deps/update.Po
gcc -DHAVE_CONFIG_H -I. -D'DBDIR="/var/lib/csync2"' -D'ETCDIR="/etc/csync2"' -I/usr/incl ude/p11-kit-1 -Wall -g -O2 -MT urlencode.o -MD -MP -MF .deps/urlencode.Tpo -c -o urlenco de.o urlencode.c
mv -f .deps/urlencode.Tpo .deps/urlencode.Po
gcc -DHAVE_CONFIG_H -I. -D'DBDIR="/var/lib/csync2"' -D'ETCDIR="/etc/csync2"' -I/usr/incl ude/p11-kit-1 -Wall -g -O2 -MT conn.o -MD -MP -MF .deps/conn.Tpo -c -o conn.o conn.c
conn.c:76:13: warning: conflicting types for ‘conn_response’ due to enum/integer mismatch; have ‘const char *(unsigned int)’ [-Wenum-int-mismatch]
76 | const char *conn_response(unsigned i)
| ^~~~~~~~~~~~~
In file included from conn.c:21:
csync2.h:150:20: note: previous declaration of ‘conn_response’ with type ‘const char *(enu m connection_response)’
150 | extern const char *conn_response(enum connection_response);

Would be grat if offline host messages can be suppressed to keep the logs clean. Thx.

cal0.warn: Mar 22 06:05:04 csync2[18186]: ERROR: Connection to remote host `x1c' failed.
cal0.warn: Mar 22 06:05:07 csync2[18186]: ERROR: Connection to remote host `htpc' failed.
cal0.warn: Mar 22 06:05:10 csync2[18186]: ERROR: Connection to remote host `game' failed.
cal0.warn: Mar 22 06:05:10 csync2[18186]: Finished with 3 errors.