Comments (25)
I just verified that setting distribute_network = yes in the aggregating server should make the audit events go to the plugins of audispd.
from aushape.
I managed to get everything working by setting log_format = ENRICHED
on the server. The audisp daemon does not stop anymore after receiving remote events.
Thank you @spbnick and @stevegrubb.
from aushape.
Hi @Keysuke, thank you for your report!
Do you run aushape under audispd, or single-shot? If it's the former, have you tried feeding the logs to aushape directly? If yes, what does it do?
If you run aushape under audispd, could you provide the contents of audispd configuration file invoking it?
Thank you.
from aushape.
Here are my configuration files, I am using audispd.
Everything works for local events. Only remote events (those coming from the auditd listening on a TCP port) do not get processed.
/etc/audisp/plugins.d/aushape.conf
active = yes
direction = out
path = /usr/bin/aushape-audispd-plugin
type = always
format = string
/usr/bin/aushape-audispd-plugin
#!/bin/sh
exec /usr/bin/aushape -l json --events-per-doc=none --fold=all -o syslog
from aushape.
Do they actually get delivered to aushape?
Have you checked if aushape produces any error output?
You can do that if e.g. you add &>> /var/log/aushape-audispd-plugin.out
to the end of aushape invocation in /usr/bin/aushape-audispd-plugin
.
Do you see the messages from remote hosts in /var/log/audit/audit.log
?
If you do, can you try feeding that log to aushape?
Thank you.
Meanwhile I'll do some testing of my own.
from aushape.
/var/log/audit/audit.log
gets the messages from local and remote entries.
By adding &>> /var/log/aushape-audispd-plugin.out
to the plugin file, I do not get any message in /var/log/aushape-audispd-plugin.out
, in any case.
Using
exec /usr/bin/aushape -l json --events-per-doc=none --fold=all -f /var/log/aushape.log
in /usr/bin/aushape-audispd-plugin
, the local events generate entries in /var/log/aushape.log
. Remote events do not.
from aushape.
Thank you, @Keysuke, I'm trying to reproduce this locally.
from aushape.
I think this is audispd not sending remote logs to aushape. To verify comment out the aushape invocation in /usr/bin/aushape-audispd-plugin
, and add cat >/var/log/audispd-pipe.log
, then check that file after restarting auditd and generating some remote logs.
I'll try to figure out why that happens.
from aushape.
@Keysuke, do you have distribute_network = yes
in /etc/audit/auditd.conf
on the server where you run aushape?
from aushape.
I set distribute_network = yes
in my server. The problem is still here : I am only getting local entries.
I also tried on a Fedora 25 aggregating server, without any success.
from aushape.
from aushape.
Sorry, the first day was pretty busy, will have to return to this tomorrow.
from aushape.
It seems that on my host, for some reason, auditd closes pipe to audispd after receiving the first remote event. Audispd shuts down along with all plugins for that reason. @Keysuke, do you see the same?
from aushape.
@stevegrubb, I dug some more into this and I see that auditd sends an event with zero-size payload to audispd, instead of the remote event it received. In response to that audispd shuts down. This is with audit 2.7.6-1.
Here is the relevant piece of strace from auditd:
writev(5, [{iov_base="\0\0\0\0\20\0\0\0\267\4\0\0F\0\0\0", iov_len=16}, {iov_base="audit(1497608176.826:6143): addr=192.168.122.40 port=56296 res=success", iov_len=70}], 2) = 86
write(1, "node=localhost.localdomain type=DAEMON_ACCEPT msg=audit(1497608176.826:6143): addr=192.168.122.40 port=56296 res=success\n", 121) = 121
epoll_ctl(8, EPOLL_CTL_ADD, 0, {EPOLLIN, {u32=0, u64=4294967296}}) = 0
epoll_wait(8, [{EPOLLIN, {u32=0, u64=4294967296}}], 64, 59743) = 1
read(0, "\376\0\0\377\0\0\0\0\0\0\23\1\216\0\0\0node=fedora24-dev type=USER_CMD msg=audit(1497608176.813:14039559): pid=5787 uid=1000 auid=1000 ses=3 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='cwd=\"/home/nkondras/projects/pkgs.devel.redhat.com/rpms/freeradius\" cmd=\""..., 8970) = 291
writev(5, [{iov_base="\0\0\0\0\20\0\0\0c\4\0\0\0\0\0\0", iov_len=16}, {iov_base="", iov_len=0}], 2) = 16
write(1, "node=fedora24-dev type=USER_CMD msg=audit(1497608176.813:14039559): pid=5787 uid=1000 auid=1000 ses=3 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='cwd=\"/home/nkondras/projects/pkgs.devel.redhat.com/rpms/freeradius\" cmd=\"whoami\" terminal"..., 275) = 275
write(0, "\376\0\0\377\0\0\0\0\0@\0\0\216\0\0\0", 16) = 16
The read
reads the remote message and the second writev
is trying to write it to audispd.
from aushape.
By restarting the auditd service on the remote server, I get activity on the centralizing server : the DAEMON_ACCEPT and the DAEMON_CLOSE events (which are created locally), are processed in aushape.
from aushape.
Curiouser and curiouser. @Keysuke, do you ever see audispd stopping after receiving the remote messages?
from aushape.
Also, @Keysuke, can you list audit package versions on both hosts? Which aushape version are you using?
from aushape.
Yes, it seems like audispd stops working after receiving remote events. Even for local events.
I'm using audit-2.6.5-3.el7_3.1.x86_64
and audispd-plugins-2.6.5-3.el7_3.1.x86_64
on both hosts.
from aushape.
Right, thank you, @Keysuke. Let's see what @stevegrubb has to say.
from aushape.
Something doesn't sound right. On the remote systems:
Need to set a node name method
configure au-remote to point to server
set format to enriched
On aggregating server
also set a node name method
set format to enriched
configure receiving port (usually 60)
open hole in firewall
distribute_network = no
at this point disable aushape
start the server and then the client
generate some events on the client
verify that they are being recorded to disk correctly with ausearch on the aggregating server.
When the remote events are verified good on disk, then enable aushape on the aggregating server and change distribute_network to yes. Restart auditd.
If any program exits, check the logs and see what its saying. Audisp and auditd are usually pretty good about saying why something is closing.
from aushape.
Thank you, @stevegrubb. I played with this for a while and found that this works as long as the aggregating server has log_format = ENRICHED
. If I set log_format = RAW
, no matter what the remote system uses, the audispd will stop right after receiving the first remote message, because it gets an event with zero-length payload. The message in the journal is simply something like this:
Jun 20 13:19:47 localhost.localdomain auditd[13720]: dispatcher 13722 reaped
Here is the code that does the exiting: https://github.com/linux-audit/audit-userspace/blob/master/audisp/audispd.c#L868
@Keysuke, could you please try setting log_format = ENRICHED
on the server and check if that works?
from aushape.
Hmm...auditd is supposed to treat each event separately and switch between protocol1 & 2 seamlessly (raw & enriched respectively). This sounds like its not so seamless. :-) I'll check into this. In the mean time, it is intended that whenever you are aggregating events, all nodes should be using enriched events so that the remote uid/gid is resolved before sending.
from aushape.
@stevegrubb, yes I agree it is better to use enriched events when aggregating. Thanks, Steve!
from aushape.
I know this issue is closed, but i wanted to mention that I fixed the auditd portion of the problem with this commit:
linux-audit/audit-userspace@878f1e1
from aushape.
Thanks a lot, Steve!
from aushape.
Related Issues (20)
- Scan the code for TODO and FIXME
- Ignore or warn about event not being trimmed to the required maximum
- Consider having field value sub-fields HOT 2
- Install schemas and Elasticsearch mapping
- Add instructions on Elasticsearch forwarding to README.md
- Rename `host` field to `node` to match audit logs better HOT 1
- Implement outputting normalized audit event data
- Make parsed data output optional
- Implement Elasticsearch mapping generation
- Mention --enable-debug configure option in README.md
- Implement support for output format, which can easily be forwarded to ElasticSearch HOT 1
- Handle repeated NETFILTER_CFG records
- Some execve events are considered invalid
- Docker VIRT_CONTROL vm field garbage
- Support building without audit normalization API
- Implement integration tests
- Error normalizing NETFILTER_CFG HOT 4
- aushape: error while loading shared libraries: libaushape.so.0: cannot open shared object file: No such file or directory HOT 1
- undeclared symbols on make
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aushape.