davidjbianco / pycef Goto Github PK

https://www.elastic.co/guide/en/beats/filebeat/master/exported-fields-cef.html

cef.name
Short description of the event.

type: keyword

@DavidJBianco Hey dude!

Great little python script :)

So - my use case is exactly what you created this for with one exception. If I am getting a cef payload from a syslog file or stream, I have a few fields of syslog before the beginning of the CEF string.

For example, notice below in bold my CEF syslog line has a date and hostname, as possible in a syslog packet.

**Nov 17 02:14:39 arcsightfwd.davidbianco.io** CEF:0|ACME|Firewall X|||Accept|Unknown| eventId=187643083 mrt=1541908269348 proto=UDP in=1200 out=1200 modelConfidence=4 severity=0 relevance=10 assetCriticality=0 priority=2 art=1542420847090 act=Allow rt=1542420847090 src=192.168.30.15 sourceZoneID=M-fU32AABABCDVFpYAT3UdQ\=\= sourceZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 192.168.0.0-192.168.255.255 spt=32488 slong=0.0 slat=0.0 ... <more payload here>

It's my assessment that in your CEF header parsing regex, it works fine with out the ^ anchoring us to the beginning of the string.

Replace the current line 19

with:

header_re = r'((CEF:\d+)([^=\\]+\|){,7})(.*)'

This still works exactly as intended, but doesn't fail if I'm trying to read in syslog lines. :)

Can you publish to pypi

How To Click Here

Thank you, Sir, for the great lib!

Have you ever considered to introduce the reverse functionality?
Taking the dictionary (e.g. created by pycef) and generate a string with proper CEF?

There are some libs to generate CEF, but something much better could be done, potentially...

For example, the following lib would not accept any non-standard extension data and (for example) DeviceEventClassID. It will either silently ignore the passed value or raise error.

https://github.com/kamushadenes/cefevent
Raise errors
By default the methods set_field() and set_prefix() returns False if the name or the value or the CEF field is invalid.
Set CEFEvent.strict=True to raise ValueError if any invalid field name / values are passed.

#############################

row = pycef.parse(some_cef_string)
c = CEFEvent(strict=True)
c.set_field('deviceVersion', row['DeviceVersion'])
c.set_field('DeviceVersion', row['DeviceVersion']) #will not be in CEF
c.set_field('deviceEventClassID', row['DeviceEventClassID']) #will not be in CEF

#############################

c = CEFEvent()
c.strict = True
c.set_field('sourceAddress', '192.168.67.500')

#result:
#CEF:0|CEF Vendor|CEF Product|1.0|0|CEF Event|5|

Motivation:

1. it is great to support custom extensions. Even if they are not in Standard, they are often used in real life
2. single package supporting both parsing and generating CEF might make sense?

• If regex fails, return None, not an empty dict. More pythonic, easier to handle when calling function
• add null logger, and some logger output
• add some end user help to tell them that regex failed, instead of returning empty dict { }

Check the below log format: The 5th and 6th placeholder has 'value|value' passed and the pycef parser breaks at that cases.
CEF:0|value|value||value|value|value|value|value| col1=value1

We were able to check this and modify three lines of code which makes it more robust for such cases and works well with the correctly formatted logs.

Please find attached the jpg representing the changes made.
Hope this will be useful for your team and others as well.

It seems like a pipe in the value of a cef field will cause the header_re parser to match to far into the CEF log.