Comments (7)
@raghurampai it seems very generic: while the library can be improved to support it while this may not be used when producing the protobuf abstracted flows.
So far no plan of supporting them.
What's the use case? What are the fields produced? What are the devices producing it? Do you have a pcap?
from goflow.
Hi just jumping in
We are using dedicated Netflow Probes like Flowmon and others. These are capable of extracting additional information such as DNS requests, HTTP information etc. This is very useful in open high performance environments where firewall, proxy etc. are no option.
This data is then transmitted in IPFIX like any other network traffic information.
Using the corresponding Enterprise ID (IANA is 0) and Field IDs configurations, this allows IPFIX to be used as a widely supported, very flexible data transmission container.
Ideally the enterprise field definitions would not be hard coded, but configurable.
The CESNET IPFIXcol implements such features.
Cheers.
from goflow.
@mathiaske Got it, thank you for the details. I believe this is beyond the scope of GoFlow which is meant to produce a unique flow format for network samples. The decoding library could definitely benefit from this though. Could you send some samples?
Actually, sFlow also supports things like DNS and HTTP, in which case, it may be interesting to have in GoFlow.
from goflow.
@lspgn, sorry for the delay in response. I don't have the pcap currently. The use case at high level is similar to what @mathiaske mentioned. To describe further, the monitoring of BNG (broadband subscriber services) will have some enterprise specific information elements, which basically needs to be processed-
https://www.juniper.net/documentation/en_US/junos/topics/concept/ipfix-mediation-bng.html
Thanks!
from goflow.
@raghurampai thank you for the details!
I will definitely look into it. Do you have a spec detailing the enterprise specifics fields?
from goflow.
We currently use some Flowmon Probes -> CESNET, Flowmon (former Inveatec) fields
We also intend to have our own fields in the future for some use cases
-> it would be greate if this would be configurable (but please not in xml)
Example: https://github.com/CESNET/ipfixcol/blob/master/base/config/ipfix-elements.xml
from goflow.
Below link has the enterprise specific fields-
https://www.juniper.net/documentation/en_US/junos/topics/reference/general/logging-dictionary-template-types-tdf.html
Yes, it would be great to have option of some sort of configurable file for goflow decoder to be able to understand the enterprise specific fields. The decoder could then decode these additional fields.
from goflow.
Related Issues (20)
- How to lookup the received netflow packets HOT 2
- Can this project be used as a library? HOT 2
- Renormalise the sampled data HOT 2
- gloflow2 doesn't multiply sampling rate with bytes/packets
- Nat Event On Output(Question) HOT 2
- Protobuf messages are corrupted or don't match the provided schema HOT 4
- Port Mirror Question HOT 2
- vendor-specific field parsing HOT 2
- Support for IPFIX dataLinkFrameSection HOT 1
- SIGSEGV when port is already in use
- GoFlow segfaults with bad input HOT 3
- How can we configure goflow to enable collector for Netflow V9 format?
- Same Time* for sflow even if they are different
- [Kafka SSL] Custom truststore and keystore.
- IPFIX Template decoding does not handle Enterprise-Specific Information Elements
- why parse sflow packet limit payload.Len() >= 8
- why goflow not decode FORMAT_ETH record?
- sflow decode record ipv4 and ipv6 data will cause binary.Read: invalid type *sflow.SampledIP_Base
- Sampler Address HOT 3
- Openvswitch ovs-vsctl connection HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from goflow.