ciscodevnet / bigmuddy-network-telemetry-stacks

A batteries-included docker-based collection of demo stacks adapting network streaming telemetry to common consumer formats.

Archived

This project has not been supported in multiple years and has been pushed to an archive state. If anyone is interested in supporting it, contact [email protected] and we can arrange to un-archive it. The final version can be found on the "final" branch.

bigmuddy-network-telemetry-stacks's Issues

Failed to load .proto

Hi,
I am using stack_kafka and XRv 6.1.2.
Logstash can store telemetry in kafka, but the JSON structure is nested and can not be visualized with kibana.
Is the next JSON structure normal?

JSON structure:

{
    "@timestamp": "2017-03-16T08:59:00.319Z",
    "@version": "1",
    "collection_end_time": 1489654386353,
    "collection_id": 142581,
    "collection_start_time": 1489654386284,
    "content": [
        {
            "content": {
                "name": "tunnel-te1",
                "timest": 1489654386351
            },
            "timest": 1489654386351
        },
        {
            "content": {
                "content": {
                    "content": [
                        {
                            "index": 0,
                            "timest": 1489654386351
                        },
                        {
                            "content": [
                                {
                                    "index": 0,
                                    "timest": 1489654386351
                                },
                                {
                                    "timest": 1489654386351,
                                    "type": "iana-if-type:mplsTunnel"
                                },
                                {
                                    "admin-status": "UP",
                                    "timest": 1489654386351
                                },
                                {
                                    "oper-status": "UP",
                                    "timest": 1489654386351
                                },
                                {
                                    "mtu": 1500,
                                    "timest": 1489654386351
                                },
                                {
                                    "last-change": 871688,
                                    "timest": 1489654386351
                                },
                                {
                                    "description": "",
                                    "timest": 1489654386351
                                },
                                {
                                    "content": [
                                        {
                                            "in-octets": 0,
                                            "timest": 1489654386351
                                        },
                                        {
                                            "out-octets": 0,
                                            "timest": 1489654386351
                                        },
                                        {
                                            "in-multicast-pkts": 0,
                                            "timest": 1489654386351
                                        },
                                        {
                                            "in-broadcast-pkts": 0,
                                            "timest": 1489654386351
                                        },
                                        {
                                            "out-multicast-pkts": 0,
                                            "timest": 1489654386351
                                        },
                                        {
                                            "out-broadcast-pkts": 0,
                                            "timest": 1489654386351
                                        },
                                        {
                                            "in-unknown-protos": 0,
                                            "timest": 1489654386351
                                        },
                                        {
                                            "in-errors": 0,
                                            "timest": 1489654386351
                                        },
                                        {
                                            "out-errors": 0,
                                            "timest": 1489654386351
                                        },
                                        {
                                            "last-clear": "2017-02-27T07:39:11Z",
                                            "timest": 1489654386351
                                        },
                                        {
                                            "in-unicast-pkts": 0,
                                            "timest": 1489654386351
                                        },
                                        {
                                            "in-discards": 0,
                                            "timest": 1489654386351
                                        },
                                        {
                                            "out-unicast-pkts": 0,
                                            "timest": 1489654386351
                                        },
                                        {
                                            "out-discards": 0,
                                            "timest": 1489654386351
                                        }
                                    ],
                                    "timest": 1489654386351
                                }
                            ],
                            "timest": 1489654386351
                        }
                    ],
                    "timest": 1489654386351
                },
                "timest": 1489654386351
            },
            "timest": 1489654386351
        }
    ],
    "encoding_path": "openconfig-interfaces:interfaces/interface",
    "host": "10.44.160.206",
    "msg_timestamp": 1489654386284,
    "node_id_str": "XRv-Telemetry",
    "port": 45019,
    "subscription_id_str": "testTOtest",
    "timest": 1489654386351
}

And I found out error in stack_kafka 's logstash.
I think that the proto file has failed to load.

/data/logstash.log

{:timestamp=>"2017-03-16T08:56:08.626000+0000", :message=>"Failed to load .proto Ruby binding source", :proto_binding_source=>"/data/proto/telemetry_bis.pb.rb", :exception=>#<RuntimeError: Field already exists for tag: 1>, :stacktrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/ruby-protocol-buffers-1.6.1/lib/protocol_buffers/runtime/message.rb:524:in `define_field'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/ruby-protocol-buffers-1.6.1/lib/protocol_buffers/runtime/message.rb:536:in `optional'", "/data/proto/telemetry_bis.pb.rb:15:in `Telemetry'", "/data/proto/telemetry_bis.pb.rb:12:in `(root)'", "org/jruby/RubyKernel.java:1059:in `load'", "/opt/logstash/vendor/local_gems/a404edd4/logstash-codec-telemetry-0.9.0/lib/logstash/codecs/telemetry.rb:1:in `(root)'", "org/jruby/RubyDir.java:252:in `glob'", "/opt/logstash/vendor/local_gems/a404edd4/logstash-codec-telemetry-0.9.0/lib/logstash/codecs/telemetry.rb:614:in `register'", "/opt/logstash/vendor/local_gems/a404edd4/logstash-codec-telemetry-0.9.0/lib/logstash/codecs/telemetry.rb:609:in `register'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/codecs/base.rb:15:in `initialize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/codecs/base.rb:48:in `clone'"], :level=>:warn}

Is there a problem on the XRv side?
runnning config of XRv 6.1.2.

telemetry model-driven
 destination-group test
  address family ipv4 10.44.160.188 port 5556
   encoding self-describing-gpb
   protocol tcp
  !
 !
 sensor-group ospf
  sensor-path Cisco-IOS-XR-ipv4-ospf-oper-sub2:OSPF-SH-INTERFACE-UP/interface-neighbor-count
 !
 sensor-group test
  sensor-path Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters
 !
 sensor-group test2
  sensor-path openconfig-interfaces:interfaces/interface
 !
 sensor-group if-oper
  sensor-path Cisco-IOS-XR-pfi-im-cmd-oper:interfaces/interface-xr/interface
 !
 sensor-group dial-in-test01
  sensor-path openconfig-interfaces:interfaces/interface
 !
 subscription testTOtest
  sensor-group-id test2 sample-interval 10000
  destination-id test
 !
 subscription sub-dial-in-test01
  sensor-group-id dial-in-test01 sample-interval 30000
 !
!

Logstash is exit after a while from restart the stack [tail -f /var/local/stack_elk/logstash_data/logstash.log ] output

{:timestamp=>"2017-03-29T04:23:09.818000+0000", :message=>"Unknown setting 'wire_format' for telemetry", :level=>:error}
{:timestamp=>"2017-03-29T04:23:09.826000+0000", :message=>"fetched an invalid config", :config=>"#\n# cisco telemetry codec\n#\n# This file is staged into the ../staging directory, updated according\n# to setup in ../environment, and copied into the host volume for\n# logstash (under $LOGSTASH_VOLUME/conf.d). File can be edited there\n# but will be overwritten if container is rebuilt.\n#\n#\ninput {\n tcp {\n\tport => 2103\n\tcodec => telemetry {\n\t \n\t mdt => true\n\t ########\n\t ### support for json message\t\n\t xform => "flat" \n\t wire_format => 2\n\t xform_flat_delimeter => ""\n\t xform_flat_keys => [\n\t\t#### format 1\n\t\t'interfacesPre601', 'RootOperInterfaces~(?.)',\n\t\t'ipslastatsPre601', 'RootOperIPSLAOperationData~(?\d+)Statistics',\n\t\t'ipslacommonPre601', 'RootOperIPSLAOperationData(?\d+)Common',\n\t\t'countersPre601', 'RootOperInfraStatistics~(?.)LatestGenericCounters',\n 'dataratesPre601', 'RootOperInfraStatistics(?.)LatestDataRate',\n\t\t'ipaddressPre601', 'RootOperIPV4ARMAddresses~(?.)(?.*)',\n\t\t'labelcontextPre601', 'RootOperMPLS_LSD~(?\d+)',\n\t\t'labelspecialPre601', 'RootOperMPLS_LSDLabel',\n\t\t'fiblabelPre601', 'RootOperFIB_MPLS(?.)LabelFIB(?\d+)(?EOS\d)',\n\t\t'fibprefixv4Pre601', 'RootOperFIB~(?.)IPv4(?.)~(?.)(?\d+)',\n\t\t'fibprefixv6Pre601', 'RootOperFIB~(?.)IPv6(?.)(?.*)(?\d+)',\n\t\t'fibprefixmplsPre601', 'RootOperFIB(?.)MPLS(?.)(?.*)(?\d+)',\n\t\t'ribprefixPre601', 'RootOperRIB(?.)IPv4Unicastdefault(?.)(?\d+)--',\n\t\t'mplstesummaryPre601', 'RootOperMPLS_TETunnelsSummary',\n\t\t'mplstetopologyPre601', 'RootOperMPLS_TETopology',\n\t\t'mplsteautoroutePre601', 'RootOperMPLS_TEAnnounceTunnelsInfoAutorouteAnnounceTable(?.)~(?.)(?\d+)(?.)',\n\n\t\t### format 2 \n\t\t'interfaces', 'RootOperInterfacesInterface~(?.)',\n\t\t'ipslastats', 'RootOperIPSLAOperationDataOperation(?\d+)Statistics',\n\t\t'ipslacommon', 'RootOperIPSLAOperationDataOperation~(?\d+)Common',\n\t\t'counters', 'RootOperInfraStatisticsInterface(?.)LatestGenericCounters',\n 'datarates', 'RootOperInfraStatisticsInterface~(?.)LatestDataRate',\n\t\t'ipaddress', 'RootOperIPV4ARMAddressesVRF(?.)Interfaces(?.)',\n\t\t'labelcontext', 'RootOperMPLS_LSDLabel~(?\d+)',\n\t\t'labelspecial', 'RootOperMPLS_LSDLabel',\n\t\t'fiblabel', 'RootOperFIB_MPLSCard~(?.)LabelFIBLabel~(?\d+)EOS(?EOS\d)',\n\t\t'fibprefixv4', 'RootOperFIBCard~(?.)IPv4Vrf~(?.)Pfx(?.)Len(?\d+)',\n\t\t'fibprefixv6', 'RootOperFIBCard~(?.)IPv6Vrf~(?.)Pfx(?.)Len(?\d+)',\n\t\t'fibprefixmpls', 'RootOperFIBCard~(?.)MPLSVrf~(?.)Pfx(?.)Len(?\d+)',\n\t\t'ribprefix', 'RootOperRIBVrf~(?.)IPv4UnicastdefaultPfx~(?.)Len(?\d+)--',\n\t\t'mplstesummary', 'RootOperMPLS_TETunnelsSummary',\n\t\t'mplstetopology', 'RootOperMPLS_TETopology',\n\t\t'mplsteautoroute', 'RootOperMPLS_TEAnnounceTunnelsInfoAutorouteAnnounceTableRouterID(?.)Protocol(?.)Area(?\d+)Instance(?.*)']\n\t #\n\t ##########\n\t ### support for gpb and gpb_kv\n\t protofiles => "/data/proto"\n\t}\n }\n udp {\n\tport => 2103\n\tcodec => telemetry_gpb {\n\n\t ############\n\t #### support only for gpb\n\t #\n\t # 'protofiles' specified path for directory holding:\n\t #\n\t # .proto files as generated on router, and post-processed\n\t # .pb.rb generated ruby bindings for the samen\n\t #\n\t # e.g. protofiles => "/data/proto"\n\t #\n\t # If you do not plan to make backward incompatible\n # changes to the .proto file, you can also simply use\n # the full version on this side safe in the knowledge\n # that it will be able to read any subset you wish to\n # generate.\n\t #\n\t # In order to generate the Ruby bindings you will need\n\t # to use a protocol compiler which supports Ruby\n\t # bindings for proto2 (e.g. ruby-protocol-buffer gem)\n\t #\n\t protofiles => "/data/proto"\n\t}\n }\n}\n\noutput {\n #\n # elasticsearch options documented here: www.elastic.io. With 2.0,\n # we default to using http, and must therefore use the client\n # port.\n #\n elasticsearch {\n\thosts => "192.168.137.43:9200"\n }\n\n #\n # Uncomment the following if you wish to attach to\n # container and watch updates go by.\n #\n # stdout {codec => rubydebug}\n #\n}\n\n", :reason=>"Something is wrong with your configuration.", :level=>:error}

stack_build with no Internet access?

Trying to run this in an isolated environment, can someone suggest the best way to build the docker images with no external connectivity?

Incompatibility with docker image: prom/pushgateway:v0.4.0

Hi,

I'm unable to get Pipeline to work with v0.4.0 of the prom/pushgateway Docker image. (Pipeline works fine with the older v0.3.1 image.)

When running prom/pushgateway v0.4.0, my pipelinle.log file is filled with the following:

`time="2017-06-15 21:34:54.519836" level=error msg="http reply" error="unexpected status code 400 while pushing to http://localhost:9091/metrics/job/telemetry/instance/pipeline_11823492967441504429" instance=pipeline jobName=telemetry pushGWAddress="http://localhost:9091" pushURL="http://localhost:9091/metrics/job/telemetry/instance/pipeline" tag=pipeline

time="2017-06-15 21:34:54.533393" level=error msg="http reply" error="unexpected status code 400 while pushing to http://localhost:9091/metrics/job/telemetry/instance/pipeline_17704715107158349698" instance=pipeline jobName=telemetry pushGWAddress="http://localhost:9091" pushURL="http://localhost:9091/metrics/job/telemetry/instance/pipeline" tag=pipeline `

Regards,

Michael