Coder Social home page Coder Social logo

narmi / cloudwatchmetricbeat Goto Github PK

View Code? Open in Web Editor NEW
17.0 3.0 6.0 34 KB

A beat for AWS CloudWatch Metrics. Located in NYC? We are hiring: https://www.narmi.com/careers/

License: Other

Makefile 8.33% Go 62.84% Python 5.52% Dockerfile 7.12% HTML 16.19%
cloudwatch-metrics metricbeat elastic elasticsearch beats go

cloudwatchmetricbeat's People

Contributors

chris-griffin avatar jasonlang80 avatar phillbaker avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

cloudwatchmetricbeat's Issues

Cloudwatchmetricbeat stoppes after first request without any errors

Could you please help me on this. Cannot use beat properly. It fails after first request.
Thanks in advance,
Leonid

root@elk:~/go/src/github.com/narmitech/cloudwatchmetricbeat# cloudwatchmetricbeat -c cloudwatchmetricbeat.yml -e -d "*"
2018/05/14 14:26:16.042086 beat.go:285: INFO Home path: [/root/go/src/github.com/narmitech/cloudwatchmetricbeat] Config path: [/root/go/src/github.com/narmitech/cloudwatchmetricbeat] Data path: [/root/go/src/github.com/narmitech/cloudwatchmetricbeat/data] Logs path: [/root/go/src/github.com/narmitech/cloudwatchmetricbeat/logs]
2018/05/14 14:26:16.044414 beat.go:186: INFO Setup Beat: cloudwatchmetricbeat; Version: 5.3.4
2018/05/14 14:26:16.044429 processor.go:44: DBG  Processors:
2018/05/14 14:26:16.044442 beat.go:192: DBG  Initializing output plugins
2018/05/14 14:26:16.044509 output.go:254: INFO Loading template enabled. Reading template file: /root/go/src/github.com/narmitech/cloudwatchmetricbeat/cloudwatchmetricbeat.template.json
2018/05/14 14:26:16.044650 output.go:265: INFO Loading template enabled for Elasticsearch 2.x. Reading template file: /root/go/src/github.com/narmitech/cloudwatchmetricbeat/cloudwatchmetricbeat.template-es2x.json
2018/05/14 14:26:16.044796 client.go:123: INFO Elasticsearch url: http://localhost:9200
2018/05/14 14:26:16.044828 outputs.go:108: INFO Activated elasticsearch as output plugin.
2018/05/14 14:26:16.044848 publish.go:238: DBG  Create output worker
2018/05/14 14:26:16.044886 publish.go:280: DBG  No output is defined to store the topology. The server fields might not be filled.
2018/05/14 14:26:16.044909 publish.go:295: INFO Publisher name: elk
2018/05/14 14:26:16.045031 async.go:63: INFO Flush Interval set to: 1s
2018/05/14 14:26:16.045050 async.go:64: INFO Max Bulk Size set to: 50
2018/05/14 14:26:16.045058 async.go:72: DBG  create bulk processing worker (interval=1s, bulk size=50)
2018/05/14 14:26:16.045268 cloudwatchmetricbeat.go:44: INFO settings: config={Period:1m0s AWSRegion:us-east-1 Prospectors:[{Id:billing Metrics:[{AWSStatistics:[Maximum] AWSDimensions:[Currency] AWSDimensionSelect:map[Currency:[USD]] AWSDimensionsSelectParam:map[] AWSNamespace:AWS/Billing AWSMetricName:EstimatedCharges RangeSeconds:86400 PeriodSeconds:0 DelaySeconds:0}]} {Id:ec2_cloudwatch Metrics:[{AWSStatistics:[Average] AWSDimensions:[InstanceId] AWSDimensionSelect:map[InstanceId:[id-09005aa854cb13b23]] AWSDimensionsSelectParam:map[] AWSNamespace:AWS/EC2 AWSMetricName:CPUUtilization RangeSeconds:0 PeriodSeconds:0 DelaySeconds:0} {AWSStatistics:[Average] AWSDimensions:[InstanceId] AWSDimensionSelect:map[InstanceId:[id-09005aa854cb13b23]] AWSDimensionsSelectParam:map[] AWSNamespace:AWS/EC2 AWSMetricName:NetworkOut RangeSeconds:0 PeriodSeconds:0 DelaySeconds:0}]} {Id:vpn_mon Metrics:[{AWSStatistics:[Average] AWSDimensions:[VpnId] AWSDimensionSelect:map[VpnId:[vpn-61687a11]] AWSDimensionsSelectParam:map[] AWSNamespace:AWS/VPN AWSMetricName:TunnelState RangeSeconds:3600 PeriodSeconds:0 DelaySeconds:0}]}]}
2018/05/14 14:26:16.045381 beat.go:221: INFO cloudwatchmetricbeat start running.
2018/05/14 14:26:16.045392 cloudwatchmetricbeat.go:63: INFO cloudwatchmetricbeat is running! Hit CTRL-C to stop it.
2018/05/14 14:26:16.045424 metrics.go:23: INFO Metrics logging every 30s
2018/05/14 14:26:46.045680 metrics.go:34: INFO No non-zero metrics in the last 30s
2018/05/14 14:27:16.045624 metrics.go:34: INFO No non-zero metrics in the last 30s
2018/05/14 14:27:16.175219 cloudwatchmetricbeat.go:194: INFO making request: params={
  Dimensions: [{
      Name: "VpnId",
      Value: "vpn-61687a11"
    }],
  EndTime: 2018-05-14 14:22:16.045654982 +0000 UTC,
  MetricName: "TunnelState",
  Namespace: "AWS/VPN",
  Period: 60,
  StartTime: 2018-05-14 13:22:16.045654982 +0000 UTC,
  Statistics: ["Average"]
}
2018/05/14 14:27:16.175281 metrics.go:51: INFO Total non-zero values:
2018/05/14 14:27:16.175293 metrics.go:52: INFO Uptime: 1m0.136432267s
2018/05/14 14:27:16.175301 beat.go:225: INFO cloudwatchmetricbeat stopped.

instillation steps still failing.

I think the code has a bug:
$:/go/src/github.com/narmitech/cloudwatchmetricbeat$ pwd
/home/jasonlang/go/src/github.com/narmitech/cloudwatchmetricbeat
$:
/go/src/github.com/narmitech/cloudwatchmetricbeat$ make
go build -i -ldflags "-X github.com/elastic/beats/libbeat/version.buildTime=2018-06-20T17:26:32Z -X github.com/elastic/beats/libbeat/version.commit=fbf0b84fb01fd271b26122dee758166b20e226ca"

github.com/narmitech/cloudwatchmetricbeat/beater

beater/cloudwatchmetricbeat.go:25:9: undefined: publisher.Client
vendor/github.com/elastic/beats/libbeat/scripts/Makefile:112: recipe for target 'cloudwatchmetricbeat' failed
make: *** [cloudwatchmetricbeat] Error 2

non recursive COPY in Dockerfile

I'm running Docker version 18.06.1-ce, build e68fc7a. When running the Dockerfile, the following copy statement isn't recursive, and appears to only transfer the Dockerfile itself albeit to the correct directory.

COPY . $GOPATH/src/github.com/narmitech/cloudwatchmetricbeat

This is most likely a Docker bug I'd bet, but just to be aware there's a problem right now.

AWS/RDS not using correct dimensions

Something Odd is going on when I attempt to use the beat to pull RDS metrics from cloudwatch. It's not using the correct Dimension.
Config:
prospectors: - id: rds-metrics metrics: - aws_namespace: "AWS/RDS" aws_dimensions: [DBClusterIdentifier] aws_dimensions_select: InstanceId: [aurora1-cluster] aws_metric_name: CPUUtilization aws_statistics: [Average]

Log:
2018-07-31T17:42:28Z INFO making request: params={ Dimensions: [{ Name: "InstanceId", Value: "aurora1-cluster" }], EndTime: 2018-07-31 17:37:28.871505767 +0000 UTC m=-119.993549520, MetricName: "CPUUtilization", Namespace: "AWS/RDS", Period: 60, StartTime: 2018-07-31 17:27:28.871505767 +0000 UTC m=-719.993549520, Statistics: ["Average"]

installation steps

can you please installation steps.. clearly...

i am getting below error..
path :- /tmp/cloudwatchmetricbeat/beater
=> go build
# _/tmp/cloudwatchmetricbeat/beater
Error:-
./cloudwatchmetricbeat.go:25:9: undefined: publisher.Client

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.