googlecloudplatform / ubbagent Goto Github PK
View Code? Open in Web Editor NEWMetering agent
License: Apache License 2.0
Metering agent
License: Apache License 2.0
The 2 python sdks make calls to C.CString(). See https://github.com/GoogleCloudPlatform/ubbagent/blob/master/sdk/python2/api.go and https://github.com/GoogleCloudPlatform/ubbagent/blob/master/sdk/python3/api.go
C.CString() allocates memory in the C heap, but the memory is never freed. https://golang.org/cmd/cgo/#hdr-Go_references_to_C
Can't determine what --v
is or should do, also it was a bit confusing that a single-letter argument was used in long style (--v
rather than -v
).
The main.go
file appears to do argument parsing, but seems to ignore --v
if specified.
In order to add workaround fix for #39
I'm Parsing config with ubbconfig.Parse, then modify it and json.Marshall it back.
This is a modified json
{
"identities": [{
"name": "gcp",
"gcp": {
"serviceAccountKey": null,
"encodedServiceAccountKey": "base64 string"
}
}],
"metrics": [{
"Name": "nodes",
"Type": "int",
"endpoints": [{
"name": "on_disk"
}, {
"name": "servicecontrol"
}],
"aggregation": {
"bufferSeconds": 60
},
"passthrough": null
}],
"endpoints": [{
"name": "on_disk",
"disk": {
"reportDir": "/tmp/ubbagent/reports",
"expireSeconds": 3600
},
"servicecontrol": null,
"pubsub": null
}, {
"name": "servicecontrol",
"disk": null,
"servicecontrol": {
"identity": "gcp",
"serviceName": "someservice",
"consumerId": "project:someproject"
},
"pubsub": null
}],
"sources": null,
"filters": null
}
It looks like ubbagent is still counting feilds woth null value
I encountered an error when I send a message one but I see multiple invocations of the same message in retry.go
the message gets dequeued after the first success, when it attempts to do so in the second attempt the dequeue fails as the message is not in the queue anymore as it was dequeued in the first attempt.
This happens in case of service control endpoint and it is not an expected behaviour
To make things worse ubbagent panics resulting in the system to crash
I believe when this happens it should only log the error and move on
So we have 2 problems at hand that we must discuss:
Command to run
make clean setup deps build
Failure:
go: downloading google.golang.org/protobuf v1.27.1
go get: installing executables with 'go get' in module mode is deprecated.
Use 'go install pkg@version' instead.
For more information, see https://golang.org/doc/go-get-install-deprecation
or run 'go help get' or 'go help install'.
# github.com/golang/dep/gps
.GOPATH/pkg/mod/github.com/golang/[email protected]/gps/constraint.go:103:21: cannot use sv (type *semver.Version) as type semver.Version in field value
.GOPATH/pkg/mod/github.com/golang/[email protected]/gps/constraint.go:122:16: invalid type assertion: c.(semver.Version) (non-interface type *semver.Constraints on left)
.GOPATH/pkg/mod/github.com/golang/[email protected]/gps/constraint.go:149:4: undefined: semver.Constraint
Makefile:127: recipe for target 'setup' failed
make: *** [setup] Error 2
Both Docker and Kubernetes provide simple mechanisms to convey a file (client secret) to a running container:
Using these mechanisms directly saves some processing steps and, I believe, aids clarity.
Would it perhaps even be possible to drop the references to the [consumer project] and robot entirely from the ubbagent config and pass these values using environment variables as is conventionally done with ADCs?
https://cloud.google.com/docs/authentication/production#auth-cloud-implicit-go
{"name":"requests",
"startTime":"2018-03-26T12:39:55",
"endTime":"2018-03-26T12:39:55",
"value":{
"int64Value": 10,
"labels":{"foo":"Bar"}
}
}
Sample input (as t.json
)
Posting this:
$ curl -X post -d @t.json http://localhost:3456/report
parsing time ""2018-03-26T12:39:55"" as ""2006-01-02T15:04:05Z07:00"": cannot parse """ as "Z07:00"/ #
Assuming timezone UTC (Z
) would have made sense, but not sure where the 07:00 offset is coming from, this looks like a bug.
The current implementation of ubbagent is requiring to provide a configuration via JSON config.
Unfortunately, marketplace-k8s-app-tools default deployer is providing encodedServiceAccountKey, consumerId etc as Kubernets secrets.
K8s secrets can be mounted into Pod as File or as ENV vars. To make consumers live easy, it would nice to have another way of providing encodedServiceAccountKey and Ids into ubbagent library.
Something Keys/Secrets files path or knows ENV vars.
Similar to https://github.com/golang/oauth2/blob/ef147856a6ddbb60760db74283d2424e98c87bff/google/default.go#L43
I'm trying to add Google Cloud Marketplace usage reporting to my app and following this doc it says
In the SDK approach, the agent must be compiled or linked into your app binary. The SDK is implemented natively for Go, with bindings for Python.
and I'm struggling to figure out how to use it. Am I supposed to use https://github.com/GoogleCloudPlatform/ubbagent/blob/master/sdk/sdk.go ? There's not much there...
Do you have any examples? I can't find anything. Any help you can provide is greatly appreciated it.
Hi everyone
I am unable to configure my application with the usage-based billing agent. My application runs on GKE.
I have the following questions :
A customer marketplace deployment hit the following issue in ubbagent:
panic: RetryingSender.maybeSend: loading from retry queue: unexpected end of JSON input
goroutine 15 [running]:
vendor/github.com/GoogleCloudPlatform/ubbagent/pipeline/senders.(*RetryingSender).maybeSend(0xc0005806e0, 0xbf97a99c50737bd1, 0x24e27478, 0x48378c0)
/vendor/github.com/GoogleCloudPlatform/ubbagent/pipeline/senders/retry.go:199 +0x7d8
vendor/github.com/GoogleCloudPlatform/ubbagent/pipeline/senders.(*RetryingSender).run(0xc0005806e0, 0xbf97a99c50737bd1, 0x24e27478, 0x48378c0)
/vendor/github.com/GoogleCloudPlatform/ubbagent/pipeline/senders/retry.go:149 +0x6a
created by vendor/github.com/GoogleCloudPlatform/ubbagent/pipeline/senders.newRetryingSender
/vendor/github.com/GoogleCloudPlatform/ubbagent/pipeline/senders/retry.go:88 +0x244
IIUC ubbagent can forward metering metrics to an endpoint for collection.
Currently this endpoint must be configured through Google Service Control.
Google's Service Control content is dated and inaccurate:
https://cloud.google.com/service-control/overview
https://github.com/googleapis/googleapis/tree/master/google/api/servicecontrol
I assumed I could use an arbitrary service (e.g. container.googleapis.com) to get to a working ubbagent
but I'm receiving 403s when I try to use these:
Permission 'servicemanagement.services.check' denied for the consumer project, forbidden
Is there a general-purpose sink that we may use for testing?
If not, am I correct in thinking that I'd need to e.g. deploy a Cloud Endpoints service to point this configuration at?
Just a comment on what I'm seeing but the default behavior of ubbagent-start
is to disable HTTP access unless a port is provided. I would expect port 3456
to be used as the default (or whatever you choose).
Additionally, it's undocumented and unclear what this application should do at all if HTTP is disabled. Suggest defaulting to port 3456 and always enabling HTTP unless explicitly disabled by configuration (in the event that's useful)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.