karimra / gnmic Goto Github PK

Print error if !=nil in prompt mode when --log or --debug are not set

Set log flag to true in case no log-file is specified and debug is true.
Meaning --debug implies an implicit --log if there is no --log-file

Add link to the documentation site (http://gnmiclient.kmrd.dev/) in the gnmiClient version output

it is considered to be a bad practice for a tool binary name to have characters which require shift press

background: https://smallstep.com/blog/the-poetics-of-cli-command-names/

In order to not clutter the output it might be feasible to change --nolog flag to --logstdout once #71 is done

I've introduced staticcheck in Testing pipeline; it found some issues - https://github.com/karimra/gnmiClient/runs/789172646?check_suite_focus=true

• cmd/get.go:148:6: func stringInList is unused (U1000)
• cmd/root.go:82:4: grpclog.SetLogger is deprecated: use SetLoggerV2. (SA1019)
• cmd/path.go:58:28: should use make(chan string) instead (S1019)
• cmd/root.go:186:22: grpc.WithTimeout is deprecated: use DialContext instead of Dial and context.WithTimeout instead. - [ ] Will be supported throughout 1.x. (SA1019)
• cmd/root.go:368:66: syscall.SIGKILL cannot be trapped (did you mean syscall.SIGTERM?) (SA1016)
• cmd/root.go:418:9: assigning the result of this type assertion to a variable (switch targetsInt := targetsInt.(type)) could eliminate type assertions in switch cases (S1034)
• cmd/root.go:420:38: could eliminate this type assertion
• cmd/root.go:424:16: could eliminate this type assertion
• cmd/root.go:446:10: assigning the result of this type assertion to a variable (switch t := t.(type)) could eliminate type assertions in switch cases (S1034)
• cmd/root.go:448:30: could eliminate this type assertion
• cmd/set.go:189:16: should use String() instead of fmt.Sprintf (S1025)
• cmd/set.go:196:16: should use String() instead of fmt.Sprintf (S1025)
• cmd/subscribe.go:107:7: ineffective break statement. Did you mean to break out of the outer loop? (SA4011)
• cmd/version.go:41:3: redundant return statement (S1023)

As dockerhub now put on some restrictions on free tier accounts (images which are not pulled/pushed in 6mo will be deleted) the proposal suggests to mirror the images to the newly launched github container registry which stays free for public projects.

This can be easily done with goreleaser by adding a fqdn with registry address as explained here

--path flag should be marked mandatory to prevent full datastore retrieval if the flag is omitted (can happen if the path is considered to be an argument)

Recently we added a requirement condition for path cli flags to avoid getting entire state when this flag is missing on the CLI.

But that has a side effect of not being able to set paths in a config file:

❯ go run . --config ~/.gnmic.yml get
Error: required flag(s) "path" not set

I propose we remove the required condition on the flag and instead check the value of the paths := viper.GetStringSlice("get-path") for emptiness

currently if the file is not present by a referenced path, the empty set request is crafted and an empty struct is printed:

❯ go run . -a 0.tcp.eu.ngrok.io:4444 --insecure -u admin -p admin --config /dev/null set --update-path "/configure/port[port-id=1/1/c1/1]/description" --update-file port_descr.json
gnmic 2020/07/08 11:55:17.985379 error reading data from file 'port_descr.json': open port_descr.json: no such file or directory
gnmic 2020/07/08 11:55:18.148917 sending gNMI SetRequest: prefix='', delete='[]', replace='[]', update='[]', extension='[]' to 0.tcp.eu.ngrok.io:4444
{
  "source": "0.tcp.eu.ngrok.io:14852",
  "time": "1970-01-01T01:00:00+01:00"
}

Get cmd panics in case there is both nil and non nil update values in the response

an empty value passed to prometheus-address is a workaround, but this can be made a default value as well as treated gracefully (without logging a missing address as an error)

subscribe interval is not set when the subs is defined in a file and not from CLI:

example of a request sent via file:

gnmic 2020/07/11 21:38:04.395276 sending gNMI SubscribeRequest: subscribe='subscribe:{prefix:{}  subscription:{path:{elem:{name:"state"}  elem:{name:"port"  key:{key:"port-id"  value:"1/1/c1/1"}}  elem:{name:"statistics"}  elem:{name:"out-octets"}}}  qos:{marking:20}}', mode='STREAM', encoding='JSON', to 0.tcp.eu.ngrok.io:x

the same sub request made via CLI:

gnmic 2020/07/11 21:37:08.030590 sending gNMI SubscribeRequest: subscribe='subscribe:{prefix:{}  subscription:{path:{elem:{name:"state"}  elem:{name:"port"  key:{key:"port-id"  value:"1/1/c1/1"}}  elem:{name:"statistics"}  elem:{name:"out-octets"}}  sample_interval:10000000000}  qos:{marking:20}}', mode='STREAM', encoding='JSON', to 0.tcp.eu.ngrok.io:x

when in prompt mode, add auto completion for flags with enum values like:
--encoding [json, bytes, proto, ascii, json_ietf],
--formt [protojson, prototext, json, event, proto]
get --type [all, config, state, operational]
subscribe --mode [once, stream, poll]
subscribe --stream-mode [on-change, sample, target-defined]

I try to gnmic client to update some values on SRLinux platform, which supports json_ietf encoding.

Here is a simple example with hostname change:

gnmic --config gnmiClient_srl.yaml set --update-path /system/name/host-name --update-value new_hostname

Debug message:

sending gNMI SetRequest: prefix='<nil>', delete='[]', replace='[]', update='[path:{elem:{name:"system"} elem:{name:"name"} elem:{name:"host-name"}} val:{json_val:"\"new_hostname\""}]', extension='[]' to 192.168.1.1:57400

error sending set request: failed sending SetRequest to '192.168.1.1:57400': rpc error: code = InvalidArgument desc = Field type (0) is not supported

If i try to change hostname using another method, everything is successfully changed
Method-1:

gnmic --config gnmiClient_srl.yaml set --update /system/name/host-name:::json_ietf:::IXR-6-4

Method-2

gnmic --config gnmiClient_srl.yaml set --update-path / --update-file change_name.json

Complete log for all combinations
log.txt

Currently only the path information is output to the log:

2020/06/03 21:07:55.469675 sending gnmi GetRequest 'path:{elem:{name:"state"} elem:{name:"system"} elem:{name:"version"}}' to 10.2.0.11:57400

It seems valuable to display the missing elements like DataType, Encoding, etc

• file
• nats
• stan
• kafka
• udp
• tcp
• influxdb

Support subscribing using the cli mode with named subscriptions already defined in a config file.
e.g: gnmic subscribe --name sub1

Currently the installation script has the following limitations:

• always download the latest version
• cant be told to download a specific version

The proposal is to base the installer script either on the Helm installer or leverage godownloader

ModelName is a repeated object in GetResponse and Subscribe. Long term plan should be to support multiple names (comma separated for instance)

I think this can be just a return of fmt.Errorf. no need to assign a new var

Originally posted by @karimra in #136 (comment)

Sometime ONCE response is not printed out to stdout:

The output shows two consecutive runs:

❯ go run . -a r1 --insecure -u admin -p admin --config /dev/null sub --path /state/system/version --mode ONCE
file_output 2020/07/07 15:42:45.454581 initialized file output: {"Cfg":{"FileName":"","FileType":"stdout"}}
gnmic 2020/07/07 15:42:45.553653 target '0.tcp.eu.ngrok.io:14852' initialized
gnmic 2020/07/07 15:42:45.555383 sending gNMI SubscribeRequest: subscribe='subscribe:{prefix:{}  subscription:{path:{elem:{name:"state"}  elem:{name:"system"}  elem:{name:"version"}}}  qos:{marking:20}  mode:ONCE}', mode='ONCE', encoding='JSON', to 0.tcp.eu.ngrok.io:14852
{
  "source": "0.tcp.eu.ngrok.io:14852",
  "subscription-name": "default",
  "timestamp": 1594129419152003783,
  "time": "2020-07-07T15:43:39.152003783+02:00",
  "prefix": "state/system/version",
  "updates": [
    {
      "Path": "version-number",
      "values": {
        "version-number": "B-20.5.R1"
      }
    },
    {
      "Path": "version-string",
      "values": {
        "version-string": "TiMOS-B-20.5.R1 both/x86_64 Nokia 7750 SR Copyright (c) 2000-2020 Nokia.\r\nAll rights reserved. All use subject to applicable license agreements.\r\nBuilt on Wed May 13 14:08:50 PDT 2020 by builder in /builds/c/205B/R1/panos/main/sros"
      }
    }
  ]
}
file_output 2020/07/07 15:42:45.753384 closing file '/dev/stdout' output



❯ go run . -a r2 --insecure -u admin -p admin --config /dev/null sub --path /state/system/version --mode ONCE
file_output 2020/07/07 15:42:49.217199 initialized file output: {"Cfg":{"FileName":"","FileType":"stdout"}}
gnmic 2020/07/07 15:42:49.352015 target '0.tcp.eu.ngrok.io:14852' initialized
gnmic 2020/07/07 15:42:49.353721 sending gNMI SubscribeRequest: subscribe='subscribe:{prefix:{}  subscription:{path:{elem:{name:"state"}  elem:{name:"system"}  elem:{name:"version"}}}  qos:{marking:20}  mode:ONCE}', mode='ONCE', encoding='JSON', to 0.tcp.eu.ngrok.io:14852
file_output 2020/07/07 15:42:49.528603 closing file '/dev/stdout' output

If make this a regular method call iso of goroutine, I do not see the race condition:
https://github.com/karimra/gnmic/blob/collector/collector/collector.go#L185

How crucial is it to have it a goroutine here? Since the o.Write() inside of t.Export is async anyway?

1. mode in subscription struct vs subscription-mode in cmd local flags
2. stream-mode in subscription struct vs stream-subscription-mode in cmd local flags

Consider instrumenting gnmiClient using prometheus

• current 30s default value is a bit high, consider changing to 10s.
• change the flag type from StringP to DurationP

Add output options like:

• file
• kafka
• nats
• stan
• influxdb
• prometheus

It seems that having the internal type of the value does not benefit the user. Any reasons to keep it?

In the output below I am referring to (*gnmi.TypedValue_JsonVal)

root@eve-ng:~# gnmiClient -a 10.1.0.11:57400 -u admin -p admin --insecure get --path /state/system/platform
timestamp: 1587670793967241885
prefix:
alias:
state/system/platform: (*gnmi.TypedValue_JsonVal) "7750 SR-1s"

inline update without type specified in the --update-path should be a valid case, yet its not

$ go run . -a 10.0.0.10:123 -u admin -p admin --debug set --update-path /test --update-value val1
2020/04/23 22:47:45 deletes(0)=[]
2020/04/23 22:47:45 updates(0)=
2020/04/23 22:47:45 replaces(0)=
2020/04/23 22:47:45 delimiter=:::
2020/04/23 22:47:45 updates-paths(1)=[/test]
2020/04/23 22:47:45 replaces-paths(0)=[]
2020/04/23 22:47:45 updates-files(0)=[]
2020/04/23 22:47:45 replaces-files(0)=[]
2020/04/23 22:47:45 updates-values(1)=[val1]
2020/04/23 22:47:45 replaces-values(0)=[]
Error: unknown type '', must be one of: [json json_ietf string int uint bool decimal float bytes ascii]

Implement udp output option, to be used primarily with ELK

Proposal:

Make timestamp info more readable for CLI users by either

1. display the timestamp info both in nanoseconds and a datetime by default
2. implement a flag that will enable this behavior

related to #48

to display the line of code where the log happenede

TBD if its needed only with debug flag or not

• Add to the project description "gNMI CLI client and collector"
• Add site reference to gnmic.kmrd.dev

As per https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-specification.md#2221-path-target
Add a target flag to get, set and subscribe commands to support the target field in requests

In current master the subscriptions made towards the node only print the first subscription (default stdout output)
code stops on receiving the second subscription request

Currently the set command if provided with --update-file or --replace-file creates the set Values as JsonVal.
Using [-e | --encoding] it's possible to control the value type, jsonVal vs jsonIetfVal

currently when --insecure is not set and the remote system doesn't have the TLS certificate the client silently exits:

gnmiClient -a 10.1.0.11:57400 -u admin -p admin get --path /state/system/version

instead a meaningful error should be emitted

If a user chooses to use multiple paths by comma separating them like

gnmiClient -a 10.2.0.11 -u admin -p nokiasr0s --insecure get \
    --path "/state/system/version,\
           /state/system/version/version-number"

the the code should strip the whitespaces after splitting by comma, otherwise the full state is retrieved

It should be possible to use nfpm and gorelaser in gh actions to add a .rpm and a .deb files to releases

• Make grpc ignore env proxies by default
• Can be activated via a bool flag

The proposal is to publish a docker image with gnmic as part of the release pipeline

As seen with IOS XR:

*gnmi.TypedValue : json_ietf_val:""
jsonData: []
gnmic 2020/07/10 17:17:45.420801 /home/karim/github.com/karimra/gnmic/cmd/get.go:124: unexpected end of JSON input

This issue tracks the progress on switching to a logger facility that can satisfy the requirements set by SetLoggerV2.

The staticcheck linter will be disabled for https://github.com/karimra/gnmiClient/blob/master/cmd/root.go#L82

This works:
gnmic -a 172.16.18.4:57400 -u svautour -p xxxxx --skip-verify --mode STREAM --stream-mode ON_CHANGE sub --path /state/router[router-name=Base]/interface[interface-name=*]/if-oper-status

This works:
gnmic -a 172.16.18.9:57400 -u svautour -p xxxx --skip-verify --encoding JSON_IETF --log --timeout 30s get --path "openconfig-interfaces:interfaces/interface"

This works:
gnmic -a 172.16.18.9:57400 -u svautour -p xxx --skip-verify --encoding JSON_IETF --log --timeout 30s --model openconfig-interfaces get --path "interfaces/interface"

This however does not work:

svautour@covengcore2:~$ gnmic -a 172.16.18.9:57400 -u svautour -p xxx --skip-verify --encoding JSON_IETF --log --timeout 30s --model openconfig-interfaces get --path "interfaces/interface[name=Loopback0]"
gnmic 2020/08/06 17:00:17.026960 sending gNMI GetRequest: prefix='', path='[elem:{name:"interfaces"} elem:{name:"interface" key:{key:"name" value:"Loopback0"}}]', type='ALL', encoding='JSON_IETF', models='[name:"openconfig-interfaces" organization:"OpenConfig working group" version:"1.0.2"]', extension='[]' to 172.16.18.9:57400
gnmic 2020/08/06 17:00:17.044347 failed sending GetRequest to 172.16.18.9:57400: failed sending GetRequest to '172.16.18.9:57400': rpc error: code = Internal desc = gNMI get-request: rpc error: code = Internal desc = lexical error: invalid char in json text.
{"openconfig-interfaces:interf
(right here) ------^

When I pull back all interfaces I get:

---snip---
"interfaces/interface": [
{
"config": {
"description": "this is lo0 intf",
"enabled": true,
"name": "Loopback0",
"type": "iana-if-type:softwareLoopback"
},
"name": "Loopback0",
"state": {
---snip---

I should be able to filter on the name key.