qmsk / snmpbot Goto Github PK
View Code? Open in Web Editor NEWGolang SNMP library + SNMP REST API
License: Mozilla Public License 2.0
Golang SNMP library + SNMP REST API
License: Mozilla Public License 2.0
It seems upon reaching INT_MAX, queries begin returning Error 500.
Log message:
Feb 18 02:14:33 monni snmpbot[28606]: WARN client: Engine<[::]:43176>: Unknown request [email protected]:161[-2147421757] recv
Referring to earlier IRC conversation, possible fix would be to apply modulo INT_MAX at
Line 58 in 8ac389a
Some common MIBs like SNMPv2-MIB
and IF-MIB
contain objects in multiple different OID trees. While the top-level OID for the SNMPv2-MIB itself is .1.3.6.1.6.3.1
, most of the interesting objects are actually under the legacy .1.3.6.1.2.1.1
system
group, and others under the 1.3.6.1.2.1.11
snmp
groups: http://www.oidview.com/mibs/0/SNMPv2-MIB.html
Probing for the .1.3.6.1.6.3.1
OID will fail if the system only supports the older system
objects, and does not support any of the newer objects defined under the MIB OID.
This affects Linux snmpd, and breaks API queries like GET /api/hosts/test/objects/?object=SNMPv2-MIB::*
- although note that GET /api/hosts/test/objects/SNMPv2-MIB::sysDescr
still works fine, since that isn't limited to probed MIBs.
Seems like some version of go added panics for t.Logf(...)
calls after test completion. This happens for our withTestClient
-> go engine.Run()
-> engine.teardown
as triggerd by the defer engine.Close()
not waiting for the engine to actually close.
panic: Log in goroutine after TestWalkBulk has completed
goroutine 120 [running]:
testing.(*common).logDepth(0xc00042c300, 0xc00009f020, 0x23, 0x3)
/home/terom/opt/go-1.15/src/testing/testing.go:738 +0x59f
testing.(*common).log(...)
/home/terom/opt/go-1.15/src/testing/testing.go:720
testing.(*common).Logf(0xc00042c300, 0xc00009eff0, 0x23, 0x0, 0x0, 0x0)
/home/terom/opt/go-1.15/src/testing/testing.go:766 +0x7e
github.com/qmsk/go-logging.testLogger.Printf(...)
/home/terom/go/pkg/mod/github.com/qmsk/[email protected]/testing.go:13
github.com/qmsk/go-logging.Logging.Debugf(...)
/home/terom/go/pkg/mod/github.com/qmsk/[email protected]/logging.go:13
github.com/qmsk/go-logging.PrefixLogging.Debugf(...)
/home/terom/go/pkg/mod/github.com/qmsk/[email protected]/prefix_logging.go:17
github.com/qmsk/snmpbot/client.(*Engine).teardown(0xc0004463c0)
/home/terom/go/src/github.com/qmsk/snmpbot/client/engine.go:68 +0x21c
github.com/qmsk/snmpbot/client.(*Engine).run(0xc0004463c0, 0x69bc40, 0xc000098030)
/home/terom/go/src/github.com/qmsk/snmpbot/client/engine.go:192 +0x2a9
github.com/qmsk/snmpbot/client.(*Engine).Run(0xc0004463c0, 0xc0002b22a0, 0xc0004463c0)
/home/terom/go/src/github.com/qmsk/snmpbot/client/engine.go:212 +0x128
created by github.com/qmsk/snmpbot/client.withTestClient
/home/terom/go/src/github.com/qmsk/snmpbot/client/client_test.go:37 +0x8e
FAIL github.com/qmsk/snmpbot/client 0.168s
The table walking ATM assumes that each GETNEXT
request returns a complete row, i.e. each entry object in the response has the same index suffix. This assumption isn't necessarily valid, there can be "holes in tables" where some column objects are missing entries:
http://net-snmp.sourceforge.net/wiki/index.php/TUT:snmpgetnext#Multiple_Values
One thing to be aware of is that there may potentially be "holes" in the table, where a particular column does not have a value in every row. The GETNEXT request will step over these missing values, and return the value of that column from the next row where it does exist. The next iteration would need to adjust to this if the walk is to keep stepping through a row at a time:
http://net-snmp.sourceforge.net/wiki/index.php/TUT:snmptable#Holes_in_Tables
The snmpgetnext tutorial includes a discussion of the idea of "holes" in a table, illustrated by an (artificially) missing value for sysORDescr.4. The snmptable command will handle such holes automatically, filling in any such missing values:
Per #17 these currently result in Mismatching VarBind[%v] OID for %v: index %v != expected %v
errors for the sparse columns, and the sparse column values after the first hole will be missing (null
in the API JSON).
It seems like the table-walking logic needs to pick the minimum index from the response, and also use that index for the other objects in the following GETNEXT
request?
New in Go 1.10: https://beta.golang.org/pkg/encoding/asn1/#MarshalWithParams
Should simplify snmp.pack
Currently MIBs have to be writte out by hand as importable Go packages:
This is painful, but so is complete SNMP/SMI/ASN.1 MBI parsing...
For GET /api/hosts/test/objects/?object=x
(or something equivalent like an object in a MIB that failed to probe):
INFO server: Query objects {} @ {test}
SIGABRT: abort
...
goroutine 9 [chan receive]:
github.com/qmsk/snmpbot/server.(*objectsHandler).query(0xc420332780, 0xc420330380, 0x1, 0x1, 0xc42032e270, 0xc42006e960)
/home/terom/go/src/github.com/qmsk/snmpbot/server/object.go:262 +0x589
github.com/qmsk/snmpbot/server.(*objectsHandler).GetREST(0xc420332780, 0x71b140, 0xc420332780, 0x7f924cae65a8, 0xc420332780)
/home/terom/go/src/github.com/qmsk/snmpbot/server/object.go:293 +0x11d
github.com/qmsk/go-web.API.handle(0x726560, 0xc4201ec300, 0x8fee00, 0xc420110a80, 0xc4200a0b00, 0xc420327bf0, 0x411bb7)
/home/terom/go/src/github.com/qmsk/go-web/rest.go:232 +0x736
github.com/qmsk/go-web.API.ServeHTTP(0x726560, 0xc4201ec300, 0x8fee00, 0xc420110a80, 0xc4200a0b00)
/home/terom/go/src/github.com/qmsk/go-web/rest.go:315 +0x74
github.com/qmsk/go-web.(*API).ServeHTTP(0xc4200441c0, 0x8fee00, 0xc420110a80, 0xc4200a0b00)
<autogenerated>:1 +0x63
net/http.StripPrefix.func1(0x8fee00, 0xc420110a80, 0xc4200a0a00)
/home/terom/opt/go-1.9.2/src/net/http/server.go:1957 +0x1a5
net/http.HandlerFunc.ServeHTTP(0xc42026cb70, 0x8fee00, 0xc420110a80, 0xc4200a0a00)
/home/terom/opt/go-1.9.2/src/net/http/server.go:1918 +0x44
net/http.(*ServeMux).ServeHTTP(0xc42026cba0, 0x8fee00, 0xc420110a80, 0xc4200a0a00)
/home/terom/opt/go-1.9.2/src/net/http/server.go:2254 +0x130
net/http.serverHandler.ServeHTTP(0xc4203cc680, 0x8fee00, 0xc420110a80, 0xc4200a0a00)
/home/terom/opt/go-1.9.2/src/net/http/server.go:2619 +0xb4
net/http.(*conn).serve(0xc4204be000, 0x8ff440, 0xc4202fc000)
/home/terom/opt/go-1.9.2/src/net/http/server.go:1801 +0x71d
created by net/http.(*Server).Serve
/home/terom/opt/go-1.9.2/src/net/http/server.go:2720 +0x288
Per some stack traces, the query goroutine is stuck in client:Client.walkGetBulk
:
goroutine 36 [runnable]:
github.com/qmsk/snmpbot/client.walkScalarVars(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/home/terom/go/src/github.com/qmsk/snmpbot/client/walk.go:7 +0x3a7
github.com/qmsk/snmpbot/client.(*Client).walkGetBulk(0xc420096370, 0x0, 0x0, 0x0, 0x952d50, 0x0, 0x0, 0xc420041e80, 0x0, 0x0)
/home/terom/go/src/github.com/qmsk/snmpbot/client/walk.go:117 +0x383
github.com/qmsk/snmpbot/client.(*Client).WalkWithScalars(0xc420096370, 0x0, 0x0, 0x0, 0x952d50, 0x0, 0x0, 0xc42002ae80, 0x0, 0x730de0)
/home/terom/go/src/github.com/qmsk/snmpbot/client/walk.go:62 +0x108
github.com/qmsk/snmpbot/client.(*Client).Walk(0xc420096370, 0x952d50, 0x0, 0x0, 0xc42002af00, 0x0, 0x0)
/home/terom/go/src/github.com/qmsk/snmpbot/client/walk.go:135 +0x91
github.com/qmsk/snmpbot/mibs.Client.WalkObjects(0xc420096370, 0x952d50, 0x0, 0x0, 0xc42002af78, 0xc420312180, 0x933800)
/home/terom/go/src/github.com/qmsk/snmpbot/mibs/client.go:43 +0x19e
github.com/qmsk/snmpbot/server.(*ObjectQuery).queryHost(0xc42032e2a0, 0xc4200962c0, 0xc42032e2b8, 0x775001)
/home/terom/go/src/github.com/qmsk/snmpbot/server/query.go:42 +0x9a
github.com/qmsk/snmpbot/server.(*ObjectQuery).query.func1(0xc42032e2a0, 0xc4200962c0)
/home/terom/go/src/github.com/qmsk/snmpbot/server/query.go:66 +0x65
created by github.com/qmsk/snmpbot/server.(*ObjectQuery).query
/home/terom/go/src/github.com/qmsk/snmpbot/server/query.go:63 +0x105
goroutine 20 [runnable]:
github.com/qmsk/snmpbot/client.makeBulkVars(0x0, 0x0, 0x0, 0x952d50, 0x0, 0x0, 0x952d50, 0x0, 0x0)
/home/terom/go/src/github.com/qmsk/snmpbot/client/client.go:182 +0x60
github.com/qmsk/snmpbot/client.(*Client).GetBulk(0xc4200a4370, 0x0, 0x0, 0x0, 0x952d50, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/home/terom/go/src/github.com/qmsk/snmpbot/client/client.go:221 +0xb6
github.com/qmsk/snmpbot/client.(*Client).walkGetBulk(0xc4200a4370, 0x0, 0x0, 0x0, 0x952d50, 0x0, 0x0, 0xc420041e80, 0x0, 0x0)
/home/terom/go/src/github.com/qmsk/snmpbot/client/walk.go:112 +0x2f9
github.com/qmsk/snmpbot/client.(*Client).WalkWithScalars(0xc4200a4370, 0x0, 0x0, 0x0, 0x952d50, 0x0, 0x0, 0xc42002be80, 0x0, 0x730de0)
/home/terom/go/src/github.com/qmsk/snmpbot/client/walk.go:62 +0x108
github.com/qmsk/snmpbot/client.(*Client).Walk(0xc4200a4370, 0x952d50, 0x0, 0x0, 0xc42002bf00, 0x0, 0x0)
/home/terom/go/src/github.com/qmsk/snmpbot/client/walk.go:135 +0x91
github.com/qmsk/snmpbot/mibs.Client.WalkObjects(0xc4200a4370, 0x952d50, 0x0, 0x0, 0xc42002bf78, 0xc4201325c0, 0x0)
/home/terom/go/src/github.com/qmsk/snmpbot/mibs/client.go:43 +0x19e
github.com/qmsk/snmpbot/server.(*ObjectQuery).queryHost(0xc4203d2bd0, 0xc4200a42c0, 0xc4203d2be8, 0x0)
/home/terom/go/src/github.com/qmsk/snmpbot/server/query.go:42 +0x9a
github.com/qmsk/snmpbot/server.(*ObjectQuery).query.func1(0xc4203d2bd0, 0xc4200a42c0)
/home/terom/go/src/github.com/qmsk/snmpbot/server/query.go:66 +0x65
created by github.com/qmsk/snmpbot/server.(*ObjectQuery).query
/home/terom/go/src/github.com/qmsk/snmpbot/server/query.go:63 +0x105
The will only probe each host once when loading a new host. The probing determines the supported host MIBs and the Online state, so if the probe fails, then the offline host will not be included in any /api/objects/...
queries, and the empty mibs will result in an empty /api/hosts/.../objects/
query set. Conversely, if the probing succeeds but the host later goes offline, then any /api/objects/...
queries will get delayed waiting for that host to timeout.
The host online states should be polled and updated at runtime to avoid those problems.
Workaround: Use PUT /api/hosts/...
to re-load the host.
Going a bit further for the long term, the host mibs might also need to get re-probed if the device is upgraded... The hostname might also need to get re-resolved if the device moves? Alternatively, just require the server to be restarted in those cases...
If mibs:Client.WalkTable()
=> client:Client.Walk()
returns snmp.EndOfMibView
for any table entry object, then the entire table walk fails. It should probably return some invalid value for that column instead.
$ ~/go/bin/snmptable public@edgeswitch-098730 IF-MIB::ifTable
2018/03/04 19:18:15 Invalid VarBind[.1.3.6.1.2.1.2.2.1.15] Value for IF-MIB::ifInUnknownProtos: SNMP VarBind Error: EndOfMibView
$ ~/go/bin/snmpwalk -debug public@edgeswitch-098730 IF-MIB::ifInUnknownProtos
DEBUG client: Engine<[::]:51693>: Run...
DEBUG client: Engine<[::]:51693>: Send: client.IO{Addr:(*net.UDPAddr)(0xc420396180), Packet:snmp.Packet{Version:1, Community:[]uint8{0x70, 0x75, 0x62, 0x6c, 0x69, 0x63}, RawPDU:asn1.RawValue{Class:0, Tag:0, IsCompound:false, Bytes:[]uint8(nil), FullBytes:[]uint8(nil)}}, PDUType:1, PDU:snmp.GenericPDU{RequestID:0, ErrorStatus:0, ErrorIndex:0, VarBinds:[]snmp.VarBind{snmp.VarBind{Name:asn1.ObjectIdentifier{1, 3, 6, 1, 2, 1, 2, 2, 1, 15}, RawValue:asn1.RawValue{Class:0, Tag:5, IsCompound:false, Bytes:[]uint8(nil), FullBytes:[]uint8(nil)}}}}}
DEBUG client: Engine<[::]:51693>: Start request [email protected]:161[0]: GetNextRequest<1.3.6.1.2.1.2.2.1.15>@172.28.2.2:161[1]
DEBUG client: Engine<[::]:51693>: Recv: client.IO{Addr:(*net.UDPAddr)(0xc420198000), Packet:snmp.Packet{Version:1, Community:[]uint8{0x70, 0x75, 0x62, 0x6c, 0x69, 0x63}, RawPDU:asn1.RawValue{Class:2, Tag:2, IsCompound:true, Bytes:[]uint8{0x2, 0x1, 0x0, 0x2, 0x1, 0x0, 0x2, 0x1, 0x0, 0x30, 0x15, 0x30, 0x13, 0x6, 0xa, 0x2b, 0x6, 0x1, 0x2, 0x1, 0x2, 0x2, 0x1, 0x10, 0x1, 0x41, 0x5, 0x0, 0xb5, 0x98, 0xe, 0x22}, FullBytes:[]uint8{0xa2, 0x20, 0x2, 0x1, 0x0, 0x2, 0x1, 0x0, 0x2, 0x1, 0x0, 0x30, 0x15, 0x30, 0x13, 0x6, 0xa, 0x2b, 0x6, 0x1, 0x2, 0x1, 0x2, 0x2, 0x1, 0x10, 0x1, 0x41, 0x5, 0x0, 0xb5, 0x98, 0xe, 0x22}}}, PDUType:2, PDU:snmp.GenericPDU{RequestID:0, ErrorStatus:0, ErrorIndex:0, VarBinds:[]snmp.VarBind{snmp.VarBind{Name:asn1.ObjectIdentifier{1, 3, 6, 1, 2, 1, 2, 2, 1, 16, 1}, RawValue:asn1.RawValue{Class:1, Tag:1, IsCompound:false, Bytes:[]uint8{0x0, 0xb5, 0x98, 0xe, 0x22}, FullBytes:[]uint8{0x41, 0x5, 0x0, 0xb5, 0x98, 0xe, 0x22}}}}}}
DEBUG client: Engine<[::]:51693>: Request [email protected]:161[0] done: GetNextRequest<1.3.6.1.2.1.2.2.1.15>@172.28.2.2:161[1]
INFO client: Client<[email protected]:161>: Request GetNextRequest<1.3.6.1.2.1.2.2.1.15>@172.28.2.2:161[1] => 1.3.6.1.2.1.2.2.1.16.1=3046641186
DEBUG client: Engine<[::]:51693>: Close...
The GET .../tables/$TABLE
API returns Entries
with partial data + Errors
if some table entry objects are invalid:
{
"ID": "IF-MIB::ifTable",
...
"Entries": [
...
{
"HostID": "public@localhost",
"Index": {
"IF-MIB::ifIndex": 5
},
"Objects": {
"IF-MIB::ifAdminStatus": "up",
"IF-MIB::ifDescr": "docker0",
...
"IF-MIB::ifInNUcastPkts": null,
...
},
"Errors": [
"Invalid VarBind[.1.3.6.1.2.1.2.2.1.12.5] index for IF-MIB::ifInNUcastPkts: index [5] starts with 5"
]
},
...
]
}
The `GET .../tables/?table=$TABLE` API does not, if any entry object results in errors, then it omits the entire row from `Entires` and returns a top-level `Errors` instead:
```json
[
{
"ID": "IF-MIB::ifTable",
...
"Entries": [
...
],
"Errors": [
{
"HostID": "public@localhost",
"Error": "Invalid VarBind[.1.3.6.1.2.1.2.2.1.12.5] index for IF-MIB::ifInNUcastPkts: index [5] starts with 5"
}
]
}
]
This is a bug in #17
There is currently no rate-limit on the number of parallel queries for a server.Host
... I'm afraid some cheaply implemented network devices will choke if they get too many concurrent requests, and the client.Client
starts retrying?
See RFC1905 #4.2.3, avoids the high query latency associated with traversing large tables one row a time with the associated round-trip delay.
The client:Client.Walk
function should know how to do either a GetNext
walk or a GetBulk
walk, possibly based on some client:Options.WalkBulk
flag?
While we're at it, the client:Client.getRequest
request splitting using MaxVars
should probably also be moved into the higher-level Walk
interface (options.WalkMaxVars
?), keeping the low-level Get
, GetNext
and GetBulk
requests simple.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.