akhenakh / statgo

Access OS metrics from Golang,

StatGo

StatGo give you access to OS metrics like network interface bandwith, cpus usage ...
It supports FreeBSD, Linux, OSX & more, it's in fact a libstatgrab binding for Golang.
Tested on FreeBSD, OSX, Linux amd64, Linux arm.

Compilation

You need at least libstatgrab 0.91, Debian & Ubuntu only have 0.90 ...

On Debian/Ubunt & OSX, you can simply install libstatgrab with the usual commands:

./configure --prefix=/usr/local
make
sudo make install

You may have to set CGO_LDFLAGS and CGO_CFLAGS environment according to your path:

export CGO_CFLAGS=-I/usr/local/include
export CGO_LDFLAGS=-L/usr/local/lib

Note: On OSX you need to install gcc to access cgo.

go get github.com/akhenakh/statgo

Usage

s := NewStat()
s.HostInfos()
OSName: Darwin
OSRelease:  14.4.0
OSVersion:  Darwin Kernel Version 14.4.0: Thu May 28 11:35:04 PDT 2015; root:xnu-2782.30.5~1/RELEASE_X86_64
Platform:   x86_64
HostName:   kamoulox
NCPUs:      4
MaxCPUs:    4
BitWidth:   64

s.CPUStats()
User:       7.500000
Kernel:     2.500000
Idle:       90.000000
IOWait      0.000000
Swap:       0.000000
Nice:       0.000000
LoadMin1:   2.206055
LoadMin5:   2.031250
LoadMin15:  1.970703

s.FSInfos()[0]
DeviceName:         /dev/disk1
FSType:             hfs
MountPoint:         /
Size:               249769230336
Used:               224367140864
Free:               25402089472
Available:          25139945472
TotalInodes:        60978814
UsedInodes:         54841132
FreeInodes:         6137682
AvailableInodes:    6137682

s.InterfaceInfos()[0]
Name:   en2
Speed:  0
Factor: 1000000
Duplex: Full Duplex
State:  UP

s.MemStats()
Total:      16649420800
Free:       4323848192
Used:       12325572608
Cache:      0
SwapTotal:  3221225472
SwapUsed:   2528378880
SwapFree:   692846592

s.NetIOStats()
IntName:    en0
TX:         2310272606
RX:         3336240203
IPackets:   114473581
OPackets:   129430304
IErrors:    0
OErrors:    0
Collisions: 0

s.ProcessStats()
Total:      343
Running:    335
Sleeping:   0
Stopped:    0
Zombie:     8

s.PagesStats()
PageIn:     90173695
PageOut:    90173695

Status

• Host infos
• cpu stats
• load average
• network interfaces infos
• mem stats
• swap stat
• io stats
• net io stats
• process count
• page stats

Contributors

• HeinOldewage

statgo's Issues

I get a not-a-number value using CPUStats

I get a NaN for some all values coming from "sg_get_cpu_percents_of" using the CPUStats().
Sorry, just read the comment above the function with the 100ms ... you can delete this issue.

goroutine leak

package main

import (
	"fmt"
	"runtime"
	"time"

	"github.com/akhenakh/statgo"
)

func main() {
	go func() {
		d := time.Duration(1) * time.Second
		for range time.Tick(d) {
			fmt.Printf("Total goroutines: %d\n", runtime.NumGoroutine())
		}
	}()

	for {
		d := time.Duration(1) * time.Second
		statgo.NewStat()
		time.Sleep(d)
	}
}

'statgrab.h' file not found

github.com/akhenakh/statgo
../gospace/src/github.com/akhenakh/statgo/cpu_stats.go:4:11: fatal error: 'statgrab.h' file not found

CPUStats: Stats do not update on consecutive calls.

The values for User,Kernel,Idle,IOWait,Swap and Nice remain constant for every NewStat().

Replacing C.sg_get_cpu_percents_of(C.sg_last_diff_cpu_percent, nil) with C.sg_get_cpu_percents_of(C.sg_new_diff_cpu_percent, nil) gets updated stats for every call of CPUStats.

getting error with statgrab.h while installig

getting following error while trying to go get

akhenakh\statgo\cpu_stats.go:4:23: fatal error: statgrab.h: No such file or directory
 // #include <statgrab.h>
                       ^
compilation terminated.

s.HostInfos() crash

gomail(30652,0xb0310000) malloc: *** error for object 0x48095d0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
SIGABRT: abort
PC=0x7fff8c8aa286 m=7
signal arrived during cgo execution

goroutine 22 [syscall, locked to thread]:
runtime.cgocall(0x4002b40, 0xc820199898, 0x0)
    /usr/local/go/src/runtime/cgocall.go:120 +0x11b fp=0xc820199868 sp=0xc820199838
github.com/akhenakh/statgo._Cfunc_sg_free_stats_buf(0x4809628, 0xc800000000)
    ??:0 +0x35 fp=0xc820199898 sp=0xc820199868
github.com/akhenakh/statgo.(*Stat).HostInfos(0xc82000b4c0, 0x1)
    /Users/baiyuxiong/go/src/github.com/akhenakh/statgo/host_info.go:45 +0x22b fp=0xc820199920 sp=0xc820199898
github.com/baiyuxiong/gomail/status.HostInfos(0x0, 0x0)
    /Users/baiyuxiong/go/src/github.com/baiyuxiong/gomail/status/state.go:15 +0x2f fp=0xc820199958 sp=0xc820199920
github.com/baiyuxiong/gomail/http.configRoutes.func2(0x4d00108, 0xc8201f49a0, 0xc8200e7880)
    /Users/baiyuxiong/go/src/github.com/baiyuxiong/gomail/http/routes.go:126 +0x348 fp=0xc820199b58 sp=0xc820199958
net/http.HandlerFunc.ServeHTTP(0x451a170, 0x4d00108, 0xc8201f49a0, 0xc8200e7880)
    /usr/local/go/src/net/http/server.go:1422 +0x3a fp=0xc820199b78 sp=0xc820199b58
net/http.(*ServeMux).ServeHTTP(0xc820014870, 0x4d00108, 0xc8201f49a0, 0xc8200e7880)
    /usr/local/go/src/net/http/server.go:1699 +0x17d fp=0xc820199bd0 sp=0xc820199b78
net/http.serverHandler.ServeHTTP(0xc820132000, 0x4d00108, 0xc8201f49a0, 0xc8200e7880)
    /usr/local/go/src/net/http/server.go:1862 +0x19e fp=0xc820199c30 sp=0xc820199bd0
net/http.(*conn).serve(0xc820150420)
    /usr/local/go/src/net/http/server.go:1361 +0xbee fp=0xc820199f88 sp=0xc820199c30
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1696 +0x1 fp=0xc820199f90 sp=0xc820199f88
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:1910 +0x3f6

NetIOStats: Misleading and Incorrect

Two problems exist with NetIOStats().
Firstly the underlying call to sg_get_network_io_stats_diff returns the stats from the last call or from system start up if it is the fist call. The result includes the system time that the stats were taken over. This field (systime) is not included in the struct returned by the function, making the results almost useless as no rate can be estimated.
SysTime: time.Duration(int(v.systime))*time.Second, can convert intervals(followup calls).

Once the systime is included a second problem is made clear.
The underlying function depends on the fact that the same OS thread will make the call. Currently different threads, depending on the scheduler, call the function.
Other functions might have the same drawback/problem, but I have not investigated this.
For example, if three call were made, the second one 10 seconds after the first and the third 1 second after the second then the stats for the third might be for 1 second (if it was called from the same thread as the second) or for 11 seconds(if it was called from the same thread as the first) or for the the system uptime(if it was called from a new thread).

stats := statgo.NewStat()
go func() {
    for _, netstats := range stats.NetIOStats() {
        fmt.Println(netstats)
    }
}()
go func() {
    time.Sleep(10 * time.Second)
    for _, netstats := range stats.NetIOStats() {
        fmt.Println(netstats)
    }
}()
go func() {
    time.Sleep(11 * time.Second)
    for _, netstats := range stats.NetIOStats() {
        fmt.Println(netstats)
    }
}()

A solution might be to start up a goroutine (in init()) that is is bound to an OS thread (using runtime.LockOSThread()) and waits for calls on a channel.
This will unfortunately require a restructure of most of the code..

Incorrect fields returned

When i try to get the network tx and rx, the returned value is grossly different from the actual tx and rx

func main() {
	s := statgo.NewStat()
	time.Sleep(time.Duration(5) * time.Second)
	v := s.NetIOStats()
	for _, k := range v {
		fmt.Println(k.String())
	}
	//fmt.Println(v)
}

Return :

IntName:	lo
TX:		0
RX:		0
IPackets:	0
OPackets:	0
IErrors:	0
OErrors:	0
Collisions:	0
Period:	5s
TimeTaken:	2022-05-09 20:24:52.275271111 +0100 WAT m=+5.002235386

IntName:	enp3s0
TX:		0
RX:		0
IPackets:	0
OPackets:	0
IErrors:	0
OErrors:	0
Collisions:	0
Period:	5s
TimeTaken:	2022-05-09 20:24:52.275282861 +0100 WAT m=+5.002247086

IntName:	wlan0
TX:		2952 // This is highly different
RX:		2841 // Same for this.
IPackets:	18
OPackets:	22
IErrors:	0
OErrors:	0
Collisions:	0
Period:	5s
TimeTaken:	2022-05-09 20:24:52.275283873 +0100 WAT m=+5.002248098

When i run statgrab in the terminal:

net.enp3s0.collisions = 0
net.enp3s0.duplex = unknown
net.enp3s0.ierrors = 0
net.enp3s0.interface_name = enp3s0
net.enp3s0.ipackets = 0
net.enp3s0.oerrors = 0
net.enp3s0.opackets = 0
net.enp3s0.rx = 0
net.enp3s0.speed = 4294967295
net.enp3s0.systime = 1652124384
net.enp3s0.tx = 0
net.enp3s0.up = true

net.lo.collisions = 0
net.lo.duplex = unknown
net.lo.ierrors = 0
net.lo.interface_name = lo
net.lo.ipackets = 996
net.lo.oerrors = 0
net.lo.opackets = 996
net.lo.rx = 87257
net.lo.speed = 0
net.lo.systime = 1652124384
net.lo.tx = 87257
net.lo.up = true

net.wlan0.collisions = 0
net.wlan0.duplex = unknown
net.wlan0.ierrors = 0
net.wlan0.interface_name = wlan0
net.wlan0.ipackets = 161105
net.wlan0.oerrors = 0
net.wlan0.opackets = 106128
net.wlan0.rx = 171768422
net.wlan0.speed = 0
net.wlan0.systime = 1652124384
net.wlan0.tx = 17610127
net.wlan0.up = true

Request for documentation

As of right now, the documentation is limited to the README and a godoc that isn't linked to from the Github page. The godoc itself doesn't provide necessary information, such as what units many metrics are given in. I'd love to use this library, but this is a big barrier.

network RX WX data is 0 at Mac

Hi,

I tried using statgo to monitor my laptop's os stat, the network IO info seems to be incorrect.(TX,RX is always zero)

Could you help to take a look? Thanks!
./migration
User: 50.000000
Kernel: 0.000000
Idle: 50.000000
IOWait: 0.000000
Swap: 0.000000
Nice: 0.000000
LoadMin1: 2.580078
LoadMin5: 2.706055
LoadMin15: 2.521973
Period: 0
TimeTaken: 2016-06-21 22:08:34.754612291 +0800 CST

OSName: Darwin
OSRelease: 15.5.0
OSVersion: Darwin Kernel Version 15.5.0: Tue Apr 19 18:36:36 PDT 2016; root:xnu-3248.50.21~8/RELEASE_X86_64
Platform: x86_64
HostName: zhangwenjingdeMacBook-Pro.local
NCPUs: 8
MaxCPUs: 8
BitWidth: 64

stat is &statgo.NetIOStats{IntName:"lo0", TX:0, RX:0, IPackets:0, OPackets:0, IErrors:0, OErrors:0, Collisions:0, Period:0, TimeTaken:time.Time{sec:63602114914, nsec:755049611, loc:(_time.Location)(0x4633c40)}}
stat is &statgo.NetIOStats{IntName:"gif0", TX:0, RX:0, IPackets:0, OPackets:0, IErrors:0, OErrors:0, Collisions:0, Period:0, TimeTaken:time.Time{sec:63602114914, nsec:755225748, loc:(_time.Location)(0x4633c40)}}
stat is &statgo.NetIOStats{IntName:"stf0", TX:0, RX:0, IPackets:0, OPackets:0, IErrors:0, OErrors:0, Collisions:0, Period:0, TimeTaken:time.Time{sec:63602114914, nsec:755375818, loc:(_time.Location)(0x4633c40)}}
stat is &statgo.NetIOStats{IntName:"en0", TX:0, RX:0, IPackets:0, OPackets:0, IErrors:0, OErrors:0, Collisions:0, Period:0, TimeTaken:time.Time{sec:63602114914, nsec:755619534, loc:(_time.Location)(0x4633c40)}}
stat is &statgo.NetIOStats{IntName:"en1", TX:0, RX:0, IPackets:0, OPackets:0, IErrors:0, OErrors:0, Collisions:0, Period:0, TimeTaken:time.Time{sec:63602114914, nsec:755722533, loc:(_time.Location)(0x4633c40)}}
stat is &statgo.NetIOStats{IntName:"en2", TX:0, RX:0, IPackets:0, OPackets:0, IErrors:0, OErrors:0, Collisions:0, Period:0, TimeTaken:time.Time{sec:63602114914, nsec:755867250, loc:(_time.Location)(0x4633c40)}}
stat is &statgo.NetIOStats{IntName:"p2p0", TX:0, RX:0, IPackets:0, OPackets:0, IErrors:0, OErrors:0, Collisions:0, Period:0, TimeTaken:time.Time{sec:63602114914, nsec:756018295, loc:(_time.Location)(0x4633c40)}}
stat is &statgo.NetIOStats{IntName:"bridge0", TX:0, RX:0, IPackets:0, OPackets:0, IErrors:0, OErrors:0, Collisions:0, Period:0, TimeTaken:time.Time{sec:63602114914, nsec:756168235, loc:(_time.Location)(0x4633c40)}}
stat is &statgo.NetIOStats{IntName:"awdl0", TX:0, RX:0, IPackets:0, OPackets:0, IErrors:0, OErrors:0, Collisions:0, Period:0, TimeTaken:time.Time{sec:63602114914, nsec:756262222, loc:(*time.Location)(0x4633c40)}}

statgrab.h: No such file or directory

Hi!
Trying to get and observe this issue:

$ go get github.com/akhenakh/statgo
go: github.com/akhenakh/statgo upgrade => v0.0.0-20171021021904-3ae2cda264c5

github.com/akhenakh/statgo

../pkg/mod/github.com/akhenakh/[email protected]/cpu_stats.go:4:23: fatal error: statgrab.h: No such file or directory
// #include <statgrab.h>

fix issue with cpu diff & offer raw counters (no percents)

thats an oof

go: downloading github.com/akhenakh/statgo v0.0.0-20171021021904-3ae2cda264c5
# github.com/akhenakh/statgo
../../go/pkg/mod/github.com/akhenakh/[email protected]/cpu_stats.go:4:11: fatal error: statgrab.h: No such file or directory
    4 | // #include <statgrab.h>
      |           ^~~~~~~~~~~~
compilation terminated.

maybe someone needs to fix the c, did the make install even installed libs on my own and still got the same error, sometimes i wonder did you forget a semicolon? 😆

sometime the CPU idle value is NaN

s := stat.NewStat()	
go func() {
		s.CPUStats()
		time.Sleep(time.Second)
		cpus := s.CPUStats()

                cpu <- (100 - float32(cpus.Idle)) / 100 // range from 0 ~ 1

	}()

The above are the codes, but Why ?bug or my codes' problem?

I add some logs:

logger.LogDebugf("CPU Idle %f %f %f %f %f %f\n", cpus.User, cpus.Kernel, cpus.Idle, cpus.IOWait, cpus.Swap, cpus.Nice)

All the cpus member are NaN.