sexibytes / sexigraf

SexiGraf is a vSphere centric Graphite appliance with a Grafana frontend.

SexiGraf is a fully open-source vSphere centric Graphite VMware appliance with a Grafana frontend. It pulls VI and vSAN metrics from VMware vCenter APIs, push them to Graphite and let Grafana produces the gorgeous dashboards we love so much!

Full changelog is available here: CHANGELOG.md

Official website for this awesome appliance is available at http://www.sexigraf.fr

VMware vSAN

The metrics used in the various VSAN dashboards are now collected every 5 minutes using to the VsanPerfQueryPerf API method of VsanPerformanceManager from the famous vSAN performance service. With some educated guesses, it is possible to access almost any metrics from the vSAN cluster. And guess what! We’re already working on more cool SexiPanels for vSAN: http://www.sexigraf.fr/vsan-sexipanels/

VMware vSphere

Fast. Very fast. That’s what we had in mind when we designed SexiGraf. When you need vSphere metrics, the obvious way is the PerformanceManager, but we need something faster so we choosed managed object properties and quickstats like ResourcePoolQuickStats. If we have no other choice, we failback to the PerformanceManager but we only query the last 15 samples of the RealTime samplingPeriod since we pull vSphere metrics every 5 minutes. http://www.sexigraf.fr/vsphere-sexipanels/  

FreeNAS/TrueNAS

Starting from version 9.10, FreeNAS/TrueNAS allows users to set a “Remote Graphite Server” target to send all the metrics harvested by Collectd. Guess what would make a nice Graphite target! http://www.sexigraf.fr/freenas-sexipanel/

Windows

Leveraging the built-in Graphite listener of SexiGraf, we introduced Windows support in version 0.99c with basic cpu-ram-hdd metrics : http://www.sexigraf.fr/windows-sexipanel/

pfSense

Because we love and use pfSense very much since the m0n0wall fork, we decided to make a great dashboard with system metrics, interface statistics but most importantly the pfinfo data which let you monitor the packet filtering service of the firewall.

S.M.A.R.T.

Since the first release we wanted to add a SMART counters dashboard because we were so inspired by the Backblaze reports. But we never found a proper way to get those stats from any kind of NAS so we decided to rely on a custom script that pushes the data to graphite using netcat 😉

Veeam Backup & Replication

Since Veeam Backup & Replication 11, a new RESTful API for VBR has been introduced so we can now query the VBR servers instead of relying on Veeam Backup Enterprise Manager beeing installed and properly configured. So we figured we should add some Veeam skills to SexiGraf!

License

This project is licensed under the MIT License - see the LICENSE.md file for details

sexigraf's Issues

VMware Multi ESX QuickStats

Hi,

There is an another issue. The VMware Multi ESX Quickstats does not present any data, and I cannot select anything in the header part.
Is it a known issue or something wrong with my local installation?

Regards,
Zoli

cron files not removed after delete-vcentry

need to remove cron files if exists before removing entry in vmware credential store

Handle multiple puller on the same vcenter

If the script takes longer than 1 or 5 minutes to complete we should avoid launching it again

carbon-cache status missing in seximenu

https://github.com/sexibytes/sexigraf/blob/master/root/seximenu/seximenu.sh

Add 'Refresh' option for offline inventory

Offline inventory is already schedule to be run every day, but we should let user a way to refresh manually if needed

Re-inventory after vcenter entry removal

It may be cleaner to re-inventory after vcenter removal from creds store to reflect the change earlier than the daily schedule

Handle retry when logon to vcenter instance

Add 3 retries with few sec sleep between

Add Resync/Rebuild/Rebalance dashboard for VSAN

Cluster Hosts basic stats dashboard

Until we decided or not to add cluster hosts stats, we can use the stats already injected in graphite.

dashboard metrics definition

hi,

After implementation and 3 day of run i still does not have data in some graphics.
This is not a graphite problem but a Grafana dashboard definition problems
in may settings, you start your query with esx..xxxx in place of vmw.$vcenter.$datacentre.$cluster.esx..xxxx
and this is resulting in bad graph.
and due to the fact that you are using local dashboard, we need to patch this directly in the json files

Long work in perspective

Best regards
Ph Koenig
Belgium

add icon next to "Upload Package" button

Add NAA latencies dashboard for VSAN

add refresh button in House Cleaner page

Time data request Error on graph

Hi team
Congratulation for this amazing tool
i've got one issue on my test lab (Vsphere 6)
on the Vmware FlambX \ HDD graph, i've a red ! with "Time data request Error "
i have the same on VMWARE esx Full stat... and on some other graph expecially on datastore or Hdd graph.
i've seen that there is some similar issue related to Grafana.. but without any detail.
any idea ?

thanks

julien

ViOfflineInventory run after vcentry addition ?

Top 5 vmdk dashboard for VSAN

strict search in VI Offline Inventory

Rounding in FlambX

Hi,

Not a big issue, but the RAM capacity in the FlambX graf, when it is more than 1 TB, is rounded to whole number. For example in case of 1,37TB tha graf shows only 1TB.

Regards,
Zoli

replace "n/a" by "N/A" in offline inventory

in order to avoid ambiguity, "N/A" should be used instead of "n/a"

Handle VirtualSAN Witness ESXi

VSAN puller script iterate in the cluster's host list but since VSAN witness vm is out of the cluster it's not include in the loop.

VSAN capacity diskgroup templating

today the capacity is host level, i would be interesting to move this to diskgroup level

Wrong logrotate file definition make log file unreadable by Log Viewer

We miss-defined logrotate for SexiGraf log, thus new log files were created with "600 root:root" rights.

ignore .gitignore file in Package Updater

web-admin.json wrong path

to move to https://github.com/sexibytes/sexigraf/tree/master/var/lib/grafana/dashboards

Add logs to VI and VSAN pull scripts

Can't see vm.left above 7j in Cluster Capacity Planning dashboard

Managing multiple vCenter add

As offline inventory query is run after a vcenter is added, we should handle the case user add multiple vCenter as multiple inventory will be run in background

support for unmanaged ESX

create unmanaged version of https://github.com/sexibytes/sexigraf/blob/master/root/ViPullStatistics.pl

New to add some 'Test' action on credential store

In order to validate user input, try to initiate some connection with provided inputs and see what's happen.

list old "orphaned" files in Stats Remover

list the files that have not been updated for a long time a potential target in the Stats Remover page

shrink sexigraf ASCII logo in seximenu

As putty windows size is 80x24 by default

Enhance the SSD stats dashboard

need "dismissed" button on SexiGraf Package Update Runner log page

and maybe a "OK" or "KO" at the end of the process

No web admin go back link from Log Viewer page

we should have a Go Back or Home link

crontab files not used after vi/vsan enabling

add apache2 and carbon-cache services in the restart cycle of the SexiMenu

add apache2 service state in seximenu

apache2 is acting as reverse proxy for graphite, grafana and serves admin tools so it might be a good idea to have it the seximenu

localhost hostname makes the script die

https://twitter.com/cragdoo/status/687655125461123072

change grafana.ini to enable dashboard saving

it will be good if you enable the sqlite3, or any other, database in the default grafana.ini.
With this, a end-user will be able to save his own dashboard or a modified one, like i do.

Offline Inventory : Resource Pool name & vm folder path

Hi,

Would be useful to get

• Resource pool name (think VCD use-case : org vDC)
• vm folder path (e.g. Cloud/tenant1/vdc_name/vapp_name ; again VCD use-case)

Thus, sexigraf becomes the offline cloud inventory "par excellence"

logs truncated in Log Viewer

remove cluster configurationEx dependency for performace impact

missing "refresh inventory" in Web Toolbox menu

Add VM quickstats dashboard

ViPullStatistics very slow on HostSystem.runtime

I'm getting very slow statistics gathering while the following requests are executed:

REQUEST: $VAR1 = '<?xml version="1.0" encoding="UTF-8"?>
   <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                     xmlns:xsd="http://www.w3.org/2001/XMLSchema"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soapenv:Body>
<RetrieveProperties xmlns="urn:vim25"><_this type="PropertyCollector">propertyCollector</_this>
<specSet><propSet><type>HostSystem</type><all>0</all><pathSet>runtime</pathSet></propSet><objectSet><obj type="HostSystem">host-12052</obj>
</objectSet></specSet></RetrieveProperties></soapenv:Body></soapenv:Envelope>
';

The whole ViPullStatistics take ~20min to complete on a single vcenter (two clusters, 4 esx per cluster, 200 VMs in total).

Any idea how to improve the execution time?

Add MAC addresses on static inventory

As this is a useful info to have, we should include mac addresses in static inventory

Incorrect path in service restart function

Hey, I just deployed the the SexiGraf Appliance (or "applicance" as the download link says!) and noticed that the path for collect.d is incorrect in the restart function (line 94/95).

Should be /etc/init.d/collectd stop (i.e. no dot)

Issues during statistics pull

There is some case when ViPullStatistics.pl generate some warning like:

/usr/local/share/perl/5.20.2/Net/Graphite.pm line120 - WARN Use of uninitialized value $hash in concatenation (.) or string at /usr/local/share/perl/5.20.2/Net/Graphite.pm line 120.

Issue fetching VSAN stats after upgrade to 0.99b

Attached VsanDisksPullStatistics.log

/var/log/syslog tail GUI

For early troubleshooting purpose, it might be handy to have the last lines of /var/log/syslog file