lukas-vlcek / bigdesk Goto Github PK
View Code? Open in Web Editor NEWLive charts and statistics for Elasticsearch cluster.
License: Apache License 2.0
Live charts and statistics for Elasticsearch cluster.
License: Apache License 2.0
Hi,
When I try to install the Bigdesk plugin in elasticsearch (0.18.6) I get the following error. Guessing the location changed? Although I see the zipball at /lukas-vlcek/bigdesk/zipball/1.0.0, ES attempts to download from /lukas-vlcek/bigdesk/zipball/v1.0.0 .
/lukas-vlcek/bigdesk/downloads returns "There aren't any downloads yet. But don't worry! You can download the source code as a zip or tarball above".
# sh -x bin/plugin -install lukas-vlcek/bigdesk/1.0.0
+ CDPATH=
+ SCRIPT=bin/plugin
+ [ -h bin/plugin ]
+ dirname bin/plugin
+ ES_HOME=bin/..
+ cd bin/..
+ pwd
+ ES_HOME=/data/elasticsearch
+ [ -x /opt/java/sdk/current/bin/java ]
+ JAVA=/opt/java/sdk/current/bin/java
+ exec /opt/java/sdk/current/bin/java -Delasticsearch -Des.path.home=/data/elasticsearch -cp /data/elasticsearch/lib/* org.elasticsearch.plugins.PluginManager -install lukas-vlcek/bigdesk/1.0.0
-> Installing lukas-vlcek/bigdesk/1.0.0...
Trying https://github.com/downloads/lukas-vlcek/bigdesk/bigdesk-1.0.0.zip...
Trying https://github.com/lukas-vlcek/bigdesk/zipball/v1.0.0...
Failed to install lukas-vlcek/bigdesk/1.0.0, reason: failed to download
I installed the plugin just fine but I can't find anywhere the url to access the pretty graphs and such I see. It'd be nice to add that to the front page somewhere.
Make better use of the screen space. Some charts are not that important and useful - make then smaller. For some characteristic consider using of http://omnipotent.net/jquery.sparkline/ for some really small charts. For charts display also actual (the most recent) values.
The node info is fetched using /_nodes in RC1 and not /_cluster/nodes as in previous versions. This appears to only be used in NodeInfo.js line 25
I get utc timezoned times. This should be my local ones (+2).
Pre ES 2.x see elastic/elasticsearch#4378
Post ES 2.x see elastic/elasticsearch#12408
It would be useful to have an option to store host and port values in URL so that users can bookmark particular settings and clusters and open BigDesk quickly for them.
Hi
I seem to only be able to see the cluster pack diagram if the cluster is named elastic search, if it is named anything else it just loads the node names on top of each other:
I'm using BigDesk 2.5.0 and ElasticSearch 1.5.0.
Is this a bug or could it be something to do with my config? when removing the "cluster.name" from elasticsearch.yml the diagram shows up fine
For more details see elastic/elasticsearch#1113
I don't see 1.4 listed in the support matrix, so I wanted to file this bug as a tracker for 1.4 support. Thanks!
Hi Lukas,
Seems 2.4.0 doesn't support es 1.2.1 ? Or am I missing something?
Spancer
I'm running BigDesk as an ES plugin, and I am forwarding port 9200 through port 80 using nginx. Ie. http://localhost:9200/ is reachable from the outside as http://ip-number/elasticsearch/
It would be nice if the default suggested endpoint is the part of the URL prior to /_plugin/bigdesk/.
Ie. if the BigDesk URL is http://ip-number/elasticsearch/_plugin/bigdesk/ then the suggested ES REST endpoint should be: http://ip-number/elasticsearch/
Some admin REST API has been changed.
Hi,
After upgrading ES to 0.20.2 i'm missing a few JVM graphs.
All 4 OS graphs and the 3 last Process graphs are grayed out.
Also the Filesystem information and graphs are missing.
We pull a lot of data from REST endpoints. This can cause a serious performance issues when connected to large clusters. We do not make use of all the data anyway for now. Especially cluster state metadata can be large.
Make sure Bigdesk try to use AJAX (with CORS) if JSONP not allowed.
See: elastic/elasticsearch#6795
Using Elasticsearch 0.90.5 and the current master version of bigdesk, I cannot seem to get detailed statistics on the nodes.
After loading the bigdesk dashboard, I click one of the nodes and nothing appears to happen. Checking the Javascript log I find:
Uncaught TypeError: <template>:1
>> CPU vendor: {{#cpu.vendor}}{{cpu.vendor}}{{/cpu.vendor}}{{^cpu.vendor}}n/a{{/cpu.vendor}}<br>CPU model: {{#cpu.model}}{{cpu.model}} ({{cpu.mhz}} MHz){{/cpu.model}}{{^cpu.model}}n/a{{/cpu.model}}<br>CPU total logical cores: {{#cpu.total_cores}}{{cpu.total_cores}}{{/cpu.total_cores}}{{^cpu.total_cores}}n/a{{/cpu.total_cores}}<br>CPU cache: {{#cpu.cache_size}}{{cpu.cache_size}}{{/cpu.cache_size}}{{^cpu.cache_size}}n/a{{/cpu.cache_size}}
This is then followed by a different explanation for the two browsers I use:
Chrome:
Cannot read property 'cpu' of undefined
Safari:
'undefined' is not an object (evaluating 'context[names[j++]]')
I assume it is the /stats API that is used so I have created a gist of that response from one of the nodes:
https://gist.github.com/martinskans/8705555
Information that could also be useful is
that the Elasticsearch nodes themselves run on Ubuntu on virtual Amazon instances. The node the call is made through is a "no data" and "no master" node which is just used as an access point to the cluster.
I will be happy to help with any more information I can provide!
Hi
We have upgraded our ES Clusters to use ElasticSearch 0.90.0.RC1. We have noticed that BigDesk cannot connect to the cluster. Is there an update of BigDesk for this version?
Hi!
In Issue #21 you said you would release a ZipFile for bigdesk. This - as it appears - hasn't happened yet. I want to use bigdesk for monitoring my ES instances on CentOS. For that I want to create an RPM. I'm normally used to Debian systems. But circumstances only allow me to use CentOS. Here is a Spec file. It is a Modification of the hadoop plugin package found on @tavisto's account under the elasticsearch-rpms repository. I would like to have it reviewed by you. In addition I would like to have a 1.0.0 release Zip on the repo. This would make the rpm building much simpler. Anyway here is the spec:
%define debug_package %{nil}
%define base_install_dir %{_javadir}/%{name}
%define site_install_dir %{base_install_dir}/plugins/bigdesk/_site
# Avoid running brp-java-repack-jars
%define __os_install_post %{nil}
Name: elasticsearch-plugin-bigdesk
Version: 1.0.0
Release: 1%{?dist}
Summary: ElasticSearch plugin to use BigDesk
Group: System Environment/Daemons
License: ASL 2.0
URL: https://github.com/elasticsearch/elasticsearch-bigdesk
Source0: https://github.com/lukas-vlcek/bigdesk/zipball/%{version}
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
BuildArch: noarch
Requires: elasticsearch >= 0.19
%description
Live charts and statistics for ElasticSearch cluster.
%prep
rm -fR %{name}-%{version}
%{__mkdir} -p %{name}-%{version}
cd %{name}-%{version}
%{__mkdir} -p plugins
unzip %{SOURCE0} -d plugins/bigdesk
%build
true
%install
rm -rf $RPM_BUILD_ROOT
cd %{name}-%{version}
%{__mkdir} -p %{buildroot}/%{site_install_dir}
%{__mkdir} -p %{buildroot}/%{site_install_dir}/css
%{__mkdir} -p %{buildroot}/%{site_install_dir}/js
%{__mkdir} -p %{buildroot}/%{site_install_dir}/js/lib
%{__mkdir} -p %{buildroot}/%{site_install_dir}/js/lib/mustache
%{__mkdir} -p %{buildroot}/%{site_install_dir}/js/lib/highcharts
%{__mkdir} -p %{buildroot}/%{site_install_dir}/js/jQueryMustache
%{__mkdir} -p %{buildroot}/%{site_install_dir}/js/jquery
%{__mkdir} -p %{buildroot}/%{site_install_dir}/js/parseUri/
%{__mkdir} -p %{buildroot}/%{site_install_dir}/images
%{__install} -D -m 755 plugins/bigdesk/index.html -t %{buildroot}/%{site_install_dir}/
%{__install} -D -m 755 plugins/bigdesk/README.textile -t %{buildroot}/%{site_install_dir}/
%{__install} -D -m 755 plugins/bigdesk/screenshot -t %{buildroot}/%{site_install_dir}/
%{__install} -D -m 755 plugins/bigdesk/css/bigdesk.css -t %{buildroot}/%{site_install_dir}/css/
%{__install} -D -m 755 plugins/bigdesk/css/reset.css -t %{buildroot}/%{site_install_dir}/css/
%{__install} -D -m 755 plugins/bigdesk/images/1license.txt -t %{buildroot}/%{site_install_dir}/images/
%{__install} -D -m 755 plugins/bigdesk/images/award_star_gold.png -t %{buildroot}/%{site_install_dir}/images/
%{__install} -D -m 755 plugins/bigdesk/images/es-logo.png -t %{buildroot}/%{site_install_dir}/images/
%{__install} -D -m 755 plugins/bigdesk/js/bigdesk.js -t %{buildroot}/%{site_install_dir}/js/
%{__install} -D -m 755 plugins/bigdesk/js/charts.js -t %{buildroot}/%{site_install_dir}/js/
%{__install} -D -m 755 plugins/bigdesk/js/lib/highcharts/highcharts.js -t %{buildroot}/%{site_install_dir}/js/lib/highcharts/
%{__install} -D -m 755 plugins/bigdesk/js/lib/jQueryMustache/License.txt -t %{buildroot}/%{site_install_dir}/js/jQueryMustache/
%{__install} -D -m 755 plugins/bigdesk/js/lib/jQueryMustache/jQueryMustache.js -t %{buildroot}/%{site_install_dir}/js/jQueryMustache/
%{__install} -D -m 755 plugins/bigdesk/js/lib/jquery/jquery-1.4.4.min.js -t %{buildroot}/%{site_install_dir}/js/lib/jquery/
%{__install} -D -m 755 plugins/bigdesk/js/lib/mustache/LICENSE -t %{buildroot}/%{site_install_dir}/js/lib/mustache/
%{__install} -D -m 755 plugins/bigdesk/js/lib/mustache/mustache.js -t %{buildroot}/%{site_install_dir}/js/lib/mustache/
%{__install} -D -m 755 plugins/bigdesk/js/lib/parseUri/parseuri.js -t %{buildroot}/%{site_install_dir}/js/lib/parseUri/
%files
%defattr(-,root,root,-)
%dir %{base_install_dir}/plugins/bigdesk
%{base_install_dir}/plugins/bigdesk/*
%changelog
* Tue Aug 14 2012 Andreas Marschke 1.0.0-0
- Initial package
Thanks and happy Hacking!
If bigdesk remains open for long periods, memory keeps growing. For example, in Chomium browser, the tab used big-desk consumed about 5Gb of RAM in about 5 hours.
To free the memory, I have to reopen and reconnect bigdesk to ElasticSearch.
Thanks for awesome Bigdesk. I want to know if i can setup any authentication to access Bigdesk app. Thanks in advance for help.
Current version seems to be running fine on 1.4.x but this version is missing from the Support matrix. Worth updating.
Tried running BigDesk against ES 1.7 and it does not seem to work...
Would like to check if 1.7 is supported, or if anyone was successful running BigDesk against their ES 1.7 cluster before i start hacking mine.
Thanks!
I installed bigdesk 2.2.0 and i keep getting errors after i select a node from the cluster.
I altered the js to stop throwing errors but im not sure what it was suppose to do.
So in the file SelectedClusterNodeView.js i added the following check at line 611:
if(stats_the_latest.node.indices.cache){
}
In the file bigdesk_charts.js i added similar checks (line 816 for example):
value: snapshot.node.indices.cache ? snapshot.node.indices.cache.filter_size_in_bytes : ""
Also had to make a find and replace to replace all "+snapshot" to "snapshot". Not sure what the "+" was for.
Hi,
Love bigdesk, by the way, but I upgraded to ES 0.90.4 (and also tried .5) and previously (with 0.90.3), bigdesk showed me the stats for my ES node but also for my 4 Logstash procs (nodes) that are running on the same server. This was really handy as I could see the heap usage for ES and logstash.
However now, (and I have latest bigdesk) it shows the ES and logstash nodes but the logstash ones are blank and show no info at all. It has the correct node name but what I notice is it shows the same PID and ID for all (the ES PID/ID).
I know bigdesk is primarily for ES so this is probably not an issue/priority for some but it was a really nice-to-have. I have attached a couple of images to show what I mean.
Is this something that could be fixed? I can see them with the kopf plugin but I preferred it in one place with bigdesk. Thanks.
For more details see elastic/elasticsearch#1115
Currently, if user switches between nodes then the history of the node is lost and new node values are loaded from scratch. It would be better to keep stats values in memory of all available nodes to allow smooth switching between nodes in UI.
My ElasticSearch cluster is listening private IP address and accepting connections only from frontend server.
How can I use big desk to monitor it in the case when ES server is not visible from the public network?
Using Chrome on Windows (will verify on other OSes soon).
While editing the default "http://localhost:9200" text in the "ES node REST endpoint" textbox, pressing Enter causes the page to refresh and the textbox revert to their default. The behavior should be execute "Connect".
Happy to see BigDesk on 0.19+
I am using ES 1.0.0 and bigdesk cloned with specific tag (v2.4.0) and Chrome browser (33.0.1750.117). After upgrade es to 1.0.0, bigdesk can't show detailed statistic info of each node.
Google Javascript console shows some messages like this:
Failed to load resource: the server responded with a status of 404 (Not Found)
The specific URI is as follows:
http://10.133.1.14:9200/_cluster/nodes/oIsi7nqjRi-aeG6luIz3pQ?all=true&callback=jQuery1710721426458330825_1393185588987&_=1393185609602
The returned message is:
jQuery1710721426458330825_1393185588987({"error":"IndexMissingException[[_cluster] missing]","status":404});
I can verify this request (same returned message) with curl command in the terminal. And if I change the above request URI to:
http://10.133.1.14:9200/_nodes/oIsi7nqjRi-aeG6luIz3pQ?all=true&callback=jQuery1710721426458330825_1393185588987&_=1393185609602
I can get lots of information returned though.
Hi!
I have a Found cluster with Bigdesk, I can see search, get & other queries but not any scan/scroll queries. Is that expected? We do mostly this type of queries and it would be great to be able to monitor them as well.
Thanks!
I was wondering, is there a way to apply some sort of authentication mechanism for accessing the bigdesk interface? Currently, anyone who knows the address will be able to access it.
Thanks
It'd be great if the refresh time could be saved in a cookie so that a browser/tab restart doesn't reset the timing down to 2 seconds, as that's not really a sane default when running bigdesk over a WAN link due to the amount of data it transfers.
Hello,
first of all thank you very much for the nice interface.
Would it be possible to add the elasticsearch version on the node's information?
Cheers,
Massimo
When starting with 1hour limit no graphs were shown. Switching back to 15min (and then to 1hour again) I can see the JVM graph.
my system graph stays empty, maybe this is a jvm issue as e.g. the cpu usage is not shown even in jvisualvm (although the mem usage is shown there)
I would like to add explicit authentication as I did for elasticsearch-head - see mobz/elasticsearch-head#57 and Asquera/elasticsearch-http-basic#6
I know it is a stupid work around for ElasticSearch inflexible header customization, but I only need ElasticSearch and two plugins (head+bigdesk). Now I cannot make it working for bigdesk. How can I add something to every request with backbone? The ajaxSetup what I was using seems to be "unavailable" for backbone and using beforeSend would need to change every request/fetch, right?
Also this trick to intercept sync does not add the header to the request for me.
Hello,
I have installed latest ES and latest version of bigdesk (as of today).
I also use Apache as proxy layer.
When I connect to the url
http://127.0.0.1:8080/_plugin/bigdesk/#nodes
I see only page basic page with connection and a message:
"No cluster connected."
After debugging I found out that one request is not created correctly for me:
Last two requests in debug are:
/cluster/health?callback=jQuery17106085601344238967_1406811659076&=1406811659388
and then following:
?callback=jQuery17106085601344238967_1406811659077&_=1406811659504
The first one will get response correctly, but the second will not provide info and will cause an error. It's missing "_cluster/health"
When I add /_cluster/health? before the second request, it is served and response is OK.
Is this issue for someone else?
I tried FF 31.0
and Chrome 35.0.1916.153
Do you have any suggestions?
Thanks,
Rado
When pointing BigDesk (current master as of a few minutes ago) against a ES cluster I get this error when clicking on Nodes:
Uncaught TypeError: Cannot read property 'getMasterNodeId' of null bigdeskApp.js:372
Backbone.Router.extend.nodes_master bigdeskApp.js:372
(anonymous function) backbone-min.js:25
(anonymous function) backbone-min.js:30
b.some.b.any underscore-min.js:14
f.extend.loadUrl backbone-min.js:30
f.extend.checkUrl backbone-min.js:30
f.event.dispatch jquery-1.7.1.min.js:3
h.handle.i jquery-1.7.1.min.js:3
With Cluster I get:
Uncaught TypeError: Cannot read property 'get' of undefined ClusterHealthView.js:24
Backbone.View.extend.initialize ClusterHealthView.js:24
g.View backbone-min.js:32
d backbone-min.js:36
clusterView.render bigdeskApp.js:67
Backbone.Router.extend.cluster bigdeskApp.js:360
(anonymous function) backbone-min.js:25
(anonymous function) backbone-min.js:30
b.some.b.any underscore-min.js:14
f.extend.loadUrl backbone-min.js:30
f.extend.checkUrl backbone-min.js:30
f.event.dispatch jquery-1.7.1.min.js:3
h.handle.i jquery-1.7.1.min.js:3
The one query I see it do in Chrome's network trace is:
http://test:9200/_cluster/health?callback=jQuery171004769798368215561_1406574553380&_=1406574561869
which when called directly returns
{
"cluster_name": "test",
"status": "green",
"timed_out": false,
"number_of_nodes": 4,
"number_of_data_nodes": 4,
"active_primary_shards": 9,
"active_shards": 18,
"relocating_shards": 0,
"initializing_shards": 0,
"unassigned_shards": 0
}
Getting a warning complaining of version mismatch. Bigdesk won't seem to let me connect after acknowledging the warning.
Installed ES 0.20.5, and lukas-vlcek/bigdesk/2.2.0
Here is the warning displayed in the popup:
Bigdesk may not work correctly!
Found ES node version: 0.20.5
Requires ES node version: 0.20.x
Some of the index info is there along with the correct node name and cluster name.
I do not see any data for system, jvm, file descriptors and cache size graph...
I was doing some load testing and noticed in the Elasticsearch logs that i was getting No SPACE left on device. But this is something i couldn't see on the GRAPHS in Bigdesk,
-->Is there a way i can see this error on Big desk?
Also the error reported
java.io.FileNotFoundException: /var/elasticsearch/elasticsearch-0.90.3/data/elasticsearch5/nodes/0/indices/dw/3/index/_dzg.si (Too many open files)
--> Is there a way to determine this error in big desk?
Thanks
Hi loving the plugin! We started using it about a month ago and it has allowed my team to get a real time view on what our nodes are doing. Hoping I can help to fix an issue we have with using it on more sizable clusters.
For any of our larger clusters we have many customers, and many indexes (~100). Calling "_cluster/state" is a guaranteed way to kill a browser. it is currently over 11 mega bytes of data. Chrome kills the page, other browsers even crash.
It maybe possible to use the filters on cluster state to reduce the size while still maintaining functionality?
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-state.html
If I switch on no metadata filter the size drops down to 665Kb. It's much faster and possibly would scale to clusters much larger than ours. Not sure that's doable.
I don't want to preemptively fork and code. If you think this is something that warrants attention let me know. I will be extremely happy to add my coding efforts.
hi
im getting JS error:
Error: Invalid value for attribute r="NaN" d3.v2.min.js:2131
ES version: 0.20.6
bigdesk version: well master, dunno where to find your versioning
installed ES and bigdesk out of the box, no data indexed yet.
just visit bigdesk like :
http://domain/context_path/_plugin/bigdesk
and then i can't get the cluster info,because bigdesk send ajax like
http://domain/_plugin/bigdesk
You used to show the IP address of the node, which was useful.
Please can we see that somewhere again?
Filter out unused data from indices status REST endpoint response.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.