couchbase / cbmonitor Goto Github PK
View Code? Open in Web Editor NEWcbmonitor
License: Apache License 2.0
cbmonitor
License: Apache License 2.0
I eventually figured this out when ns_collector gave me the message:
seriesly.exceptions.ConnectionError: Connection refused to "http://127.0.0.1:3133/"
But it would be nice if the README made this more clear up front.
Traceback (most recent call last):
File "cbtop", line 9, in
from libcbtop.main import main
File "priority15/libcbtop/main.py", line 9, in
import visit_cb as vc
File "priority15/libcbtop/visit_cb.py", line 11, in
from tabula.table import Table
File "virtualenvs/priority15/lib/python2.6/site-packages/tabula/table.py", line 6, in
from section import Section
File "virtualenvs/priority15/lib/python2.6/site-packages/tabula/section.py", line 323
{k:v-1 for k,v in self.irt.iteritems() if v > row_num})
^
SyntaxError: invalid syntax
Currently this is defined manually:
function (doc, meta) {
emit(meta.id, doc);
}
And it must be integrated into dashboard functionality.
I've got a workflow that seems to work for me, but it involves a lot of deleting everything and starting over, and I don't know if thats right. Typically right now I do:
To start new test, start over at number 1.
If I deviate from these steps I usually get data from previous tests showing up, or errors about cluster names already existing, collectors crashing, etc.
Right now cluster tree looks like
cluster
|_
server
|_
bucket
which is wrong because bucket is part of cluster.
I created my own config file and ran:
bin/ns_collector laptop.cfg
This appears to do something, a cbmonitor.couch file shows up, and the file size grows over time.
Next I started web UI:
bin/webapp runserver
When I view http://localhost:8000/ there are no clusters visible. In the demo you said something about normally not needing to add them manually. Is there another step I should perform here instead?
Next I went ahead and tried adding it manually. I added the information about the cluster running on my laptop. It seems to discover all the correct information about this cluster. Next if I click over to the Metrics area, select a cluster, node, and the first bucket "beer-sample". I see only one metric:
swap usage
If try to drag it to the view window on the right, nothing happens.
If select any other bucket, I don't see any metrics at all.
I see query for one of our metrics going to seriesly as:
In my seriesly db this returns data like:
{
"1361547184000": [
null
],
"1361547194000": [
null
],
"1361547204000": [
null
],
"1361547214000": [
null
],
"1361547225000": [
null
],
"1361547227000": [
7303
],
"1361547235000": [
null
],
"1361547237000": [
7303
],
"1361547245000": [
null
],
"1361547248000": [
7303
],
"1361547255000": [
null
],
"1361547259000": [
7303
],
"1361547265000": [
null
],
"1361547269000": [
7303
],
"1361547275000": [
null
],
"1361547280000": [
7303
],
"1361547285000": [
null
],
"1361547291000": [
7303
],
"1361547295000": [
null
],
"1361547301000": [
7303
],
"1361547305000": [
null
],
"1361547312000": [
7303
],
"1361547315000": [
null
],
"1361547323000": [
7303
],
"1361547326000": [
null
],
"1361547335000": [
7303
],
"1361547346000": [
7303
]
}
I believe the large number of nulls is because I have many different kinds of metrics being stored (also from multiple clusters).
What if we stored data more like:
{
"meta": {
"cluster": cluster-name (optional)
"node": node-name (optional)
"bucket": bucket-name (optional)
},
"metric": {
...
}
}
Then in we'd use both ptr and filters in the query:
ptr=/metric/curr_items_tot
f=/meta/cluster
fv=laptop
f=/meta/bucket
vb=beer-sample
f=/meta/node
fv=127.0.0.1
This would also be more extendable for different kinds of metric meta-data in the future.
Also would like to ping @dustin for input on this.
Baseline builds, sorting preference and number of items per page.
If I run this command to list stats keys and filter:
curl http://localhost:8091/pools/default/buckets/beer-sample/stats | jq '.op.samples | keys' | grep replication
I see the following:
"replication_active_vbreps",
"replication_changes_left",
"replication_commit_time",
"replication_data_replicated",
"replication_docs_checked",
"replication_docs_rep_queue",
"replication_docs_written",
"replication_num_checkpoints",
"replication_num_failedckpts",
"replication_size_rep_queue",
"replication_waiting_vbreps",
"replication_work_time",
"replications/f4268e62702130298bf87f17cc481219/beer-sample/bs/active_vbreps",
"replications/f4268e62702130298bf87f17cc481219/beer-sample/bs/changes_left",
"replications/f4268e62702130298bf87f17cc481219/beer-sample/bs/data_replicated",
"replications/f4268e62702130298bf87f17cc481219/beer-sample/bs/docs_checked",
"replications/f4268e62702130298bf87f17cc481219/beer-sample/bs/docs_rep_queue",
"replications/f4268e62702130298bf87f17cc481219/beer-sample/bs/docs_written",
"replications/f4268e62702130298bf87f17cc481219/beer-sample/bs/num_checkpoints",
"replications/f4268e62702130298bf87f17cc481219/beer-sample/bs/num_failedckpts",
"replications/f4268e62702130298bf87f17cc481219/beer-sample/bs/size_rep_queue",
"replications/f4268e62702130298bf87f17cc481219/beer-sample/bs/time_committing",
"replications/f4268e62702130298bf87f17cc481219/beer-sample/bs/time_working",
"replications/f4268e62702130298bf87f17cc481219/beer-sample/bs/waiting_vbreps",
However, none of these show up in the UI.
Currently we plot all charts, this is too expensive. It takes time to create even empty PNG image. We must be more selective.
For instance, we can exclude stats for all nodes by default and plot only cluster-wide metrics. Also we need "report presets" with a list of typical metrics that must be plotted. Like "View performance test", "XDCR performance test", "System test", and etc.
refactoring
Current rest api supports one metric per request:
build=2.0.0-1723-rel-enterprise&metric=Latency, ms&value=555
I'm running Chrome 24.0.1312.57, when I drag metric to right-hand side nothing happens.
When I switched to Firefox it works correctly.
The specific case I have in mind when we create an new XDCR, new stats are introduced for this. I think with the current model there is no way to discover these new metrics once data collection has started.
2013-02-21 10:45:26 - WARNING - {"name": "Ensure this value has at most 64 characters (it has 75)."}
[21/Feb/2013 10:45:26] "POST /cbmonitor/add_metric_or_event/ HTTP/1.1" 400 68
2013-02-22 07:33:41 - WARNING - columns name, cluster_id, server_id, bucket_id are not unique
[22/Feb/2013 07:33:41] "POST /cbmonitor/add_metric_or_event/ HTTP/1.1" 400 61
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.