hastic-zzz / hastic-server Goto Github PK

Current release-archive size is ~55MB
We should reduce it somehow

• What changes should we do in UI and requirements from it
• Algorithm implementation

@rozetko, @VargBurz just add some thoughts at least

We should send stacktrace and some additional data somewhere on errors for easier debugging

It's a problem starting without webpack:

You get this error when you've just created anomaly and there are no segments yet

All of your models: drops, peaks, ... don`t give a shit about deletion.

I think we should rethink this part: https://github.com/hastic/hastic-server#known-bugs--issues

Let's check what is actual and make issues here on github if we have bugs.

https://nodejs.org/api/child_process.html#child_process_options_detached

I am not sure that we close it properly @rozetko
let's check it

With usage "General approach", user gets error from python
like follows:

It is simpler version of https://github.com/hastic/hastic-server/issues/44
where we don't make proper classification, we just select the pattern with best fit.

Currently, dataset doesn't get updated after first downloading
We should update it on every learning

Sometimes you want to detect when you don't have a pattern in your data and get notifications about it

Any thoughts?

Steps to reproduce:

• clone repo
• go to analytics/
• pip3 install -r requirements.txt
• python3 server.py

You would see error:

Traceback (most recent call last):
  File "server.py", line 7, in <module>
    from worker import Worker
  File "/mnt/c/Users/rozetko/git/hastic-server/analytics/worker.py", line 2, in <module>
    from anomaly_model import AnomalyModel
  File "/mnt/c/Users/rozetko/git/hastic-server/analytics/anomaly_model.py", line 2, in <module>
    from data_provider import DataProvider
  File "/mnt/c/Users/rozetko/git/hastic-server/analytics/data_provider.py", line 1, in <module>
    import pandas as pd
  File "/home/rozetko/.local/lib/python3.5/site-packages/pandas/__init__.py", line 42, in <module>
    from pandas.core.api import *
  File "/home/rozetko/.local/lib/python3.5/site-packages/pandas/core/api.py", line 10, in <module>
    from pandas.core.groupby import Grouper
  File "/home/rozetko/.local/lib/python3.5/site-packages/pandas/core/groupby/__init__.py", line 2, in <module>
    from pandas.core.groupby.groupby import (
  File "/home/rozetko/.local/lib/python3.5/site-packages/pandas/core/groupby/groupby.py", line 42, in <module>
    from pandas.core.dtypes.missing import isna, isnull, notna, _maybe_fill
ImportError: cannot import name 'isna'

• Make it work only after labeling (like drops)
• Make it find peaks similar to labeled
• Find / create test dataset
• Write tests

Now anomaly name is case-sensitive
If you create anomaly with capital letter(s) in name - you get "Not found" or "Internal error" alert
We should change all names to lower-case

I like how Grafana does it:

https://grafana.com/grafana/download?platform=linux

just installation from source and everything works

Guess we don't need 2 types of build because node 6.14 build would work in any 6.14+ version.
Also, documentation with different types of build looks confusing.
@jonyrock what do you think?

If you make a new pattern with name "drop" and you already have "drop" in database, learning state would start

We want to send data source where we will find correlated metrics automatically. When just reports than metrics which analytic found.
Sounds futuristic , but lets make subtasks.

I believe that analytics should know about how extract data. It is server responsibility.
We need to send data for initial learning and then data about new data from datasource.

There are many benefits from this.

@rozetko lets stop querying grafana from python

For example, we need to detect patterns in for few metrics together like in merge sort.
Another example is: https://github.com/hastic/hastic-server/issues/44 where apply model to predict class.
So we need to keep in memory model for few pattern predictors.

• Module for multiple metrics analysis
• Alerting refactoring

We haven't defined how system should behave when we have no data. We need to understand requirements to the system first.

It would be nice to see a proper error message in panel alert instead of just "Internal error"

We want to support old versions of Node.js
I think we better setup CI for this and update it in docker

We still use anomaly names in some cases instead of IDs
It is really confusing
We should use only IDs

We want to make node.js part the service which responsible for data processing.
So python doesn't write to anywhere.

@rozetko
I think our docker file is not actual. We use requirements.txt and least

There is a bug when indices get transformed to time

In this screenshot all segments have the same length in indices, but the one on the left has a lot larger length

We can't scale with files. Obviously.

We update alerts.json to send "notifications" about new detected alerts. It is crazy.
I think it is not the only place.

The module https://github.com/hastic/hastic-server/blob/8fce88d76a795656e73262704df735f625717978/server/src/services/analytics.ts

has both transport & logic in one code. Lets separate it.

It would be also nice to have a way to debug python code

We want to use zeromq for messaging between node.js / python

It will allow to separate processes and debug easier

• ZeroMQ Node/Python worker refactoring
• Docs to new installation
• Remove redundant deps after refactoring
• Return analytics status
• Fix building for production

Any behavior even if is defined should be rethinked . I think it is the case where user should define which strategy to choose.

we use any - like definitions for task, but we can do it more accurate

let task = {
    type: 'learn',
    anomaly_id: anomalyId,
    pattern,
    segments: segments
  };

I am sorry to say that, but I need to use google to find instructions about how should I install python and pip.

I just want to start to use software. Is there a nice way to install Python/pip on my machine without googling ? :)

For example, with node.js you only need to link to: https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions and that's it. Just one command.

@rozetko what do you think?

We can face a problem where user has an older version of hastic-panel and we need to return version of hastic-server. Also it is necessary for bug reports.

So we need to

• keep track of current version of server with is the same as release
• return server version when get status (root) url

The first time we get this problem: https://github.com/hastic/hastic-server/pull/65

We now want to have pattern for multiple series.

We need to make

• Algorithms with multiple series
• Node Server part
• UI

We should get HASTIC_API_KEY from config if it exists. Otherwise, use environment variable.

Imagine log which looks like this:

[ANALYTICS] something
[ANALYTICS] something more
[SERVER] oh oh
[SERVER] more logs
[ANALYTICS] learning

@rozetko

We have entities like:

• anomaly
• pattern
• segment
• and so on
  It can be hard to understand them sometimes.
  We should create doc with their interpretation.

Currently only InfluxDB datasource is supported
We should find a way to support other Grafana's datasources

https://github.com/hastic/hastic-server/blob/0b857996d8e2e7844c3002698d53d32a92b884d5/analytics/server.py#L36

We need to wrap task result payload in res with metadata with taskId and result of execution.
so it's like:

{
"taskId": 123123,
"status": "ok",
"data": {
    ... payload data ...
  } 
}

It would be nice to have Kalman filter for example before we do analytics

Steps to reproduce:

• create anomaly
• find it in data/anomalies/all_anomalies.json
• restart server
• create new anomaly

Now there is only new anomaly in all_anomalies.json
It happens because loadAnomaliesMap() method doesn't get executed on anomaly insert

https://pip.readthedocs.io/en/1.1/requirements.html

Based on https://github.com/hastic/hastic-server/issues/53#issuecomment-402556227

We need to add documentation on how to build release version

SImillar to drops. Just want to see it working

When server already started learning / prediction task and you delete anomaly - it doesn't know about this fact and continue doing the task => you have to wait for it to finish before it starts analysing another anomaly