Comments (2)
A note about metrics-clojure would be helpful, too.
from mulog.
Hi @timofeytt
You are correct saying that Riemann is somehow overlapping with µ/log, in fact both systems are event-based systems although in Riemann the basic event is a metric event (an event that describes or samples a metric).
In µ/log, each event is a free-form, pure, event which means that you as in Reimann you have a bunch of categorical properties (tags) which you can use to "slice & dice" the events and group them the way you want, but, in opposition to Riemann, µ/log doesn't constrain the user to a single numerical field.
If an event needs multiple numerical properties to describe it fully in µ/log you can pack this information in a single event.
Another difference is that the core of Riemann is a streaming and aggregation engine which allows you to turn raw data into high level (meaningful) insights. µ/log (at this stage) is just a client to produce the raw info.
It is entirely possible to write a µ/log publisher to send µ/log events to Riemann in its expected format.
Regarding metrics-clojure
the difference is more fundamental. metrics-clojure
, like many other libraries, is basic of a metering system. Events happen on a remote system and get aggregated at the source, then, time to time, the metric is sampled and the sample is sent to a collection system. Because the events are aggregated at the source, you are not able to slice & dice the metrics at query time unless you have expressly captured that particular category.
I'm very familiar with this approach, I used it for many years and I even wrote a Clojure wrapper for it (TRACKit!). Some tools like Prometheus try to overcome the lack of categorical dimensions providing a hybrid approach, but still not as rich as µ/log.
The benefits of switching to an event-based system are enormous although not very apparent at the start.
Instrumenting your code with a metrics
library and produce a rich set of metrics is a very tedious and time consuming.
For example, if you instrument only your webservice request handlers with µ/log you could ask the following questions:
- how many requests I've received in the last week
- how many requests by day/hour/minute/second
- how many requests by user over time
- how many requests by endpoint overtime
- how many requests were failures (4xx or 5xx)
- of the error request, how many were for a specific endpoint
- which user issued the failing requests
- what do they have different than the successful requests
- what's the latency distribution of the successful request vs the failed requests
- which content-type/content-encoding was used
- what's the distribution of the failures by host/jvm
- what's the status JVM metrics (GC/memory/etc) of failing hosts during that time.
- what's the repartition of the latencies between internal processing and external connections (db query, caches, etc)
and much more. All this from 1 single good log instrumentation.
To achieve the same with a metrics system you will need several dozens of metrics to be collected and published.
µ/log works incredibly well with Elasticsearch which is an amazing tool to slice and dice the data the way you need.
One side of Elasticsearch which is not very well known is that Elasticsearch has a very fast and robust aggregation engine as well.
The final point is that traditional systems consider logs different from metrics and different from traces (the 3 pillars of observability), in reality, they are all different forms of events. For example, the same events that you can use for the logs and to capture metrics can represent traces. In µ/log, if you add a Zipkin publisher you get the traces collected and visualised as follow:
all this just come from simple µ/log instrumentation.
from mulog.
Related Issues (20)
- Character code for `µ` HOT 2
- Does mulog publisher for zipkin support Annotations? HOT 1
- Extract file/line info from callsites?
- µ/log publisher for OpenTelemetry HOT 6
- Logging maps HOT 7
- Merge map keys into µ/log statement HOT 2
- Add payload length to the zipkin publisher HOT 1
- AWS Cloudwatch logs are not generally "discoverable" HOT 2
- Publish errors with Cloudwatch Logs HOT 8
- Slack Publisher Documentation implies support for Block Kit messages HOT 1
- Errors serializing to JSON are unreasonably difficult to debug HOT 2
- Publishers continually retrying HOT 3
- Silent failure with trace and :capture HOT 1
- `:data-stream` shows up as index in Elasticsearch 8 HOT 3
- :mulog/publisher-error NullPointerException HOT 4
- Cascading errors in elasticsearch publisher when using multi publisher HOT 8
- mulog-elasticsearch does not work with data-streams HOT 3
- suggestion: lmax disruptor queues HOT 2
- ElasticSearch `type-mangle` fn causes type problems HOT 1
- Put event name in options map HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mulog.