dotnetcore / mocha Goto Github PK
View Code? Open in Web Editor NEWMocha is an application performance monitor tools based on OpenTelemetry, which also provides a scalable platform for observability data analysis and storage.
License: MIT License
mocha's People
Contributors
Stargazers
Watchers
Forkers
eventhorizon-cli azhi98 mygit-2023 liuhaoyang tsaiggo knightyyj dotnettreasury gmh5225 webb-z zpczpc benlampson kawhiwei yepiaoran17 joesdu bluetianx tersonly myfirstway wangjie868686 zlzforever xiaoheivirtual netcore-jroger lingbohome jackieyangyi rchcommmocha's Issues
[task] distributor 实现 OTLP Trace gRPC Server
- 在 distributor 中以 submodule 方式引入 opentelemetry-proto
- 实现 trace_service.proto来接收 Trace 的数据
[task] proposal design of the distributor component
The proposal design of the Distributor needs to be completed as a priority, as all future tasks rely on it.
[task] Trace storage model and ITraceStorage interface definition
- Determine the storage model for trace data
- Ability to efficiently query corresponding span data using trace IDs
- Ability to filter and query trace lists using service name, operation, or tags, as shown below
Chinese User Guide
Build a Chinese User Guide Document for Chinese User Guide and Integrated into system
[task] compatible with Jaeger query protocol
Grafana ships with built-in support for Jaeger, which provides open source, end-to-end distributed traces.
Once Mocha is compatible with the Jaeger query protocol, it will be possible to directly use Grafana to explore traces.
Here are references to the Jaeger plugin and jaeger query protocol:
[task] proposal design of the standalone mode
It has the following two purposes:
- Provide a low-cost deploy option for small-scale services
- Make it easier for Mocha developers to standalone mode for local testing.
[task] define the buffer api used by distributor
[proposal] mocha : 云原生 APM 和大规模可观测性数据处理平台
背景
近年来，可观测性概念被提出和逐渐流行，OpenTelemetry 也逐渐成为最流行的可观测性框架，OTel 很好的解决了多语言系统中Metrics\Trace\Log 的收集和标准化问题，但对于如何存储和分析使用收集到的 M.T.L 数据，业界并没有统一的方案，一般来说大家需要
- 使用 Jaeger、Prometheus、Loki 等不同的后端系统搭配，但引入了相当的复杂度
- 引入 SkyWalking 、ElasticAPM 等 APM 系统
- 使用 Datadog、SLS 等 SaaS 可观测性平台，需要支付昂贵的流量和数据存储费用
同时上述提到的开源 APM 后端，除 SkyWalking [Java实现]外，其余无一例外使用 Golang 实现。
而从 .NET 5 以来，到目前的 .NET 8，每一个版本都对 CLR 和 BCL做了大量性能优化和提供了面向高性能场景的新语言特性，.NET 的演进很适合开发高性能的云原生中间件。所以我们发起 mocha 项目，使用 .NET 实现一个面向大规模可观测性数据分析和存储的平台。
Mocha 的定位：基于 OpenTelemetry 的 APM 系统，同时提供可伸缩的可观测性数据分析和存储平台。
平台功能
Mocha 将要提供的功能集合：
- APM 和 分布式追踪
- 服务概览、R.E.D 指标和可用性监控
- 服务拓扑
- 调用监控，包括 HTTP、RPC、Cache、DB、MQ 等
- 调用链路查询和检索
- 基础设施监控
- 主机监控
- 容器和 Kubernetes 监控
- 主流中间件监控
- 日志
- 日志查询
- 日志聚合分析
- 报警
- 报警规则管理
- 报警通知
- M.T.L 数据探索 [Data Explore / Inspect]
技术架构
Mocha 由下面的部分组成
- Mocha Distributor Cluster：作为 mocha 系统的数据入口，负责接收 OTel SDK 和 Collector 上报的数据，并通过一致性Hash 将数据路由到对应的 aggregator 节点上。为了保证数据不丢失，最终 Distributor 应该具备本地 FIFO 队列的能力。
- Mocha Aggregator Cluster：mocha 的核心组件，通过读取预配置或者用户配置的 aggr rule dsl 生成对应的 streaming data flow 并执行。Aggregator 是具备分布式 shuffle 的能力的有状态组件，需要将自身信息注册到ETCD中。
- Storage：mocha M.T.L 存储，可以选用开源存储组件，如 ClickHouse、ElasticSearch、victoriametrics 等。
- Mocha Querier + Grafana: 从存储查询数据并提供给 grafana 做展示。因此要兼容 promql / jeager / loki 等数据源的查询。
- Mocha Manager : 包括 manager server、dashboard和ETCE组件，集群元数据和 M.T.L 数据分析规则存储。
- OTel SDK / Collector : 开源 OpenTelemetry 采集套件。
[task] Implement trace storage using Entity Framework Core and MySQL
In v0.1 , we prioritize MySQL as the trace storage. It has the following two purposes:
- Provide a low-cost trace storage option for small-scale services
- Make it easier for Mocha developers to use MySQL storage for local testing.
Build error for commit a6910fe417b66ed76dc2a2326416d3a366c4a708
For: a6910fe (recursive checkout)
Running on Windows 10 with Docker Desktop latest, docker compose up -d fails (in Windows Terminal with PS 7.4.1)
=> ERROR [distributor build 10/10] RUN dotnet build "Mocha.Distributor.csproj" -c Release -o /app/build 5.7s
=> [jaeger-query build 8/10] COPY . . 0.1s
=> [jaeger-query build 9/10] WORKDIR /src/src/Mocha.Query.Jaeger 0.0s
=> CANCELED [jaeger-query build 10/10] RUN dotnet build "Mocha.Query.Jaeger.csproj" -c Release -o /app/build 4.9s
------
> [distributor build 10/10] RUN dotnet build "Mocha.Distributor.csproj" -c Release -o /app/build:
0.592 MSBuild version 17.8.3+195e7f5a3 for .NET
1.133 Determining projects to restore...
2.182 All projects are up-to-date for restore.
4.762 Mocha.Protocol.Generated -> /app/build/Mocha.Protocol.Generated.dll
5.551 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(6,7): error CS0246: The type or namespace name 'OpenTelemetry' could not be found (are you missing a using directive or an assembly reference?) [/src/src/Mocha.Core/Mocha.Core.csproj]
5.551 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(7,7): error CS0246: The type or namespace name 'OpenTelemetry' could not be found (are you missing a using directive or an assembly reference?) [/src/src/Mocha.Core/Mocha.Core.csproj]
5.552 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(8,7): error CS0246: The type or namespace name 'OpenTelemetry' could not be found (are you missing a using directive or an assembly reference?) [/src/src/Mocha.Core/Mocha.Core.csproj]
5.552 /src/src/Mocha.Core/Storage/ISpanWriter.cs(5,7): error CS0246: The type or namespace name 'OpenTelemetry' could not be found (are you missing a using directive or an assembly reference?) [/src/src/Mocha.Core/Mocha.Core.csproj]
5.552 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(14,46): error CS0305: Using the generic type 'Span<T>' requires 1 type arguments [/src/src/Mocha.Core/Mocha.Core.csproj]
5.552 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(14,57): error CS0246: The type or namespace name 'Resource' could not be found (are you missing a using directive or an assembly reference?) [/src/src/Mocha.Core/Mocha.Core.csproj]
5.552 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(59,57): error CS0305: Using the generic type 'Span<T>' requires 1 type arguments [/src/src/Mocha.Core/Mocha.Core.csproj]
5.552 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(69,55): error CS0305: Using the generic type 'Span<T>' requires 1 type arguments [/src/src/Mocha.Core/Mocha.Core.csproj]
5.552 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(82,57): error CS0246: The type or namespace name 'KeyValue' could not be found (are you missing a using directive or an assembly reference?) [/src/src/Mocha.Core/Mocha.Core.csproj]
5.578
5.579 Build FAILED.
5.580
5.580 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(6,7): error CS0246: The type or namespace name 'OpenTelemetry' could not be found (are you missing a using directive or an assembly reference?) [/src/src/Mocha.Core/Mocha.Core.csproj]
5.580 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(7,7): error CS0246: The type or namespace name 'OpenTelemetry' could not be found (are you missing a using directive or an assembly reference?) [/src/src/Mocha.Core/Mocha.Core.csproj]
5.580 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(8,7): error CS0246: The type or namespace name 'OpenTelemetry' could not be found (are you missing a using directive or an assembly reference?) [/src/src/Mocha.Core/Mocha.Core.csproj]
5.580 /src/src/Mocha.Core/Storage/ISpanWriter.cs(5,7): error CS0246: The type or namespace name 'OpenTelemetry' could not be found (are you missing a using directive or an assembly reference?) [/src/src/Mocha.Core/Mocha.Core.csproj]
5.580 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(14,46): error CS0305: Using the generic type 'Span<T>' requires 1 type arguments [/src/src/Mocha.Core/Mocha.Core.csproj]
5.580 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(14,57): error CS0246: The type or namespace name 'Resource' could not be found (are you missing a using directive or an assembly reference?) [/src/src/Mocha.Core/Mocha.Core.csproj]
5.580 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(59,57): error CS0305: Using the generic type 'Span<T>' requires 1 type arguments [/src/src/Mocha.Core/Mocha.Core.csproj]
5.580 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(69,55): error CS0305: Using the generic type 'Span<T>' requires 1 type arguments [/src/src/Mocha.Core/Mocha.Core.csproj]
5.580 /src/src/Mocha.Core/Models/Trace/OTelToMochaSpanConversionExtensions.cs(82,57): error CS0246: The type or namespace name 'KeyValue' could not be found (are you missing a using directive or an assembly reference?) [/src/src/Mocha.Core/Mocha.Core.csproj]
5.581 0 Warning(s)
5.581 9 Error(s)
5.582
5.582 Time Elapsed 00:00:04.82
------
failed to solve: process "/bin/sh -c dotnet build \"Mocha.Distributor.csproj\" -c $BUILD_CONFIGURATION -o /app/build" did not complete successfully: exit code: 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.