maksim-paskal / envoy-control-plane Goto Github PK
View Code? Open in Web Editor NEWLightweight Envoy control plane
License: Apache License 2.0
envoy-control-plane's People
Contributors
Stargazers
Watchers
Forkers
dghermanenvoy-control-plane's Issues
Envoy restart with critical exception
Sometime some envoy's sidecar restarts with error
[2022-02-14 23:28:53.199][22][critical][main] [source/exe/terminate_handler.cc:12] std::terminate called! (possible uncaught exception, see trace)
[2022-02-14 23:28:53.199][22][critical][backtrace] [./source/server/backtrace.h:91] Backtrace (use tools/stack_decode.py to get line numbers):
[2022-02-14 23:28:53.199][22][critical][backtrace] [./source/server/backtrace.h:92] Envoy version: a9d72603c68da3a10a1c0d021d01c7877e6f2a30/1.21.0/Clean/RELEASE/BoringSSL
[2022-02-14 23:28:53.218][22][critical][backtrace] [./source/server/backtrace.h:96] #0: Envoy::TerminateHandler::logOnTerminate()::$_0::operator()() [0x55d953a74f0e]
[2022-02-14 23:28:53.228][22][critical][backtrace] [./source/server/backtrace.h:98] #1: [0x55d953a74dd9]
[2022-02-14 23:28:53.237][22][critical][backtrace] [./source/server/backtrace.h:96] #2: std::__terminate() [0x55d953f25433]
[2022-02-14 23:28:53.246][22][critical][backtrace] [./source/server/backtrace.h:96] #3: std::__1::__function::__func<>::operator()() [0x55d9535f67a3]
[2022-02-14 23:28:53.258][22][critical][backtrace] [./source/server/backtrace.h:96] #4: event_process_active_single_queue [0x55d953915220]
[2022-02-14 23:28:53.268][22][critical][backtrace] [./source/server/backtrace.h:96] #5: event_base_loop [0x55d953913f11]
[2022-02-14 23:28:53.278][22][critical][backtrace] [./source/server/backtrace.h:96] #6: Envoy::Server::InstanceImpl::run() [0x55d95318261c]
[2022-02-14 23:28:53.287][22][critical][backtrace] [./source/server/backtrace.h:96] #7: Envoy::MainCommonBase::run() [0x55d951d5cd64]
[2022-02-14 23:28:53.297][22][critical][backtrace] [./source/server/backtrace.h:96] #8: Envoy::MainCommon::main() [0x55d951d5d5d6]
[2022-02-14 23:28:53.307][22][critical][backtrace] [./source/server/backtrace.h:96] #9: main [0x55d951d5979c]
[2022-02-14 23:28:53.310][22][critical][backtrace] [./source/server/backtrace.h:96] #10: __libc_start_main [0x7f4eda35dbf7]
[2022-02-14 23:28:53.310][22][critical][backtrace] [./source/server/backtrace.h:104] Caught Aborted, suspect faulting address 0x6500000016
[2022-02-14 23:28:53.310][22][critical][backtrace] [./source/server/backtrace.h:91] Backtrace (use tools/stack_decode.py to get line numbers):
[2022-02-14 23:28:53.310][22][critical][backtrace] [./source/server/backtrace.h:92] Envoy version: a9d72603c68da3a10a1c0d021d01c7877e6f2a30/1.21.0/Clean/RELEASE/BoringSSL
[2022-02-14 23:28:53.310][22][critical][backtrace] [./source/server/backtrace.h:96] #0: __restore_rt [0x7f4eda73f980]
[2022-02-14 23:28:53.320][22][critical][backtrace] [./source/server/backtrace.h:98] #1: [0x55d953a74dd9]
[2022-02-14 23:28:53.320][22][critical][backtrace] [./source/server/backtrace.h:96] #2: std::__terminate() [0x55d953f25433]
[2022-02-14 23:28:53.320][22][critical][backtrace] [./source/server/backtrace.h:96] #3: std::__1::__function::__func<>::operator()() [0x55d9535f67a3]
[2022-02-14 23:28:53.320][22][critical][backtrace] [./source/server/backtrace.h:96] #4: event_process_active_single_queue [0x55d953915220]
[2022-02-14 23:28:53.320][22][critical][backtrace] [./source/server/backtrace.h:96] #5: event_base_loop [0x55d953913f11]
[2022-02-14 23:28:53.320][22][critical][backtrace] [./source/server/backtrace.h:96] #6: Envoy::Server::InstanceImpl::run() [0x55d95318261c]
[2022-02-14 23:28:53.320][22][critical][backtrace] [./source/server/backtrace.h:96] #7: Envoy::MainCommonBase::run() [0x55d951d5cd64]
[2022-02-14 23:28:53.320][22][critical][backtrace] [./source/server/backtrace.h:96] #8: Envoy::MainCommon::main() [0x55d951d5d5d6]
[2022-02-14 23:28:53.320][22][critical][backtrace] [./source/server/backtrace.h:96] #9: main [0x55d951d5979c]
[2022-02-14 23:28:53.320][22][critical][backtrace] [./source/server/backtrace.h:96] #10: __libc_start_main [0x7f4eda35dbf7]
slow kubernetes API call results to zero entrypoints
if kunernetes API handles calls very slow (> 2s) on control-plane no entrypoints to handle traffic
StreamRoutes gRPC config stream closed
Got this warnings on envoy proxy, it's hapends every ~ 2 minues
envoy-test1_1 | [2021-06-23 11:05:16.540][16][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:101] StreamRoutes gRPC config stream closed: 13,
envoy-test1_1 | [2021-06-23 11:05:16.540][16][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:101] StreamListeners gRPC config stream closed: 13,
envoy-test1_1 | [2021-06-23 11:05:16.540][16][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:101] StreamEndpoints gRPC config stream closed: 13,
envoy-test1_1 | [2021-06-23 11:05:16.540][16][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:101] StreamClusters gRPC config stream closed: 13,
envoy-test1_1 | [2021-06-23 11:05:16.540][16][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:101] StreamEndpoints gRPC config stream closed: 13,
if run control-plane with GODEBUG=http2debug=2 got this lines before warnings
envoy-control-plane_1 | 2021/06/23 11:05:16 http2: Framer 0xc0004062a0: wrote GOAWAY len=22 LastStreamID=9 ErrCode=ENHANCE_YOUR_CALM Debug="too_many_pings"
Service created with manual endpoints throws panic
Service
kind: Service
apiVersion: v1
metadata:
name: my-es
spec:
clusterIP: None
---
kind: Endpoints
apiVersion: v1
metadata:
name: my-es
subsets:
- addresses:
- ip: 172.22.111.250
- ip: 172.22.149.230
ports:
- port: 9200Exception
k8s.io/apimachinery/pkg/util/runtime.logPanic({0x1cc6080?, 0x334b600})
k8s.io/[email protected]/pkg/util/runtime/runtime.go:75 +0x99
k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0x3381e80?})
k8s.io/[email protected]/pkg/util/runtime/runtime.go:49 +0x75
panic({0x1cc6080, 0x334b600})
runtime/panic.go:884 +0x213
github.com/maksim-paskal/envoy-control-plane/pkg/configstore.(*ConfigStore).getEnvoyMetaFromEndpoint(0xc0005c8910?, {{0xc00075d040, 0xf}, {0x0, 0x0}, 0x0, 0x0})
github.com/maksim-paskal/envoy-control-plane/pkg/configstore/configStore.go:434 +0x68
github.com/maksim-paskal/envoy-control-plane/pkg/configstore.(*ConfigStore).getLocalityLbEndpoints(0xc0005d7880)
github.com/maksim-paskal/envoy-control-plane/pkg/configstore/configStore.go:375 +0x7f1
github.com/maksim-paskal/envoy-control-plane/pkg/configstore.(*ConfigStore).saveLastEndpoints(0xc0005d7880, {0x22dcaf0?, 0xc0000e3770})
github.com/maksim-paskal/envoy-control-plane/pkg/configstore/configStore.go:466 +0x207
github.com/maksim-paskal/envoy-control-plane/pkg/configstore.New({0x22dcaf0, 0xc0000e3770}, 0xc000bad320)
github.com/maksim-paskal/envoy-control-plane/pkg/configstore/configStore.go:96 +0x5e5
github.com/maksim-paskal/envoy-control-plane/pkg/configmapsstore.NewConfigMap({0x22dcaf0, 0xc0000e3770}, 0xc000578360)
github.com/maksim-paskal/envoy-control-plane/pkg/configmapsstore/configMapsStore.go:90 +0x697
github.com/maksim-paskal/envoy-control-plane/internal.Start.func3(0x0?)
github.com/maksim-paskal/envoy-control-plane/internal/internal.go:111 +0x27
github.com/maksim-paskal/envoy-control-plane/pkg/api.(*client).RunKubeInformers.func4({0x1fcd3e0?, 0xc000578360})
github.com/maksim-paskal/envoy-control-plane/pkg/api/informers.go:122 +0x11b
k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnAdd(...)
k8s.io/[email protected]/tools/cache/controller.go:243
k8s.io/client-go/tools/cache.(*processorListener).run.func1()
k8s.io/[email protected]/tools/cache/shared_informer.go:973 +0x148
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0x30?)
k8s.io/[email protected]/pkg/util/wait/backoff.go:226 +0x3e
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000072f38?, {0x22be620, 0xc000037f80}, 0x1, 0xc00046bd40)
k8s.io/[email protected]/pkg/util/wait/backoff.go:227 +0xb6
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0x0?, 0x3b9aca00, 0x0, 0x0?, 0x0?)
k8s.io/[email protected]/pkg/util/wait/backoff.go:204 +0x89
k8s.io/apimachinery/pkg/util/wait.Until(...)
k8s.io/[email protected]/pkg/util/wait/backoff.go:161
k8s.io/client-go/tools/cache.(*processorListener).run(0xc00038e120)
k8s.io/[email protected]/tools/cache/shared_informer.go:967 +0x6b
k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1()
k8s.io/[email protected]/pkg/util/wait/wait.go:72 +0x5a
created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start
k8s.io/[email protected]/pkg/util/wait/wait.go:70 +0x85
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1a2e488]
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.