Comments (9)
rg0now
commented on September 26, 2024
This usually means that the backend Service in the UDPRoute does not cover the server's pod IP the client wants to connect to. Make sure that all your WebRTC media server pods are actually part of one of your backend Services, otherwise STUNner will deny access (as per the above message) for security reasons.
from stunner.
tyohan
commented on September 26, 2024
Based on the service endpoint in the attached screenshot below, the service targets the right pod. That inlive-stream-xxx pod is one of the WebRTC media servers I'm trying to connect with; after attempting to ensure that the service points to the right pod, the client still can't connect to the WebRTC media server.
Another message from udp-gateway also printed this one:
16:26:54.477950 allocation.go:283: turn INFO: No Permission or Channel exists for 10.1.0.143:50005 on allocation 10.1.0.78:36504 (suppressed 2 log events)
Can you also help with that message? Where can I read the code that is responsible for printing both messages?
Thank you
from stunner.
rg0now
commented on September 26, 2024
Unfortunately, this input is not enough to pinpoint the problem. Can you please post the full stunnerd logs (preferably at a high loglevel like all:TRACE) and the operator logs?
Can you also help with that message? Where can I read the code that is responsible for printing both messages?
These are coming from pion/turn, the TURN implementation we're using. The issue is that your client is trying to reach a peer via STUNner that it has not obtained a permission for. This error can be a normal conclusion of the previous problems. Once you post the full logs, we'll know more.
from stunner.
tyohan
commented on September 26, 2024
Below are the logs from deploying the media plane pod to my cluster. This pod is dynamically deployed when an API endpoint is called. When the API endpoint is called, it will deploy the media plane pod, create a service, and stunner UDP route in the same time. Based on the operator logs below, the UDP route is trying to connect to the service, but the service was just recently created and is probably not ready yet. What I know is that all K8s create resource APIs that are async, and that's why the service is probably not found when the UDP route is created.
My next question is, when the UDP route is created, must the service exist, or will it retry until it gets the service?
This is the operator logs:
2024-02-05T10:15:27.496920941Z INFO renderer STUNner dataplane Deployment ready {"generation": 2058, "deployment": "{\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"creationTimestamp\":null,\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"replicas\":1,\"selector\":{\"matchExpressions\":[{\"key\":\"app\",\"operator\":\"In\",\"values\":[\"stunner\"]},{\"key\":\"stunner.l7mp.io/related-gateway-name\",\"operator\":\"In\",\"values\":[\"udp-gateway\"]},{\"key\":\"stunner.l7mp.io/related-gateway-namespace\",\"operator\":\"In\",\"values\":[\"stunner\"]}]},\"template\":{\"metadata\":{\"creationTimestamp\":null,\"labels\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"}},\"spec\":{\"containers\":[{\"name\":\"stunner-daemon\",\"image\":\"l7mp/stunnerd:0.17.0\",\"command\":[\"stunnerd\"],\"args\":[\"-w\",\"--udp-thread-num=16\"],\"env\":[{\"name\":\"STUNNER_ADDR\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.podIP\"}}},{\"name\":\"STUNNER_NAME\",\"value\":\"udp-gateway\"},{\"name\":\"STUNNER_NAMESPACE\",\"value\":\"stunner\"},{\"name\":\"STUNNER_CONFIG_ORIGIN\",\"value\":\"http://10.1.0.45:13478\"}],\"resources\":{\"limits\":{\"cpu\":\"2\",\"memory\":\"512Mi\"},\"requests\":{\"cpu\":\"500m\",\"memory\":\"128Mi\"}},\"livenessProbe\":{\"httpGet\":{\"path\":\"/live\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"readinessProbe\":{\"httpGet\":{\"path\":\"/ready\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"imagePullPolicy\":\"Always\"}],\"terminationGracePeriodSeconds\":3600}},\"strategy\":{}},\"status\":{}}"}
2024-02-05T10:15:27.496946258Z INFO renderer STUNner data plane configuration ready {"generation": 2058, "config": "{version=\"v1\",admin:{name=\"stunner/udp-gateway\",logLevel=\"all:INFO\",health-check=\"http://:8086\"},auth:{realm=\"turn.example.com\",type=\"static\",username=\"<SECRET>\",password=\"<SECRET>\"},listeners=[\"stunner/udp-gateway/udp-listener\":{turn://0.0.0.0:3478,public=x.x.x.x:3478,cert/key=-/-,routes=[stunner/dev-hub,stunner/hub]}],clusters=[\"stunner/dev-hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.182,10.152.183.173]},\"stunner/hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.180,10.152.183.71]}]}"}
2024-02-05T10:15:27.496972678Z INFO cds-server processing config update event {"generation": 2058, "update": "update (gen: 2058): upsert-queue: gway-cls: 1, gway: 1, route: 2, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T10:15:27.496986564Z INFO updater processing update event {"generation": 2058, "update": "update (gen: 2058): upsert-queue: gway-cls: 1, gway: 1, route: 2, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T10:20:32.820877807Z INFO node-controller reconciling {"node": "/node01"}
2024-02-05T10:20:32.820952899Z INFO node-controller failed to find node with valid external address {"reason": "end of node list reached after searching through 1 node(s)"}
2024-02-05T10:20:33.071347387Z INFO renderer rendering configuration {"generation": 2059, "event": "render"}
2024-02-05T10:20:33.071373076Z INFO renderer commencing dataplane render {"mode": "managed"}
2024-02-05T10:20:33.071382353Z INFO renderer rendering configuration {"gateway-class": "/stunner-gatewayclass"}
2024-02-05T10:20:33.0715874Z INFO renderer creating public service for gateway {"service": "stunner/udp-gateway", "gateway": "stunner/udp-gateway", "service": "{\"kind\":\"Service\",\"apiVersion\":\"v1\",\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"uid\":\"a8ce9f53-960b-4588-a3f3-ca625bbeb53a\",\"resourceVersion\":\"12285358\",\"creationTimestamp\":\"2024-01-29T17:40:57Z\",\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"metallb.universe.tf/allow-shared-ip\":\"x.x.x.x\",\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"ports\":[{\"name\":\"udp-listener\",\"protocol\":\"UDP\",\"port\":3478,\"targetPort\":3478,\"nodePort\":32298}],\"selector\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"clusterIP\":\"10.152.183.130\",\"clusterIPs\":[\"10.152.183.130\"],\"type\":\"LoadBalancer\",\"sessionAffinity\":\"None\",\"loadBalancerIP\":\"x.x.x.x\",\"externalTrafficPolicy\":\"Cluster\",\"ipFamilies\":[\"IPv4\"],\"ipFamilyPolicy\":\"SingleStack\",\"allocateLoadBalancerNodePorts\":true,\"internalTrafficPolicy\":\"Cluster\"},\"status\":{\"loadBalancer\":{\"ingress\":[{\"ip\":\"x.x.x.x\"}]}}}"}
2024-02-05T10:20:33.071655007Z INFO renderer update (gen: 2059): upsert-queue: gway-cls: 0, gway: 1, route: 2, routeV1A2: 0, svc: 1, confmap: 0, dp: 0 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 0
2024-02-05T10:20:33.071876124Z INFO renderer STUNner dataplane Deployment ready {"generation": 2059, "deployment": "{\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"creationTimestamp\":null,\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"replicas\":1,\"selector\":{\"matchExpressions\":[{\"key\":\"app\",\"operator\":\"In\",\"values\":[\"stunner\"]},{\"key\":\"stunner.l7mp.io/related-gateway-name\",\"operator\":\"In\",\"values\":[\"udp-gateway\"]},{\"key\":\"stunner.l7mp.io/related-gateway-namespace\",\"operator\":\"In\",\"values\":[\"stunner\"]}]},\"template\":{\"metadata\":{\"creationTimestamp\":null,\"labels\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"}},\"spec\":{\"containers\":[{\"name\":\"stunner-daemon\",\"image\":\"l7mp/stunnerd:0.17.0\",\"command\":[\"stunnerd\"],\"args\":[\"-w\",\"--udp-thread-num=16\"],\"env\":[{\"name\":\"STUNNER_ADDR\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.podIP\"}}},{\"name\":\"STUNNER_NAME\",\"value\":\"udp-gateway\"},{\"name\":\"STUNNER_NAMESPACE\",\"value\":\"stunner\"},{\"name\":\"STUNNER_CONFIG_ORIGIN\",\"value\":\"http://10.1.0.45:13478\"}],\"resources\":{\"limits\":{\"cpu\":\"2\",\"memory\":\"512Mi\"},\"requests\":{\"cpu\":\"500m\",\"memory\":\"128Mi\"}},\"livenessProbe\":{\"httpGet\":{\"path\":\"/live\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"readinessProbe\":{\"httpGet\":{\"path\":\"/ready\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"imagePullPolicy\":\"Always\"}],\"terminationGracePeriodSeconds\":3600}},\"strategy\":{}},\"status\":{}}"}
2024-02-05T10:20:33.071900901Z INFO renderer STUNner dataplane configuration ready {"generation": 2059, "config": "{version=\"v1\",admin:{name=\"stunner/udp-gateway\",logLevel=\"all:INFO\",health-check=\"http://:8086\"},auth:{realm=\"turn.example.com\",type=\"static\",username=\"<SECRET>\",password=\"<SECRET>\"},listeners=[\"stunner/udp-gateway/udp-listener\":{turn://0.0.0.0:3478,public=x.x.x.x:3478,cert/key=-/-,routes=[stunner/dev-hub,stunner/hub]}],clusters=[\"stunner/dev-hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.182,10.152.183.173]},\"stunner/hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.180,10.152.183.71]}]}"}
2024-02-05T10:20:33.071927581Z INFO cds-server processing config update event {"generation": 2059, "update": "update (gen: 2059): upsert-queue: gway-cls: 1, gway: 1, route: 2, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T10:20:33.071952328Z INFO updater processing update event {"generation": 2059, "update": "update (gen: 2059): upsert-queue: gway-cls: 1, gway: 1, route: 2, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T10:21:31.132396986Z INFO udproute-controller reconciling {"resource": "stunner/inlive-stream-55"}
2024-02-05T10:21:31.132494069Z INFO udproute-controller no Service found for UDPRoute backend {"udproute": "stunner/inlive-stream-55", "namespace": "inlive", "name": "inlive-stream-55"}
2024-02-05T10:21:31.13249997Z INFO udproute-controller no Endpoints found for UDPRoute backend {"udproute": "stunner/inlive-stream-55", "namespace": "inlive", "name": "inlive-stream-55"}
2024-02-05T10:21:31.135617038Z INFO udproute-controller reconciling {"resource": "inlive/inlive-stream-55"}
2024-02-05T10:21:31.135691659Z INFO udproute-controller no Endpoints found for UDPRoute backend {"udproute": "stunner/inlive-stream-55", "namespace": "inlive", "name": "inlive-stream-55"}
2024-02-05T10:21:31.142910722Z INFO udproute-controller reconciling {"resource": "inlive/inlive-stream-55"}
2024-02-05T10:21:31.382685661Z INFO renderer rendering configuration {"generation": 2060, "event": "render"}
2024-02-05T10:21:31.382721138Z INFO renderer commencing dataplane render {"mode": "managed"}
2024-02-05T10:21:31.382733491Z INFO renderer rendering configuration {"gateway-class": "/stunner-gatewayclass"}
2024-02-05T10:21:31.382936173Z INFO renderer creating public service for gateway {"service": "stunner/udp-gateway", "gateway": "stunner/udp-gateway", "service": "{\"kind\":\"Service\",\"apiVersion\":\"v1\",\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"uid\":\"a8ce9f53-960b-4588-a3f3-ca625bbeb53a\",\"resourceVersion\":\"12285358\",\"creationTimestamp\":\"2024-01-29T17:40:57Z\",\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"metallb.universe.tf/allow-shared-ip\":\"x.x.x.x\",\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"ports\":[{\"name\":\"udp-listener\",\"protocol\":\"UDP\",\"port\":3478,\"targetPort\":3478,\"nodePort\":32298}],\"selector\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"clusterIP\":\"10.152.183.130\",\"clusterIPs\":[\"10.152.183.130\"],\"type\":\"LoadBalancer\",\"sessionAffinity\":\"None\",\"loadBalancerIP\":\"x.x.x.x\",\"externalTrafficPolicy\":\"Cluster\",\"ipFamilies\":[\"IPv4\"],\"ipFamilyPolicy\":\"SingleStack\",\"allocateLoadBalancerNodePorts\":true,\"internalTrafficPolicy\":\"Cluster\"},\"status\":{\"loadBalancer\":{\"ingress\":[{\"ip\":\"x.x.x.x\"}]}}}"}
2024-02-05T10:21:31.38302467Z INFO renderer update (gen: 2060): upsert-queue: gway-cls: 0, gway: 1, route: 3, routeV1A2: 0, svc: 1, confmap: 0, dp: 0 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 0
2024-02-05T10:21:31.383274722Z INFO renderer STUNner dataplane Deployment ready {"generation": 2060, "deployment": "{\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"creationTimestamp\":null,\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"replicas\":1,\"selector\":{\"matchExpressions\":[{\"key\":\"app\",\"operator\":\"In\",\"values\":[\"stunner\"]},{\"key\":\"stunner.l7mp.io/related-gateway-name\",\"operator\":\"In\",\"values\":[\"udp-gateway\"]},{\"key\":\"stunner.l7mp.io/related-gateway-namespace\",\"operator\":\"In\",\"values\":[\"stunner\"]}]},\"template\":{\"metadata\":{\"creationTimestamp\":null,\"labels\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"}},\"spec\":{\"containers\":[{\"name\":\"stunner-daemon\",\"image\":\"l7mp/stunnerd:0.17.0\",\"command\":[\"stunnerd\"],\"args\":[\"-w\",\"--udp-thread-num=16\"],\"env\":[{\"name\":\"STUNNER_ADDR\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.podIP\"}}},{\"name\":\"STUNNER_NAME\",\"value\":\"udp-gateway\"},{\"name\":\"STUNNER_NAMESPACE\",\"value\":\"stunner\"},{\"name\":\"STUNNER_CONFIG_ORIGIN\",\"value\":\"http://10.1.0.45:13478\"}],\"resources\":{\"limits\":{\"cpu\":\"2\",\"memory\":\"512Mi\"},\"requests\":{\"cpu\":\"500m\",\"memory\":\"128Mi\"}},\"livenessProbe\":{\"httpGet\":{\"path\":\"/live\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"readinessProbe\":{\"httpGet\":{\"path\":\"/ready\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"imagePullPolicy\":\"Always\"}],\"terminationGracePeriodSeconds\":3600}},\"strategy\":{}},\"status\":{}}"}
2024-02-05T10:21:31.383307794Z INFO renderer STUNner dataplane configuration ready {"generation": 2060, "config": "{version=\"v1\",admin:{name=\"stunner/udp-gateway\",logLevel=\"all:INFO\",health-check=\"http://:8086\"},auth:{realm=\"turn.example.com\",type=\"static\",username=\"<SECRET>\",password=\"<SECRET>\"},listeners=[\"stunner/udp-gateway/udp-listener\":{turn://0.0.0.0:3478,public=x.x.x.x:3478,cert/key=-/-,routes=[stunner/inlive-stream-55,stunner/dev-hub,stunner/hub]}],clusters=[\"stunner/inlive-stream-55\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.152.183.251]},\"stunner/dev-hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.182,10.152.183.173]},\"stunner/hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.180,10.152.183.71]}]}"}
2024-02-05T10:21:31.383339504Z INFO cds-server processing config update event {"generation": 2060, "update": "update (gen: 2060): upsert-queue: gway-cls: 1, gway: 1, route: 3, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T10:21:31.383349082Z INFO updater processing update event {"generation": 2060, "update": "update (gen: 2060): upsert-queue: gway-cls: 1, gway: 1, route: 3, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T10:21:32.42207051Z INFO udproute-controller reconciling {"resource": "inlive/inlive-stream-55"}
2024-02-05T10:21:32.672389744Z INFO renderer rendering configuration {"generation": 2061, "event": "render"}
2024-02-05T10:21:32.672418358Z INFO renderer commencing dataplane render {"mode": "managed"}
2024-02-05T10:21:32.672429038Z INFO renderer rendering configuration {"gateway-class": "/stunner-gatewayclass"}
2024-02-05T10:21:32.67260537Z INFO renderer creating public service for gateway {"service": "stunner/udp-gateway", "gateway": "stunner/udp-gateway", "service": "{\"kind\":\"Service\",\"apiVersion\":\"v1\",\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"uid\":\"a8ce9f53-960b-4588-a3f3-ca625bbeb53a\",\"resourceVersion\":\"12285358\",\"creationTimestamp\":\"2024-01-29T17:40:57Z\",\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"metallb.universe.tf/allow-shared-ip\":\"x.x.x.x\",\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"ports\":[{\"name\":\"udp-listener\",\"protocol\":\"UDP\",\"port\":3478,\"targetPort\":3478,\"nodePort\":32298}],\"selector\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"clusterIP\":\"10.152.183.130\",\"clusterIPs\":[\"10.152.183.130\"],\"type\":\"LoadBalancer\",\"sessionAffinity\":\"None\",\"loadBalancerIP\":\"x.x.x.x\",\"externalTrafficPolicy\":\"Cluster\",\"ipFamilies\":[\"IPv4\"],\"ipFamilyPolicy\":\"SingleStack\",\"allocateLoadBalancerNodePorts\":true,\"internalTrafficPolicy\":\"Cluster\"},\"status\":{\"loadBalancer\":{\"ingress\":[{\"ip\":\"x.x.x.x\"}]}}}"}
2024-02-05T10:21:32.672682626Z INFO renderer update (gen: 2061): upsert-queue: gway-cls: 0, gway: 1, route: 3, routeV1A2: 0, svc: 1, confmap: 0, dp: 0 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 0
2024-02-05T10:21:32.672885078Z INFO renderer STUNner dataplane Deployment ready {"generation": 2061, "deployment": "{\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"creationTimestamp\":null,\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"replicas\":1,\"selector\":{\"matchExpressions\":[{\"key\":\"app\",\"operator\":\"In\",\"values\":[\"stunner\"]},{\"key\":\"stunner.l7mp.io/related-gateway-name\",\"operator\":\"In\",\"values\":[\"udp-gateway\"]},{\"key\":\"stunner.l7mp.io/related-gateway-namespace\",\"operator\":\"In\",\"values\":[\"stunner\"]}]},\"template\":{\"metadata\":{\"creationTimestamp\":null,\"labels\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"}},\"spec\":{\"containers\":[{\"name\":\"stunner-daemon\",\"image\":\"l7mp/stunnerd:0.17.0\",\"command\":[\"stunnerd\"],\"args\":[\"-w\",\"--udp-thread-num=16\"],\"env\":[{\"name\":\"STUNNER_ADDR\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.podIP\"}}},{\"name\":\"STUNNER_NAME\",\"value\":\"udp-gateway\"},{\"name\":\"STUNNER_NAMESPACE\",\"value\":\"stunner\"},{\"name\":\"STUNNER_CONFIG_ORIGIN\",\"value\":\"http://10.1.0.45:13478\"}],\"resources\":{\"limits\":{\"cpu\":\"2\",\"memory\":\"512Mi\"},\"requests\":{\"cpu\":\"500m\",\"memory\":\"128Mi\"}},\"livenessProbe\":{\"httpGet\":{\"path\":\"/live\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"readinessProbe\":{\"httpGet\":{\"path\":\"/ready\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"imagePullPolicy\":\"Always\"}],\"terminationGracePeriodSeconds\":3600}},\"strategy\":{}},\"status\":{}}"}
2024-02-05T10:21:32.672911848Z INFO renderer STUNner dataplane configuration ready {"generation": 2061, "config": "{version=\"v1\",admin:{name=\"stunner/udp-gateway\",logLevel=\"all:INFO\",health-check=\"http://:8086\"},auth:{realm=\"turn.example.com\",type=\"static\",username=\"<SECRET>\",password=\"<SECRET>\"},listeners=[\"stunner/udp-gateway/udp-listener\":{turn://0.0.0.0:3478,public=x.x.x.x:3478,cert/key=-/-,routes=[stunner/inlive-stream-55,stunner/dev-hub,stunner/hub]}],clusters=[\"stunner/inlive-stream-55\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.249,10.152.183.251]},\"stunner/dev-hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.182,10.152.183.173]},\"stunner/hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.180,10.152.183.71]}]}"}
2024-02-05T10:21:32.672941254Z INFO cds-server processing config update event {"generation": 2061, "update": "update (gen: 2061): upsert-queue: gway-cls: 1, gway: 1, route: 3, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T10:21:32.672971591Z INFO updater processing update event {"generation": 2061, "update": "update (gen: 2061): upsert-queue: gway-cls: 1, gway: 1, route: 3, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
And the gateway logs:
09:49:59.136396 reconcile.go:112: stunner INFO: setting loglevel to "all:INFO"
09:49:59.136435 reconcile.go:176: stunner INFO: reconciliation ready: new objects: 0, changed objects: 1, deleted objects: 0, started objects: 0, restarted objects: 0
09:49:59.136445 reconcile.go:180: stunner INFO: status: READY, realm: turn.replaced.com, authentication: static, listeners: stunner/udp-gateway/udp-listener: [turn-udp://10.1.0.78:3478<0:0>], active allocations: 0
10:21:31.383928 reconcile.go:112: stunner INFO: setting loglevel to "all:INFO"
10:21:31.383976 reconcile.go:176: stunner INFO: reconciliation ready: new objects: 1, changed objects: 1, deleted objects: 0, started objects: 0, restarted objects: 0
10:21:31.383986 reconcile.go:180: stunner INFO: status: READY, realm: turn.replaced.com, authentication: static, listeners: stunner/udp-gateway/udp-listener: [turn-udp://10.1.0.78:3478<0:0>], active allocations: 0
10:21:32.116444 handlers.go:25: stunner-auth INFO: static auth request: username="inlive" realm="turn.replaced.com" srcAddr=10.1.0.114:55365
10:21:32.673512 reconcile.go:112: stunner INFO: setting loglevel to "all:INFO"
10:21:32.673553 reconcile.go:176: stunner INFO: reconciliation ready: new objects: 0, changed objects: 2, deleted objects: 0, started objects: 0, restarted objects: 0
10:21:32.673563 reconcile.go:180: stunner INFO: status: READY, realm: turn.replaced.com, authentication: static, listeners: stunner/udp-gateway/udp-listener: [turn-udp://10.1.0.78:3478<0:0>], active allocations: 1
10:21:32.726985 allocation.go:283: turn INFO: No Permission or Channel exists for 10.1.0.249:50005 on allocation 10.1.0.78:36118
10:21:32.727017 allocation.go:283: turn INFO: No Permission or Channel exists for 10.1.0.249:50005 on allocation 10.1.0.78:36118
10:21:32.927715 allocation.go:283: turn INFO: No Permission or Channel exists for 10.1.0.249:50005 on allocation 10.1.0.78:36118
10:26:32.003346 reconcile.go:112: stunner INFO: setting loglevel to "all:INFO"
10:26:32.003371 reconcile.go:176: stunner INFO: reconciliation ready: new objects: 0, changed objects: 1, deleted objects: 1, started objects: 0, restarted objects: 0
10:26:32.003377 reconcile.go:180: stunner INFO: status: READY, realm: turn.replaced.com, authentication: static, listeners: stunner/udp-gateway/udp-listener: [turn-udp://10.1.0.78:3478<0:0>], active allocations: 1
10:30:46.478421 reconcile.go:112: stunner INFO: setting loglevel to "all:INFO"
10:30:46.478447 reconcile.go:176: stunner INFO: reconciliation ready: new objects: 0, changed objects: 1, deleted objects: 0, started objects: 0, restarted objects: 0
10:30:46.478453 reconcile.go:180: stunner INFO: status: READY, realm: turn.replaced.com, authentication: static, listeners: stunner/udp-gateway/udp-listener: [turn-udp://10.1.0.78:3478<0:0>], active allocations: 1
10:35:53.286683 reconcile.go:112: stunner INFO: setting loglevel to "all:INFO"
10:35:53.286719 reconcile.go:176: stunner INFO: reconciliation ready: new objects: 0, changed objects: 1, deleted objects: 0, started objects: 0, restarted objects: 0
10:35:53.286729 reconcile.go:180: stunner INFO: status: READY, realm: turn.replaced.com, authentication: static, listeners: stunner/udp-gateway/udp-listener: [turn-udp://10.1.0.78:3478<0:0>], active allocations: 0
from stunner.
tyohan
commented on September 26, 2024
An update. I was created UDP route before create the service. After I switch them and create service first before create the UDP route, the log of no service found in operator logs has gone. Below are the operator log with the new resource create order:
operator logs:
2024-02-05T11:47:17.40082965Z INFO renderer commencing dataplane render {"mode": "managed"}
2024-02-05T11:47:17.400842684Z INFO renderer rendering configuration {"gateway-class": "/stunner-gatewayclass"}
2024-02-05T11:47:17.40111653Z INFO renderer creating public service for gateway {"service": "stunner/udp-gateway", "gateway": "stunner/udp-gateway", "service": "{\"kind\":\"Service\",\"apiVersion\":\"v1\",\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"uid\":\"a8ce9f53-960b-4588-a3f3-ca625bbeb53a\",\"resourceVersion\":\"12285358\",\"creationTimestamp\":\"2024-01-29T17:40:57Z\",\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"metallb.universe.tf/allow-shared-ip\":\"x.x.x.x\",\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"ports\":[{\"name\":\"udp-listener\",\"protocol\":\"UDP\",\"port\":3478,\"targetPort\":3478,\"nodePort\":32298}],\"selector\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"clusterIP\":\"10.152.183.130\",\"clusterIPs\":[\"10.152.183.130\"],\"type\":\"LoadBalancer\",\"sessionAffinity\":\"None\",\"loadBalancerIP\":\"x.x.x.x\",\"externalTrafficPolicy\":\"Cluster\",\"ipFamilies\":[\"IPv4\"],\"ipFamilyPolicy\":\"SingleStack\",\"allocateLoadBalancerNodePorts\":true,\"internalTrafficPolicy\":\"Cluster\"},\"status\":{\"loadBalancer\":{\"ingress\":[{\"ip\":\"x.x.x.x\"}]}}}"}
2024-02-05T11:47:17.401210304Z INFO renderer update (gen: 2079): upsert-queue: gway-cls: 0, gway: 1, route: 2, routeV1A2: 0, svc: 1, confmap: 0, dp: 0 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 0
2024-02-05T11:47:17.401512302Z INFO renderer STUNner dataplane Deployment ready {"generation": 2079, "deployment": "{\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"creationTimestamp\":null,\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"replicas\":1,\"selector\":{\"matchExpressions\":[{\"key\":\"app\",\"operator\":\"In\",\"values\":[\"stunner\"]},{\"key\":\"stunner.l7mp.io/related-gateway-name\",\"operator\":\"In\",\"values\":[\"udp-gateway\"]},{\"key\":\"stunner.l7mp.io/related-gateway-namespace\",\"operator\":\"In\",\"values\":[\"stunner\"]}]},\"template\":{\"metadata\":{\"creationTimestamp\":null,\"labels\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"}},\"spec\":{\"containers\":[{\"name\":\"stunner-daemon\",\"image\":\"l7mp/stunnerd:0.17.0\",\"command\":[\"stunnerd\"],\"args\":[\"-w\",\"--udp-thread-num=16\"],\"env\":[{\"name\":\"STUNNER_ADDR\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.podIP\"}}},{\"name\":\"STUNNER_NAME\",\"value\":\"udp-gateway\"},{\"name\":\"STUNNER_NAMESPACE\",\"value\":\"stunner\"},{\"name\":\"STUNNER_CONFIG_ORIGIN\",\"value\":\"http://10.1.0.45:13478\"}],\"resources\":{\"limits\":{\"cpu\":\"2\",\"memory\":\"512Mi\"},\"requests\":{\"cpu\":\"500m\",\"memory\":\"128Mi\"}},\"livenessProbe\":{\"httpGet\":{\"path\":\"/live\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"readinessProbe\":{\"httpGet\":{\"path\":\"/ready\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"imagePullPolicy\":\"Always\"}],\"terminationGracePeriodSeconds\":3600}},\"strategy\":{}},\"status\":{}}"}
2024-02-05T11:47:17.4015494Z INFO renderer STUNner dataplane configuration ready {"generation": 2079, "config": "{version=\"v1\",admin:{name=\"stunner/udp-gateway\",logLevel=\"all:INFO\",health-check=\"http://:8086\"},auth:{realm=\"turn.example.com\",type=\"static\",username=\"<SECRET>\",password=\"<SECRET>\"},listeners=[\"stunner/udp-gateway/udp-listener\":{turn://0.0.0.0:3478,public=x.x.x.x:3478,cert/key=-/-,routes=[stunner/dev-hub,stunner/hub]}],clusters=[\"stunner/dev-hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.182,10.152.183.173]},\"stunner/hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.180,10.152.183.71]}]}"}
2024-02-05T11:47:17.401592009Z INFO cds-server processing config update event {"generation": 2079, "update": "update (gen: 2079): upsert-queue: gway-cls: 1, gway: 1, route: 2, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T11:47:17.401609682Z INFO updater processing update event {"generation": 2079, "update": "update (gen: 2079): upsert-queue: gway-cls: 1, gway: 1, route: 2, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T11:52:22.874162731Z INFO node-controller reconciling {"node": "/node01"}
2024-02-05T11:52:22.874238201Z INFO node-controller failed to find node with valid external address {"reason": "end of node list reached after searching through 1 node(s)"}
2024-02-05T11:52:23.124608365Z INFO renderer rendering configuration {"generation": 2080, "event": "render"}
2024-02-05T11:52:23.12462727Z INFO renderer commencing dataplane render {"mode": "managed"}
2024-02-05T11:52:23.124636507Z INFO renderer rendering configuration {"gateway-class": "/stunner-gatewayclass"}
2024-02-05T11:52:23.124812783Z INFO renderer creating public service for gateway {"service": "stunner/udp-gateway", "gateway": "stunner/udp-gateway", "service": "{\"kind\":\"Service\",\"apiVersion\":\"v1\",\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"uid\":\"a8ce9f53-960b-4588-a3f3-ca625bbeb53a\",\"resourceVersion\":\"12285358\",\"creationTimestamp\":\"2024-01-29T17:40:57Z\",\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"metallb.universe.tf/allow-shared-ip\":\"x.x.x.x\",\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"ports\":[{\"name\":\"udp-listener\",\"protocol\":\"UDP\",\"port\":3478,\"targetPort\":3478,\"nodePort\":32298}],\"selector\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"clusterIP\":\"10.152.183.130\",\"clusterIPs\":[\"10.152.183.130\"],\"type\":\"LoadBalancer\",\"sessionAffinity\":\"None\",\"loadBalancerIP\":\"x.x.x.x\",\"externalTrafficPolicy\":\"Cluster\",\"ipFamilies\":[\"IPv4\"],\"ipFamilyPolicy\":\"SingleStack\",\"allocateLoadBalancerNodePorts\":true,\"internalTrafficPolicy\":\"Cluster\"},\"status\":{\"loadBalancer\":{\"ingress\":[{\"ip\":\"x.x.x.x\"}]}}}"}
2024-02-05T11:52:23.124894324Z INFO renderer update (gen: 2080): upsert-queue: gway-cls: 0, gway: 1, route: 2, routeV1A2: 0, svc: 1, confmap: 0, dp: 0 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 0
2024-02-05T11:52:23.125133416Z INFO renderer STUNner dataplane Deployment ready {"generation": 2080, "deployment": "{\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"creationTimestamp\":null,\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"replicas\":1,\"selector\":{\"matchExpressions\":[{\"key\":\"app\",\"operator\":\"In\",\"values\":[\"stunner\"]},{\"key\":\"stunner.l7mp.io/related-gateway-name\",\"operator\":\"In\",\"values\":[\"udp-gateway\"]},{\"key\":\"stunner.l7mp.io/related-gateway-namespace\",\"operator\":\"In\",\"values\":[\"stunner\"]}]},\"template\":{\"metadata\":{\"creationTimestamp\":null,\"labels\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"}},\"spec\":{\"containers\":[{\"name\":\"stunner-daemon\",\"image\":\"l7mp/stunnerd:0.17.0\",\"command\":[\"stunnerd\"],\"args\":[\"-w\",\"--udp-thread-num=16\"],\"env\":[{\"name\":\"STUNNER_ADDR\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.podIP\"}}},{\"name\":\"STUNNER_NAME\",\"value\":\"udp-gateway\"},{\"name\":\"STUNNER_NAMESPACE\",\"value\":\"stunner\"},{\"name\":\"STUNNER_CONFIG_ORIGIN\",\"value\":\"http://10.1.0.45:13478\"}],\"resources\":{\"limits\":{\"cpu\":\"2\",\"memory\":\"512Mi\"},\"requests\":{\"cpu\":\"500m\",\"memory\":\"128Mi\"}},\"livenessProbe\":{\"httpGet\":{\"path\":\"/live\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"readinessProbe\":{\"httpGet\":{\"path\":\"/ready\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"imagePullPolicy\":\"Always\"}],\"terminationGracePeriodSeconds\":3600}},\"strategy\":{}},\"status\":{}}"}
2024-02-05T11:52:23.125163923Z INFO renderer STUNner dataplane configuration ready {"generation": 2080, "config": "{version=\"v1\",admin:{name=\"stunner/udp-gateway\",logLevel=\"all:INFO\",health-check=\"http://:8086\"},auth:{realm=\"turn.example.com\",type=\"static\",username=\"<SECRET>\",password=\"<SECRET>\"},listeners=[\"stunner/udp-gateway/udp-listener\":{turn://0.0.0.0:3478,public=x.x.x.x:3478,cert/key=-/-,routes=[stunner/dev-hub,stunner/hub]}],clusters=[\"stunner/dev-hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.182,10.152.183.173]},\"stunner/hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.180,10.152.183.71]}]}"}
2024-02-05T11:52:23.12519466Z INFO cds-server processing config update event {"generation": 2080, "update": "update (gen: 2080): upsert-queue: gway-cls: 1, gway: 1, route: 2, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T11:52:23.125216089Z INFO updater processing update event {"generation": 2080, "update": "update (gen: 2080): upsert-queue: gway-cls: 1, gway: 1, route: 2, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T11:55:57.219539741Z INFO udproute-controller reconciling {"resource": "stunner/inlive-stream-56"}
2024-02-05T11:55:57.46974478Z INFO renderer rendering configuration {"generation": 2081, "event": "render"}
2024-02-05T11:55:57.469776689Z INFO renderer commencing dataplane render {"mode": "managed"}
2024-02-05T11:55:57.469787319Z INFO renderer rendering configuration {"gateway-class": "/stunner-gatewayclass"}
2024-02-05T11:55:57.47001471Z INFO renderer creating public service for gateway {"service": "stunner/udp-gateway", "gateway": "stunner/udp-gateway", "service": "{\"kind\":\"Service\",\"apiVersion\":\"v1\",\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"uid\":\"a8ce9f53-960b-4588-a3f3-ca625bbeb53a\",\"resourceVersion\":\"12285358\",\"creationTimestamp\":\"2024-01-29T17:40:57Z\",\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"metallb.universe.tf/allow-shared-ip\":\"x.x.x.x\",\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"ports\":[{\"name\":\"udp-listener\",\"protocol\":\"UDP\",\"port\":3478,\"targetPort\":3478,\"nodePort\":32298}],\"selector\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"clusterIP\":\"10.152.183.130\",\"clusterIPs\":[\"10.152.183.130\"],\"type\":\"LoadBalancer\",\"sessionAffinity\":\"None\",\"loadBalancerIP\":\"x.x.x.x\",\"externalTrafficPolicy\":\"Cluster\",\"ipFamilies\":[\"IPv4\"],\"ipFamilyPolicy\":\"SingleStack\",\"allocateLoadBalancerNodePorts\":true,\"internalTrafficPolicy\":\"Cluster\"},\"status\":{\"loadBalancer\":{\"ingress\":[{\"ip\":\"x.x.x.x\"}]}}}"}
2024-02-05T11:55:57.470102863Z INFO renderer update (gen: 2081): upsert-queue: gway-cls: 0, gway: 1, route: 3, routeV1A2: 0, svc: 1, confmap: 0, dp: 0 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 0
2024-02-05T11:55:57.470344209Z INFO renderer STUNner dataplane Deployment ready {"generation": 2081, "deployment": "{\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"creationTimestamp\":null,\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"replicas\":1,\"selector\":{\"matchExpressions\":[{\"key\":\"app\",\"operator\":\"In\",\"values\":[\"stunner\"]},{\"key\":\"stunner.l7mp.io/related-gateway-name\",\"operator\":\"In\",\"values\":[\"udp-gateway\"]},{\"key\":\"stunner.l7mp.io/related-gateway-namespace\",\"operator\":\"In\",\"values\":[\"stunner\"]}]},\"template\":{\"metadata\":{\"creationTimestamp\":null,\"labels\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"}},\"spec\":{\"containers\":[{\"name\":\"stunner-daemon\",\"image\":\"l7mp/stunnerd:0.17.0\",\"command\":[\"stunnerd\"],\"args\":[\"-w\",\"--udp-thread-num=16\"],\"env\":[{\"name\":\"STUNNER_ADDR\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.podIP\"}}},{\"name\":\"STUNNER_NAME\",\"value\":\"udp-gateway\"},{\"name\":\"STUNNER_NAMESPACE\",\"value\":\"stunner\"},{\"name\":\"STUNNER_CONFIG_ORIGIN\",\"value\":\"http://10.1.0.45:13478\"}],\"resources\":{\"limits\":{\"cpu\":\"2\",\"memory\":\"512Mi\"},\"requests\":{\"cpu\":\"500m\",\"memory\":\"128Mi\"}},\"livenessProbe\":{\"httpGet\":{\"path\":\"/live\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"readinessProbe\":{\"httpGet\":{\"path\":\"/ready\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"imagePullPolicy\":\"Always\"}],\"terminationGracePeriodSeconds\":3600}},\"strategy\":{}},\"status\":{}}"}
2024-02-05T11:55:57.470378623Z INFO renderer STUNner dataplane configuration ready {"generation": 2081, "config": "{version=\"v1\",admin:{name=\"stunner/udp-gateway\",logLevel=\"all:INFO\",health-check=\"http://:8086\"},auth:{realm=\"turn.example.com\",type=\"static\",username=\"<SECRET>\",password=\"<SECRET>\"},listeners=[\"stunner/udp-gateway/udp-listener\":{turn://0.0.0.0:3478,public=x.x.x.x:3478,cert/key=-/-,routes=[stunner/inlive-stream-56,stunner/dev-hub,stunner/hub]}],clusters=[\"stunner/dev-hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.182,10.152.183.173]},\"stunner/hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.180,10.152.183.71]},\"stunner/inlive-stream-56\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.152.183.87]}]}"}
2024-02-05T11:55:57.470417716Z INFO cds-server processing config update event {"generation": 2081, "update": "update (gen: 2081): upsert-queue: gway-cls: 1, gway: 1, route: 3, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T11:55:57.470427874Z INFO updater processing update event {"generation": 2081, "update": "update (gen: 2081): upsert-queue: gway-cls: 1, gway: 1, route: 3, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T11:55:57.91787645Z INFO udproute-controller reconciling {"resource": "inlive/inlive-stream-56"}
2024-02-05T11:55:58.168959173Z INFO renderer rendering configuration {"generation": 2082, "event": "render"}
2024-02-05T11:55:58.168975153Z INFO renderer commencing dataplane render {"mode": "managed"}
2024-02-05T11:55:58.168982997Z INFO renderer rendering configuration {"gateway-class": "/stunner-gatewayclass"}
2024-02-05T11:55:58.169144306Z INFO renderer creating public service for gateway {"service": "stunner/udp-gateway", "gateway": "stunner/udp-gateway", "service": "{\"kind\":\"Service\",\"apiVersion\":\"v1\",\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"uid\":\"a8ce9f53-960b-4588-a3f3-ca625bbeb53a\",\"resourceVersion\":\"12285358\",\"creationTimestamp\":\"2024-01-29T17:40:57Z\",\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"metallb.universe.tf/allow-shared-ip\":\"x.x.x.x\",\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"ports\":[{\"name\":\"udp-listener\",\"protocol\":\"UDP\",\"port\":3478,\"targetPort\":3478,\"nodePort\":32298}],\"selector\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"clusterIP\":\"10.152.183.130\",\"clusterIPs\":[\"10.152.183.130\"],\"type\":\"LoadBalancer\",\"sessionAffinity\":\"None\",\"loadBalancerIP\":\"x.x.x.x\",\"externalTrafficPolicy\":\"Cluster\",\"ipFamilies\":[\"IPv4\"],\"ipFamilyPolicy\":\"SingleStack\",\"allocateLoadBalancerNodePorts\":true,\"internalTrafficPolicy\":\"Cluster\"},\"status\":{\"loadBalancer\":{\"ingress\":[{\"ip\":\"x.x.x.x\"}]}}}"}
2024-02-05T11:55:58.16921701Z INFO renderer update (gen: 2082): upsert-queue: gway-cls: 0, gway: 1, route: 3, routeV1A2: 0, svc: 1, confmap: 0, dp: 0 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 0
2024-02-05T11:55:58.16943798Z INFO renderer STUNner dataplane Deployment ready {"generation": 2082, "deployment": "{\"metadata\":{\"name\":\"udp-gateway\",\"namespace\":\"stunner\",\"creationTimestamp\":null,\"labels\":{\"stunner.l7mp.io/owned-by\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"},\"ownerReferences\":[{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"Gateway\",\"name\":\"udp-gateway\",\"uid\":\"d1cf77dc-ac96-492e-b9df-31d9d47ad24b\"}]},\"spec\":{\"replicas\":1,\"selector\":{\"matchExpressions\":[{\"key\":\"app\",\"operator\":\"In\",\"values\":[\"stunner\"]},{\"key\":\"stunner.l7mp.io/related-gateway-name\",\"operator\":\"In\",\"values\":[\"udp-gateway\"]},{\"key\":\"stunner.l7mp.io/related-gateway-namespace\",\"operator\":\"In\",\"values\":[\"stunner\"]}]},\"template\":{\"metadata\":{\"creationTimestamp\":null,\"labels\":{\"app\":\"stunner\",\"stunner.l7mp.io/related-gateway-name\":\"udp-gateway\",\"stunner.l7mp.io/related-gateway-namespace\":\"stunner\"},\"annotations\":{\"stunner.l7mp.io/related-gateway-name\":\"stunner/udp-gateway\"}},\"spec\":{\"containers\":[{\"name\":\"stunner-daemon\",\"image\":\"l7mp/stunnerd:0.17.0\",\"command\":[\"stunnerd\"],\"args\":[\"-w\",\"--udp-thread-num=16\"],\"env\":[{\"name\":\"STUNNER_ADDR\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.podIP\"}}},{\"name\":\"STUNNER_NAME\",\"value\":\"udp-gateway\"},{\"name\":\"STUNNER_NAMESPACE\",\"value\":\"stunner\"},{\"name\":\"STUNNER_CONFIG_ORIGIN\",\"value\":\"http://10.1.0.45:13478\"}],\"resources\":{\"limits\":{\"cpu\":\"2\",\"memory\":\"512Mi\"},\"requests\":{\"cpu\":\"500m\",\"memory\":\"128Mi\"}},\"livenessProbe\":{\"httpGet\":{\"path\":\"/live\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"readinessProbe\":{\"httpGet\":{\"path\":\"/ready\",\"port\":8086,\"scheme\":\"HTTP\"},\"timeoutSeconds\":1,\"periodSeconds\":15,\"successThreshold\":1,\"failureThreshold\":3},\"imagePullPolicy\":\"Always\"}],\"terminationGracePeriodSeconds\":3600}},\"strategy\":{}},\"status\":{}}"}
2024-02-05T11:55:58.169462475Z INFO renderer STUNner dataplane configuration ready {"generation": 2082, "config": "{version=\"v1\",admin:{name=\"stunner/udp-gateway\",logLevel=\"all:INFO\",health-check=\"http://:8086\"},auth:{realm=\"turn.example.com\",type=\"static\",username=\"<SECRET>\",password=\"<SECRET>\"},listeners=[\"stunner/udp-gateway/udp-listener\":{turn://0.0.0.0:3478,public=x.x.x.x:3478,cert/key=-/-,routes=[stunner/dev-hub,stunner/hub,stunner/inlive-stream-56]}],clusters=[\"stunner/dev-hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.182,10.152.183.173]},\"stunner/hub\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.180,10.152.183.71]},\"stunner/inlive-stream-56\":{type=\"STATIC\",protocol=\"UDP\",endpoints=[10.1.0.199,10.152.183.87]}]}"}
2024-02-05T11:55:58.169488723Z INFO cds-server processing config update event {"generation": 2082, "update": "update (gen: 2082): upsert-queue: gway-cls: 1, gway: 1, route: 3, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
2024-02-05T11:55:58.169522967Z INFO updater processing update event {"generation": 2082, "update": "update (gen: 2082): upsert-queue: gway-cls: 1, gway: 1, route: 3, routeV1A2: 0, svc: 1, confmap: 1, dp: 1 / delete-queue: gway-cls: 0, gway: 0, route: 0, routeV1A2: 0, svc: 0, confmap: 0, dp: 0 / config-queue: 1"}
And the gateway log
11:37:06.117350 reconcile.go:112: stunner INFO: setting loglevel to "all:INFO"
11:37:06.117383 reconcile.go:176: stunner INFO: reconciliation ready: new objects: 0, changed objects: 1, deleted objects: 0, started objects: 0, restarted objects: 0
11:37:06.117393 reconcile.go:180: stunner INFO: status: READY, realm: turn.example.com, authentication: static, listeners: stunner/udp-gateway/udp-listener: [turn-udp://10.1.0.78:3478<0:0>], active allocations: 0
11:47:17.402316 reconcile.go:112: stunner INFO: setting loglevel to "all:INFO"
11:47:17.402365 reconcile.go:176: stunner INFO: reconciliation ready: new objects: 0, changed objects: 1, deleted objects: 0, started objects: 0, restarted objects: 0
11:47:17.402377 reconcile.go:180: stunner INFO: status: READY, realm: turn.example.com, authentication: static, listeners: stunner/udp-gateway/udp-listener: [turn-udp://10.1.0.78:3478<0:0>], active allocations: 0
11:55:57.470979 reconcile.go:112: stunner INFO: setting loglevel to "all:INFO"
11:55:57.471041 reconcile.go:176: stunner INFO: reconciliation ready: new objects: 1, changed objects: 1, deleted objects: 0, started objects: 0, restarted objects: 0
11:55:57.471051 reconcile.go:180: stunner INFO: status: READY, realm: turn.example.com, authentication: static, listeners: stunner/udp-gateway/udp-listener: [turn-udp://10.1.0.78:3478<0:0>], active allocations: 0
11:55:58.169989 reconcile.go:112: stunner INFO: setting loglevel to "all:INFO"
11:55:58.170027 reconcile.go:176: stunner INFO: reconciliation ready: new objects: 0, changed objects: 2, deleted objects: 0, started objects: 0, restarted objects: 0
11:55:58.170036 reconcile.go:180: stunner INFO: status: READY, realm: turn.example.com, authentication: static, listeners: stunner/udp-gateway/udp-listener: [turn-udp://10.1.0.78:3478<0:0>], active allocations: 0
11:55:58.289758 handlers.go:25: stunner-auth INFO: static auth request: username="inlive" realm="turn.example.com" srcAddr=10.1.0.114:51112
11:55:58.296422 allocation.go:283: turn INFO: No Permission or Channel exists for 10.1.0.199:50005 on allocation 10.1.0.78:34464 (suppressed 7 log events)
11:55:58.296444 allocation.go:283: turn INFO: No Permission or Channel exists for 10.1.0.199:50005 on allocation 10.1.0.78:34464
11:55:58.497181 allocation.go:283: turn INFO: No Permission or Channel exists for 10.1.0.199:50005 on allocation 10.1.0.78:34464
11:55:59.300481 allocation.go:283: turn INFO: No Permission or Channel exists for 10.1.0.199:50005 on allocation 10.1.0.78:34464 (suppressed 7 log events)
from stunner.
tyohan
commented on September 26, 2024
Another question to clarify:
I saw that when I created the UDP route successfully, it triggered the reconciliation onbthe UDP gateway. I wonder if I should wait for a moment, until everything is ready to relay the connection to the new pod. Because my app will trying to connect instantly once the creation of the pod, service, and UDP route through the API endpoint has completed.
from stunner.
rg0now
commented on September 26, 2024
As per UDPRoute and Service availability: you don't have to manually synchronize the creation of the resources (in fact, in Kubernetes you never have to). If the UDPRoute is created before the Service STUNner will wait until the Service becomes ready and, once this happens, it updates the dataplane config accordingly. You can create anything in any order and things will just work.
from stunner.
rg0now
commented on September 26, 2024
This is terribly weird:
11:55:58.289758 handlers.go:25: stunner-auth INFO: static auth request: username="inlive" realm="turn.example.com" srcAddr=10.1.0.114:51112
11:55:58.296422 allocation.go:283: turn INFO: No Permission or Channel exists for 10.1.0.199:50005 on allocation 10.1.0.78:34464 (suppressed 7 log events)
Your client creates an allocation with the (NATted) source address 10.1.0.114:51112 but then it tries to send traffic from another source address (10.1.0.199:50005). This should never happen unless something weird is going on with your client or cloud load-balancer. In order to confirm this, please repeat the experiment with setting the dataplane loglevel to all:TRACE and re-post the logs. Hopefully we'll see where things go south.
Also, in order to guarantee that your WebRTC code is OK you can redeploy with a well-tested Kubernetes distro, like Minikube or a k3s, and check if everything works there. We have a good understanding of how things should happen there and then we could extrapolate from that to your specific cloud provider (which is what, BTW?).
from stunner.
rg0now
commented on September 26, 2024
Closing this for now because this seems like a LoadBalancer issue (still weird). If you have new input then please reopen, we'd like to know the solution.
from stunner.
Related Issues (20)
- Extra question about horizontally scaled Stunner HOT 3
- Example app udp-greeter.yaml not working - help needed HOT 10
- v0.16.0 - Websocket error HOT 3
- v0.16.0 - Stunnerd pods get into state where they won't respond to TURN requests HOT 1
- Allow Gateways to request a specific NodePort in the automatically created Service HOT 7
- TURN connection breaks when the backend pod enters graceful shutdown HOT 13
- `stunnerctl config` does not fall back to the default namespace
- Help testing on AKS (Azure) HOT 1
- Media plane: Asymmetric ICE connection issues: no allocation found HOT 6
- Deployment in headless mode does not resolve public ip address of client HOT 4
- Turncat example not working on EKS HOT 11
- error install on container : kubernate and helm HOT 3
- Turncat does not include SNI header when using TURN/TLS HOT 5
- UDPRoute with media servers that dynamically choose the listener port HOT 3
- Operator failed if tolerations were specified. HOT 1
- stunner.l7mp.io/service-type: ClusterIP doesn't work HOT 2
- Lost connections while scaling HOT 7
- [Feature request] Option to deploy the control plane as a DaemonSet HOT 1
- OCI Network Loadbalancer only supports sessionAffinity: None for v0.20.0 HOT 2
- docs: readthedocs is stuck on v0.18.0 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from stunner.