Use of Nodeport instead of LoadBalancer about stunner

[rg0now]

Actually, this should automatically work: as long as STUNner cannot find a public LoadBalancer IP for the Service it creates to expose your Gateway it will automatically fall back to the NodePort.

Below is a sample output for a case when this happens: the public IP 34.116.220.190 belongs to one of our Kubernetes nodes in the GKE cluster this output is taken from and the public port is 30501, a typical port in the default NodePort range:

$ cmd/stunnerctl/stunnerctl running-config stunner/stunnerd-config
STUN/TURN authentication type:  plaintext
STUN/TURN username:             user-1
STUN/TURN password:             pass-1
Listener 1
        Name:   udp-listener
        Listener:       udp-listener
        Protocol:       UDP
        Public address: 34.116.220.190
        Public port:    30501

Note that currently there is no way to enforce the public port for the NodePort service (i.e., 30501), this will be chosen automatically by Kubernetes. There are workarounds, but they are somewhat cumbersome. Note also that you will have to manually open the NodePort range (32000-32767) on the cloud firewall (this is automatically done when you use LoadBalancer services).

[cdkey22]

Hello,
This is interesting but actually i don't see any NodePort Service (for stunner) in the list of generated service (using k get svc --all-namespaces).

Is this normal ?
When I run stunnerctl i see that :

STUN/TURN authentication type:	static
STUN/TURN username:		user-1
STUN/TURN password:		pass-1
Listener 1
	Name:	stunner/udp-gateway/udp-listener
	Listener:	stunner/udp-gateway/udp-listener
	Protocol:	UDP
	Public port:	30200

Can you tell me more ?
What append if I create on my hand a new NodePort service using the same charcteristics than the loadbalancer one ?
Tks in advance.

[rg0now]

It seems that STUNner has already picked up the NodePort service. The way you know this is by looking at the public port: when the public port is in the default Nodeport range (32000-32767) then most probably STUNner has failed to find a LoadBalancer service and fell back to NodePort.

The curious thing here is that there is no public IP: this means that your Kubernetes nodes either run without a public IP (are you using a GKE private cluster?) or the public IP of your nodes is somehow encoded in a nonstandard way in the Kubernetes Node resources.

The easiest way to debug this is to look at the output of kubectl get nodes -o yaml and looking at the status field. Is there an external IP? Is it routable?

As per manually creating the service that exposes STUNner: that is a perfectly valid way to expose STUNner precisely the way you want if you don't like the service created automatically by STUNner. That being said, manually creating the NodePort service will not solve your problem until we find out why there are no public IPs assigned by Kubernetes to your nodes. And when we find out, STUNner will automatically find them too and you wouldn't need the manual service any more...:-)

[cdkey22]

Hello,
Ok thanks for your answer. Actually each nodes of my cluster are using public IP but there is something strange in the status of the node.
kc get nodes -o json | jq .items[].status.addresses
[ { "address": "51.210.190.***", "type": "InternalIP" }, { "address": "dev-b2-30-node-1b7067", "type": "Hostname" }
We see that the type is InternalIP. Maybe this is this that make the matching not working ?

Any way I tried using a TCP loadbalancer for testing, and even if it's not perfect it seems to work (in tutorials, not in my real case).
I was wondering if there is some documentation about the UdpRoute parameters that we can use. I've still not understood how we avoid to create a service openning the full range udp port (32000 -65567) in the clusterIP service. Do you have any clue about that.
Thanks in advance.

[rg0now]

Actually each nodes of my cluster are using public IP but there is something strange in the status of the node.
[...]
We see that the type is InternalIP. Maybe this is this that make the matching not working ?

Exactly. This is really weird: your InternalIP seems like a perfectly functional routable IP, why is it marked as "internal" then? STUNner will refuse to pick up an internal IP as a public address for obvious reasons.

Any way I tried using a TCP loadbalancer for testing, and even if it's not perfect it seems to work (in tutorials, not in my real case).

I'd recommend using TCP only as the last resort. For now, just set the client-side ICEServer configs with the InternalIP of one of your nodes and use the NodePort as above and you should be good to go. Meanwhile, contact your cloud operator and tell them their handling of Kubernetes nodes' external IPs needs some fixing.

I was wondering if there is some documentation about the UdpRoute parameters that we can use. I've still not understood how we avoid to create a service openning the full range udp port (32000 -65567) in the clusterIP service. Do you have any clue about that.

I think there's some misunderstanding here so let's get things straight:

• Public address/port: if you use NodePort you have to open the entire NodePort range on the external firewall of your cluster, otherwise the cloud operator will block external access to these ports. LoadBalancer services automatically handle this.
• Internal UDPRoute address/port: STUNner allows connections to the backend Service in the UDPRPoute on any protocol-port pair so you don't need to open "the full range udp port (32000-65567) in the clusterIP service". We rely on this hack until support for port ranges in Services lands in Kubernetes.

I've just added a warning to the README to document this behavior, thanks for calling my attention to this.

[cdkey22]

Ok that's very interesting.

By the way i'm trying to proxy my sip call to a freeswitch server.
The idea is that I can avoid to open the differents rtc_media_port via a node-port as I'm doing it now (using restriction of port 30000-30100).
It's ok for me, the turn server is correctly recognized etc.
The ip provided as a relay in the sdp seems to be ok provided private ip adress of stunner pod (i think it's normal).
I guess the next thing to do is understanding why freeswitch doesn't wan't to use this private IP to establish connection.

Thank's for all

[rg0now]

I guess the next thing to do is understanding why freeswitch doesn't wan't to use this private IP to establish connection.

I'm not sure we can be of big help on this matter but we'd be happy to learn more about your use case. Join our Discord and tell us about your use of STUNner, maybe someone will show up who understands freeswitch!

Closing this for now, feel free to reopen if anything interesting comes up.

[smeng9]

Hi @rg0now you have mentioned

There are workarounds, but they are somewhat cumbersome

Do you know such workarounds for me to choose a static Nodeport rather than letting Kubernetes randomly assign one?

[rg0now]

There are workarounds, but they are somewhat cumbersome

The workaround that I had in mind was manually creating the service that exposes STUNner and enforcing the fix NodePort in the manually created Service. Here are some simple steps to do that:

• First make sure STUNner's automatically created Service is internal (we will create the external Service manually, no need for two Services). This can be done by adding the annotation stunner.l7mp.io/service-type: ClusterIP, which will instruct STUNner to create a ClusterIP Service instead of the default LoadBalancer.
• Then create a copy of the Service created by STUNner under a new name (new name is important, otherwise STUNner will reset your modifications), set the Service type to NodePort, and set a fixed NodePort value to something you opened on the firewall.
• Create a manual ICE server config for your clients: the public IP would be the external IP of any of your nodes and the public port would be the chosen NodePort. This will be static, but it would at least work. Ephemeral authentication would need additional work though.

Since you are not the first one requesting for a way to fix the NodePort, I've created an wishlist item covering this. Once we get some free cycles we'll get to implement this, stay tuned.