Comments (14)
I am also having the same issue. Its not about DNS. Its the routing to the service ip cidr (10.96.0.0/12) is missing in the container.
from multus-cni.
@DigiDaz I edited your issue with Mardown for understanding. Please have a look how to include the "resolvConf": "/etc/resolv.conf"
in the https://github.com/containernetworking/plugins/tree/master/plugins/ipam/host-local. What is your /run/flannel/subnet.env
? Kubernetes will get only Flannel IP, which is the master plugin in this case.
from multus-cni.
@rkamudhan Thank you, my /run/flannel/subnet.env is:
FLANNEL_NETWORK=10.244.0.0/16
FLANNEL_SUBNET=10.244.1.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=true
I will look at the resolv.conf stuff but do I not need to access the dns at 10.96.0.10?
from multus-cni.
Setting the resolvConf option does not work for me, I thought my syntax may be wrong but I can echo it from the command line perfectly fine:
echo '{ "name": "default", "ipam": {"type": "host-local","subnet": "137.74.152.128/26","rangeStart": "137.74.152.129","rangeEnd": "137.74.152.189","resolvConf": "/etc/resolv.conf","routes": [{ "dst": "0.0.0.0/0","gw":"137.74.152.190" }],"gateway": "137.74.152.190"} }' | CNI_COMMAND=ADD CNI_CONTAINERID=example CNI_NETNS=/dev/null CNI_IFNAME=dummy0 CNI_PATH=. ./host-local
{
"cniVersion": "0.2.0",
"ip4": {
"ip": "137.74.152.130/26",
"gateway": "137.74.152.190",
"routes": [
{
"dst": "0.0.0.0/0",
"gw": "137.74.152.190"
}
]
},
"dns": {
"nameservers": [
"127.0.0.1",
"213.186.33.99"
],
"search": [
"ovh.net"
]
}
}
In a new container:
cat /etc/resolv.conf
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local ovh.net
options ndots:5
more /etc/cni/net.d/10-multus.conf
{
"name": "multus-demo",
"type": "multus",
"delegates": [
{
"type": "macvlan",
"master": "eth1",
"mode": "bridge",
"ipam": {
"type": "host-local",
"subnet": "137.74.152.128/26",
"rangeStart": "137.74.152.129",
"rangeEnd": "137.74.152.189",
"resolvConf": "/etc/resolv.conf",
"routes": [
{ "dst": "0.0.0.0/0" }
],
"gateway": "137.74.152.190"
}
},
{
"type": "flannel",
"masterplugin": true,
"delegate": {
"isDefaultGateway": false
}
}
]
}
from multus-cni.
@DigiDaz
From Host-local documentation:
host-local IPAM allocates IPv4 and IPv6 addresses out of a specified address range. Optionally, it can include a DNS configuration from a resolv.conf file on the host.
resolvConf (string, optional): Path to a resolv.conf on the host to parse and return as the DNS configuration
Container must have the following data
"dns": {
"nameservers": [
"127.0.0.1",
"213.186.33.99"
],
"search": [
"ovh.net"
]
}
Are you sure the application is not rewriting any thing in the container /etc/resolv.conf
?
from multus-cni.
This is the container resolv.conf
cat /etc/resolv.conf
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local ovh.net
options ndots:5
This is host resolv.conf:
more /etc/resolv.conf
nameserver 127.0.0.1
nameserver 213.186.33.99
search ovh.net
from multus-cni.
@DigiDaz Is that all nodes in your cluster have same "resolvConf": "/etc/resolv.conf"
. Can you check the /etc/resolv.conf
in the node, where the container is running. Can you display both your nodes and running containers /etc/resolv.conf
from multus-cni.
I only have two:
The master:
more /etc/resolv.conf
nameserver 213.186.33.99
search local
and the one the containers are being created on:
nameserver 127.0.0.1
nameserver 213.186.33.99
search ovh.net
The containers all have:
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local ovh.net
options ndots:5
It baffles me because because I can do the echo '{ "name": "default", "ipam": {"type": "host-local","subnet": "137.74.152.128/26, etc, etc from the cli on the hosts and see what looks like the correct info.
Is there anyway this could be getting set as expected and then overridden at a later stage of the container creation process?
from multus-cni.
kubectl run -i --tty busybox --image=busybox --restart=Never -- sh
If you don't see a command prompt, try pressing enter.
/ # cat /etc/resolv.conf
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local ovh.net
options ndots:5
/ # ifconfig
eth0 Link encap:Ethernet HWaddr 66:6B:72:07:09:AA
inet addr:10.244.1.4 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::646b:72ff:fe07:9aa/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1450 Metric:1
RX packets:5 errors:0 dropped:0 overruns:0 frame:0
TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:438 (438.0 B) TX bytes:690 (690.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
net0 Link encap:Ethernet HWaddr EA:1F:65:03:49:07
inet addr:137.74.152.136 Bcast:0.0.0.0 Mask:255.255.255.192
inet6 addr: fe80::e81f:65ff:fe03:4907/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1142 errors:0 dropped:0 overruns:0 frame:0
TX packets:44 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:68520 (66.9 KiB) TX bytes:2304 (2.2 KiB)
/ # route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 137.74.152.190 0.0.0.0 UG 0 0 0 net0
10.244.0.0 10.244.1.1 255.255.0.0 UG 0 0 0 eth0
10.244.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
137.74.152.128 0.0.0.0 255.255.255.192 U 0 0 0 net0
from multus-cni.
Can you display kubectl get svc --all-namespaces
and find your service with name kube-dns
. I think you running kude-dns with cluster-IP 10.96.0.10
from multus-cni.
Yes, that seems the case, is that not correct?
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 1d
default wordpress-mysql ClusterIP None <none> 3306/TCP 23h
kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP 1d
kube-system tiller-deploy ClusterIP 10.107.196.214 <none> 44134/TCP 1d
from multus-cni.
@DigiDaz Please set the dnspolicy in the pod refer this webpage for more details https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/ to your deployment mode and requirement. But it is good to raise this issue in CNI community and clarifies the issue.
from multus-cni.
Hi @saravanakumar-periyasamy, Can you display the route in your container with Multus and without Multus along with your CNI conf and Network object you used in pod spec. I want to make sure Multus is not giving any preference in the routing.
from multus-cni.
@DigiDaz -- could you please close this issue? If you still have concerns, feel free to re-start the conversation and we'll address them, thank you!
from multus-cni.
Related Issues (20)
- Deploying v4.0.2-thick to mixed amd64 and arm64 environment HOT 2
- Having a hard time to understand the spec to delegate the IP assignments (and mac address) to pod HOT 2
- Improve k8s event output when the annotation contains an error HOT 3
- Support of Virtual IP or floating IP HOT 1
- not able to see multiple interfaces other than default & loop back HOT 3
- Weird "interface name net1 already exists" on a build from master HOT 2
- should use K8S_POD_NAMESPACE HOT 2
- I installed multus according to the guide, but I found that the image was not pulled normally. What can I do about this? HOT 1
- How to use host-device plugin inside microk8s HOT 1
- Communication between pods running on multiple nodes HOT 3
- multus upgrade from 3.x to 4.x (thin plugin) causes pods start up issues HOT 2
- thin_entrypoint with `--cleanup-config-on-exit=true` is constantly creating and deleting config and kubeconfig HOT 2
- Race issue after node reboot HOT 7
- CNIDeviceInfoFile are not generated once upgraded to v4.0 HOT 3
- Failed to create pod sandbox after installing Multus HOT 1
- Using --multus-master-cni-file-name with thick plugin
- Provide arm64 builds for thick packages HOT 7
- Master CNI file flag option isn't working for thin plugin HOT 3
- IP Range Multiple Allocation in DHCP HOT 1
- MACSEC communication between nodes in K8s environment HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from multus-cni.