如何让固件支持通过opkg 安装 dockerman about luci-app-dockerman

[lisaac]

嗯，按照需要，添加内核选项就可以

[lisaac]

请尝试使用 make menuconfig/kernel_menuconfig 调整config 后再测试

[lisaac]

#36

[kiddin9]

check-config 测试结果:

Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: enabled
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_CPUSETS: enabled
- CONFIG_MEMCG: enabled
- CONFIG_KEYS: enabled
- CONFIG_VETH: enabled (as module)
- CONFIG_BRIDGE: enabled
- CONFIG_BRIDGE_NETFILTER: enabled (as module)
- CONFIG_NF_NAT_IPV4: missing
- CONFIG_IP_NF_FILTER: enabled (as module)
- CONFIG_IP_NF_TARGET_MASQUERADE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_IPVS: enabled (as module)
- CONFIG_IP_NF_NAT: enabled (as module)
- CONFIG_NF_NAT: enabled (as module)
- CONFIG_NF_NAT_NEEDED: missing
- CONFIG_POSIX_MQUEUE: enabled

Optional Features:
- CONFIG_USER_NS: enabled
- CONFIG_SECCOMP: enabled
- CONFIG_CGROUP_PIDS: enabled
- CONFIG_MEMCG_SWAP: missing
- CONFIG_MEMCG_SWAP_ENABLED: missing
- CONFIG_LEGACY_VSYSCALL_NONE: enabled
    (containers using eglibc <= 2.13 will not work. Switch to
     "CONFIG_VSYSCALL_[NATIVE|EMULATE]" or use "vsyscall=[native|emulate]"
     on kernel command line. Note that this will disable ASLR for the,
     VDSO which may assist in exploiting security vulnerabilities.)
- CONFIG_BLK_CGROUP: enabled
- CONFIG_BLK_DEV_THROTTLING: missing
- CONFIG_IOSCHED_CFQ: missing
- CONFIG_CFQ_GROUP_IOSCHED: missing
- CONFIG_CGROUP_PERF: missing
- CONFIG_CGROUP_HUGETLB: missing
- CONFIG_NET_CLS_CGROUP: enabled
- CONFIG_CGROUP_NET_PRIO: missing
- CONFIG_CFS_BANDWIDTH: missing
- CONFIG_FAIR_GROUP_SCHED: enabled
- CONFIG_RT_GROUP_SCHED: enabled
- CONFIG_IP_VS: enabled (as module)
- CONFIG_IP_VS_NFCT: enabled
- CONFIG_IP_VS_RR: enabled (as module)
- CONFIG_EXT4_FS: enabled
- CONFIG_EXT4_FS_POSIX_ACL: missing
- CONFIG_EXT4_FS_SECURITY: missing
    enable these ext4 configs if you are using ext3 or ext4 as backing filesystem
- Network Drivers:
  - "overlay":
    - CONFIG_VXLAN: enabled (as module)
      Optional (for encrypted networks):
      - CONFIG_CRYPTO: enabled
      - CONFIG_CRYPTO_AEAD: enabled
      - CONFIG_CRYPTO_GCM: enabled (as module)
      - CONFIG_CRYPTO_SEQIV: enabled (as module)
      - CONFIG_CRYPTO_GHASH: enabled (as module)
      - CONFIG_XFRM: enabled
      - CONFIG_XFRM_USER: enabled (as module)
      - CONFIG_XFRM_ALGO: enabled (as module)
      - CONFIG_INET_ESP: enabled (as module)
      - CONFIG_INET_XFRM_MODE_TRANSPORT: missing
  - "ipvlan":
    - CONFIG_IPVLAN: missing
  - "macvlan":
    - CONFIG_MACVLAN: enabled (as module)
    - CONFIG_DUMMY: enabled (as module)
  - "ftp,tftp client in container":
    - CONFIG_NF_NAT_FTP: enabled (as module)
    - CONFIG_NF_CONNTRACK_FTP: enabled (as module)
    - CONFIG_NF_NAT_TFTP: enabled (as module)
    - CONFIG_NF_CONNTRACK_TFTP: enabled (as module)
- Storage Drivers:
  - "aufs":
    - CONFIG_AUFS_FS: missing
  - "btrfs":
    - CONFIG_BTRFS_FS: enabled (as module)
    - CONFIG_BTRFS_FS_POSIX_ACL: missing
  - "devicemapper":
    - CONFIG_BLK_DEV_DM: enabled (as module)
    - CONFIG_DM_THIN_PROVISIONING: missing
  - "overlay":
    - CONFIG_OVERLAY_FS: enabled
  - "zfs":
    - /dev/zfs: missing
    - zfs command: missing
    - zpool command: missing

Limits:
cat: can't open '/proc/sys/kernel/keys/root_maxkeys': No such file or directory
./test.sh: line 347: [: -le: unary operator expected
cat: can't open '/proc/sys/kernel/keys/root_maxkeys': No such file or directory

谢谢

[kiddin9]

Generally Necessary中显示缺少了CONFIG_NF_NAT_IPV4和CONFIG_NF_NAT_NEEDED
但是openwrt .config中找不到这两项的设置项
我看报错日志也并不是因为这两项导致的启动失败.

[lisaac]

应该在kernel config中查找
另外报错好像是Storage Drivers中没有选择对应模块

[kiddin9]

我在错误日志中发现了这一条

failed to start daemon: Error initializing network controller: Error creating default "bridge" network: Failed to program NAT chain: Failed to inject DOCKER in PREROUTING chain: iptables failed: iptables --wait -t nat -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER: iptables v1.8.4 (legacy): Couldn't load match `addrtype':No such file or directory

我重新编译了一版带dockerman的固件,发现CONFIG_NF_NAT_IPV4和CONFIG_NF_NAT_NEEDED这两项也是missing,但是却可以正常启动.

[kiddin9]

报错信息说的是缺少 br-netfilter 导致的
但是我在kernel config中 开启了
CONFIG_NETFILTER=y
CONFIG_NETFILTER_ADVANCED=y
CONFIG_BRIDGE_NETFILTER=m

后台安装dockerman也有自动安装kmod-br-netfilter依赖

Wed Jun 17 16:45:12 2020 daemon.err modprobe: failed to find a module named bridge
Wed Jun 17 16:45:12 2020 kern.warn kernel: [ 331.187589] br_netfilter: Unknown symbol __skb_ext_del (err -2)
Wed Jun 17 16:45:12 2020 kern.warn kernel: [ 331.188212] br_netfilter: Unknown symbol ipv6_mod_enabled (err -2)
Wed Jun 17 16:45:12 2020 kern.warn kernel: [ 331.188844] br_netfilter: Unknown symbol skb_ext_add (err -2)
Wed Jun 17 16:45:12 2020 daemon.err modprobe: 1 module could not be probed
Wed Jun 17 16:45:12 2020 daemon.err modprobe: - br_netfilter
Wed Jun 17 16:45:12 2020 daemon.err dockerd[27539]: time="2020-06-17T16:45:12.710995760+08:00" level=warning msg="Running modprobe bridge br_netfilter failed with message: , error: exit status 255"
Wed Jun 17 16:45:12 2020 daemon.err modprobe: failed to find dependency x_tables
Wed Jun 17 16:45:12 2020 daemon.err modprobe: 1 module could not be probed
Wed Jun 17 16:45:12 2020 daemon.err modprobe: - xt_addrtype
Wed Jun 17 16:45:12 2020 daemon.err modprobe: failed to find dependency x_tables
Wed Jun 17 16:45:12 2020 daemon.err modprobe: 1 module could not be probed
Wed Jun 17 16:45:12 2020 daemon.err modprobe: - xt_addrtype
Wed Jun 17 16:45:12 2020 daemon.err modprobe: failed to find dependency x_tables
Wed Jun 17 16:45:12 2020 daemon.err modprobe: 1 module could not be probed
Wed Jun 17 16:45:12 2020 daemon.err modprobe: - xt_addrtype
Wed Jun 17 16:45:12 2020 daemon.err modprobe: failed to find dependency x_tables
Wed Jun 17 16:45:12 2020 daemon.err modprobe: 1 module could not be probed
Wed Jun 17 16:45:12 2020 daemon.err modprobe: - xt_addrtype
Wed Jun 17 16:45:12 2020 daemon.err modprobe: failed to find dependency x_tables
Wed Jun 17 16:45:12 2020 daemon.err modprobe: 1 module could not be probed
Wed Jun 17 16:45:12 2020 daemon.err modprobe: - xt_addrtype
Wed Jun 17 16:45:12 2020 daemon.err modprobe: failed to find dependency x_tables
Wed Jun 17 16:45:12 2020 daemon.err modprobe: 1 module could not be probed
Wed Jun 17 16:45:12 2020 daemon.err modprobe: - xt_addrtype
Wed Jun 17 16:45:12 2020 daemon.err dockerd[27539]: time="2020-06-17T16:45:12.793154239+08:00" level=warning msg="Could not load necessary modules for IPSEC rules: protocol not supported"
Wed Jun 17 16:45:12 2020 daemon.info modprobe: nf_conntrack is already loaded
Wed Jun 17 16:45:12 2020 daemon.info modprobe: nf_conntrack_netlink is already loaded
Wed Jun 17 16:45:12 2020 daemon.err modprobe: failed to find dependency x_tables
Wed Jun 17 16:45:12 2020 daemon.err modprobe: 1 module could not be probed
Wed Jun 17 16:45:12 2020 daemon.err modprobe: - xt_addrtype
Wed Jun 17 16:45:12 2020 daemon.err modprobe: failed to find dependency x_tables
Wed Jun 17 16:45:12 2020 daemon.err modprobe: 1 module could not be probed
Wed Jun 17 16:45:12 2020 daemon.err modprobe: - xt_addrtype
Wed Jun 17 16:45:12 2020 daemon.err modprobe: failed to find dependency x_tables
Wed Jun 17 16:45:12 2020 daemon.err modprobe: 1 module could not be probed
Wed Jun 17 16:45:12 2020 daemon.err modprobe: - xt_addrtype
Wed Jun 17 16:45:12 2020 daemon.err modprobe: failed to find dependency x_tables
Wed Jun 17 16:45:12 2020 daemon.err modprobe: 1 module could not be probed
Wed Jun 17 16:45:12 2020 daemon.err modprobe: - xt_addrtype
Wed Jun 17 16:45:13 2020 daemon.err dockerd[27539]: failed to start daemon: Error initializing network controller: Error creating default "bridge" network: Failed to program NAT chain: Failed to inject DOCKER in PREROUTING chain: iptables failed: iptables --wait -t nat -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER: iptables v1.8.4 (legacy): Couldn't load match addrtype':No such file or directory Wed Jun 17 16:45:13 2020 daemon.err dockerd[27539]: Wed Jun 17 16:45:13 2020 daemon.err dockerd[27539]: Try iptables -h' or 'iptables --help' for more information.

[lisaac]

docker 运行要满足 Generally Necessary 中的模块， 否则即使跑起来，也会出现各种错误，另外根据自己的需要，选择内核模块。

至于 addrtype 貌似是 xt-addrtype 模块。

PS: 如果你要测试 docker 运行时的 log，可以在关闭 docker daemon 的情况下，直接在命令行中运行 dockerd，并查看结果。

[kiddin9]

原来是IPV6的锅,编译的时候加上IPV6就可以了.
但是直接编译进固件的docker去掉ipv6依赖也能正常运行,
感谢