technical documentation about openmptcprouter

[Ysurac]

A complete technical documentation will be made when technical part will be finished :)

[olaulau]

of course, I'm just very interested, and don't understand all of this.

[lars18th]

Hi @Ysurac ,

I change from #27 to this thread, as you pointed.

VPS and OpenMPTCProuter connection is done with shadowsocks, a socks proxy. So it's TCP is over socks5.
VPN, when activated, is used to redirect ports from the VPS to the router and for ICMP and UDP (if shadowsocks is not used for that). Glorytun VPN is used over TCP.
Nginx is only used when multiple VPS are used, this permit VPS failover.

Ok. So as an overview this is the current architecture?

Layer 2 (with only one VPS server):

                                  /--- WAN1 ---> Public IP1 ---\
[LAN] ------ [OpenMPTCProuter] --<                              > [VPS server]
                                  \--- WAN2 ---> Public IP2 ---/

Please, correct me if something is wrong!

Then, my problem is to understand the Layer 3 architecture. Please, help me to clarify these issues:

• The TCP traffic from the LAN to Internet goes through what? Transparent redirection to SS? In this case, I don't know how it's used the MPTCP protocol.
• The UDP and IGMP traffic from the LAN to the Internet goes through the VPN? Why, if the SS can manage UDP traffic?
• The forward traffic from the VPS to the LAN is through the VPN. Correct?
• All the VPN traffic is TCP. So, it goes all the times over the SS connection?

I'm trying to undertand the global architecture. 😕

[Ysurac]

It's current architecture.

• Every TCP traffic is redirected to ShadowSocks, It's TCP over socks, but it's TCP. So this use MPTCP.
• By default, all UDP traffic is redirected to ShadowSocks using TPROXY. At least for now. Everything else use Glorytun VPN when activated, because it's used as default route.
• True
• False, the VPS IP is bypassed, so it's direct connection.

[lars18th]

Hi @Ysurac ,

Thank you for clarification! 😄

Then at time all traffic goes over TCP and through the multiple ShadowSocks connections.

In this case, I have some ideas that perhaps you like to consider:

• Split the Transport layer from the Application layer. That is, at time you send all the traffic over the SS tunnels. These tunnels represents the Transport layer, as opposed to the Application layer that is the traffic generated by the LAN clients. Why do this split? Because then you can use different connection types. For example, imagine that you like to connect over WAN1 and over a TUNNEL (this represents now the WAN2). In the first connection perhaps you like to use encryption, and in the second only offuscation. Then will be preferable to not rely on ShawdowSocks as the only method for sending TCP traffic over MPTCP connections. Futhermore, then you can compile and use the ShadowSocks without encryption and rely on the underlaying transport for the security layer.

• Another idea is to support different client Application protocols. For example, at time you redirect all LAN traffic (TCP and UDP) over the SOCKS5 server provided by the ShadowSocks protocol. This can have inconvenients with some application protocols. Then you can provide multiple "entry" points. For example: SOCKS5, HTTP proxy, and Transparent Redirection. For the SOCKS5 server you have the ShadowSocks. For the Transparent Redirection the TPROXY (why not redsocks instead of it?). And for HTTP proxy you can run the same nginx in the client side, or use some multi-protocol proxy (like https://github.com/guyingbo/shadowproxy). And for a real UDP traffic perhaps you like to route over a real UDP transport layer (VPN over UDP, direct tunnel, KCP https://github.com/xtaci/kcptun, etc.)

I hope you like to consider these ideas.
But, without diagrams is difficult to understand the ideas. 😕

[Ysurac]

• Not possible. Shadowsocks is used to aggregate all connections, so you must connect to the same shadowsocks server. Not sure this would provide anything to use an encryption for a connection and an other for an other connection. Shadowsocks without encryption will be available later, even if I don't know any use case yet.
• Socks5 is used to avoid TCP over TCP. In this case TCP congestion doesn't work well. I don't use an UDP tunnel for now because UDP is sometimes blocked or limited by some ISP but choice will be available soon (and an TCP VPN can use MPTCP).

Why would I use redsocks ? It's using TPROXY for UDP.

[Ysurac]

For second point, you mean to add a HTTP proxy and socks5 proxy available to lan connection ? Socks5 proxy port can already be open using the firewall.

[lars18th]

Hi @Ysurac ,

Shadowsocks without encryption will be available later, even if I don't know any use case yet.

The user case is when the WAN-x connection is over a TUNNEL or VPN. So, please implement the option for Shadowsocks without encryption.

Socks5 is used to avoid TCP over TCP

Sure! For this reason I suggest to use tunnels or udp-based-vpn's for the underlaying transport.

I don't use an UDP tunnel for now because UDP is sometimes blocked or limited by some ISP but choice will be available soon (and an TCP VPN can use MPTCP).

Great! I prefer to have one (or more) udp channels for UDP traffic; and only force UDP-over-TCP as a last change.

Why would I use redsocks ? It's using TPROXY for UDP.

Only in case of UDP-over-TCP.

For second point, you mean to add a HTTP proxy and socks5 proxy available to lan connection ?

Yes! For regular WEB traffic (even HTTPS), that is HTTP proxy, will be preferable (more optimized) to execute a local NGINX that uses SS-MPTCP enabled to reach the VPS server, instead of a direct SOCKS5 proxy. No?

And for the DNS server, I suggest to replace the current Unbound to a pure DNS server with TCP/HTTP support. For example, see the DNSCrypt Proxy (https://github.com/jedisct1/dnscrypt-proxy). Then you can configure it to use all the time the TCP transport for reach a remote DNS server. You agree?

[Ysurac]

• This will be done. But not sure a WAN over TUNNEL or VPN will work really well with MPTCP...
• UDP will not use MPTCP if not over TCP, but this will be added as manual settings
• OK, but TPROXY is used for UDP over socks5, no really need of redsocks for that
• Socks5 is better than a HTTP proxy that redirect to socks5.
• DNSCrypt need some servers that are not unicast. It's even less optimized as the current solution that use root server. It's always possible to use any DNS server using ss-tunnel.

[lars18th]

Hi @Ysurac ,

This will be done. But not sure a WAN over TUNNEL or VPN will work really well with MPTCP...

This will depends on the type of TUNNEL or VPN used. When using UDP or GRE, or similar protocol, as the encapsulation underlaying protocol, then I'm sure no problem will appear when the correct MTU value is used. So, please keep the option for use the configured MTU value in each path.

UDP will not use MPTCP if not over TCP, but this will be added as manual settings

Sure! So the idea is let the user to select if use UDP over MPTCP all the time; or select one path for routing UDP. And in this last case, I suggest to use a priority method: path 1, path 2, path N. This will be more versatile.

OK, but TPROXY is used for UDP over socks5, no really need of redsocks for that

OK.

Socks5 is better than a HTTP proxy that redirect to socks5.

This depends. But in any case, if you leave the option for runing one Nginx server in the client side, then you leave to the user the option for using it or not. You can redirect all TCP traffic over the Socks5 server, but leave the option for serving proxies (HTTP & Socks5) in local address. You agree?

DNSCrypt need some servers that are not unicast. It's even less optimized as the current solution that use root server. It's always possible to use any DNS server using ss-tunnel.

And this is my request! Leave the user to choose what DNS server use in the client side. If you can, please leave the option for install 3rd party DNS resolvers. For example, I prefer to use DNSCrypt and route all DNS traffic over TCP. Another time, you agree?

Thank you for this great project! 😄

[Ysurac]

• I added the none encryption for shadowsocks
• Not sure for select path... For now Multipath selected as master is used.
• No sure for http proxy.
• Options to install many packages will be added later, even if I don't like DNSCrypt at all.

[lars18th]

I added the none encryption for shadowsocks

Yes, I see it. Thank you!
I'm doing some research to disable it also in the VPN (as the Glorytun uses "mud", and this project has the encryption hardcoded).

Not sure for select path... For now Multipath selected as master is used.

If you route UDP traffice over the VPN, then Glorytun (as I feel) is using multipath in the sense of "best ROT path".

No sure for http proxy.

I'll explain more: I like to use this project as a "secondary" router. That is, not as my "primary" router. In the configuration of my stock primary router I have a secondary default gateway upstream route that is the OpenMPTCPRouter (configured with one IP address in my LAN segment that isn't the default gw address). And this "secondary default gateway route" in my primary router has a higher cost. So, in a regular state, my LAN traffic goes to my primary router and reach the Internet using the default connection. But, when the default connection goes off (manually or automatically), then all LAN traffic goes to the primary router and it forwards all the traffic to the OpenMPTCPRouter, that then routes all the traffic over the multipath connection to the VPS. In this scenerio, when the default connection is ON, I like to maintain the use of the OpenMPTCPRouter with explicit PROXY connections. That is, the LAN IP of the OpenMPTCPRouter needs to serve the SOCKS5 (1080/TCP) and HTTP (8080/TCP) proxy ports.

This is the main reason for one HTTP proxy in the client side. It's sufficient to run one Nginx in the client side using the SOCKS5 proxy. That's all!

Options to install many packages will be added later, even if I don't like DNSCrypt at all.

Great! This will be sufficient for me. Only remember to leave the option to enable/disable included services (for example the stock DNS server); or at minimum the option for running it in a different port.

Regards.

[Ysurac]

• Disabling encryption on the VPN would be really strange... No usage for that.
• Default route is Multipath master by default. And VPN if used. When I will add (and test) UDP VPN, it will use the default route too.
• maybe later
• It's already possible to disable unbound. dnsmasq forward DNS request to port 5353 that is unbound by default, can also be a ss-tunnel.

[lars18th]

Disabling encryption on the VPN would be really strange... No usage for that.

Not really! If your underlaying pathes are encrypted (using tunnels+encryption), then it has sense to disable encryption in the ShadowSocks side. But this is then only true for TCP traffic. All UDP (plus incoming forward traffic) is encrypted using the Glorytun VPN. This is the reason for disabling the encryption in the VPN.

maybe later (HTTP proxy).

OK!

It's already possible to disable unbound. dnsmasq forward DNS request to port 5353 that is unbound by default, can also be a ss-tunnel.

So, then I only need to run DNSCrypt in 5353 and it will be used. Great! And all 53/UDP (and 53/TCP) will be transparently routed to the DNS server in the 5353 port? Yeah! 😄

[Ysurac]

Why would you use a VPN if you already use tunnels+encryption ? VPN over VPN ? In this case don't run glorytun and configure the already running VPN.

Dnsmasq forward all requests, but this can also be configured in the LuCI interface where you want, even on a server listening on another ip.

[lars18th]

Why would you use a VPN if you already use tunnels+encryption ? VPN over VPN ? In this case don't run glorytun and configure the already running VPN.

I try to explain it:

• Open-MPTCP-Router uses ShadowSocks for "tunneling" all TCP traffic over multiple TCP connections using different paths.
• But, what about the UDP traffic, ICMP traffic, and forwarding traffic? In this case, as I understand all of such traffic goes through the Glorytun VPN. This is good because it uses the "best-one-path" for the traffic, and when one path fails, it uses another.
• So, if you disable the Glorytun VPN then you lost this functionality. For this reason my idea is to support to run it without encryption (as a simple "tunneling" mode) when your underlaying WAN paths are alreasy encrypted.

This has sense for you?

Dnsmasq forward all requests, but this can also be configured in the LuCI interface where you want, even on a server listening on another ip.

That's a perfect solution! 👍

[Ysurac]

The default route is always set to master or, if master down, to a working path (if any). This feature is not lost when glorytun VPN is down. Traffic will use default route, and if already encrypted no need of glorytun for that.

[lars18th]

Traffic (UDP/ICMP) will use default route...

Yes, but please review the "mud" project description: https://github.com/angt/mud
It says: "It enables the distribution of packets on multiple paths while maintaining a low latency".

So, when using the Glorytun VPN you use the best path for such UDP/ICMP traffic, and not the "master" route. And if you don't use at all then VPN, then you lost this function. Futhermore, you lost the Forwading TCP functionality.

Why then not have the "option" for disabling the encryption with Glorytun?
And yes, the current code doesn't have this functionality. It's required a patch for a simple XOR in mud.

Regards.

[Ysurac]

The version of glorytun used by OpenMPTCProuter doesn't use mud. Mud is used by the UDP version.

Glorytun TCP with encryption can be used inside a tunnel, this will really not have big speed impact.

[lars18th]

The version of glorytun used by OpenMPTCProuter doesn't use mud. Mud is used by the UDP version.
Glorytun TCP with encryption can be used inside a tunnel, this will really not have big speed impact.

Then, this is the reason to have a better documentation!
Please, review this:

Open-MPTCP-Router uses:

• TCP Traffic: Transparent TPROXY-->ShadowSocks SOCKS5-->MPTCP-->VPS server-->Internet.
• SOCKS Proxy: ShadowSocks SOCKS5-->MPTCP-->VPS server-->Internet.
• UDP Traffic: Direct forward UDP or/and UDP-over-TCP with TPROXY-->ShadowSocks SOCKS5-->MPTCP-->VPS server-->Internet.
• ICMP Traffic: Glorytun UDP-->direct forward UDP-->VPS server-->Internet.
• Forward TCP Traffic: Internet-->VPS server-->Glorytun TCP-->ShadowSocks SOCKS5-->LAN.
• DNS Server: unbound-->ShadowSocks SOCKS5-->MPTCP-->VPS server-->Default DNS.
• ???

[lars18th]

Hi @Ysurac ,

Regarding the use of Glorytun...

• You're using it only as "tunnel" solution? If this is true, what about supporting other simple VPN tools?
• If you use Glorytun TCP for multipath support for UDP-over-TCP, why not support also the Glorytun-MUD version for routing the UDP traffic?

I ask this because the use of encryption has a very high impact in performance. And when using it with already encrypted tunnels, in this scenario it's completly useless.

[Ysurac]

As I already said, documentation will be done for OpenMPTCProuter 1.0, it's alpha version for now. All may change before stable version.

• TCP traffic is redirected to socks 5 proxy, but without TPROXY, TPROXY is used for UDP.
• SOCKS ok
• UDP traffic use shadowsocks redirected with TPROXY by default, but this may change in the future.
• ICMP traffic (and anything that is not TCP or UDP) use Glorytun TCP (when enabled), so it's ICMP over TCP here.
• forward : no shadowsocks here. Internet ->VPS->glorytun TCP->OpenMPTCProuter
• DNS server: by default root server are used, it's UDP so it's over shadowsocks by default too.

Glorytun:

• I use glorytun because it's fast, small, support MPTCP and it's working quite well.
• Glorytun mud is UDP VPN, no TCP or MPTCP here. But I already said that glorytun UDP support will be added and supported for both OpenMPTCProuter and OpenMPTCProuter VPS.

You make some test of chacha20 encryption impact ? And it's only for ICMP, who care ?

[lars18th]

Hi @Ysurac ,

As I already said, documentation will be done for OpenMPTCProuter 1.0, it's alpha version for now. All may change before stable version.

I know. So, I only like to comment here my experiencies with this project. Don't worry.

Regarding the current implementation:

• TCP: redirected over SOCKS5 proxy (ShadowSocks) with MPTCP enabled.
• UDP: option 1) redirected with TSOCKS over SOCKS5 proxy (SS) with MPTCP.
• UDP: option 2) forwarded using Glorytun TCP with MPTCP.
• UDP: option 3) forwarded using Glorytun UDP (in the near future).
• UDP: option 4) directly forwarded (without encryption?).
• ICMP/others: forwarded using Glorytun TCP with MPTCP.
• Incoming TCP Forward: over Glorytun TCP (with MPTCP enabled).
• DNS: standalone server over SOCKS5 proxy (ShadowSocks) with MPTCP enabled.

It's this true at time?

My suggestion is to replace this current "in-App-custom-Protocol" model with a "layer-level" model. For example:

• TCP transport: ShadowSocks with MPTCP enabled (encryption optional).
• UDP transport A: over TCP with TSOCKS .
• UDP transport B: over TUN device.
• ICMP/other transport: over TUN device.

Then as for the Application level:

• SOCKS server (TCP and UDP): over TCP transport.
• HTTP server (HTTP and HTTPS): over TCP transport.
• TCP transparent redirection: over TCP transport.
• DNS server: over TCP, UDP A or UDP B (user selectable).
• Incoming TCP forwarding: over TUN device.
• Future services: the transport will be user selectable if it's not a TCP native service (SIP, etc.).

And regarding the TUN device:

• Option 1: Glorytun over TCP (over SS with MPTCP).
• Option 2: Glorytun-mud with TSOCKS over TCP (perhaps useless).
• Option 3: Glorytun-mud with direct UDP connection (over one path only).
• Option 4: IPIP tunnel (layer 3 simple tunnel).
• Option 5: or any other TUN device provided by the user.

And why the IPIP protocol? Because if you use Glorytun as a simple "tunneling protocol" (without NAT) then you can use this simple protocol for setup the TUN device. This is lightweight, doesn't use encryption and it's native in OpenWRT.

And regarding the option of a custom option for setup the TUN device this will leave to the user the option for use any VPN solution that s/he likes to use.

I use glorytun because it's fast, small, support MPTCP and it's working quite well.

And it can be a great solution when using only MPTCP transport. But, think on this: imagine that you like to route some protocol (like SIP) over UDP in one path only, and other protocols (like UDT) over the Glorytun MPTCP. If you use this "layer-level" model then the user can select over wich transport protocol route each service. In fact, perhaps you will need to support more than one TUN device (tun0, tun1, tun2, etc.) and the user can then "map" each non TCP service to the prefered TUN device. And then, the "tun0" can be using the Glorytun-TCP, the "tun1" the Glorytun-mud, the "tun2" the IPIP tunnel, etc.

You make some test of chacha20 encryption impact ? And it's only for ICMP, who care ?

I care when routing other protocols that aren't not TCP. At time, the user can't control wich services use UDP-over-MPTCP or not. Futhermore, when using the TCP Incoming Forwarding all the time the Glorytun is used... and as the encryption is enforced... for an incomming SSH connection... over a tunneled already encrypted connection... you will end with a three-encryption-layers. A waste of resources!

I hope you like to thing on this approach... even if you doesn't like it! Perhaps you can found a better solution.
Regards.

[github-actions]

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 5 days