subutai-io / p2p Goto Github PK

For EZSSH we need to support MacOS

Installations on Ubintu-VMWare-Win, Windows-VMWare-Win: can not ping container from host, can not ping hoist from RH

karaflogs2016-06-03 10-39-26.txt
error_03.06.2016.txt
info_03.06.2016.txt

journalctl -u p2p shows nothing

p2p_log.txt

We need to refactor existing packages so they're using correct names and looks clear. Right now we have a mess of mixed packages.
Directory structure should be reviewed too.

System Configuration
Windows 10 x64
OpenVPN 2.3.10-l601
TAP Windows 9.21.1

P2P Log
[INFO] 2016/03/14 19:33:39 Starting RPC Listener on 52523 port
[INFO] 2016/03/14 19:33:44 Generate MAC for TAP device: 06:fa:b9:17:8c:9e
[INFO] 2016/03/14 19:33:44 Looking for available network interfaces
[INFO] 2016/03/14 19:33:44 Interface Ethernet: 192.168.0.100/16. Type: Global Unicast. Saving
[INFO] 2016/03/14 19:33:44 Interface Loopback Pseudo-Interface 1: ::1/28. Type: Loopback. Ignoring
[INFO] 2016/03/14 19:33:44 Interface Loopback Pseudo-Interface 1: 127.0.0.1/16. Type: Loopback. Ignoring
[INFO] 2016/03/14 19:33:44 1 interfaces were saved
[INFO] 2016/03/14 19:33:44 No AES key were provided. Traffic encryption is disabled
[ERROR] 2016/03/14 19:33:44 Failed to load config: open config.yaml: The system cannot find the file specified.
[INFO] 2016/03/14 19:33:44 Acquired control over TAP interface: Ethernet 2
[INFO] 2016/03/14 19:33:44 vptp1 TAP Device created
[INFO] 2016/03/14 19:33:44 Configuring Ethernet 2. IP: 10.10.10.1 Mask: 255.255.255.0
[INFO] 2016/03/14 19:33:44 Executing: netsh interface ip set address "Ethernet 2" static 10.10.10.1 255.255.255.0
[ERROR] 2016/03/14 19:33:44 Failed to properly configure TAP device with netsh: exit status 1
[INFO] 2016/03/14 19:33:44 Started UDP Listener at port 51721
[INFO] 2016/03/14 19:33:44 DHT operating in CLIENT mode
[INFO] 2016/03/14 19:33:44 Connecting to a router dht1.subut.ai:6881
[INFO] 2016/03/14 19:33:44 Ready to peer discovery via dht1.subut.ai:6881 [52.34.217.204:6881]
[INFO] 2016/03/14 19:33:44 Handshaked. Starting listener
[INFO] 2016/03/14 19:33:44 Bootstraping via 52.34.217.204:6881

When we want to create fresh swarm we need to know which IP subnets are used on a remote peer to avoid conflicts.

Currently each peer requests list of connected peers from DHT. We need DHT to notify peers about new participants. This will reduce traffic.

User may want to request invoke of a p2p start command on a remote peer.

When client have any IPv6 unicast interface in the system it will kill DHT during handshake procedure.

In a situation, when every peer was accidentally disconnected from DHT (e.g. network connection problems) information about their network mask and distributed IPs are erased and it can't be restored after reconnect (unless application was restarted). This needs to be fixed

After creating Subutai crosspeer environment, p2p daemon crash and connection cannot be established.

• 2xRH in LAN;
• encryption enabled;
• One RH with static IP, one with dynamic.

p2p.txt
p2p.txt

Logs from crosspeer initiator:
karaflogs2016-03-25 13-10-43.txt

We need to have a configure script to check availability of UPX and goupx.

We need to make sure, that DHT doesn't support some of previous realizations. We need to send this information during connection and refuse connections if version is not supported

After setting up connection without -fwd option, after some time connection is lost inside peer network, cannot ping or connect to any port to devices within peer network.

Restart with -fwd resolves the problem

After SS installed on vm (Ubuntu 16.04 host-VMWare(NAT)-Win10 have message
.
Can connect to SS, register peer, create environment
have many errors in tray log
error_02.06.2016.txt

karaflogs2016-06-02 17-05-20.txt

1. Installed Subutai peer using windows installer-dev.
2. Created environment
3. Trying to connect to the container using tray application (EZ-SSH).
4. After clicking connect to the environment nothing happens for few seconds.
5. Withing this short period of time executing p2p debug and getting panic on p2p daemon:

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x98 pc=0x4052c2]

goroutine 33 [running]:
panic(0x8b42a0, 0xc082008070)
        C:/Go/src/runtime/panic.go:464 +0x3f4
main.(*Procedures).Debug(0xc082009840, 0xc0822006a0, 0xc082200780, 0x0, 0x0)
        D:/Go/src/github.com/subutai-io/p2p/instance.go:335 +0x592
reflect.Value.call(0x878940, 0x908948, 0x13, 0x9478a0, 0x4, 0xc082221ed8, 0x3, 0x3, 0x0, 0x0, ...)
        C:/Go/src/reflect/value.go:435 +0x1214
reflect.Value.Call(0x878940, 0x908948, 0x13, 0xc082221ed8, 0x3, 0x3, 0x0, 0x0, 0x0)
        C:/Go/src/reflect/value.go:303 +0xb8
net/rpc.(*service).call(0xc082007d80, 0xc082007840, 0xc0821f49f0, 0xc082076300, 0xc0821ce6a0, 0x7d0640, 0xc0822006a0, 0x16, 0x7d0760, 0xc082200780, ...)
        C:/Go/src/net/rpc/server.go:383 +0x1c9
created by net/rpc.(*Server).ServeCodec
        C:/Go/src/net/rpc/server.go:477 +0x4a4

p2p debug gives following at that moment:

C:\Users\user>p2p debug
[ERROR] Failed to run RPC request: read tcp 127.0.0.1:60436->127.0.0.1:52523: wsarecv: An existing connection was forcibly closed by the remote host.

P2P daemon works only with active connections and does not store it somewhere. And in case if service was restarted or crashed, P2P daemon after restart has clear configuration and does not restore previous connections.

It will be incredibly useful if P2P daemon will restore all active connection after start.

Currently WindowsTAP driver works with only one TAP device and new devices can be added with a BAT-file. We need to add TAP device management for windows system

We need to find a good place on where to look for config.yaml file. Currently it search in PATH which is not acceptable. For MacOS and Linux we may want to use a directory inside /etc or in user specific configuration directory

I created env using 3 peers:

• Askat Peer (behind NAT)
• Sydyk Peer (behind NAT)
• AWS Germany (accessible directly)

I can successfully ping containers to each other (see ping-ssh-check.txt). But SSH fails between them (askat-peer-p2p-logs.txt). Seems this issue is mostly between peers behind NAT.

Note: for each command's response I waited ~20 sec. If there was none I typed Ctrl-C.

ping-ssh-check.txt
askat-peer-p2p-logs.txt

We need to bind ID to some clients based on their GPG fingerprint.

For EZSSH we need to support Windows operating systems
@Lezh1k is blocked by this issue

We've discover a bug when key are not being switched after timeout period has been passed. New key is still stored in the memory, but old one is used instead.

cannot integrate with peers without -fwd flag

To reproduce you need to crate x-peer and restart remote VM
On initiator try to ping or ssh remote containers.

In a situation when DHT has no control peers registerd it will send empty control peer is response, which is handled by client as a normal control peer. We need to avoid that.

Daemon after some time periodfailed with "slice bounds out of range". Network consisted of 2 peers. No particular activity over network were made, just created network.
Here is last log.

p2p-daemon.txt

SendTo method iterates over each peer to find a destination peer based on a MAC address.

There is a possibility that multiple tunnels for same connection will be created by control peer

We need to switch from slice of peers to map of peers with UUID as a key.
Also, we need to remove switch operator and use callbacks in Listen method

If for some reason fails to communicate to a DHT server for some time - it is going to be removed from DHT list. This needs to be fixed.

If an RH is rebooted, then p2p instances can not be deleted, after deletion they reappear.

When control peer have peer that was not set to "Ready" state - pings will not be sent to them. This causes dead peers to not be disconnected by timeout

I created environment two days ago, the container on remote peer (not in LAN, over WAN) is not accessible via p2p. It is running and accessible via SS.
p2p.txt

Current introduction schemes may cause peer to skip received data about other network participants. Therefore, instead of having only single MT_INTRO we need also to have a intro request message type

1. Run p2p daemon
2. Launch 2 terminals.
3. Launch in each terminal start command with ip = "dhcp" (try to join to existing swarm).
4. See crash report. Reproduces everytime. As we know this issue related to not thread-safe map.

For some reason DHCP can't give IP and mask to some machines.

P2P should use UPnP technology for creation direct connection between two peers.
It allows to configure router with required port forwarding rule to establish direct connection.
It will reduce load for proxy nodes and will increase connection speed between peers.

We're using third-party library for generating UUID. We need to implement our own function to get rid from importing the whole library for one specific function

During handshake with a control peer unencrypted message will make p2p panic.

Behavior
If you have P2P running - you cannot start any OpenVPN connection
If you have any active OpenVPN connections you are getting panic on command p2p start -ip ...

Log of exception
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x8 pc=0x4c805f]

goroutine 7 [running]:
github.com/subutai-io/p2p/lib.openDevice(0xc082128d00, 0x5, 0xc08202b468, 0x0, 0x0)
C:/Projects/go/src/github.com/subutai-io/p2p/lib/tuntap_windows.go:147 +0x25f
github.com/subutai-io/p2p/lib.Open(0xc082128d00, 0x5, 0x1, 0x8b1680, 0x0, 0x0)
C:/Projects/go/src/github.com/subutai-io/p2p/lib/tuntap_windows.go:258 +0x3e
main.(_PTPCloud).AssignInterface(0xc082062c60, 0xc082009e30, 0xa, 0xc08210e3e0, 0x11, 0xc082009e50, 0xd, 0xc082128d00, 0x5, 0x0, ...)
C:/Projects/go/src/github.com/subutai-io/p2p/p2p.go:80 +0x3ef
main.p2pmain(0xc082009e30, 0xa, 0xc082009e50, 0xd, 0xc08210e3e0, 0x11, 0xc082128d00, 0x5, 0x0, 0x0, ...)
C:/Projects/go/src/github.com/subutai-io/p2p/p2p.go:283 +0x11a3
main.(_Procedures).Run(0xc0820d6450, 0xc082070dc0, 0xc08210e2c0, 0x0, 0x0)
C:/Projects/go/src/github.com/subutai-io/p2p/instance.go:201 +0x3bf
reflect.Value.call(0x801880, 0x883510, 0x13, 0x8bba50, 0x4, 0xc08202bed8, 0x3, 0x3, 0x0, 0x0, ...)
c:/go/src/reflect/value.go:432 +0x1211
reflect.Value.Call(0x801880, 0x883510, 0x13, 0xc08202bed8, 0x3, 0x3, 0x0, 0x0, 0x0)
c:/go/src/reflect/value.go:300 +0xb8
net/rpc.(_service).call(0xc0820d0700, 0xc0820d0200, 0xc082009700, 0xc0820dc280, 0xc082003ae0, 0x76e4e0, 0xc082070dc0, 0x16, 0x76e480, 0xc08210e2c0, ...)
c:/go/src/net/rpc/server.go:383 +0x1c8
created by net/rpc.(_Server).ServeCodec
c:/go/src/net/rpc/server.go:477 +0x4b3

goroutine 1 [sleep]:
time.Sleep(0x12a05f200)
c:/go/src/runtime/time.go:59 +0x107
main.Daemon(0x8bb4b8, 0x5, 0x0, 0x0, 0x0, 0x0)
C:/Projects/go/src/github.com/subutai-io/p2p/main.go:339 +0x641
main.main()
C:/Projects/go/src/github.com/subutai-io/p2p/main.go:118 +0x162e

goroutine 17 [syscall]:
os/signal.loop()
c:/go/src/os/signal/signal_unix.go:22 +0x1f
created by os/signal.init.1
c:/go/src/os/signal/signal_unix.go:28 +0x3e

goroutine 18 [IO wait]:
net.runtime_pollWait(0x2a312b0, 0x72, 0xc0820096b0)
c:/go/src/runtime/netpoll.go:157 +0x67
net.(_pollDesc).Wait(0xc0820342f0, 0x72, 0x0, 0x0)
c:/go/src/net/fd_poll_runtime.go:73 +0x41
net.(_ioSrv).ExecIO(0xc0820d8030, 0xc0820341e0, 0x8c5f30, 0x8, 0xc082003a60, 0xc08201ad80, 0x0, 0x0)
c:/go/src/net/fd_windows.go:182 +0x177
net.(_netFD).acceptOne(0xc082034180, 0xc0820841c0, 0x2, 0x2, 0xc0820341e0, 0x220000, 0x0, 0x0)
c:/go/src/net/fd_windows.go:564 +0x26c
net.(_netFD).accept(0xc082034180, 0x0, 0x0, 0x0)
c:/go/src/net/fd_windows.go:594 +0x173
net.(_TCPListener).AcceptTCP(0xc0820d8040, 0xc0820ede60, 0x0, 0x0)
c:/go/src/net/tcpsock_posix.go:254 +0x54
net.(_TCPListener).Accept(0xc0820d8040, 0x0, 0x0, 0x0, 0x0)
c:/go/src/net/tcpsock_posix.go:264 +0x44
net/http.(*Server).Serve(0xc0820f0060, 0x2a30308, 0xc0820d8040, 0x0, 0x0)
c:/go/src/net/http/server.go:1887 +0xba
net/http.Serve(0x2a30308, 0xc0820d8040, 0x0, 0x0, 0x0, 0x0)
c:/go/src/net/http/server.go:1765 +0x8c
created by main.Daemon
C:/Projects/go/src/github.com/subutai-io/p2p/main.go:323 +0x55a

goroutine 19 [chan receive]:
main.Daemon.func1(0xc0820f0000)
C:/Projects/go/src/github.com/subutai-io/p2p/main.go:332 +0x85
created by main.Daemon
C:/Projects/go/src/github.com/subutai-io/p2p/main.go:337 +0x62e

goroutine 6 [IO wait]:
net.runtime_pollWait(0x2a311f0, 0x72, 0xc082009e80)
c:/go/src/runtime/netpoll.go:157 +0x67
net.(_pollDesc).Wait(0xc0820348f0, 0x72, 0x0, 0x0)
c:/go/src/net/fd_poll_runtime.go:73 +0x41
net.(_ioSrv).ExecIO(0xc0820d8030, 0xc0820347e0, 0x8bdbc8, 0x7, 0x9847c8, 0xc082078201, 0x0, 0x0)
c:/go/src/net/fd_windows.go:182 +0x177
net.(_netFD).Read(0xc082034780, 0xc08210a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
c:/go/src/net/fd_windows.go:463 +0x17e
net.(_conn).Read(0xc082024080, 0xc08210a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
c:/go/src/net/net.go:172 +0xeb
bufio.(_Reader).fill(0xc08200e420)
c:/go/src/bufio/bufio.go:97 +0x1f0
bufio.(_Reader).Read(0xc08200e420, 0xc0820096e0, 0x1, 0x9, 0x0, 0x0, 0x0)
c:/go/src/bufio/bufio.go:207 +0x267
io.ReadAtLeast(0x3a4a50, 0xc08200e420, 0xc0820096e0, 0x1, 0x9, 0x1, 0x0, 0x0, 0x0)
c:/go/src/io/io.go:298 +0xed
io.ReadFull(0x3a4a50, 0xc08200e420, 0xc0820096e0, 0x1, 0x9, 0xc0820fd000, 0x0, 0x0)
c:/go/src/io/io.go:316 +0x69
encoding/gob.decodeUintReader(0x3a4a50, 0xc08200e420, 0xc0820096e0, 0x9, 0x9, 0x0, 0x1, 0x0, 0x0)
c:/go/src/encoding/gob/decode.go:121 +0x99
encoding/gob.(_Decoder).recvMessage(0xc082078280, 0xc08202f408)
c:/go/src/encoding/gob/decoder.go:76 +0x65
encoding/gob.(_Decoder).decodeTypeSequence(0xc082078280, 0x984e00, 0xc082078280)
c:/go/src/encoding/gob/decoder.go:140 +0x4e
encoding/gob.(_Decoder).DecodeValue(0xc082078280, 0x771ae0, 0xc08210e2e0, 0x16, 0x0, 0x0)
c:/go/src/encoding/gob/decoder.go:208 +0x164
encoding/gob.(_Decoder).Decode(0xc082078280, 0x771ae0, 0xc08210e2e0, 0x0, 0x0)
c:/go/src/encoding/gob/decoder.go:185 +0x290
net/rpc.(_gobServerCodec).ReadRequestHeader(0xc0820878f0, 0xc08210e2e0, 0x0, 0x0)
c:/go/src/net/rpc/server.go:403 +0x58
net/rpc.(_Server).readRequestHeader(0xc0820d0200, 0x3a4aa0, 0xc0820878f0, 0x0, 0x0, 0xc08210e2e0, 0xc08202f700, 0x0, 0x0)
c:/go/src/net/rpc/server.go:576 +0x97
net/rpc.(_Server).readRequest(0xc0820d0200, 0x3a4aa0, 0xc0820878f0, 0xc0820d0200, 0xc082009700, 0xc0820dc280, 0x0, 0x0, 0x0, 0x0, ...)
c:/go/src/net/rpc/server.go:543 +0x92
net/rpc.(_Server).ServeCodec(0xc0820d0200, 0x3a4aa0, 0xc0820878f0)
c:/go/src/net/rpc/server.go:462 +0x93
net/rpc.(_Server).ServeConn(0xc0820d0200, 0x3a49f0, 0xc082024080)
c:/go/src/net/rpc/server.go:454 +0x4f5
net/rpc.(_Server).ServeHTTP(0xc0820d0200, 0x3a4940, 0xc0820b5810, 0xc0820842a0)
c:/go/src/net/rpc/server.go:694 +0x50a
net/http.(_ServeMux).ServeHTTP(0xc082086d80, 0x3a4940, 0xc0820b5810, 0xc0820842a0)
c:/go/src/net/http/server.go:1699 +0x184
net/http.serverHandler.ServeHTTP(0xc0820f0060, 0x3a4940, 0xc0820b5810, 0xc0820842a0)
c:/go/src/net/http/server.go:1862 +0x1a5
net/http.(_conn).serve(0xc0820b5760)
c:/go/src/net/http/server.go:1361 +0xbf5
created by net/http.(*Server).Serve
c:/go/src/net/http/server.go:1910 +0x3fd

In order to use efficient crossplatform building scheme we need to rewrite code related to TAP devices functionality and separate parts related to Windows and other operating systems, because Windows realization doesn't compatible with out app architecture

https://confluence.critical-factor.com/x/OIHBAQ

Log shows following information:

Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving
Mar 19 06:16:51 rh1458367464.gw.intra.lan ubuntu-core-launcher[884]: [INFO] 2016/03/19 06:16:51 Received control peer . Saving

At that moment pings doesn't work between instances.

p2p instance highly CPU loaded at that moment:

Test case:

1. Create env with any template.
2. After ready status try ssh into container.
   Result: Connection time out.

We need to make it work or find alternatives