Coder Social home page Coder Social logo

python-p2p-network's Issues

Bug in connect_with_node function in node.py

The connect_with_node function returns None when a connection is successfully established since there is no return statement before the end of the function in that case, however, I would expect it to return True to indicate that the connection was successful.

Am I misunderstanding what the return value represents (in that case an extra line elaborating on this in the function's docstring would help) or is there just a return statement missing?

Import "p2pnetwork.node" could not be resolved

Hello! I'm trying to use this library but I can't seem to import it. I ran the pip (and pip3 to test) install correctly and no errors were shown, and even if I run pip list it shows p2pnetwork. Does anyone have an idea as to why I'm getting this import error?
Thank you!

New feature: Plugins

I got the idea to improve the framework with plugins to implement the required functionality instead of extending the class. The main application starts the node and adds the required plugins to the scene that is required to implement the application. Implementation of a discovery methodolgy is than simply done by the implementation of a plugin. Furthermore, sharing is made easier, because these pugins can be easily used by others. Therefore, this functionality will enable the open source creation of the functionality. I am also able to supply basic plugins to the scene like encryption, blockchain and discovery things. I will start developing this functionality and also try to create an example.

Connect to a node already connected to

I was wondering why a node can't connect to a node that is already connected to the former. See this line. Lets say that nodeA has an outbound connection to nodeB. Why can't nodeB have an outbound connection to nodeA?

Perhaps I don't fully understand how connections work. Lets say that we have the following connections: nodeA->nodeB->nodeC, where -> denotes a connection, being the node on the left the outbound one and the right the inbound. Summarised as:

  • nodeA has an outbound connection to nodeB
  • nodeB has an inbound connection to nodeA
  • nodeB has an outbound connection to nodeC
  • nodeC has an inbound connection to nodeB

If in the previous setup a nodeB calls send_to_nodes, will it reach nodeA?

Thanks!

I have link question between "node.py" and "nodeconnection.py"

Hi!

In nodeconnection.py, we have a Nodeconnection class, it has 5 parameters, respectively are main_node, sock, id, host, port.
But in node.py, "create_new_connection" method in node class has 4 parameters. And it return class that has 4 parameters.
I don't understand this behavior. Or am I missing something?

Thank you!

Misspelled debug messages

I found misspelled word on nodeconnection.py Its on line 170

self.main_node.debug_print('datatype used is not valid plese use str, dict (will be send as json) or bytes')

Crash if node id malformed

If a connecting client sends a malformed id/port message, server will throw an uncaught exception.

In run(), I threw a try/except around the split to catch the error, but maybe there are better solutions?

                   if ":" in connected_node_id:
                        print("Got connected node id", connected_node_id)
                        try:
                                (connected_node_id, connected_node_port) = connected_node_id.split(':') # When a node is connected, it sends it id!
                        except Exception as e:
                                print(e)
                                connection.close()
                                continue

Issue: unit test test_node_max_connections failed

I could not succeed the unit test test_node_max_connections. The length of the node_1_inbound is not right, seems like a bug of some sort. I have debugged the code and seems the connection is lost between node0 and node1 after sending data from the nodes. Before sending data, the inbound connections of node1 are correct.

node_0.send_to_nodes('hello from node 0')
time.sleep(2)


#place where node seems to lose it's connection
node_1.send_to_nodes('hello from node 1')
time.sleep(2)

node_2.send_to_nodes('hello from node 2')
time.sleep(2)

#test that is false.
self.assertEqual(node_1_inbound, 2, "Node 1 should have two connections from node_0 and node_2!")

After some deeper debugging it seems the node in send_to_node is in self.all_node however the if statement is still false.

if n in self.all_nodes:
  n.send(data, compression=compression)
else:
  self.debug_print("Node send_to_node: Could not send the data, node is not found!")

Besides I also get the following warning however, I get this warning before the connection is lost.

ResourceWarning: unclosed <socket.socket fd=11, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 49489)>

I will continue debugging later this week, will let know if I find something.

how to use send_to_node api?

i want to send msg to an specify node with code

node_1.send_to_node(node_3,data="message: Hi there from 1!\n") ,but it not works,while send_to_nodes can works well,is there any demo show how to do?

Length protocol instead of EOT

Hey Maurice,

Currently, I'm working on my own cryptocurrency project in Python by using a modified version of this package. Isn't a length protocol between the socket where, the first 4 bytes will be sent to determine the length of the upcoming package, then the end of transmission protocol better than the EOT protocol? If you want to send binary data to the socket isn't there a possibility that the binary contains the byte sequence EOT?

Currently, I'm implementing this in my own project however, I have not tested it fully yet.

how to

node_1.send_to_node(node_3,data="message: Hi there from 1!\n")

Pip3 version is out of date and unstable

The latest version of p2pnetwork in pip3 is 0.0.3 and has at least one bug in nodeconnection.py. Particularly on line 54 there is a bug that has since been patched on the development and master branches self.info = dict -> self.info = {}, but a project like python-p2p-secure-node that specifies version 0.0.3 can crash when a "pong" is received.

I have question about the "nodeconnection.py"

The Event Flag is initialized in this file.

self.terminate_flag = threading.Event()

And it is false. This means that the thread is in a wait state.
But in run function:

while not self.terminate_flag.is_set(): ...

It mean the program will run when the flag is False, But in fact in this time we want it wait.
I want to say that why not while self.terminate_flag.is_set():.
Because it seems more common sense.

"RuntimeError: cannot join current thread" while sending a message to another node

What's should happen
I need a node to send a response to another node. This is how my code looks:

def add_node_to_network(self, node, conn_type):
    super().connect_with_node(node.get_host(), node.get_port())
    if conn_type == 'indirect':
        super().send_to_node(node, {'type' : 'join_request_resp'})
        self.peer_book[self.UUID + 1] = node
        print("Node {} was added to the peer book".format(self.UUID+1))
        print(self.nodesIn)
        super().send_to_nodes({'type' : 'new_peer', 'node' : node, 'UUID' : self.UUID + 1})
        print("Sourcing information about new peer...")

What actually happened
I get this response

connect_with_node(127.0.0.1, 51687)
Node 1 was added to the peer book
[<NodeConnection(Thread-4, started 123145562787840)>]
p2p_event_node_inbound_closed: Thread-4
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/anaconda3/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/Projekty/IPDS-RELATED/IPDS-Python/src/../lib/TcpServerNode.py", line 433, in run
    self.nodeServer.event_node_message(self, data)
  File "src/gateway_server.py", line 48, in event_node_message
    self.add_node_to_network(node, conn_type)
  File "src/gateway_server.py", line 63, in add_node_to_network
    super().send_to_nodes({'type' : 'new_peer', 'node' : node, 'UUID' : self.UUID + 1})
  File "/Projekty/IPDS-RELATED/IPDS-Python/src/../lib/TcpServerNode.py", line 181, in send_to_nodes
    self.send_to_node(n, data)
  File "/Projekty/IPDS-RELATED/IPDS-Python/src/../lib/TcpServerNode.py", line 192, in send_to_node
    self.delete_closed_connections()
  File "/Projekty/IPDS-RELATED/IPDS-Python/src/../lib/TcpServerNode.py", line 153, in delete_closed_connections
    n.join()
  File "/anaconda3/lib/python3.7/threading.py", line 1029, in join
    raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread

It seems that there is an error in the send_to_node function. I looked in your code and I didn't find anything. Maybe I'm doing something wrong.

There's more
There is another weird thing I noticed. I tried to use get_port() function you provided in your code, but it returns a different port (in my case 56310 when I specified 4560). I've tried writing the port manually in the code, but it didn't fix the issue.

PS. I see that self and super() both work. I tried both with the same effect.

Unit tests do work on Windows ... not on Linux

The unit tests that are created to test all the functionality of the package do work on Windows. On Linux it fails unfortunately. This should be investigated and solved. The following output is shown when the tests are runned on Linux:

$ python3 setup.py test
running test
Searching for nose
Best match: nose 1.3.7
Processing nose-1.3.7-py3.7.egg

Using /home/maurice/projects/python-p2p-network/.eggs/nose-1.3.7-py3.7.egg
running egg_info
writing p2pnetwork.egg-info/PKG-INFO
writing dependency_links to p2pnetwork.egg-info/dependency_links.txt
writing top-level names to p2pnetwork.egg-info/top_level.txt
reading manifest file 'p2pnetwork.egg-info/SOURCES.txt'
writing manifest file 'p2pnetwork.egg-info/SOURCES.txt'
running build_ext
test_extending_class_of_node (p2pnetwork.tests.test_node.TestNode)
Testing the class implementation of the Node. ... ok
test_node_communication (p2pnetwork.tests.test_node.TestNode)
Test whether the connected nodes are able to send messages to each other. ... ok
test_node_complete (p2pnetwork.tests.test_node.TestNode)
Testing the complete sequence of the Node based on Samuel complete_test.py. ... ERROR
/usr/lib/python3.7/unittest/case.py:643: ResourceWarning: unclosed <socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('0.0.0.0', 0)>
  outcome.errors.clear()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
test_node_connection (p2pnetwork.tests.test_node.TestNode)
Testing whether two Node instances are able to connect with each other. ... ERROR
/usr/lib/python3.7/unittest/case.py:643: ResourceWarning: unclosed <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('0.0.0.0', 0)>
  outcome.errors.clear()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.7/unittest/case.py:643: ResourceWarning: unclosed <socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 10001)>
  outcome.errors.clear()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
test_node_events (p2pnetwork.tests.test_node.TestNode)
Testing the events that are triggered by the Node. ... ERROR
test_node_send_data_dict (p2pnetwork.tests.test_nodeconnection.TestNode)
Testing whether NodeConnections handle sending dict well enough. ... ERROR
/usr/lib/python3.7/unittest/case.py:643: ResourceWarning: unclosed <socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 8001)>
  outcome.errors.clear()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
test_node_send_data_str (p2pnetwork.tests.test_nodeconnection.TestNode)
Testing whether NodeConnections handle sending str well enough. ... ERROR

======================================================================
ERROR: test_node_complete (p2pnetwork.tests.test_node.TestNode)
Testing the complete sequence of the Node based on Samuel complete_test.py.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/tests/test_node.py", line 121, in test_node_complete
    node_0 = Node('127.0.0.1', 8000, node_callback)
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/node.py", line 69, in __init__
    self.init_server()
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/node.py", line 92, in init_server
    self.sock.bind((self.host, self.port))
OSError: [Errno 98] Address already in use
-------------------- >> begin captured stdout << ---------------------
Initialisation of the Node on port: 8000 on node (ed8bab37d7f15e30da8720e96bcad500f2f8025ff06cbee97eb82bba8498baa444b3937d6bcae3945622421f38a27739cbfa91a3a6a22380caba42d7fff3b79d)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_node_connection (p2pnetwork.tests.test_node.TestNode)
Testing whether two Node instances are able to connect with each other.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/tests/test_node.py", line 21, in test_node_connection
    node2 = Node("localhost", 10002)
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/node.py", line 69, in __init__
    self.init_server()
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/node.py", line 92, in init_server
    self.sock.bind((self.host, self.port))
OSError: [Errno 98] Address already in use
-------------------- >> begin captured stdout << ---------------------
Initialisation of the Node on port: 10001 on node (da332f1ea0d8ef643c9fae02837ee76911d8d4bb0c9edc3ac12ffb7078c69a46cdc1cc61dc2b1f9aa4abe86720dd959a5af6ceb383812bef0c58281c62cc7c90)
Initialisation of the Node on port: 10002 on node (0c265613937e77eb17a9039f3d4c7ac9b11a6b591b5e12077241f197f92f56eabeb23c3ce0b0c925154a3085ef76c0528b8897ddfb115ce3915e574c12923db9)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_node_events (p2pnetwork.tests.test_node.TestNode)
Testing the events that are triggered by the Node.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/tests/test_node.py", line 207, in test_node_events
    node_0 = Node('127.0.0.1', 8000, node_callback)
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/node.py", line 69, in __init__
    self.init_server()
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/node.py", line 92, in init_server
    self.sock.bind((self.host, self.port))
OSError: [Errno 98] Address already in use
-------------------- >> begin captured stdout << ---------------------
Initialisation of the Node on port: 8000 on node (bcd13e8d7236a4bdb27a1edd3f0d188b1f0122f80891f6977922ff5ce51618e3a8a05f5b47a715b632e58a80185cff967e2a582f4ddef9dabf32ebfc6cefae36)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_node_send_data_dict (p2pnetwork.tests.test_nodeconnection.TestNode)
Testing whether NodeConnections handle sending dict well enough.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/tests/test_nodeconnection.py", line 100, in test_node_send_data_dict
    node2 = MyTestNode("127.0.0.1", 8002)
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/tests/test_nodeconnection.py", line 88, in __init__
    super(MyTestNode, self).__init__(host, port, None)
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/node.py", line 69, in __init__
    self.init_server()
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/node.py", line 92, in init_server
    self.sock.bind((self.host, self.port))
OSError: [Errno 98] Address already in use
-------------------- >> begin captured stdout << ---------------------
Initialisation of the Node on port: 8001 on node (2fe5423ad2fa43b023063a3a54f3fb6e28768c574ecf14de8621135195d87591327191cdf905d03a05e70c7021e9fe658ae30188cd08939dc201ee029239c6fd)
Initialisation of the Node on port: 8002 on node (aa90124307d53a8e8ed99bbd8d624c1adc5668a78830c58238480fac76762c2be37d55992637c04cd7ae5800f9235777f053a55d365d06a3e7feb2513b236b93)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_node_send_data_str (p2pnetwork.tests.test_nodeconnection.TestNode)
Testing whether NodeConnections handle sending str well enough.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/tests/test_nodeconnection.py", line 36, in test_node_send_data_str
    node2 = MyTestNode("127.0.0.1", 8002)
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/tests/test_nodeconnection.py", line 26, in __init__
    super(MyTestNode, self).__init__(host, port, None)
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/node.py", line 69, in __init__
    self.init_server()
  File "/home/maurice/projects/python-p2p-network/p2pnetwork/node.py", line 92, in init_server
    self.sock.bind((self.host, self.port))
OSError: [Errno 98] Address already in use
-------------------- >> begin captured stdout << ---------------------
Initialisation of the Node on port: 8001 on node (371fae118c3037f1a2f2cdec60f1f2729a0da7b2b6433f639f43dca722ecd5c928bdb4d1b5de66d5c82b8322d2fc6300caa974ded873e13a1eb7238e98d67ab7)
Initialisation of the Node on port: 8002 on node (752f24de3d4dfaf4ec126ada2f13c278c7992a383cd21b248600df0daac3ea07c7c685810166766ac4b6348b2946ae9d79d2a0dacd324cfdded5d806ca877008)

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 7 tests in 28.110s

FAILED (errors=5)
Test failed: <unittest.runner.TextTestResult run=7 errors=5 failures=0>
error: Test failed: <unittest.runner.TextTestResult run=7 errors=5 failures=0>

how

node_1.send_to_node(node_3,data="message: Hi there from 1!\n")

Info about node discovery

Hello,
please can you confirm that the library doesn't have any mechanism for nodes discovery?
If so, do you have any repo or best practice for storing a "global" database containing a list of available nodes?
Is there any other best practice for obtaining the same result (node discovery and connection)

Thank you.
Regards,
Marco

implementing compression to transfer large files over socket.

I am adding the functionality of compressing data to send and decompressing it on the other end to reduce bandwidth in the network.
help me to achieve this.
example code is here.

def dataencode(objectToEncode):
    data_out = jsonpickle.encode(objectToEncode, unpicklable=True)
    data_out = data_out.encode("utf-8")
    compressed_data = compress(data_out)
    return jsonpickle.encode(compressed_data, unpicklable=True)

OS Error when stopping nodes

Firstly, I want to thank everyone involved with this library for the wonderful work that has been done. I am an undergraduate grader for a networking course, and I had planned on writing a library similar to this one to create a peer2peer chat client, but once I had found this lib I decided to implement it instead because of its clear documentation and ease of use.

Secondly, I've noticed an error when I stop a node. My script will crash with an OS error. OS Error: [Errno 9]: Bad file descriptor . I believe this is due to the use of sock.settimeout(None) in the Node class's self.run() method, at line 308. I was curious what the thought process was behind using None, as opposed to an integer or float, for the timeout duration. Would changing the timeout duration to a float, like 0.01 for instance, have a negative impact on the rest of the class?

Possible vulnerability on the node unique ID

MD5 is used on the source code to create the node unique ID, however, md5 is a vulnerable algorithm (collision attacks). To improve the security of the protocol, I suggest to change it to a secure and efficient algorithm (Blake2, for example).

Non UTF-8 messages will throw exception

If a connected client sends non-UTF8 data, run() will throw an uncaught exception. I added a try/except to catch it, but maybe better options?

                    try:
                        connected_node_id   = connection.recv(4096).decode('utf-8')
                    except Exception as e:
                        print(e)
                        connection.close()
                        continue

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.