mcpyproject / mcpy Goto Github PK
View Code? Open in Web Editor NEWA open source Minecraft server written 100% in Python
License: GNU Affero General Public License v3.0
A open source Minecraft server written 100% in Python
License: GNU Affero General Public License v3.0
Bug Description
Chat is not being logged by logging import
To Reproduce
Expected behaviour
As in the code logging.info(message)
in Connection.py its expected to be logged as info
Host system:
Describe the bug
Encountering 2 exceptions when running startup.py
, EOFError and a TypeError
To Reproduce
Expected behavior
The server starts
Screenshots
INFO:root:Getting release list...
INFO:root:McPy found! Running it...
[2021-01-25 11:45:48,759 - INFO - MainThread] Trying to initialize the Blackfire probe
[2021-01-25 11:45:48,760 - INFO - MainThread] Blackfire not installed: passing
[2021-01-25 11:45:48,760 - INFO - MainThread] Starting queues...
[2021-01-25 11:45:48,765 - INFO - MainThread] Started queues!
[2021-01-25 11:45:48,765 - INFO - MainThread] Trying to find number of available cores
[2021-01-25 11:45:48,765 - WARNING - MainThread] Falling back to multiprocessing cpu_count to calc cores. Most likely getaffinity is not supported on your OS
[2021-01-25 11:45:48,765 - INFO - MainThread] Found 2 cores available!
[2021-01-25 11:45:48,766 - INFO - MainThread] Starting worker ID 15824
[2021-01-25 11:45:48,771 - INFO - MainThread] Started worker.
[2021-01-25 11:45:48,882 - INFO - MainThread] Starting networking worker
Traceback (most recent call last):
File "McPy/main.py", line 238, in <module>
main()
File "McPy/main.py", line 221, in main
networkingProcess.start()
File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\context.py", line 326, in _Popen
return Popen(process_obj)
File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
reduction.dump(process_obj, to_child)
File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_cffi_backend.FFI' object
[2021-01-25 11:45:49,267 - INFO - MainThread] Trying to initialize the Blackfire probe
[2021-01-25 11:45:49,268 - INFO - MainThread] Blackfire not installed: passing
[2021-01-25 11:45:49,268 - INFO - MainThread] Starting queues...
[2021-01-25 11:45:49,272 - INFO - MainThread] Started queues!
[2021-01-25 11:45:49,273 - INFO - MainThread] Worker ID 15824 has started up.
[2021-01-25 11:45:49,375 - INFO - MainThread] Trying to initialize the Blackfire probe
[2021-01-25 11:45:49,376 - INFO - MainThread] Blackfire not installed: passing
[2021-01-25 11:45:49,376 - INFO - MainThread] Starting queues...
[2021-01-25 11:45:49,380 - INFO - MainThread] Started queues!
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input
Host system:
Additional context
If it makes any difference, I installed Twisted
from https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted using Twisted‑20.3.0‑cp38‑cp38‑win32.whl
The current backend Quarry is good for quick and simple server implementations and handles nearly all of the networking for the server. However, I think we could benefit from developing our own networking backend, based off of asyncio.
AsyncIO in Python is significantly more performant in a single thread than a similarly written synchronous server implementation and can match the performance of a threaded server, all without the overhead of using separate threads (thread-safety, overhead, limited access to resources, etc). Other advantages of splitting from Quarry is that we are not bound to any of its constructs; for example its server classes, its API format, and its version support.
Per the starting discussion on Discord, here's my thoughts about how asyncio networking could be implemented:
The networking thread will consist of its own process (to avoid GIL bottleneck w/ main thread), and will run its own asyncio event loop to do networking. The networking thread will start the server, and create a networking queue for events. The networking queue will take in events that happen relating to players (eg. skeleton shot you, block was removed, etc). These will use internal event objects (probably dataclasses) and the networking thread will sort them out as needed and dispatch events to individual players' queues, at which point the event loop will pick up the events on each player and actually write the packet out to them and stuff.
Then, when a new player connects, the networking thread will manage the entire process of server list ping and login and such, until the player has mostly logged in. Then it will send an internal "player join" event back to the main server thread, and the server will give it the appropriate chunks to load.
Incoming data from the player relating to the main server thread (such as world movement or whatever) will be sent back to the main server with an event queue for the main server thread.
Things that don't involve the main server thread at all will never get passed to it; I can only think of this being chat and server commands, but perhaps PvP could be in this category as well.
That's my initial thought as to how async networking could be done. It passes a lot of the server functions into it but unless the server has 200 players on it I don't think it'd be too bad. Then again, we could also go with a simpler implementation where only incoming data events and outgoing data events are handled, and everything is done in the main server thread. This should also be something we consider in addition to my thoughts above- it would probably make the overall server implementation simpler and easier at the cost of more work in the main server thread.
Either way we take this, I am down to support the development and oversight of this project on a new branch of the repo (for example named asyncio-networking
), as I have almost two years of prior asyncio experience developing complex Discord bots.
I think it will be correct if we add a file in which it will be possible to change the properties of the server, for example, its port, the maximum number of players, etc.
On macOS platforms, the maximum allowable size of a multiprocessing.Queue
is 32767.
A check should probably be added for this in NetworkController (classes/network/Connection.py
) before the creation of OUT_QUEUE
.
https://discord.com/channels/729914898607243334/729914898607243337/787017743878193232
e: wait I think I goofed
Here is some things we should parse with the Parser:
Coming onto the project, documentation is currently lacking.
I was told an accessible resource was @Geolykt who at one point did develop for a time.
I'm working through understanding the existing code base and with WorldGenerator.py
.
The Pull requests and commit statements suggest that world generation has been executed before concurrently with development, but I can't find any example of a main.py
which executed any of the functions inside of WorldGenerator
@Geolykt , are you able to provide any information? I do know you are not active on this project anymore.
I wanted to change this message in main.py just as a test
[2021-01-19 21:35:11,815 - INFO - MainThread] Trying to initialize the Blackfire probe
so I edited this line:
logging.info("Trying to initialize the Blackfire probe")
and changed it to
logging.info("Trrying to initialize the Blackfire probe")
and it did nothing?! even check sure it was saved and opened it in nano instead of vscode and it was changed but it still does nothing
even commented it out then rebooted still nothing I can't change anything..? I want to try adding new features since the code base is small enough for me to understand
Describe the bug
When pinging the server with version 1.15.2 or 1.16.4, an error is displayed on the server
To Reproduce
Steps to reproduce the behavior:
Expected behavior
No error
Error
This error occures when pinging with 1.16.4 (Error from @Geolykt)
PlayerNetwork{127.0.0.1} | ERROR | No name known for packet: (754, 'status', 'upstream', 0)
Traceback (most recent call last):
File "/home/Geolykt/.local/lib/python3.9/site-packages/quarry/net/protocol.py", line 207, in get_packet_name
return packets.packet_names[key]
KeyError: (754, 'status', 'upstream', 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/Geolykt/.local/lib/python3.9/site-packages/quarry/net/protocol.py", line 243, in data_received
name = self.get_packet_name(buff.unpack_varint())
File "/home/Geolykt/.local/lib/python3.9/site-packages/quarry/net/protocol.py", line 209, in get_packet_name
raise ProtocolError("No name known for packet: %s" % (key,))
quarry.net.protocol.ProtocolError: No name known for packet: (754, 'status', 'upstream', 0)
Host system:
Additional context
Maybe a quarry issue ?
Describe the bug
the startup.py file uses a different logging format than the rest of the project
To Reproduce
Steps to reproduce the behavior:
python3 startup.py
Expected behavior
Uniform logging format
Host system:
Additional context
Current output is:
INFO:root:Making sure pip is installed...
Defaulting to user installation because normal site-packages is not writeable
Looking in links: /tmp/tmp4e2vgybv
Requirement already up-to-date: setuptools in /usr/lib/python3.9/site-packages (49.1.3)
Requirement already up-to-date: pip in /usr/lib/python3.9/site-packages (20.2.2)
INFO:root:Installed.
INFO:root:Getting release list...
INFO:root:McPy found! Running it...
root | INFO | Trying to initialize the Blackfire probe
root | INFO | Blackfire not installed: passing
root | INFO | Starting queues...
root | INFO | Started queues!
root | INFO | Trying to find number of available cores
root | INFO | Found 2 cores available!
root | INFO | Starting worker ID 84619
root | INFO | Started worker.
root | INFO | Worker ID 84619 has started up.
root | INFO | Starting networking worker
root | INFO | Started worker.
root | INFO | Startup done! Listening on 0.0.0.0:25565
^Croot | INFO | Shutting server down!
root | INFO | Worker ID 84619 has completed all tasks.
Traceback (most recent call last):
File "/home/Geolykt/git/McPy/startup.py", line 174, in <module>
mcpyProcess = subprocess.check_call(data)
File "/usr/lib64/python3.9/subprocess.py", line 368, in check_call
retcode = call(*popenargs, **kwargs)
File "/usr/lib64/python3.9/subprocess.py", line 351, in call
return p.wait(timeout=timeout)
File "/usr/lib64/python3.9/subprocess.py", line 1185, in wait
return self._wait(timeout=timeout)
File "/usr/lib64/python3.9/subprocess.py", line 1915, in _wait
(pid, sts) = self._try_wait(0)
File "/usr/lib64/python3.9/subprocess.py", line 1873, in _try_wait
(pid, sts) = os.waitpid(self.pid, wait_flags)
KeyboardInterrupt
If you run main.py
without dependencies installed, a bunch of import errors are raised, and this could be difficult for inexperienced end-users to understand with all of the paths.
Adding a check in main.py
, or even classes/mcPy/McPy.py
would assist the end user in understanding that they need to run dependencies.py
.
Something similar to https://stackoverflow.com/questions/16294819/check-if-my-python-has-all-required-packages/45474387#45474387 could be useful
Adding a plugin system is integral to this project, and we will need to develop it soon, maybe even before (or during) the world generation. Some things we will need:
This might go hand-in-hand with a command implementation (outlined in #23), so this may need to be pushed back until then.
Describe the bug
System not working
To Reproduce
Steps to reproduce the behavior:
root | INFO | Trying to initialize the Blackfire probe
root | INFO | Blackfire not installed: passing
root | INFO | Starting queues...
root | INFO | Started queues!
root | INFO | Trying to find number of available cores
root | WARNING | Falling back to multiprocessing cpu_count to calc cores. Most likely getaffinity is not supported on your OS
root | INFO | Found 2 cores available!
root | INFO | Starting worker ID 24407
root | INFO | Started worker.
root | INFO | Starting networking worker
Traceback (most recent call last):
File "/Users/Grant/McPy/main.py", line 238, in
main()
File "/Users/Grant/McPy/main.py", line 221, in main
networkingProcess.start()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in init
super().init(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_cffi_backend.FFI' object
root | INFO | Trying to initialize the Blackfire probe
root | INFO | Blackfire not installed: passing
root | INFO | Starting queues...
root | INFO | Started queues!
Traceback (most recent call last):
File "", line 1, in
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/synchronize.py", line 110, in setstate
self._semlock = _multiprocessing.SemLock._rebuild(*state)
FileNotFoundError: [Errno 2] No such file or directory
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Host system:
Additional context
Add any other context about the problem here.
Information
This issue is not an issue, it's more about informations / questions for the Command Implementation in McPy
Before Implementing Commands
I think we should handle commands when we have the general architecture of McPy.
By general architecture, I want to say having network in his correct package, worlds in another package, some Unit Tests etc.
How it could be implemented
Command
This is not an official implementation, just an example of how it could be implemented
To play Minecraft you need to be able to move.
At the moment the server just ignores incoming movement packets, and the player is stuck at a preset location. I aim to change this. This should work basically like vanilla.
Github actions are really useful. It's used in a lot of projects and it could be cool to add it here, in McPy
Some examples could be:
See above, it was frequently requested, so it only makes sense to add it (and helps me to debug things when I have no internet (which is almost always))
Setuptools is a library that allow us (developers) to build, package and distribute Python packages.
I think we should use setuptools to build and distribute McPy to pypi.org (pip)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.