mcpyproject / mcpy Goto Github PK

Bug Description
Chat is not being logged by logging import

To Reproduce

1. Send Text in Minecraft
2. No logging for message

Expected behaviour
As in the code logging.info(message) in Connection.py its expected to be logged as info

Host system:

• MacOS
• Big Sur (X86)
• Python 3.8.5 64-bit

Describe the bug
Encountering 2 exceptions when running startup.py, EOFError and a TypeError

To Reproduce

1. Run startup.py

Expected behavior
The server starts

Screenshots

INFO:root:Getting release list...
INFO:root:McPy found! Running it...
[2021-01-25 11:45:48,759 - INFO - MainThread] Trying to initialize the Blackfire probe
[2021-01-25 11:45:48,760 - INFO - MainThread] Blackfire not installed: passing
[2021-01-25 11:45:48,760 - INFO - MainThread] Starting queues...
[2021-01-25 11:45:48,765 - INFO - MainThread] Started queues!
[2021-01-25 11:45:48,765 - INFO - MainThread] Trying to find number of available cores
[2021-01-25 11:45:48,765 - WARNING - MainThread] Falling back to multiprocessing cpu_count to calc cores. Most likely getaffinity is not supported on your OS
[2021-01-25 11:45:48,765 - INFO - MainThread] Found 2 cores available!
[2021-01-25 11:45:48,766 - INFO - MainThread] Starting worker ID 15824
[2021-01-25 11:45:48,771 - INFO - MainThread] Started worker.
[2021-01-25 11:45:48,882 - INFO - MainThread] Starting networking worker
Traceback (most recent call last):
  File "McPy/main.py", line 238, in <module>
    main()
  File "McPy/main.py", line 221, in main
    networkingProcess.start()
  File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\context.py", line 326, in _Popen
    return Popen(process_obj)
  File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_cffi_backend.FFI' object
[2021-01-25 11:45:49,267 - INFO - MainThread] Trying to initialize the Blackfire probe
[2021-01-25 11:45:49,268 - INFO - MainThread] Blackfire not installed: passing
[2021-01-25 11:45:49,268 - INFO - MainThread] Starting queues...
[2021-01-25 11:45:49,272 - INFO - MainThread] Started queues!
[2021-01-25 11:45:49,273 - INFO - MainThread] Worker ID 15824 has started up.
[2021-01-25 11:45:49,375 - INFO - MainThread] Trying to initialize the Blackfire probe
[2021-01-25 11:45:49,376 - INFO - MainThread] Blackfire not installed: passing
[2021-01-25 11:45:49,376 - INFO - MainThread] Starting queues...
[2021-01-25 11:45:49,380 - INFO - MainThread] Started queues!
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

Host system:

• OS: Windows
• OS version: Version 2004, OS Build 19041.746, 64 Bit
• Python version 3.8.3, 32 Bit

Additional context
If it makes any difference, I installed Twisted from https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted using Twisted‑20.3.0‑cp38‑cp38‑win32.whl

The current backend Quarry is good for quick and simple server implementations and handles nearly all of the networking for the server. However, I think we could benefit from developing our own networking backend, based off of asyncio.

AsyncIO in Python is significantly more performant in a single thread than a similarly written synchronous server implementation and can match the performance of a threaded server, all without the overhead of using separate threads (thread-safety, overhead, limited access to resources, etc). Other advantages of splitting from Quarry is that we are not bound to any of its constructs; for example its server classes, its API format, and its version support.

Per the starting discussion on Discord, here's my thoughts about how asyncio networking could be implemented:

The networking thread will consist of its own process (to avoid GIL bottleneck w/ main thread), and will run its own asyncio event loop to do networking. The networking thread will start the server, and create a networking queue for events. The networking queue will take in events that happen relating to players (eg. skeleton shot you, block was removed, etc). These will use internal event objects (probably dataclasses) and the networking thread will sort them out as needed and dispatch events to individual players' queues, at which point the event loop will pick up the events on each player and actually write the packet out to them and stuff.

Then, when a new player connects, the networking thread will manage the entire process of server list ping and login and such, until the player has mostly logged in. Then it will send an internal "player join" event back to the main server thread, and the server will give it the appropriate chunks to load.

Incoming data from the player relating to the main server thread (such as world movement or whatever) will be sent back to the main server with an event queue for the main server thread.

Things that don't involve the main server thread at all will never get passed to it; I can only think of this being chat and server commands, but perhaps PvP could be in this category as well.

That's my initial thought as to how async networking could be done. It passes a lot of the server functions into it but unless the server has 200 players on it I don't think it'd be too bad. Then again, we could also go with a simpler implementation where only incoming data events and outgoing data events are handled, and everything is done in the main server thread. This should also be something we consider in addition to my thoughts above- it would probably make the overall server implementation simpler and easier at the cost of more work in the main server thread.

Either way we take this, I am down to support the development and oversight of this project on a new branch of the repo (for example named asyncio-networking), as I have almost two years of prior asyncio experience developing complex Discord bots.

I think it will be correct if we add a file in which it will be possible to change the properties of the server, for example, its port, the maximum number of players, etc.

On macOS platforms, the maximum allowable size of a multiprocessing.Queue is 32767.

A check should probably be added for this in NetworkController (classes/network/Connection.py) before the creation of OUT_QUEUE.

https://discord.com/channels/729914898607243334/729914898607243337/787017743878193232

e: wait I think I goofed

Here is some things we should parse with the Parser:

• Block (Important point !!!!)
• Effect
• Enchantment
• Entity
• Particle
• Biome
• Sound effect
• Item
• Menu
• Recipe type
• Villager type
• Villager Profession

Coming onto the project, documentation is currently lacking.

I was told an accessible resource was @Geolykt who at one point did develop for a time.

I'm working through understanding the existing code base and with WorldGenerator.py.

The Pull requests and commit statements suggest that world generation has been executed before concurrently with development, but I can't find any example of a main.py which executed any of the functions inside of WorldGenerator

@Geolykt , are you able to provide any information? I do know you are not active on this project anymore.

https://cdn.discordapp.com/attachments/729914898607243337/741775653921357884/unknown.png

I wanted to change this message in main.py just as a test

[2021-01-19 21:35:11,815 - INFO - MainThread] Trying to initialize the Blackfire probe

so I edited this line:
logging.info("Trying to initialize the Blackfire probe")

and changed it to
logging.info("Trrying to initialize the Blackfire probe")

and it did nothing?! even check sure it was saved and opened it in nano instead of vscode and it was changed but it still does nothing

even commented it out then rebooted still nothing I can't change anything..? I want to try adding new features since the code base is small enough for me to understand

Describe the bug
When pinging the server with version 1.15.2 or 1.16.4, an error is displayed on the server

To Reproduce
Steps to reproduce the behavior:

1. Launch Minecraft 1.15.2
2. Go to Multiplayer
3. An error is displayed on the server

Expected behavior
No error

Error
This error occures when pinging with 1.16.4 (Error from @Geolykt)

PlayerNetwork{127.0.0.1} | ERROR | No name known for packet: (754, 'status', 'upstream', 0)
Traceback (most recent call last):
  File "/home/Geolykt/.local/lib/python3.9/site-packages/quarry/net/protocol.py", line 207, in get_packet_name
    return packets.packet_names[key]
KeyError: (754, 'status', 'upstream', 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/Geolykt/.local/lib/python3.9/site-packages/quarry/net/protocol.py", line 243, in data_received
    name = self.get_packet_name(buff.unpack_varint())
  File "/home/Geolykt/.local/lib/python3.9/site-packages/quarry/net/protocol.py", line 209, in get_packet_name
    raise ProtocolError("No name known for packet: %s" % (key,))
quarry.net.protocol.ProtocolError: No name known for packet: (754, 'status', 'upstream', 0)

Host system:

• OS: Windows
• OS version: 10

Additional context
Maybe a quarry issue ?

Describe the bug
the startup.py file uses a different logging format than the rest of the project

To Reproduce
Steps to reproduce the behavior:

1. Run python3 startup.py

Expected behavior
Uniform logging format

Host system:

• OS: Fedora
• OS version: 33 (Kernel version 5.10.9-201.fc33.x86_64)
• Python version: 3.9.1

Additional context
Current output is:

INFO:root:Making sure pip is installed...
Defaulting to user installation because normal site-packages is not writeable
Looking in links: /tmp/tmp4e2vgybv
Requirement already up-to-date: setuptools in /usr/lib/python3.9/site-packages (49.1.3)
Requirement already up-to-date: pip in /usr/lib/python3.9/site-packages (20.2.2)
INFO:root:Installed.
INFO:root:Getting release list...
INFO:root:McPy found! Running it...
root | INFO | Trying to initialize the Blackfire probe
root | INFO | Blackfire not installed: passing
root | INFO | Starting queues...
root | INFO | Started queues!
root | INFO | Trying to find number of available cores
root | INFO | Found 2 cores available!
root | INFO | Starting worker ID 84619
root | INFO | Started worker.
root | INFO | Worker ID 84619 has started up.
root | INFO | Starting networking worker
root | INFO | Started worker.
root | INFO | Startup done! Listening on 0.0.0.0:25565
^Croot | INFO | Shutting server down!
root | INFO | Worker ID 84619 has completed all tasks.
Traceback (most recent call last):
  File "/home/Geolykt/git/McPy/startup.py", line 174, in <module>
    mcpyProcess = subprocess.check_call(data)
  File "/usr/lib64/python3.9/subprocess.py", line 368, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/usr/lib64/python3.9/subprocess.py", line 351, in call
    return p.wait(timeout=timeout)
  File "/usr/lib64/python3.9/subprocess.py", line 1185, in wait
    return self._wait(timeout=timeout)
  File "/usr/lib64/python3.9/subprocess.py", line 1915, in _wait
    (pid, sts) = self._try_wait(0)
  File "/usr/lib64/python3.9/subprocess.py", line 1873, in _try_wait
    (pid, sts) = os.waitpid(self.pid, wait_flags)
KeyboardInterrupt

If you run main.py without dependencies installed, a bunch of import errors are raised, and this could be difficult for inexperienced end-users to understand with all of the paths.

Adding a check in main.py, or even classes/mcPy/McPy.py would assist the end user in understanding that they need to run dependencies.py.

Something similar to https://stackoverflow.com/questions/16294819/check-if-my-python-has-all-required-packages/45474387#45474387 could be useful

Adding a plugin system is integral to this project, and we will need to develop it soon, maybe even before (or during) the world generation. Some things we will need:

• Event Handler
• Plugin Loader
• API for Server Functions

This might go hand-in-hand with a command implementation (outlined in #23), so this may need to be pushed back until then.

Describe the bug
System not working

To Reproduce
Steps to reproduce the behavior:
root | INFO | Trying to initialize the Blackfire probe
root | INFO | Blackfire not installed: passing
root | INFO | Starting queues...
root | INFO | Started queues!
root | INFO | Trying to find number of available cores
root | WARNING | Falling back to multiprocessing cpu_count to calc cores. Most likely getaffinity is not supported on your OS
root | INFO | Found 2 cores available!
root | INFO | Starting worker ID 24407
root | INFO | Started worker.
root | INFO | Starting networking worker
Traceback (most recent call last):
File "/Users/Grant/McPy/main.py", line 238, in
main()
File "/Users/Grant/McPy/main.py", line 221, in main
networkingProcess.start()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in init
super().init(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_cffi_backend.FFI' object
root | INFO | Trying to initialize the Blackfire probe
root | INFO | Blackfire not installed: passing
root | INFO | Starting queues...
root | INFO | Started queues!
Traceback (most recent call last):
File "", line 1, in
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/synchronize.py", line 110, in setstate
self._semlock = _multiprocessing.SemLock._rebuild(*state)
FileNotFoundError: [Errno 2] No such file or directory

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Host system:

• OS: [Macos Catalina Latest]
• OS version: [10.15.6]
• Python version [3.8.5/Pycharm 2020.2]

Additional context
Add any other context about the problem here.

Information
This issue is not an issue, it's more about informations / questions for the Command Implementation in McPy

Before Implementing Commands
I think we should handle commands when we have the general architecture of McPy.
By general architecture, I want to say having network in his correct package, worlds in another package, some Unit Tests etc.

How it could be implemented

• In his own package
• Having a file per command
• Commands should extends global class Command
  • Functions should have @Command.execute annotation. McPy will automagically check which function will be called when a player execute a command
  • here is an example of teleport command

This is not an official implementation, just an example of how it could be implemented

To play Minecraft you need to be able to move.
At the moment the server just ignores incoming movement packets, and the player is stuck at a preset location. I aim to change this. This should work basically like vanilla.

Github actions are really useful. It's used in a lot of projects and it could be cool to add it here, in McPy

Some examples could be:

• An action that test the whole project in differents versions of python
• An action that test the coverage of the project
• An action that check if the user that make a PR has signed our User Agrements for this project
• Etc

See above, it was frequently requested, so it only makes sense to add it (and helps me to debug things when I have no internet (which is almost always))

Setuptools is a library that allow us (developers) to build, package and distribute Python packages.

I think we should use setuptools to build and distribute McPy to pypi.org (pip)