wilfred-dev / wilfred Goto Github PK

A lot of the code is currently not consistent. Before #34 is implemented, I think a lot of the methods (i.e. servers.py, server_config.py, config_parser.py and images.py) should be rewritten to be more consistent and that the different method functions should be split into smaller modules so that the application as a whole can be imported as an API (and separate the CLI from the Wilfred API).

With this implemented, a more general and predictable extension system could be implemented. I also propose the CLI should be completely separated from the core API itself.

Is your feature request related to a problem? Please describe.
The current way of managing data is not really sustainable for long-term development.

Describe the solution you'd like
We need to have a local database that supports models and migrations. This could be done with sqlalchemy and alembic (or similar).

Describe alternatives you've considered
Also yoyo-migrations but alembic seems to have better integration with sqlalchemy.

Is your feature request related to a problem? Please describe.
Wilfred only allows servers to be created and run on the same host as the interface itself is installed on.

Describe the solution you'd like
Using a custom Docker host, Wilfred should be able to create and manage containers that are running on a separate host. The problem is that Wilfred needs to have some sort of access to the actual filesystem outside of the containers as well. Perhaps this could be done using just pure SSH?

Describe alternatives you've considered
This could also be accomplished by using a daemon-and-master configuration. I do not think that would be a good idea though, as it's too far away from the main purpose of Wilfred (a simple CLI for managing game servers).

Additional context
No.

Final adjustments to the Wilfred API. Currently, the API and the CLI are very much intertwined and they should be separated more clearly. Everything should go through the API. As it is now, database actions are required to be performed manually and are not performed via the API.

Environment

• Installation type: all
• Python version: 3.6 (tested with 3.7 also)
• Wilfred version: v0.7.0

Steps to Reproduce

Create a new server while wilfred top is running or run wilfred servers in another terminal during the installation.

Expected Behavior

The status should be installing.

Observed Behavior

The status is stopped, even though wilfred top can report both CPU usage and RAM usage.

Environment

• Installation type: both
• Python version: all
• Wilfred version: master

Steps to Reproduce

Try to access SERVER_MEMORY and SERVER_PORT from the installation script.

Expected Behavior

The variables to be set accordingly.

Observed Behavior

The installation script does not currently recognize these variables, only the startup command.

Is your feature request related to a problem? Please describe.
YAML and JSON configs are not editable by Wilfred.

Describe the solution you'd like
Ability to edit them using wilfred config <name>.

Describe alternatives you've considered
None.

Additional context
Related to #27.

Environment

• Installation type: both
• Python version: all supported
• Wilfred version: master

Steps to Reproduce

Leave the stop_command empty in the image configuration. Run wilfred stop <server>.

Expected Behavior

Recognize that the image parameter stop_command is empty and either exit the application or kill the container.

Observed Behavior

Running wilfred stop <server> just waits for the container to die infinitely.

Is your feature request related to a problem? Please describe.
If you want to specify a custom startup command, it's not possible currently. You'd have to create a duplicate of the image with a modified command.

Describe the solution you'd like
Ability to set a custom startup command on a per server basis.

Environment

• Installation type: snap and pip
• Python version: 3.7.3
• Wilfred version: v3.0.0

Steps to Reproduce

1. Create a server that does not require any environment variables (such as minecraft-bungeecord).
2. Try to delete it using wilfred delete <name>.

Expected Behavior

Server to be deleted without exceptions.

Observed Behavior

Raises sqlalchemy exception, it tries to delete environment variables which do not exist. Wilfred does not handle this.

See traceback below.

Are you sure you want to do this? All data will be permanently deleted. [y/N]: y
Traceback (most recent call last):
  File "/home/vilhelm/.local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1999, in delete
    state = attributes.instance_state(instance)
AttributeError: 'NoneType' object has no attribute '_sa_instance_state'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/wilfred", line 10, in <module>
    sys.exit(main())
  File "/home/vilhelm/.local/lib/python3.7/site-packages/wilfred/wilfred.py", line 88, in main
    cli()
  File "/home/vilhelm/.local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/vilhelm/.local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/vilhelm/.local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/vilhelm/.local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/vilhelm/.local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/vilhelm/.local/lib/python3.7/site-packages/wilfred/wilfred.py", line 409, in delete
    servers.remove(server)
  File "/home/vilhelm/.local/lib/python3.7/site-packages/wilfred/servers.py", line 116, in remove
    session.query(EnvironmentVariable).filter_by(server_id=server.id).first()
  File "/home/vilhelm/.local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2001, in delete
    raise exc.UnmappedInstanceError(instance)
sqlalchemy.orm.exc.UnmappedInstanceError: Class 'builtins.NoneType' is not mapped

Is your feature request related to a problem? Please describe.
Sometimes you want to forcefully perform some actions that right now require a confirmation prompt.

Describe the solution you'd like
A -f, --force flag would be useful for the wilfred kill and wilfred delete commands if you just want to perform the action without confirmation.

Describe alternatives you've considered
None.

Additional context
None.

Is your feature request related to a problem? Please describe.
Images can quickly become outdated. Wilfred does not currently auto-refresh images.

Describe the solution you'd like
Auto-refresh on upgrade and like every week or something.

Describe alternatives you've considered
Integrate the image fetching upon server restart and/or server creation.

Additional context
N/A

Is your feature request related to a problem? Please describe.
Sometimes you just want to send a single command to a server. Or you want a backup script to send a notification to the players on the server.

Describe the solution you'd like
Ability to send a single command to the STDIN of a server, preferably with script support (i.e. no prompts, use piping instead).

Is your feature request related to a problem? Please describe.

Sometimes you just want to gracefully stop and then start a server, which is something that requires two commands at the moment.

Describe the solution you'd like

It would be nice if a "restart" command could do both, in one command.

Is your feature request related to a problem? Please describe.
As of right now, setting up a bungeecord server is a lengthy and complicated process.

Describe the solution you'd like
I would like to be able to set up multiple paper servers and bungeecord all without touching any config files.

Describe alternatives you've considered
None.

Additional context
None.

Environment

• Installation type: only applies to pip, snap installations show up correctly
• Python version: 3.7.3 (I suspect this is for all supported versions)
• Wilfred version: v0.4.0

Steps to Reproduce

Run wilfred --version on pip installation (using v0.4.0).

Expected Behavior

Show version, commit hash and date accordingly.

Observed Behavior

This is the output:

✨ wilfred version v0.4.0/stable (commit develop) built YYYY-MM-DD

When Travis builds the package for Pip releasing, these values should be updated but has clearly not been.

Is your feature request related to a problem? Please describe.
The implementation from #19 lacks any migrations of any kind, even if it's a lot better than it was before. In order to ensure the project can envolve, the database needs to be more flexible.

Describe the solution you'd like
Database migrations. I've tried to implement alembic but I'll have to look more into how to use the API (alembic is mainly a CLI).

Describe alternatives you've considered
yoyo-migrations, but it doesn't seem to support SQLAlchemy.

Additional context
This was meant to be included in #16 and consequentially #19, but I decided to put the whole database migration thing on hold for a bit.

Is your feature request related to a problem? Please describe.
When using the edge channel on the snap package, it can be hard to track which version you are on.

Describe the solution you'd like
Display the commit hash upon running wilfred --version.

Describe alternatives you've considered
None.

Additional context
Should be possible to do within Travis when building for pip and within the snapcraft.yml when building for the snap package.

Is your feature request related to a problem? Please describe.
Right now, each server has a unique ID. The server root directory is named after this ID, e.g. g43nrf8q. It's hard to navigate and find this directory.

Describe the solution you'd like
A shorthand CLI command to directly cd into the server root directory. Perhaps wilfred cd <name> or wilfred files <name> or something like that.

Describe alternatives you've considered
Not using ID's and using the name of the server as the directory name. The downside is that exceptions can occur if a new server is created with the same name as a previously deleted server and any remaining files are still left in a directory with that name.

Additional context
This was mentioned by @alvarlagerlof a while back.

Edit environment variables for already existing servers.

Only variables that are not marked as install_only.

Waiting for the classic confinement to be approved.

See the forum thread.

Wilfred needs a logo.

Is your feature request related to a problem? Please describe.
If something goes wrong when Wilfred tries to create a new server (e.g. create the Docker container) and something goes wrong, Wilfred will exit with a very nondescript exception message since v0.6.0. This makes it hard to debug what caused the issue.

Describe the solution you'd like
By default, Wilfred should be a little bit more specific when it comes to printing what went wrong. Then I'd also like to have a verbose argument (-v) and Wilfred can just do the normal Python traceback.

Describe alternatives you've considered
None.

Additional context
This issue was introduced in v0.6.0. Prior versions handle exceptions differently.

Is your feature request related to a problem? Please describe.

Wilfred does not currently support Windows.

Describe the solution you'd like

Using "Docker Desktop", there is actually a way to run Docker on Windows. If possible, Windows should also be supported by Wilfred.

Describe alternatives you've considered

Ignore Windows 😄

Environment

• Installation type: all
• Python version: 3.6.12 (should be all)
• Wilfred version: master and v0.7.1 tested

Steps to Reproduce

1. Install Wilfred on a computer that does not have Docker installed.
2. Try to run wilfred

Expected Behavior

A short message stating that Docker is missing/DockerException exception message.

Observed Behavior

A long and unnecessary traceback.

Is your feature request related to a problem? Please describe.
I'd like to be able to have different namespaces so that I can have different groups of servers

Describe the solution you'd like
I'd be nice if they are stored in different folders or be prefixed in the folder names. For the cli, you could have a command to switch current namespace and something like -n or --namespace to run a specific command in another namespace.

Describe alternatives you've considered
None

Additional context
None

Is your feature request related to a problem? Please describe.

Travis is not configured to build and publish a wheel package when there is really no reason not to.

Describe the solution you'd like

Either add this to the setup.cfg.

[bdist_wheel]
universal = 1

Or update the python command Travis executes when building.

python setup.py sdist bdist_wheel

Is your feature request related to a problem? Please describe.
Some things like my Overviewer map is thightly coupled to my server, yet does not fit as a general feature in wilfred.

Describe the solution you'd like
I'd like to be able to develop python extensions that use some kind of API that Wilfred exposes. The API could for example expose:

• list of servers
• server state
• server start/stop
• server path
• sever console (for announcing rendering)

With only these ones, you'd be able to make a plugin for an Overviewer map.

Describe alternatives you've considered
Integrating support for Overviewer directly in Wilfred.

Additional context
None.

Environment

• Installation type: all
• Python version: all
• Wilfred version: master

Steps to Reproduce

Change image API version and try to perform image refresh.

Expected Behavior

Skip image parsing and refresh images before parsing.

Observed Behavior

Wilfred tries to parse the images before refreshing resulting in the application exiting before the refresh process initiates.

Is your feature request related to a problem? Please describe.
Backing up servers as of now requires custom scripts for smooth and efficient operation. Even then, the result may be unstable and error-prone.

Describe the solution you'd like
I'd like a setting that turns on backups for a server, with options for different intervals and automatic shut-down and restart of the servers during the process. It would be nice if only the diff is stored, as servers have a tendency to grow to 10s of gigabytes of data.

Describe alternatives you've considered
None.

Additional context
None.

Is your feature request related to a problem? Please describe.
Hard to navigate to a server when you don't know the server's ID.

Describe the solution you'd like
Rename the folder names to {name}-{id} which avoids the "unique duplicate" problem while still being quite memorable (when using cd you can just tab to autocomplete).

Describe alternatives you've considered
Something like wilfred cd but that turned out to be rather difficult.

Additional context
Closes #43

The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.

Is your feature request related to a problem? Please describe.
When I am working on a new image in another branch, I’d like to fetch images from another branch than master. Or if I am working in a fork, I’d like to be able to fetch the fork instead of the official one.

Describe the solution you'd like
I want wilfred images --refresh to accept branch and/or URL as alternative image source.

Describe alternatives you've considered
None.

Additional context
I found the need for this when working on wilfred-dev/images#2

Is your feature request related to a problem? Please describe.
It's tedious to write backup scripts, task schedulers etc.

Describe the solution you'd like
I would like Wilfred to be able to schedule tasks, either for one-time use or for recurring periodic use (e.g. cronjob) and have a set of built-in tasks such as managed backups. Perhaps this should be implemented as an extension (see #34) but I believe that this could be implemented directly to Wilfred for now and then separated once the extension system is implemented.

Describe alternatives you've considered
None.

Additional context
This feature would make it possible to schedule backups and fix #24

Environment

• Installation type: all
• Python version: all
• Wilfred version: v0.4.1

Steps to Reproduce

1. Create a new Minecraft Paper/Vanilla server.
2. Start it.

Expected Behavior

I expect it to replace the server-port setting as that is linked to the environment variable named SERVER_PORT.

Observed Behavior

Wilfred raises an exception and exits without starting the server.

💥 Error This setting is linked to an environment variable and is therefore not editable directly

Write unit tests to avoid issues.

Is your feature request related to a problem? Please describe.
If you decide to use a custom startup command, there is no way of resetting to the default startup command. Instead, you have to look the default command for that specific image and then manually set it.

Describe the solution you'd like
Some kind of way to reset back to the default startup command easily.

Describe alternatives you've considered
None.

Additional context
None.

Environment

• Installation type: all
• Python version: 3.6.9
• Wilfred version: master

Steps to Reproduce

Set a long custom startup command. Run wilfred servers.

Expected Behavior

List of servers, formatted in table.

Observed Behavior

The long command breaks the layout of the table.

Environment

• Installation type: brew (should apply to all)
• Python version: 3.7
• Wilfred version: v0.7.0

Steps to Reproduce

1. Run wilfred top in one terminal.
2. Create a new Minecraft: Bedrock Edition server in another terminal (wilfred create --console).
3. Observe the wilfred top terminal when the installation finishes and the server starts.

Expected Behavior

When the installation is finished, the status should just change.

Observed Behavior

ID        Name    Image UID            Port    RAM  Status    Custom startup    CPU    RAM usage
d06wlnug  test    minecraft-bedrock   25565   1024  stopped   None              99%    1759 MB
Traceback (most recent call last):
  File "/home/linuxbrew/.linuxbrew/bin/wilfred", line 33, in <module>
    sys.exit(load_entry_point('wilfred==0.7.0', 'console_scripts', 'wilfred')())
  File "/home/linuxbrew/.linuxbrew/Cellar/wilfred/0.7.0/libexec/lib/python3.7/site-packages/wilfred/wilfred.py", line 149, in main
    cli()
  File "/home/linuxbrew/.linuxbrew/Cellar/wilfred/0.7.0/libexec/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/linuxbrew/.linuxbrew/Cellar/wilfred/0.7.0/libexec/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/linuxbrew/.linuxbrew/Cellar/wilfred/0.7.0/libexec/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/linuxbrew/.linuxbrew/Cellar/wilfred/0.7.0/libexec/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/linuxbrew/.linuxbrew/Cellar/wilfred/0.7.0/libexec/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/linuxbrew/.linuxbrew/Cellar/wilfred/0.7.0/libexec/lib/python3.7/site-packages/wilfred/wilfred.py", line 713, in top
    data = servers.pretty_data(cpu_load=True, memory_usage=True)
  File "/home/linuxbrew/.linuxbrew/Cellar/wilfred/0.7.0/libexec/lib/python3.7/site-packages/wilfred/api/servers.py", line 129, in pretty_data
    cpu_count = len(d["cpu_stats"]["cpu_usage"]["percpu_usage"])
KeyError: 'percpu_usage'

Environment

• Installation type: snap & pip
• Python version: 3.6.9 (affects all supported versions)
• Wilfred version: since v0.5.0

Steps to Reproduce

Create a server with wilfred create --detach and attach to console wilfred console <name> when the server is still installing.

Expected Behavior

The log to be streamed with input disabled.

Observed Behavior

The following exception is raised.

Viewing server console of test (id yyl8lsr5) - input disabled, installing
Traceback (most recent call last):
  File "run.py", line 16, in <module>
    main()
  File "/home/vilhelm/src/github/wilfred-dev/wilfred/wilfred/wilfred.py", line 107, in main
    cli()
  File "/home/vilhelm/.local/share/virtualenvs/wilfred-SNdVLYox/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/vilhelm/.local/share/virtualenvs/wilfred-SNdVLYox/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/vilhelm/.local/share/virtualenvs/wilfred-SNdVLYox/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/vilhelm/.local/share/virtualenvs/wilfred-SNdVLYox/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/vilhelm/.local/share/virtualenvs/wilfred-SNdVLYox/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/vilhelm/src/github/wilfred-dev/wilfred/wilfred/wilfred.py", line 501, in server_console
    server, disable_user_input=True if server.status == "installing" else False
  File "/home/vilhelm/src/github/wilfred-dev/wilfred/wilfred/servers.py", line 216, in console
    if not _thread._running:
UnboundLocalError: local variable '_thread' referenced before assignment

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like
I'd like to have a command that is able to change as much of server config files as possible, so that editing individual files is not necessary. This way, changing config for multiple servers is fast and easy.

Describe alternatives you've considered
None.

Additional context
None.

When changing the port of a server, configuration files are not updated.

Environment

• Installation type: pip (both affected)
• Python version: 3.7.3 (all should be affected)
• Wilfred version: v0.5.1

Steps to Reproduce

Create a server that is certain to immediately crash after server start. Start it with wilfred start <name> --console.

Expected Behavior

A "server not running" error to be printed.

Observed Behavior

A long docker.errors.NotFound exception traceback is printed. This is because there is a slight delay between the statement that checks if the server is running and the statement that retrieves the container logs. Perhaps a try-except statement can be added to the container log retrieval as well.

Environment

• Installation type: both
• Python version: 3.6.9
• Wilfred version: master

Steps to Reproduce

Try to edit a config setting that is linked to an environment variable.

Expected Behavior

Should not be editable, should not be exposed in the same way as the other settings.

Observed Behavior

Editable but instantly updated when the server is restarted.

(suggested by @alvarlagerlof)

Is your feature request related to a problem? Please describe.
I have a bunch of plugins that uses their own config files. Updating multiple files is tedious.

Describe the solution you'd like
I'd be nice to be able to include a relative path to a file in the config command.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Environment

• Installation type: snap/pip
• Python version: 3.6.9 (should affect all)
• Wilfred version: v0.5.1 (should affect all releases with wilfred top)

Steps to Reproduce

Set a long startup command. Run wilfred top. Resize your terminal width so that Wilfred should start truncating the long startup command. Also, try running wilfred servers and the same problem will occur.

Expected Behavior

The length of the startup command should not be so long that the server line continues on the next line.

Observed Behavior

The truncation is not aggressive enough, a line break is inserted which makes the whole formatting of wilfred top look off if you have several servers running.

Update: since v0.7.1 this seems to not truncate at all.

Is your feature request related to a problem? Please describe.
If creating a server that takes a long time to install, waiting for installation can be tedious.

Describe the solution you'd like
Add an option, like --detach that allows the installation to occur in the background. Would require the interface to handle the installation state better.

The user should also be able to just CTRL+C if the installation takes too long, without the installation container dying.

Is your feature request related to a problem? Please describe.
When running multiple servers, getting an overview of chat cpu load, memory usage, chat activity and player count is not possible without custom code.

Describe the solution you'd like
A command that fills the screen and updates in real-time, similar to something like top.

Describe alternatives you've considered
There are web-based UIs for managing servers, but I want to be able to do it from the command line.

Additional context
None.

Travis PyPI deployment is broken. Check the output for 6599926 for more information. For release v0.6.0, I just manually built and uploaded it.