Running on Unraid using Docker.

Attempting to check the Siad status and I receive this error

siad

Sia Daemon v1.5.6
Git Revision 0b253ab18
Loading...
ERROR: listen tcp 127.0.0.1:9980: bind: address already in use
listen tcp 127.0.0.1:9980: bind: address already in use

When Sia is running in a container we don't see its output. Let's redirect both stdout and errout to files within the data directory.

• Create a Docker image.
• Add image to Docker Hub.
• Set up automatic builds.

Docker is already sending the right signal to siad and it's starting the graceful shutdown. The issue is that shutting down the renter is quite slow sometimes and docker is not patient enough for it. We need to ensure it waits for the full shutdown and doesn't trigger any timeouts that interrupt it. On the other hand we need to be vigilant to not allow a crash to block the container from ever shutting down.

There is a big issue in regards to how the SIA_API_PASSWORD environment variable is set in the current Docker image.

By setting ARG SIA_API_PASSWORD="/root/.sia" you are setting the password to "/root/.sia" not the path.

You can validate it by executing the following commands on a new docker data folder :

mkdir sia-data

docker run \
   --detach \
   --volume $(pwd)/sia-data:/sia-data \
   --publish 127.0.0.1:9980:9980 \
   --publish 9981:9981 \
   --publish 9982:9982 \
   --name sia-container \
    nebulouslabs/sia

curl -A "Sia-Agent" -u "":"/root/.sia" --data "encryptionpassword=test&force=false" "127.0.0.1:9980/wallet/init"

curl -A "Sia-Agent" -u "":"/root/.sia" --data "encryptionpassword=test" "127.0.0.1:9980/wallet/unlock"

curl -A "Sia-Agent" -u "":"/root/.sia" "127.0.0.1:9980/wallet/address"

Been running for a week or so just fine, and now I get the following when trying to start:

Sia Daemon v1.4.11
Git Revision 026f95f65
Loading...
(1/6) Loading siad...
(2/6) Loading gateway...
(3/6) Loading consensus...
(4/6) Loading transaction pool...
(5/6) Loading wallet...
(6/6) Loading host...
API is now available, synchronous startup completed in 5.633 seconds
Finished full setup in 23.168 seconds
panic: runtime error: index out of range [2] with length 2

goroutine 1028 [running]:
gitlab.com/NebulousLabs/Sia/modules/host.revisionFromRequest(0x70c196fab90ffa5a, 0xa91b4eb77e886c89, 0xbd1f5af35971db60, 0xd02d491b8bd04269, 0x0, 0xc0001643c0, 0x2, 0x4, 0x2, 0xffffffffffffffff, ...)
	gitlab.com/NebulousLabs/Sia@/modules/host/payment.go:252 +0x432
gitlab.com/NebulousLabs/Sia/modules/host.(*Host).managedPayByContract(0xc000096700, 0x103c1c0, 0xc00236a000, 0x0, 0x0, 0x0, 0x0)
	gitlab.com/NebulousLabs/Sia@/modules/host/payment.go:92 +0x461
gitlab.com/NebulousLabs/Sia/modules/host.(*Host).ProcessPayment(0xc000096700, 0x103c1c0, 0xc00236a000, 0xc00025a300, 0x0, 0x0, 0x0)
	gitlab.com/NebulousLabs/Sia@/modules/host/payment.go:31 +0x20f
gitlab.com/NebulousLabs/Sia/modules/host.(*Host).managedRPCUpdatePriceTable(0xc000096700, 0x103c1c0, 0xc00236a000, 0xc000344f90, 0x0)
	gitlab.com/NebulousLabs/Sia@/modules/host/rpcupdatepricetable.go:126 +0x27b
gitlab.com/NebulousLabs/Sia/modules/host.(*Host).threadedHandleStream(0xc000096700, 0x103c1c0, 0xc00236a000)
	gitlab.com/NebulousLabs/Sia@/modules/host/network.go:389 +0x6a3
gitlab.com/NebulousLabs/siamux.(*SiaMux).threadedAccept.func2(0xc0000534c0, 0xc00236a000, 0xc00000c780)
	gitlab.com/NebulousLabs/[email protected]/listener.go:186 +0x3c
created by gitlab.com/NebulousLabs/siamux.(*SiaMux).threadedAccept
	gitlab.com/NebulousLabs/[email protected]/listener.go:185 +0x450

Sia Daemon v1.3.7
Git Revision 126260536
Loading...
(0/6) Loading siad...
(1/6) Loading gateway...
(2/6) Loading consensus...
(3/6) Loading transaction pool...
(4/6) Loading wallet...
(5/6) Loading host...
(6/6) Loading renter...
Finished loading in 10.0945764 seconds

Building a docker image from tag 1.5.1 fails with error:

 ---> Running in a1824cd9be34
ln: /mnt/sia: No such file or directory
The command '/bin/sh -c ln -s "$SIA_DATA_DIR" /mnt/sia' returned a non-zero code: 1

We use the shell form of ENTRYPOINT which does not pass signals to the running process. This means that stopping the container will not signal siad to perform a graceful shutdown.

Tasks:

• verify that shell ENTRYPOINT doesn't receive signals
• verify that exec ENTRYPOINT does receive signals
• explore other ways to execute in the container, e.g. CMD
• change the Dockerfile accordingly

Due to debugging constraints when working with a pre-compiled binaries we should provide a dev image that:

• has /bin/bash shell (compared to limited /bin/ash from alpine)
• when logged in to shell: has all the tools like curl, wget, zip, vim, nano etc installed (@DavidVorick can you collaborate on that list)
• when logged in to shell: has a local copy of Sia repo so anyone can change a line and run make to recompile the binaries and restart a container (needs full go environment)
• when logged in to shell: has go env path in the PATH so you can run siac and siad after you recompile the binaries with make

The idea is to be able to debug production servers like this:

1. turn off health-check and wait for the server to be out of load balancer
2. run docker stop sia to stop the sia container
3. change the image in docker-compose.yml to dev
4. run docker exec -it sia /bin/bash to ssh to sia container
5. you should be in ~/Sia git directory where the master is checked out
6. you change branch or edit your changes and run make to compile binaries
7. run exit to exit sia container shell
8. run docker restart sia to restart sia container with your changes on it

It'd be better to restart socat or exit the container.

We can check if port 9980 is working. However, since 9980 might not respond while loading, it's not easy to find whether socat dies or siad is still loading.

I need to set --profile flag, but currently, it's not easy to set flags except for --modules, --sia-directory, and --api-addr. It'd be helpful if run.sh passes given flags to siad.

Currently the docker image is build for linux/amd64. It would really help to have it build for more architectures e.g. linux/arm (raspberry pi 2) or linux/arm64 (raspberry pi 3/4).

Hello,

I got my sia host running since 1 month now i got it crashed and it's not starting showing up that error:

sia_1 | ERROR: [server is unable to create the Sia node; unable to create host; could not unmarshal hash: encoding/hex: invalid byte: U+0060 '`']

sia_1 | [server is unable to create the Sia node; unable to create host; could not unmarshal hash: encoding/hex: invalid byte: U+0060 '`']

What it can be?

When I pull with docker to the latest version sia remains with siad at 1.5.0.

May you please look into it?

Docker host running on Unraid server. The host seems to mess up contract manager file every once in a while. This is a third time it has happened in 1 year of running the host.

Stack Trace or error message

Sia Daemon v1.5.6
Git Revision 0b253ab18
Loading...
(1/9) Loading siad...
(2/9) Loading gateway...
(3/9) Loading consensus...
(4/9) Loading transaction pool...
(5/9) Loading wallet...
(6/9) Loading feemanager...
(7/9) Loading host...
ERROR: [server is unable to create the Sia node; unable to create host; [error during contract manager startup; error while loading contract manager atomic data; error loading the contract manager settings file; unable to read persisted json object from disk: unexpected end of JSON input]; [error while loading contract manager atomic data; error loading the contract manager settings file; unable to read persisted json object from disk: unexpected end of JSON input]]

[server is unable to create the Sia node; unable to create host; [error during contract manager startup; error while loading contract manager atomic data; error loading the contract manager settings file; unable to read persisted json object from disk: unexpected end of JSON input]; [error while loading contract manager atomic data; error loading the contract manager settings file; unable to read persisted json object from disk: unexpected end of JSON input]]

Expected Behavior

Load up and run as normal

Observed Behavior

Error during host loading.

How to reproduce it (as minimally and precisely as possible)

Random occurence. No idea what is causing it. No idea what would be the proper fix without deleting the contract manager file.

Environment

Docker running on Unraid

• Sia version:

v1.5.6

• OS:

Unraid (Linux)

In order to have skynet and consensus working in full speed wirh an SSD is there a way to have them in a spearate volume?

Is it possible to pass it as an argument when we execute the docker file?

The image won't allow environment variables to be passed in. Features dependent on being able to work with environment variables, such as auto wallet unlock, don't work.

Hi,

Would it be possible to add possibility to provide password/seed via docker secret to increase security?