nixcloud / nixcloud-webservices Goto Github PK

View Code? Open in Web Editor NEW

169.0 169.0 26.0 2.84 MB

This nixpkgs extension, called nixcloud-webservices, focuses on ease of deployment of web-related technologies.

Home Page: https://nixcloud.io

License: Other

Nix 96.69% Python 1.79% Shell 1.15% Lua 0.14% Sieve 0.23%

nixcloud-webservices's People

Contributors

Stargazers

Watchers

nixcloud-webservices's Issues

Firewall ports open for reverse proxy

Currently, the user needs to indicate manually in the configuration that port 80 and port 443 need to be opened. However, if the reverse proxy is used, this implies that those ports are to be opened.

Proposed solution:
Add ports 80 and 443 firewall rules when reverse proxy is active.

nixcloud.email: evaluation error when using nginx simultaniously

When using the example config, I'm getting

error: cannot coerce null to a string, at /home/justin/git/nixcloud-webservices/modules/services/reverse-proxy/default.nix:224:7
(use ‘--show-trace’ to show detailed location information)

This is a test case, reproducing the issue:

import <nixpkgs/nixos/tests/make-test.nix> ({ pkgs, ...}:

{
  name = "nixcloud-email";

  nodes = {

    server =
      { config, pkgs, lib, ... }:

      let
        ipAddress = "8.19.10.3";
        ipv6Address = "201:48:11:403::1:1";
      in {
        imports = [
          <nixcloud-webservices/modules>
        ];
        nixcloud.email= {
          enable = true;
          domains = [ "lastlog.de" "dune2.de" ];
          ipAddress = ipAddress;
          ip6Address = ipv6Address;
          hostname = "mail.lastlog.de";
          users = [
            # see https://wiki.dovecot.org/Authentication/PasswordSchemes
            { name = "js"; domain = "lastlog.de"; password = "{SHA256-CRYPT}$<<<removed by qknight>>>"; }
            { name = "foo1"; domain = "dune2.de"; password = "{PLAIN}asdfasdfasdfasdf"; }
          ];
        };
        services.nginx = {
          enable = true;
          virtualHosts = {
            "example.com" = {
              locations."/" = {
                root = "/webroot/";
              };
              forceSSL = true;
              enableACME = true;
            };
          };
        };

      };
  };

  testScript = ''
    startAll;

  '';
})

Put it in test-email.nix and call it with

nix-build test-email.nix -I nixcloud-webservices=/path/to/your/nixcloud-webservices/ -I nixpkgs=https://d3g5gsiof5omrk.cloudfront.net/nixos/17.09/nixos-17.09.3154.b47b4f3dd3c/nixexprs.tar.xz

where nixpkgs is current nixos-17.09 and nixcloud-webservices current master.

Permission error certificate TLS-acmeSupplied

I wanted to upgrade my test instance but I'm consistently gettting a permission error on the certificate install phase for all certs:

8z9z3qyyybqqhvyk0z7fnkqymfvm287q-unit-script-nixcloud.TLS-acmeSupplied-test.com-start[12817]: /nix/store/8z9z3qyyybqqhvyk0z7fnkqymfvm287q-unit-script-nixcloud.TLS-acmeSupplied-test.com-start: line 2: cd: /var/lib/nixcloud/TLS/test.com/acmeSupplied: Permission denied

rspamd not working (on 20.09) due to option users.users error

After working around #76 (I mention this because it may be caused by a regression on newer nixos versions, and for this issue looks a bit similar to the issue mentioned here and patched here.

The error I'm seeing:

error: The option value `users.users' in `/nix/store/ihx06ak2767z80ri7i5wamiacnd16asp-source/nixos/modules/services/mail/rspamd.nix' is not of type `attribute set of submodules'.

I have users defined (only) as:

{ 
   nixcloud.email ={
      # ...
      users = [
        { name = "johndoe"; domain = "mydomain.me"; password = "{PLAIN}hahatest12345"; }
      ];
   };
}

Switch from Nixpkgs TLS to Nixcloud TLS fails

I went from

{ config, pkgs, ...}:{
  services.nginx = {
    enable = true;
    virtualHosts = {
      "erictapen.de" = {
        locations."/" = {
          root = "/webroot/erictapen.de/";
        };
        enableACME = true;
        forceSSL = true;
      };
    };
  };
}

{ config, pkgs, ...}:{
  services.nginx = {
    enable = true;
    virtualHosts = {
      "erictapen.de" = {
        locations."/" = {
          root = "/webroot/erictapen.de/";
        };
        forceSSL = true;
        sslCertificate = config.nixcloud.TLS.certs."erictapen.de-ACME".tls_certificate;
        sslCertificateKey = config.nixcloud.TLS.certs."erictapen.de-ACME".tls_certificate_key;
      };
    };
  };
}

When deploying with NixOps this fails:

...
myhost> warning: the following units failed: acme-erictapen.de-ACME.service
...

I have not enough capacity to provide a redacted log of acme-erictapen.de-ACME.service, so I will send it to @qknight, so that he can post snippets without sensible information in it.

Nix info:

system: "x86_64-linux"
host os: Linux 4.14.35, NixOS, 18.03 (Impala)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.0
nixpkgs: /home/justin/nixpkgs

Documentation for database abstraction

Documentation request

There is no documentation yet for the database abstraction.
One may notice a database target and a db-server target for every service created with nixcloud, but it is not yet clear how to add anything to them.

Secure POP not activated

The Secure (TLS) version of POP normally runs on port 995, and allows direct connection over TLS rather than StartTLS (which can be MitM). The mail abstraction does yet seem to run this?

20.09: preliminary self-signed certificates fail to generate

[...]
Oct 29 13:58:26 nixos nixcloud.TLS-acmeSuppliedPreliminary-<snip>-start[528]: Done creating a self signed certificate
Oct 29 13:58:26 nixos nixcloud.TLS-acmeSuppliedPreliminary-<snip>-post-start[728]: cp: cannot stat '/tmp/server.key': No such file or directory

Really weird. The ExecStart didn't fail, so server.key should be there

Leaps in path not working

When an instance of leaps is create elsewhere than on the root folder of a domain, e.g. using path = "/somedir", the resulting service available on example.com/somedir opens document non-editable - at least when using TLS.
Working theory is that the webservice is not mapped the right way...

nixcloud-webservices 18.09 doesn't seem to enforce nixos/nixpkgs 18.09

Quoting comment from chat:

today, I finally also upgraded my personal mailserver to nixcloud-webservices/email 18.09 with our new TLS stack and stumbled upon an issue, of which I'm not sure, whether it can be prevented properly/Nix-ish…
…I switched my nixcloud-webservices channel from 18.03 to 18.09, but forgot to do the same for my regular nixos channel and stateVersion, which resulted in nixcloud.TLS pulling in an ancient lego release which ended up failing due to a bug which is fixed in 18.03…
…shouldn't nixos-webservices-18.09 somehow enforce the proper/required release of NixOS/nixpkgs/stateVersion?

Leaps inheritance of websockets

Inheritance is incomplete. There are three types defined in modules/services/reverse-proxy/options.nix:

type = types.enum [ "redirect_to_https" "on" "off" ];

When you want to use redirect_to_https in combination with leaps, you get an error:

The option value nixcloud.webservices.leaps.demo.proxyOptions.websockets.ws.http.mode' in /some/path/nixcloud-webservicesl/modules/web/core/webserver.nix' is not a one of "on", "off".

(I think nix has a language error in there as well, "a one of" should be "one of")

Would it not make sense instead of having to turn "http" off to have an option under https [ "on" "off" "exclusive" ] so you would only have to set that?

nixcloud.email: example config runs VM test when built

This is my host config:

{ config, pkgs, ... }:
{

  nixcloud.email= {
    enable = true;
    domains = [ "erictapen.de" ];
    ipAddress = "REDACTED";
    ip6Address = "REDACTED";
    hostname = "mail.erictapen.de";
    enableSpamassassin = false;

    users = [
      { name = "mail"; domain = "erictapen.de"; password = "{PLAIN}passpass"; }
    ];
  };

}

I'm on nixos-18.03 and nixcloud-webservices 8c9a230.

When deploying this, vm-test-run-email is built. This is annoying, as it takes alot of resources on my laptop. I there a reason for it?

The option `nixcloud.email.enableACME` does not exist.

It's described in here.

My nixcloud-webservices checkout is 41616c7.

Add extraConfig option for dovecot2

I have a lot of folders and I use IDLE, so I need a high mail_max_userip_connections, but there's no way to add it to dovecot2's config without editing nixcloud-webservices

nixos-19.09: containers test fails

fails with:

client# [ 920.629694] systemd[1]: Started Cleanup of Temporary Directories.
mail1: running command: nc -z localhost 80
mail1: exit status 1
(0.01 seconds)
mail1: running command: nc -z localhost 80
mail1: exit status 1
(0.01 seconds)
mail1: running command: nc -z localhost 80
mail1: exit status 1
(0.01 seconds)
mail1: running command: nc -z localhost 80
mail1: exit status 1
(0.01 seconds)
mail1: running command: nc -z localhost 80
mail1: exit status 1
(0.01 seconds)
mail1: running command: nc -z localhost 80
mail1: exit status 1
(0.01 seconds)
mail1: running command: nc -z localhost 80
mail1: exit status 1
(0.01 seconds)
mail1: running command: nc -z localhost 80
mail1: exit status 1
(0.01 seconds)
mail1: running command: nc -z localhost 80
mail1: exit status 1
(0.01 seconds)
mail1: running command: nc -z localhost 80
mail1: exit status 1
(0.01 seconds)
mail1: running command: nc -z localhost 80
mail1: exit status 1
(0.01 seconds)
mail1: running command: nc -z localhost 80
mail1: exit status 1
(0.01 seconds)
(908.57 seconds)
The nixcloud test 'email' has failed with error 'action timed out after -1 seconds at /nix/store/f053vwv6hdkhl7xxaqcx1dmb5gqd88s0-nixos-test-driver/lib/perl5/site_perl/Machine.pm line 247, <ANONIO> line 1246.' but in case the machine was too slow (virtualized, not enough ram, too much cpu load, etc) then you can also disable the tests by adding 'nixcloud.tests.enable = false;' to your /etc/nixos/configuration.nix and still use our software.
cleaning up
killing client (pid 6)
killing mail2 (pid 18)
killing mail1 (pid 30)
killing dns (pid 42)
(0.00 seconds)
Attempt to end element "nest" with "logfile" tag at /nix/store/f053vwv6hdkhl7xxaqcx1dmb5gqd88s0-nixos-test-driver/lib/perl5/site_perl/Logger.pm line 28.
END failed--call queue aborted, <ANONIO> line 1246.
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
builder for '/nix/store/mc7pj1z8g1x25s0ip2c5adzg45g1fmvd-vm-test-run-email.drv' failed with exit code 22
error: build of '/nix/store/mc7pj1z8g1x25s0ip2c5adzg45g1fmvd-vm-test-run-email.drv' failed

Problem: The option `nixcloud.reverse-proxy.extraMappings.[definition 1-entry 1].port' is used but not defined.

Steps to reproduce

nixos-rebuild switch --upgrade after upgrading from 18.03 ( upgraded in stateVersion + nix-channels)

Expected behaviour

Executes without errors as in 18.03
This is still experimental software so breakage is expected.

Actual behaviour

# nixos-rebuild switch --upgrade
unpacking channels...
created 5 symlinks in user environment
unpacking channels...
created 5 symlinks in user environment
these paths will be fetched (1.70 MiB download, 8.27 MiB unpacked):
  /nix/store/3mfhdscydl8rgvsszxs0sv70afx76sh5-nix-2.1.1-man
  /nix/store/a6zdbsflqj152v16l8xds18yakckls2p-nix-2.1.1
  /nix/store/h1g9yw9859wvx4rx8z8qgsnfyad7wy2z-nixos-rebuild
copying path '/nix/store/3mfhdscydl8rgvsszxs0sv70afx76sh5-nix-2.1.1-man' from 'https://cache.nixos.org'...
copying path '/nix/store/a6zdbsflqj152v16l8xds18yakckls2p-nix-2.1.1' from 'https://cache.nixos.org'...
copying path '/nix/store/h1g9yw9859wvx4rx8z8qgsnfyad7wy2z-nixos-rebuild' from 'https://cache.nixos.org'...
building Nix...
building the system configuration...
error: The option `nixcloud.reverse-proxy.extraMappings.[definition 1-entry 1].port' is used but not defined.
(use '--show-trace' to show detailed location information)

# nix-channel --list
nixcloud-webservices https://hydra.nixcloud.io/channel/custom/nixcloud-webservices/release-18.09/nixcloud-webservices
nixos https://nixos.org/channels/nixos-18.09

leaps: Lost connection

the leaps websocket connects but after one minute of idle time it looses the connection.

probably a timeout in the reverse proxy which i assumed to be fixed but maybe i forgot to commit?

https://stackoverflow.com/questions/28828332/gorilla-websocket-disconnects-after-a-minute

Domain hydra.nixcloud.io has no dns record

I was using the webmail service you wrote, but I've been having errors when trying to update because your domain doesn't appear to exist anymore.

documentation: add remote repo inclusion recipe (as preferred installation method?)

As a nix noob I (unnecessarily) struggled with this one, although it's arguably the most logical use case to directly start using nixcloud: a "zero install" method by just referring to the remote repo in the .nix file using nixcloud.

I.e. I would have expected that a solution that does the below is actually documented as the first and preferred "installation" method, since it makes the most sense for remote declarative deployments.

After some trial and horror I ended up with:

{
  imports = [ "${fetchGit https://github.com/nixcloud/nixcloud-webservices}/modules/" ];
  nixcloud.tests.enable = false;
  # only now can you start using nixcloud.... options
}

which arguably doesn't support referring to a specific tag or commit and would therefore be less robust upon rebuilds.

An alternative which also appeared to work is the following, which doesn't have the above problem but is much more convoluted (it could maybe be simplified?):

  let
    ncws = (import <nixpkgs> {}).fetchFromGitHub {
          owner = "nixcloud";
          repo = "nixcloud-webservices";
          rev = "7e421fed1cb6dc460468d5917bb93b559606c7b6";
          sha256 = "sha256:0y3kpajq10ixh5xnijfsq2aqpf9f7yij5bxl7ynh3n9dj44ksflb";
    };
  in {
    imports = [ "${ncws}" {} ];
    nixcloud.tests.enable = false;

nixcloud.container systemd patch

motivation

it would be helpful to have https://github.com/nixcloud/nixcloud-container/blob/master/bin/helper/8447.patch in nixpkgs:

master (unstable)
18.03

the problem is that without this patch upstream we have to rebuild all software which should be deployed inside the LXC base container.

Missing submodules in Nixos

It is no longer possible to build nixcloud-webservices with current nixos-unstable:

building Nix...
building the system configuration...
error: attribute 'submodule' missing, at /home/username/repo/nixcloud-webservices/modules/web/core/webserver.nix:66:39
(use '--show-trace' to show detailed location information)

allowPing setting

The ping setting for the mail module causes conflicts with other modules... seems unlikely to be essential to the functioning of the mail module, and could perhaps be removed?

error: The option networking.firewall.allowPing' has conflicting definitions, in /nix/store/1ddrbljzx3i7j8y5yyy495nb97q7xdrd-nixcloud-webservices/nixcloud-webservices/modules/services/email/nixcloud-email.nix' and `/etc/nixos/redacted.nix'

webmail not working (on 20.09)

I don't know whether it's a reqression for newer versions (I'm a new user), but when I try to set up a very basic mail config, I get the following error:

error: The option `nixcloud.webservices.roundcube.mail.mydomain.me.meta.description' defined in `/nix/store/5pikrdbaqj8jszb1dwj723b61mw29zj3-source/modules/web/services/roundcube' does not exist.

where mydomain.me is the domain defined under nixcloud.email.domains = [ "mydomain.me" ];

The only way I found to get rid of this error is to disable webmail with nixcloud.email.webmail.enable = false; (all of which is undocumented BTW).

Configure addition reverse-proxy options with static-darkhttpd

I'm using static-darkhttpd to serve a static frontend like this:

  nixcloud.webservices.static-darkhttpd.frontend = {
    enable = true;
    root = /etc/nixos/modules/frontend-static;

    proxyOptions = {
      inherit domain;

      port = frontend_port;
      path = "/";
      TLS = domain;
      https.mode = "on";
    };
  };

This works very well.
Now I want to add a custom option to the reverse-proxy entry for this frontend, namely something like try_files $url $url.html to allow users to access the pages served without the .html extension.

(How) is this possible?

Support staging/non-default endpoint for ACME operations

To prevent running into rate-limits during tests/for debugging or to keep experimental certificates out of CT logs to which LetsEncrypt publishes, it would be quite helpful to be able to use the LetsEncrypt staging API.

Furthermore, in case (hopefully) some other CAs start offering ACME or someone runs a company internal ACME CA (e.g. using Boulder), a custom ACME API URL would have to be defined.

I'm not sure yet what to suggest in terms of the structure to use here:

Introduce an ACME-staging and an ACME-custom variant for nixcloud.TLS.certs.<name>.mode or rather introduce suboptions like staging = true/false and apiUrl?

security.acme already provides a production = true/false option which could be used for parts of this.

nixcloud.TLS fails in nixos-19.09

failing test

client# [    9.214698] dhcpcd[513]: Failed to reload-or-try-restart ntpd.service: Unit ntpd.service not found.
client# [    9.216372] dhcpcd[513]: Failed to reload-or-try-restart openntpd.service: Unit openntpd.service not found.
client# [    9.218409] dhcpcd[513]: Failed to reload-or-try-restart chronyd.service: Unit chronyd.service not found.
client# [    9.230513] dhcpcd[513]: forked to background, child pid 731
client# [    9.233495] systemd[1]: Started DHCP Client.
client# [    9.237300] systemd[1]: Reached target Network is Online.
client# [    9.239422] systemd[1]: Reached target Multi-User System.
client# [    9.241140] systemd[1]: Startup finished in 3.692s (kernel) + 5.533s (userspace) = 9.225s.
client# [    9.285373] login[661]: pam_unix(login:session): session opened for user root by LOGIN(uid=0)
client# [    9.301032] systemd[1]: Created slice user-0.slice.
client# [    9.304311] systemd[1]: Starting User Runtime Directory /run/user/0...
client# [    9.316311] systemd-logind[662]: New session 1 of user root.
client# [    9.335350] systemd[1]: Started User Runtime Directory /run/user/0.
client# [    9.340311] systemd[1]: Starting User Manager for UID 0...
client# [    9.348063] systemd[733]: pam_unix(systemd-user:session): session opened for user root by (uid=0)
client# [    9.463161] systemd[733]: Reached target Paths.
client# [    9.464749] systemd[733]: Reached target Sockets.
client# [    9.475381] systemd[733]: Reached target Timers.
client# [    9.476567] systemd[733]: Reached target Basic System.
client# [    9.484086] systemd[1]: Started User Manager for UID 0.
client# [    9.487049] login[738]: ROOT LOGIN  on '/dev/tty1'
client# [    9.492425] systemd[1]: Started Session 1 of user root.
client# [    9.495084] systemd[733]: Reached target Main User Target.
client# [    9.497534] systemd[733]: Startup finished in 104ms.
client# [    9.542223] dhcpcd[731]: eth0: soliciting an IPv6 router
client: running command: systemctl --no-pager show "default.target"
client: exit status 0
(0.03 seconds)
(10.69 seconds)
letsencrypt: waiting for unit ‘boulder.service’
letsencrypt: running command: systemctl --no-pager show "boulder.service"
letsencrypt: exit status 0
(0.03 seconds)
letsencrypt: running command: systemctl list-jobs --full 2>&1
letsencrypt: exit status 0
(0.02 seconds)
letsencrypt: running command: systemctl --no-pager show "boulder.service"
letsencrypt: exit status 0
(0.03 seconds)
(0.08 seconds)
The nixcloud test 'nixcloud.TLS' has failed with error 'unit ‘boulder.service’ is inactive and there are no pending jobs' but in case the machine was too slow (virtualized, not enough ram, too much cpu load, etc) then you can also disable the tests by adding 'nixcloud.tests.enable = false;' to your /etc/nixos/configuration.nix and still use our software.
cleaning up
killing letsencrypt (pid 6)
killing client (pid 59)
killing webserver (pid 31)
(0.00 seconds)
Attempt to end element "nest" with "logfile" tag at /nix/store/f053vwv6hdkhl7xxaqcx1dmb5gqd88s0-nixos-test-driver/lib/perl5/site_perl/Logger.pm line 28.
END failed--call queue aborted, <__ANONIO__> line 600.
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
builder for '/nix/store/9b056k9qgv5k6h9zdbm03q9s8057km2x-vm-test-run-nixcloud.TLS.drv' failed with exit code 22
error: build of '/nix/store/9b056k9qgv5k6h9zdbm03q9s8057km2x-vm-test-run-nixcloud.TLS.drv' failed

last commit

client# [    7.932803] nscd[728]: 728 monitoring file `/etc/resolv.conf` (5)
client# [    7.934374] nscd[728]: 728 monitoring directory `/etc` (2)
client# [    7.935938] nscd[728]: 728 monitoring file `/etc/services` (6)
client# [    7.937568] nscd[728]: 728 monitoring directory `/etc` (2)
client# [    7.939068] nscd[728]: 728 disabled inotify-based monitoring for file `/etc/netgroup': No such file or directory
client# [    7.941105] nscd[728]: 728 stat failed for file `/etc/netgroup'; will try again later: No such file or directory
client# [    7.949224] dhcpcd[512]: Failed to reload-or-try-restart ntpd.service: Unit ntpd.service not found.
client# [    7.950540] dhcpcd[512]: Failed to reload-or-try-restart openntpd.service: Unit openntpd.service not found.
client# [    7.951963] dhcpcd[512]: Failed to reload-or-try-restart chronyd.service: Unit chronyd.service not found.
client# [    7.960952] dhcpcd[512]: forked to background, child pid 743
client# [    7.963186] systemd[1]: Started DHCP Client.
client# [    7.966270] systemd[1]: Reached target Network is Online.
client# [    7.967863] systemd[1]: Reached target Multi-User System.
client# [    7.969158] systemd[1]: Startup finished in 3.202s (kernel) + 4.755s (userspace) = 7.957s.
client# [    8.001996] login[673]: pam_unix(login:session): session opened for user root by LOGIN(uid=0)
client# [    8.015055] systemd[1]: Created slice user-0.slice.
client# [    8.022066] systemd[1]: Starting User Runtime Directory /run/user/0...
client# [    8.025866] systemd-logind[674]: New session 1 of user root.
client# [    8.040041] systemd[1]: Started User Runtime Directory /run/user/0.
client# [    8.043885] systemd[1]: Starting User Manager for UID 0...
client# [    8.047697] systemd[745]: pam_unix(systemd-user:session): session opened for user root by (uid=0)
client# [    8.133738] systemd[745]: Reached target Paths.
client# [    8.135004] systemd[745]: Reached target Sockets.
client# [    8.146503] systemd[745]: Reached target Timers.
client# [    8.151065] systemd[745]: Reached target Basic System.
client# [    8.153827] login[750]: ROOT LOGIN  on '/dev/tty1'
client# [    8.158096] systemd[1]: Started User Manager for UID 0.
client# [    8.168185] systemd[1]: Started Session 1 of user root.
client# [    8.169527] dhcpcd[743]: eth0: soliciting an IPv6 router
client# [    8.170467] systemd[745]: Reached target Main User Target.
client# [    8.171576] systemd[745]: Startup finished in 81ms.
client: running command: systemctl --no-pager show "default.target"
client: exit status 0
(0.02 seconds)
(9.57 seconds)
letsencrypt: waiting for unit ‘boulder.service’
letsencrypt: running command: systemctl --no-pager show "boulder.service"
letsencrypt: exit status 0
(0.02 seconds)
letsencrypt: running command: systemctl list-jobs --full 2>&1
letsencrypt: exit status 0
(0.02 seconds)
letsencrypt: running command: systemctl --no-pager show "boulder.service"
letsencrypt: exit status 0
(0.02 seconds)
(0.06 seconds)
The nixcloud test 'nixcloud.TLS' has failed with error 'unit ‘boulder.service’ is inactive and there are no pending jobs' but in case the machine was too slow (virtualized, not enough ram, too much cpu load, etc) then you can also disable the tests by adding 'nixcloud.tests.enable = false;' to your /etc/nixos/configuration.nix and still use our software.
cleaning up
killing webserver (pid 48)
killing letsencrypt (pid 6)
killing client (pid 67)
(0.00 seconds)
Attempt to end element "nest" with "logfile" tag at /nix/store/kh5dfx8r6pjqp6f1n1shkyifh6xpjbhz-nixos-test-driver/lib/perl5/site_perl/Logger.pm line 28.
END failed--call queue aborted, <__ANONIO__> line 597.
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
builder for '/nix/store/gyz6c5bim59c1p12pch1gfl2bq4bcv9j-vm-test-run-nixcloud.TLS.drv' failed with exit code 22
error: build of '/nix/store/gyz6c5bim59c1p12pch1gfl2bq4bcv9j-vm-test-run-nixcloud.TLS.drv' failed
f4b14cea55078b35c755e103015e0d0e8e4943a8 is the first bad commit
commit f4b14cea55078b35c755e103015e0d0e8e4943a8
Author: Félix Baylac-Jacqué <[email protected]>
Date:   Fri Oct 18 19:13:04 2019 +0200

    nixos/tests/letsencrypt: use Pebble instead of Boulder
    
    Let's encrypt bumped ACME to V2. We need to update our nixos test to
    be compatible with this new protocol version.
    
    We decided to drop the Boulder ACME server in favor of the more
    integration test friendly Pebble.
    
    - overriding cacert not necessary
    - this avoids rebuilding lots of packages needlessly
    - nixos/tests/acme: use pebble's ca for client tests
    - pebble always generates its own ca which has to be fetched
    
    (cherry picked from commit 0c0af28cd59766f961efefea8ad4d14343a82821)

 nixos/modules/security/acme.nix                    |   5 +
 nixos/tests/acme.nix                               |  18 +-
 ...ange-ACME-directory-endpoint-to-directory.patch |  25 ++
 nixos/tests/common/letsencrypt/common.nix          |  26 +-
 nixos/tests/common/letsencrypt/default.nix         | 378 ++---------------
 nixos/tests/common/letsencrypt/mkcerts.nix         |   2 +-
 nixos/tests/common/letsencrypt/snakeoil-certs.nix  | 451 +++++++++++----------
 7 files changed, 313 insertions(+), 592 deletions(-)
 create mode 100644 nixos/tests/common/letsencrypt/0001-Change-ACME-directory-endpoint-to-directory.patch
bisect run success

nixos-19.09: directories test fails

In nixos-19.03 it was still working but in 19.09 it fails, starting with this commit:

The nixcloud test 'directories' has failed with error 'command `stat -c %U /super/n/e/s/t/e/d/owner.txt' did not succeed (exit code 1)' but in case the machine was too slow (virtualized, not enough ram, too much cpu load, etc) then you can also disable the tests by adding 'nixcloud.tests.enable = false;' to your /etc/nixos/configuration.nix and still use our software.
cleaning up
killing machine (pid 594)
(0.00 seconds)
Attempt to end element "nest" with "logfile" tag at /nix/store/6pby4z1h1068dd1zw3758ppglwlkanyf-nixos-test-driver/lib/perl5/site_perl/Logger.pm line 28.
END failed--call queue aborted, <__ANONIO__> line 270.
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
builder for '/nix/store/gqzirha38kbm6z4vjppiipwnwx27q6kz-vm-test-run-directories.drv' failed with exit code 22
error: build of '/nix/store/gqzirha38kbm6z4vjppiipwnwx27q6kz-vm-test-run-directories.drv' failed
5b0db07ee1c86376755a7e91fc4d1cfda2a7cff6 is the first bad commit
commit 5b0db07ee1c86376755a7e91fc4d1cfda2a7cff6
Author: Jörg Thalheim <[email protected]>
Date:   Fri Feb 22 08:23:48 2019 +0000

    systemd: 239.20190219 -> 241.20190221

 pkgs/os-specific/linux/systemd/default.nix | 59 ++++++++----------------------
 1 file changed, 16 insertions(+), 43 deletions(-)
bisect run success

nixcloud.email example results in system using self signed TLS certificate "Warwickshire"

I tried nixcloud.email another time, this time on a "vanilla" NixOS host. This is my host config:

{ config, pkgs, ... }:
{

  nixcloud.email= {
    enable = true;
    domains = [ "erictapen.de" ];
    ipAddress = "REDACTED";
    ip6Address = "REDACTED";
    hostname = "mail.erictapen.de";
    enableSpamassassin = false;

    users = [
      { name = "mail"; domain = "erictapen.de"; password = "{PLAIN}passpass"; }
    ];
  };

}

I'm on nixos-18.03 and nixcloud-webservices 8c9a230.

Everything seems to work, but when I try to fetch my Mails with offlineimap, I get

OfflineIMAP 7.1.5
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v2.57 (bundled), Python v2.7.14, OpenSSL 1.0.2o  27 Mar 2018
Account sync erictapen:
 *** Processing account erictapen
 Establishing connection to mail.erictapen.de:993 (erictapen-Remote)
 ERROR: Unknown SSL protocol connecting to host 'mail.erictapen.de' for repository 'erictapen-Remote'. OpenSSL responded:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)
 *** Finished account 'erictapen' in 0:00
ERROR: Exceptions occurred during the run!
ERROR: Unknown SSL protocol connecting to host 'mail.erictapen.de' for repository 'erictapen-Remote'. OpenSSL responded:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)

Traceback:
  File "/nix/store/abz63s1hsf38m9b9k1yvigxcs4l535pg-offlineimap-7.1.5/lib/python2.7/site-packages/offlineimap/accounts.py", line 283, in syncrunner
    self.__sync()
  File "/nix/store/abz63s1hsf38m9b9k1yvigxcs4l535pg-offlineimap-7.1.5/lib/python2.7/site-packages/offlineimap/accounts.py", line 359, in __sync
    remoterepos.getfolders()
  File "/nix/store/abz63s1hsf38m9b9k1yvigxcs4l535pg-offlineimap-7.1.5/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", line 452, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/nix/store/abz63s1hsf38m9b9k1yvigxcs4l535pg-offlineimap-7.1.5/lib/python2.7/site-packages/offlineimap/imapserver.py", line 630, in acquireconnection
    exc_info()[2])
  File "/nix/store/abz63s1hsf38m9b9k1yvigxcs4l535pg-offlineimap-7.1.5/lib/python2.7/site-packages/offlineimap/imapserver.py", line 544, in acquireconnection
    af=self.af,
  File "/nix/store/abz63s1hsf38m9b9k1yvigxcs4l535pg-offlineimap-7.1.5/lib/python2.7/site-packages/offlineimap/imaplibutil.py", line 194, in __init__
    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)
  File "/nix/store/abz63s1hsf38m9b9k1yvigxcs4l535pg-offlineimap-7.1.5/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 2183, in __init__
    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)
  File "/nix/store/abz63s1hsf38m9b9k1yvigxcs4l535pg-offlineimap-7.1.5/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 361, in __init__
    self.open(host, port)
  File "/nix/store/abz63s1hsf38m9b9k1yvigxcs4l535pg-offlineimap-7.1.5/lib/python2.7/site-packages/offlineimap/imaplibutil.py", line 202, in open
    super(WrappedIMAP4_SSL, self).open(host, port)
  File "/nix/store/abz63s1hsf38m9b9k1yvigxcs4l535pg-offlineimap-7.1.5/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 2196, in open
    self.ssl_wrap_socket()
  File "/nix/store/abz63s1hsf38m9b9k1yvigxcs4l535pg-offlineimap-7.1.5/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 548, in ssl_wrap_socket
    self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version)
  File "/nix/store/nx3jw576gqw01iiijgsav39w2qa4cni2-python-2.7.14/lib/python2.7/ssl.py", line 943, in wrap_socket
    ciphers=ciphers)
  File "/nix/store/nx3jw576gqw01iiijgsav39w2qa4cni2-python-2.7.14/lib/python2.7/ssl.py", line 611, in __init__
    self.do_handshake()
  File "/nix/store/nx3jw576gqw01iiijgsav39w2qa4cni2-python-2.7.14/lib/python2.7/ssl.py", line 840, in do_handshake
    self._sslobj.do_handshake()

A run of openssl reveals:

$ openssl s_client -connect mail.erictapen.de:993 -crlf
CONNECTED(00000003)
depth=1 C = UK, ST = Warwickshire, L = Leamington, O = OrgName, OU = Security Department, CN = example.com
verify error:num=19:self signed certificate in certificate chain
---
Certificate chain
 0 s:/C=UK/ST=Warwickshire/L=Leamington/O=OrgName/OU=IT Department/CN=example.com
   i:/C=UK/ST=Warwickshire/L=Leamington/O=OrgName/OU=Security Department/CN=example.com
 1 s:/C=UK/ST=Warwickshire/L=Leamington/O=OrgName/OU=Security Department/CN=example.com
   i:/C=UK/ST=Warwickshire/L=Leamington/O=OrgName/OU=Security Department/CN=example.com
...

I would expect, that the example config contains not a self signed certificate but one from Let's encrypt? Also after a quick glance at the source, I wonder wether this Warwickshire cert should be used for anything other than nginx config file validation?

Feature request: noDB flag

It doesn't make sense (especially on systems with scarce resources) to have many instances of similar services looking after databases that will never not exist. My test setup has a limited amount of inodes, which is eaten up by a very large nix store. That situation is likely to be quite common in cheap hosting. Database software is big (>3000 inodes for postgresql in the nix store) so this helps ...

leaps: websocket behind reverse-proxy broken

issue description:

leaps uses a websocket for the document editing. because the Host header is not set, leaps will return a CORS error which can be fixed by adding: proxy_set_header Host foo.de; into the reverse-proxy configuration.

the problem is that:

proxyOptions.http. | proxyOptions.https.

flags
record

would need to compute config.proxyOptions.domain which is not possible at the moment.

maybe it would make sense to have extraFlags or also make flags and record a function which is evaluated before the config is generated inside options.nix from the reverse-proxy.

user "reverse-proxy" "reverse-proxy";
error_log stderr;
daemon off;
events {}
http {
        server {
                listen 80;
                listen [::]:80;
                server_name foo.de;
                location /myleaps {
                        rewrite     ^   https://$server_name$request_uri? permanent;
                }
        }
        server {
                listen 80;
                listen [::]:80;
                server_name example.org;
                location /wiki {
                        rewrite     ^   https://$server_name$request_uri? permanent;
                }
        } server {
                ssl on;
                listen 443 ssl;
                listen [::]:443 ssl;
                server_name foo.de;
                ssl_certificate /var/lib/nixcloud/TLS/foo.de/selfsigned/fullchain.pem;
                ssl_certificate_key /var/lib/nixcloud/TLS/foo.de/selfsigned/key.pem;
                location /myleaps {
                proxy_pass http://${location.ip}:${toString location.port}${location.path};
                }
                location /myleaps/leaps/ws {
                        # https websocket default flags
                        proxy_http_version 1.1;
                        proxy_set_header Upgrade $http_upgrade;
                        proxy_set_header Connection "upgrade";
                        proxy_set_header Host foo.de;
                        proxy_set_header X-Forwarded-For $remote_addr;
                        proxy_pass http://127.0.0.1:3031/myleaps/leaps/ws;
                }
        }
        server {
                ssl on;
                listen 443 ssl;
                listen [::]:443 ssl;
                server_name example.org;
                ssl_certificate /var/lib/nixcloud/TLS/example.org/selfsigned/fullchain.pem;
                ssl_certificate_key /var/lib/nixcloud/TLS/example.org/selfsigned/key.pem;
                location /wiki {
                        # https default flags
                        proxy_set_header Host $host;
                        proxy_set_header X-Real-IP $remote_addr;
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_set_header X-Forwarded-Proto $scheme;
                        proxy_pass http://127.0.0.1:5050/wiki;
                }
        }
}

Document and test email aliases

While testing nixcloud-webservices I noticed that nixcloud mail documentation mentions the possibility of aliases but does not actually tell you how to create an alias.

nixcloud.TLS default email

It would be nice if there was a way to specify a global default email in addition to the per-cert ones

$service-$name-apache.service is started, despite failing $service-$name-webserver-init.service

How to reproduce:
Implement a webservice with:
webserver.variant = "apache";
and
webserver.startupScript = "exit 1";

Even though $service-$name-webserver-init.service will fail now expectedly due to exit 1, $service-$name-apache.service will start nonetheless.

STARTTLS encryption on managesieve cannot be enabled anymore

24101d6 (#45) enabled SSL by default on managesieve (port 4190), but as far as I understand no plaintext alternative port is provided, so STARTTLS is not possible.

Since the managesieve spec does not define a SSL port, some managesieve clients (claws mail and sieve-connect for example) only support STARTTLS, and have no way to connect to the nixcloud-webservices' managesieve server (unless encryption is disabled altogether with enableTLS = false;).

darkhttpd static webserver by design needs external mimetype indication

I've tested the nixcloud abstractions for static hosting, and the current darkhttpd back-end seems to not be able to properly serve SVG files with their mimetype (design decision of the author of darkhttpd). Some browser then refuse to show these, which would result in brokenness.

In the pull request I made before (#2), this was covered by reusing the apache mime.types. There were a number of other useful options to tweak the behaviour of darkhttpd. In the setup I suggested as well - including turning directory listing off, location of logs, etc. This allowed for convenient user customisation while keeping the service definition fully generic. In fact, there was not much specific to darkhttpd at all.

serviceCommand = "${pkgs.${config.serviceName}}/bin/${serviceName}";
serviceParameters = [
```
 "${toString config.staticDir}"
```
```
 "--addr" "127.0.0.1"
```

 "--port" "${toString config.proxyOptions.port}"

```
 "--no-server-id"
```
```
 "--daemon"
```
```
 "--no-keepalive"
```

 "--mimetypes" "${pkgs.apacheHttpd}/conf/mime.types"

] ++ optional (!config.directoryListing) "--no-listing"

 ++ optional (config.log) "--log ${config.logFile}";

Import with fetchFromGitHub

Hi,

I'd like to import this into configuration.nix using importFromGithub, but i get infinite recursion.

Is this something that's possible to do?

Thanks.

20.09: missing /var/{cache,log}/nginx directories prevent nixcloud.reverse-proxy to start

https://nixos.org/manual/nixos/stable/release-notes.html#sec-release-20.09

The Nginx log directory has been moved to /var/log/nginx, the cache directory to /var/cache/nginx. The option services.nginx.stateDir has been removed.

Infinite recursion on nixos 21.05

It seems to be related to email.

To reproduce, try to build this file in the nixcloud-webservices repo:

(import <nixpkgs/nixos> { configuration =
  { config, pkgs, ... }:

  {
    imports = [ ./. ];
    nixcloud.email.enable = true;
    nixcloud.email.domains = ["example.org"];
  };
}).system

webservices with static documents in ${config.stateDir}/www

motivation

static-darkhttpd and static-nginx should have an example index.html (and also for documents created by the user later on) served inside a www directory which should be created inside ${config.stateDir}.
leaps: create example document with text so that when the user visits the webservice for the first time sees something to edit instead of having to add this manually

for both static-darkhttpd and static-nginx the test.nix uses hardcoded values in the test:

see https://raw.githubusercontent.com/nixcloud/nixcloud-webservices/master/modules/web/services/static-darkhttpd/test.nix

perl-Razor2-Client-Agent-2.84.drv fails to build on unstable

problem description

when using nixcloud.email on unstable perl-Razor2-Client-Agent-2.84.drv won't build.

nixos-version 
18.09.6c064e6-nixcloud_54c425e (Jellyfish)

nixos-rebuild switch
...
cp deHTMLxs.pm ../blib/lib/Razor2/Preproc/deHTMLxs.pm
AutoSplitting ../blib/lib/Razor2/Preproc/deHTMLxs.pm (../blib/lib/auto/Razor2/Preproc/deHTMLxs)
Running Mkbootstrap for Razor2::Preproc::deHTMLxs ()
chmod 644 "deHTMLxs.bs"
cc -c   -D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/no-such-path/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2   -DVERSION=\"2.18\" -DXS_VERSION=\"2.18\" -fPIC "-I/nix/store/ggb7k5x9855j10dz99467djx4rplg32b-perl-5.24.3/lib/perl5/5.24.3/x86_64-linux-thread-multi/CORE"   _deHTMLxs.c
"/nix/store/ggb7k5x9855j10dz99467djx4rplg32b-perl-5.24.3/bin/perl" "/nix/store/ggb7k5x9855j10dz99467djx4rplg32b-perl-5.24.3/lib/perl5/5.24.3/ExtUtils/xsubpp"  -typemap "/nix/store/ggb7k5x9855j10dz99467djx4rplg32b-perl-5.24.3/lib/perl5/5.24.3/ExtUtils/typemap" -typemap "typemap"  deHTMLxs.xs > deHTMLxs.xsc && mv deHTMLxs.xsc deHTMLxs.c
cc -c   -D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/no-such-path/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2   -DVERSION=\"2.18\" -DXS_VERSION=\"2.18\" -fPIC "-I/nix/store/ggb7k5x9855j10dz99467djx4rplg32b-perl-5.24.3/lib/perl5/5.24.3/x86_64-linux-thread-multi/CORE"   deHTMLxs.c
rm -f ../blib/arch/auto/Razor2/Preproc/deHTMLxs/deHTMLxs.so
cc  -shared -O2 -L/nix/store/84h2zni7h805k0i1ys2bba3dsp1cqnhh-glibc-2.26-131/lib -fstack-protector-strong _deHTMLxs.o deHTMLxs.o  -o ../blib/arch/auto/Razor2/Preproc/deHTMLxs/deHTMLxs.so      \
        \

chmod 755 ../blib/arch/auto/Razor2/Preproc/deHTMLxs/deHTMLxs.so
"/nix/store/ggb7k5x9855j10dz99467djx4rplg32b-perl-5.24.3/bin/perl" -MExtUtils::Command::MM -e 'cp_nonempty' -- deHTMLxs.bs ../blib/arch/auto/Razor2/Preproc/deHTMLxs/deHTMLxs.bs 644
Manifying 1 pod document
make[1]: Leaving directory '/tmp/nix-build-perl-Razor2-Client-Agent-2.84.drv-0/Razor2-Client-Agent-2.84/Razor2-Preproc-deHTMLxs'
cp bin/razor-report blib/script/razor-report
"/nix/store/ggb7k5x9855j10dz99467djx4rplg32b-perl-5.24.3/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/razor-report
cp bin/razor-revoke blib/script/razor-revoke
"/nix/store/ggb7k5x9855j10dz99467djx4rplg32b-perl-5.24.3/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/razor-revoke
cp bin/razor-client blib/script/razor-client
"/nix/store/ggb7k5x9855j10dz99467djx4rplg32b-perl-5.24.3/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/razor-client
cp bin/razor-check blib/script/razor-check
"/nix/store/ggb7k5x9855j10dz99467djx4rplg32b-perl-5.24.3/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/razor-check
cp bin/razor-admin blib/script/razor-admin
"/nix/store/ggb7k5x9855j10dz99467djx4rplg32b-perl-5.24.3/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/razor-admin
"/nix/store/ggb7k5x9855j10dz99467djx4rplg32b-perl-5.24.3/bin/perl" "-MExtUtils::Command::MM" -e pod2man "--" \
docs/razor-agent.conf.pod \
blib/man5/razor-agent.conf.5 \
docs/razor-agents.pod \
blib/man5/razor-agents.5 \
docs/razor-whitelist.pod \
blib/man5/razor-whitelist.5
Manifying 3 pod documents
Manifying 4 pod documents
Manifying 2 pod documents
running tests
check flags: SHELL=/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash VERBOSE=y test
make[1]: Entering directory '/tmp/nix-build-perl-Razor2-Client-Agent-2.84.drv-0/Razor2-Client-Agent-2.84/Razor2-Preproc-deHTMLxs'
Running Mkbootstrap for Razor2::Preproc::deHTMLxs ()
chmod 644 "deHTMLxs.bs"
Manifying 1 pod document
make[1]: Leaving directory '/tmp/nix-build-perl-Razor2-Client-Agent-2.84.drv-0/Razor2-Client-Agent-2.84/Razor2-Preproc-deHTMLxs'
make[1]: Entering directory '/tmp/nix-build-perl-Razor2-Client-Agent-2.84.drv-0/Razor2-Client-Agent-2.84/Razor2-Preproc-deHTMLxs'
PERL_DL_NONLAZY=1 "/nix/store/ggb7k5x9855j10dz99467djx4rplg32b-perl-5.24.3/bin/perl "-I../blib/lib" "-I../blib/arch" test.pl
/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash: -c: line 0: unexpected EOF while looking for matching `"'
/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash: -c: line 1: syntax error: unexpected end of file
make[1]: *** [Makefile:796: test_dynamic] Error 1
make[1]: Leaving directory '/tmp/nix-build-perl-Razor2-Client-Agent-2.84.drv-0/Razor2-Client-Agent-2.84/Razor2-Preproc-deHTMLxs'
make: *** [Makefile:1091: subdirs-test] Error 2
builder for '/nix/store/pdx6lyy6s253m2h2fx25d5n55s2q7008-perl-Razor2-Client-Agent-2.84.drv' failed with exit code 2
cannot build derivation '/nix/store/jvfyqy6imv1h5586mkplfyyd7fczrab3-perl-SpamAssassin-3.4.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/jw45gc2xk2ml16jg5kjakby3hw1rwwxs-perl-SpamAssassin-3.4.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/2rh6326q70vzfvbgg77mixw7q30lassq-postfix-master.cf.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/n80q09zr6z6cf6ah61fiy880xylhn9w4-postfix-master.cf.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/38h1pf8gx561ck7xvzwn15cd2i50vwp9-system-path.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/ixb4fcrama6iqvvp7l406zcg8mr4czan-system-path.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/8m3qfc39nv846klh6v8ymgpsfsz1066v-unit-script.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/nd1prm8pxky3b99wvac3g329lq1agv94-unit-script.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/2lrvq13fjb4h4ndzbgxnhj8v72jmic7p-unit-spamd.service.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/vlwkfn4jmx5azl27rxn2nqp50a2vj8gp-unit-spamd.service.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/41mk76xlnczbkm60r8k4ysc9qwlmkmyi-nixos-system-hydra.nixcloud.io-18.09.6c064e6-nixcloud_4c7635e.drv': 1 dependencies couldn't be built
error: build of '/nix/store/41mk76xlnczbkm60r8k4ysc9qwlmkmyi-nixos-system-hydra.nixcloud.io-18.09.6c064e6-nixcloud_4c7635e.drv' failed

Retain trailing slash in reverse-proxy location entry

Currently, the nginx config generated by the reverse-proxy abstraction does not properly translate path settings into location entries - in my particular case, I need a trailing slash in the location entry to only match subroutes of a given path.

Expected behaviour:

path = "/api/" generates location /api/

Observed behaviour:

path = "/api/" generates location /api

I didn't check, but there could be more functionality affected by this issue (for example regexes), so the path setting should be translated into the location entry as closely as possible.

nixcloud.TLS cert owner option

Most services do not run as root, but the certs are root-owned with permissions 0700

20.09: loaOf in users.users and users.groups makes nixcloud-webservices throw an error

In 20.03 it just gave a warning:

trace: warning: In file /nix/store/kqx95nfx4wafnzxkiyjnr6xdj9mfpsdn-nixos-20.03.tar.gz/nixos/modules/config/users-groups.nix
a list is being assigned to the option config.users.users.                                                                              
This will soon be an error as type loaOf is deprecated.                                                                                 
See https://github.com/NixOS/nixpkgs/pull/63103 for more information.                                                                   
Do                                                                                                                                      
  users.users =                                                                                                                         
    { reverse-proxy = {...}; }                                                                                                          
instead of                                                                                                                              
  users.users =                                                                                                                         
    [ { name = "reverse-proxy"; ...} ]                                                                                                  
                                                                                                                                        
trace: warning: In file /nix/store/kqx95nfx4wafnzxkiyjnr6xdj9mfpsdn-nixos-20.03.tar.gz/nixos/modules/config/users-groups.nix
a list is being assigned to the option config.users.groups.                                                                             
This will soon be an error as type loaOf is deprecated.                                                                                 
See https://github.com/NixOS/nixpkgs/pull/63103 for more information.                                                                   
Do                                                                                                                                      
  users.groups =                                                                                                                        
    { reverse-proxy = {...}; }                                                                                                          
instead of                                                                                                                              
  users.groups =                                                                                                                        
    [ { name = "reverse-proxy"; ...} ]

But in 20.09 the old type was removed and this results in an error.

To reproduce, set nixcloud.reverse-proxy.enable = true

edit: same error in other places later in the evaluation

stateful webservices

motivation

services.nginx and services.httpd basically implement stateful webservices (CGI scripts in /www and not in the /nix/store).
services.httpd implementing extraSubservices is an exception and was the main inspiration for what nixcloud-webservices

however, i would love to use nixcloud-webservices also to have a frontend/backend with stateful CGI documents support:

this makes hacking easier
developers and operators from other systems might prefer this for some legacy reason
developers migrating a legacy service (stateful) to a (stateless) service might find such a feature handy

conclusion

i feel there is really the need for to support both: supporting nixcloud-webservices features as those in combination to stateful CGI documents

nixcloud-webservices breaks nixos-rebuild switch update nix 1.x -> nix 2.x

description

nix-env --version
nix-env (Nix) 1.11.16

nixos-version
18.03pre126729.2e4aded3669 (Impala)

when nixcloud.webservices is in use, then the upgrade from 2e4aded3669 to nixos-18.03pre130932.cc4677c36ee fails with this error message:

nixos-rebuild switch
building Nix...
warning: unknown setting 'signed-binary-caches'
warning: unknown setting 'signed-binary-caches'
building the system configuration...
warning: unknown setting 'signed-binary-caches'
trace: warning: The option `ipAddress' defined in `/etc/nixos/submodule/nixcloud-webservices/modules/virtualisation/container.nix' has been changed to `ipv4.addresses' that has a different type. Please read `ipv4.addresses' documentation and update your configuration accordingly.
trace: warning: The option `prefixLength' defined in `/etc/nixos/submodule/nixcloud-webservices/modules/virtualisation/container.nix' has been changed to `ipv4.addresses' that has a different type. Please read `ipv4.addresses' documentation and update your configuration accordingly.
trace: warning: The option `ipAddress' defined in `/etc/nixos/submodule/nixcloud-webservices/modules/virtualisation/container.nix' has been changed to `ipv4.addresses' that has a different type. Please read `ipv4.addresses' documentation and update your configuration accordingly.
trace: warning: The option `prefixLength' defined in `/etc/nixos/submodule/nixcloud-webservices/modules/virtualisation/container.nix' has been changed to `ipv4.addresses' that has a different type. Please read `ipv4.addresses' documentation and update your configuration accordingly.
trace: warning: The option `ipv6Address' defined in `/etc/nixos/configuration.nix' has been changed to `ipv6.addresses' that has a different type. Please read `ipv6.addresses' documentation and update your configuration accordingly.
trace: warning: The option `ipv6PrefixLength' defined in `/etc/nixos/configuration.nix' has been changed to `ipv6.addresses' that has a different type. Please read `ipv6.addresses' documentation and update your configuration accordingly.
trace: warning: The option `ipAddress' defined in `/etc/nixos/configuration.nix' has been changed to `ipv4.addresses' that has a different type. Please read `ipv4.addresses' documentation and update your configuration accordingly.
trace: warning: The option `prefixLength' defined in `/etc/nixos/configuration.nix' has been changed to `ipv4.addresses' that has a different type. Please read `ipv4.addresses' documentation and update your configuration accordingly.
these derivations will be built:
  /nix/store/5zr9g33sbqsy5nxlkphjx9vmkfhhgsbn-closure-info.drv
  /nix/store/m4rsn5mmfnxs68m814x3fxs6x2g1i4cd-run-nixos-vm.drv
  /nix/store/b4kj6blf5q020apw2fvxyx7hrgapnh47-nixos-vm.drv
  /nix/store/sh30831dmn09y6ss79mkm63b149f7ppb-nixos-test-driver-reverse-proxy.drv
  /nix/store/0bc0bh4jja7hzj5zgvgxqfxvfris0wgd-vm-test-run-reverse-proxy.drv
  /nix/store/0md4hwa56012mpnpfgrs75i9984s47ya-nixos-system-status.nixcloud.io-18.03pre130932.cc4677c36ee.drv
building path(s) ‘/nix/store/p1q9wljslx8ssx986y5yg3pms3q1d42r-closure-info’
/nix/store/aj9rz4k3xvg8q9fzqfnb0q123707zp0n-builder: line 1: .attrs.sh: No such file or directory
builder for ‘/nix/store/5zr9g33sbqsy5nxlkphjx9vmkfhhgsbn-closure-info.drv’ failed with exit code 1
cannot build derivation ‘/nix/store/m4rsn5mmfnxs68m814x3fxs6x2g1i4cd-run-nixos-vm.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/b4kj6blf5q020apw2fvxyx7hrgapnh47-nixos-vm.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/sh30831dmn09y6ss79mkm63b149f7ppb-nixos-test-driver-reverse-proxy.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/0bc0bh4jja7hzj5zgvgxqfxvfris0wgd-vm-test-run-reverse-proxy.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/0md4hwa56012mpnpfgrs75i9984s47ya-nixos-system-status.nixcloud.io-18.03pre130932.cc4677c36ee.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/0md4hwa56012mpnpfgrs75i9984s47ya-nixos-system-status.nixcloud.io-18.03pre130932.cc4677c36ee.drv’ failed

this is caused by our unit test for nixcloud.reverse-proxy.

hacky solution

comment out the test in modules/services/reverse-proxy/default.nix
do a nixos-rebuild switch and NOT reboot
comment the test back in
rerun nixos-rebuild switch -> test is executed and works!

Non-root ownership of state dirs

By default, the state directories are owned by root.
This seems wrong, these should be owned by the owner of the process that is supposed to write in those directories.

**Proposed solution: **
At creation of state directory, assign rights to the right owner. Use setgid.

user/group names get too long

issue description

user/group names are getting too long:

the limit is 32 characters according to the useradd manpage

systemctl status static-darkhttpd-options-search-darkhttpd

echo static-darkhttpd-options-search-darkhttpd | wc
      1       1      42

the problem is that systemd won't be able to start the services with a 'too' long username and bail out with a:

Mar 08 18:58:01 nixos systemd[1]: /nix/store/0fd31nk9n2yankhvg5hq2z7nxa7gcr29-unit-static-darkhttpd-iiiiiiii-darkhttpd.service/static-darkhttpd-iiiiiiii-darkhttpd.service:14: Invalid user/group name or numeric ID: static-darkhttpd-iiiiiiii-webserver
Mar 08 18:58:01 nixos systemd[1]: static-darkhttpd-iiiiiiii-darkhttpd.service: Cannot add dependency job, ignoring: Unit static-darkhttpd-iiiiiiii-darkhttpd.service is not loaded properly: Exec format error.

nixpkgs

NixOS/nixpkgs#36556

nixcloud-webservices

@aszlig i'd like to apply this patch

diff --git a/modules/web/core/webserver.nix b/modules/web/core/webserver.nix
index 6d0ebb4..c7d5e12 100644
--- a/modules/web/core/webserver.nix
+++ b/modules/web/core/webserver.nix
@@ -59,7 +59,7 @@ in {
     };
 
     webserver.user = lib.mkOption {
-      default = "webserver";
+      default = "";
       type = lib.types.str;
       description = "The main user name which executes this webservice.";
     };
@@ -76,7 +76,7 @@ in {
     };
 
     webserver.group = lib.mkOption {
-      default = "webserver";
+      default = "";
       type = lib.types.str;
       description = "The main group name which executes this webservice.";
     };

When testing the documented setup (https://github.com/nixcloud/nixcloud-webservices/blob/master/documentation/nixcloud.email.md#basic-example), using smtp over the submission port to send email to external domains is not possible. This seems to require additional sasl settings, which need to be added to get a working setup.

Otherwise, it seems only email to the own domain can be sent.

nixcloud / nixcloud-webservices Goto Github PK

nixcloud-webservices's People

Contributors

Stargazers

Watchers

Forkers

nixcloud-webservices's Issues

Documentation request

Steps to reproduce

Expected behaviour

Actual behaviour

motivation

failing test

last commit

motivation

problem description

Expected behaviour:

Observed behaviour:

motivation

conclusion

description

hacky solution

issue description

nixpkgs

nixcloud-webservices

Recommend Projects

Recommend Topics

Recommend Org