wunderio / wundermachina Goto Github PK

View Code? Open in Web Editor NEW

6.0 42.0 8.0 1012 KB

Vagrant & Ansible stuff to build up virtual environment on your local

License: MIT License

Shell 11.51% Ruby 3.31% Python 41.56% HTML 0.11% Dockerfile 0.56% Jinja 42.95%

wundermachina's Introduction

WunderMachina

Ansible roles for setting up environment to server Drupal sites

wundermachina's People

Contributors

Stargazers

Watchers

Forkers

joesb mgalang hkirsman mikaelkundert evilfurryone onnimonni tomimikola de7768

wundermachina's Issues

It should be possible to add support for external cdn like cloudflare

Should be possible to define additional set_real_ip_from addresses to pass original request ip

Add NVM

Let developers write bash scripts to install their favorite node versions and do stuff.

Backup role is broken

I have a feeling that after the latest update to the role the backup is no longer properly done if provisioned based on that.

I took a look at sites that built it based on the previous version and noticed that this commit messes up the logic.
cbf5086

No longer are the backups created into

{{ backup_location }}/drupal/database and then also deleted according to the backup_keep_max_days because the backups are just thrown into the {{ backup_location }}/.tgz format.

Can the role be update to work like it used to with also having the file backup option?

Support for multicore SOLR

We have a project where we would need multicore SOLR installation where both dev and stage sites are on the same server.

What is the license for this project?

Build error during solr unpacking.

I keep getting this build error. Running 'vagrant provision' again resolves the issue but would be nice if it worked the first try. :)

Possibility to define custom non-drupal hosts in nginx

Enable sticky sesssions in varnish 4

Make backup cron interval configurable

Currently default is once a day at 01:00, should be possible to define the crun intervals.

upcloud-firewall: Add descriptions to firewall rules

When defining a specific rule I would like to be able to add a description to that firewall rule so that it`s clear what is it for when looking at the upcloud control panel.

Ensure correct time and timezone settings

Make backup script exclude data tables configurable

Currently we have fixed cache_form and search_index tables excluded. It should be possible to define those individually.

Papertrail logs

Verify all relevant logs get tracked automatically and setup drupal.log for syslog logging

TASK [devtools : PHP | Set up xdebug.ini] fails on issues/92 branch

failed: [192.168.10.11] (item=({u'section': u'XDebug'}, {u'val': 1, u'key': u'xdebug.remote_enable'})) => {"failed": true, "item": [{"section": "XDebug"}, {"key": "xdebug.remote_enable", "val": 1}], "msg": "Destination /etc/php.d/zzz-xdebug.ini does not exist !", "rc": 257}
failed: [192.168.10.11] (item=({u'section': u'XDebug'}, {u'val': u'dbgp', u'key': u'xdebug.remote_handler'})) => {"failed": true, "item": [{"section": "XDebug"}, {"key": "xdebug.remote_handler", "val": "dbgp"}], "msg": "Destination /etc/php.d/zzz-xdebug.ini does not exist !", "rc": 257}
failed: [192.168.10.11] (item=({u'section': u'XDebug'}, {u'val': 1, u'key': u'xdebug.remote_connect_back'})) => {"failed": true, "item": [{"section": "XDebug"}, {"key": "xdebug.remote_connect_back", "val": 1}], "msg": "Destination /etc/php.d/zzz-xdebug.ini does not exist !", "rc": 257}
failed: [192.168.10.11] (item=({u'section': u'XDebug'}, {u'val': 9000, u'key': u'xdebug.remote_port'})) => {"failed": true, "item": [{"section": "XDebug"}, {"key": "xdebug.remote_port", "val": 9000}], "msg": "Destination /etc/php.d/zzz-xdebug.ini does not exist !", "rc": 257}
failed: [192.168.10.11] (item=({u'section': u'XDebug'}, {u'val': 0, u'key': u'xdebug.remote_autostart'})) => {"failed": true, "item": [{"section": "XDebug"}, {"key": "xdebug.remote_autostart", "val": 0}], "msg": "Destination /etc/php.d/zzz-xdebug.ini does not exist !", "rc": 257}

php-fpm role fails in template file

./provision.sh -t php-fpm stage
...

TASK [php-fpm : template] ******************************************************************************************************************
fatal: [hostname]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleUndefinedVariable: 'unicode object' has no attribute 'key'"}

Traced the issue to a template file...

{% for variable in php_env_vars %}
env[{{ variable.key }}] = "{{ variable.value }}"
{% endfor %}

... that loads the following:

php_env_vars:
  - key: "WKV_SITE_ENV"
    value: "{{ wkv_site_env }}"

❯ ansible --version
ansible 2.3.0.0

Won't build in HHVM branch

---> Package mesa-libGLU.x86_64 0:9.0.0-4.el7 will be installed
--> Finished Dependency Resolution
Error: Package: ImageMagick-last-perl-6.9.3.3-1.el7.remi.x86_64 (remi)
Requires: ImageMagick-last-libs(x86-64) = 6.9.3.3-1.el7.remi
Installed: ImageMagick-last-libs-6.9.3.4-1.el7.remi.x86_64 (@remi-safe)
ImageMagick-last-libs(x86-64) = 6.9.3.4-1.el7.remi
Available: ImageMagick-last-libs-6.9.3.2-1.el7.remi.x86_64 (remi)
ImageMagick-last-libs(x86-64) = 6.9.3.2-1.el7.remi
Available: ImageMagick-last-libs-6.9.3.3-1.el7.remi.x86_64 (remi)
ImageMagick-last-libs(x86-64) = 6.9.3.3-1.el7.remi
Error: Package: ImageMagick-last-c++-6.9.3.3-1.el7.remi.x86_64 (remi)
Requires: ImageMagick-last-libs(x86-64) = 6.9.3.3-1.el7.remi
Installed: ImageMagick-last-libs-6.9.3.4-1.el7.remi.x86_64 (@remi-safe)
ImageMagick-last-libs(x86-64) = 6.9.3.4-1.el7.remi
Available: ImageMagick-last-libs-6.9.3.2-1.el7.remi.x86_64 (remi)
ImageMagick-last-libs(x86-64) = 6.9.3.2-1.el7.remi
Available: ImageMagick-last-libs-6.9.3.3-1.el7.remi.x86_64 (remi)
ImageMagick-last-libs(x86-64) = 6.9.3.3-1.el7.remi
Error: Package: ImageMagick-last-djvu-6.9.3.3-1.el7.remi.x86_64 (remi)
Requires: ImageMagick-last-libs(x86-64) = 6.9.3.3-1.el7.remi
Installed: ImageMagick-last-libs-6.9.3.4-1.el7.remi.x86_64 (@remi-safe)
ImageMagick-last-libs(x86-64) = 6.9.3.4-1.el7.remi
Available: ImageMagick-last-libs-6.9.3.2-1.el7.remi.x86_64 (remi)
ImageMagick-last-libs(x86-64) = 6.9.3.2-1.el7.remi
Available: ImageMagick-last-libs-6.9.3.3-1.el7.remi.x86_64 (remi)
ImageMagick-last-libs(x86-64) = 6.9.3.3-1.el7.remi
Error: Package: ImageMagick-last-6.9.3.3-1.el7.remi.x86_64 (remi)
Requires: ImageMagick-last-libs(x86-64) = 6.9.3.3-1.el7.remi
Installed: ImageMagick-last-libs-6.9.3.4-1.el7.remi.x86_64 (@remi-safe)
ImageMagick-last-libs(x86-64) = 6.9.3.4-1.el7.remi
Available: ImageMagick-last-libs-6.9.3.2-1.el7.remi.x86_64 (remi)
ImageMagick-last-libs(x86-64) = 6.9.3.2-1.el7.remi
Available: ImageMagick-last-libs-6.9.3.3-1.el7.remi.x86_64 (remi)
ImageMagick-last-libs(x86-64) = 6.9.3.3-1.el7.remi
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest

Feature request: possibility to define extra packages to be installed

Use drush-shim instead of installing drush globally

This is suggested officially by drush install instructions.
Will use project specific drush versions by default.

Note, makes drush not work outside project folders!

pip: command not found

Using a d8 box:

[vagrant@local current]$ pip install git-up
-bash: pip: command not found

Please could you include pip by default?

It'd also be nice to upgrade from Python 2.7.5 (released 15th May 2013) to the most recent 2.7.13 (released 17th December 2016).

Allow automated updates of ip addresses in real_ip_from

Services like Cloudflare update their ip address ranges regularly so we need to have a way to automatically update the ip addresses defined in real_ip_from variable. We should also make sure that the list does not get overridden with old values when we provision some changes to configs via Ansible run.

I think we can use Cloudflare here as an example to start building from. They offer their ip address ranges as a simple txt file that can be found from here https://www.cloudflare.com/ips/

geerlingguy/centos7 1.1.3 box is no longer available

Currently we force the vagrant box version to 1.1.3, because 1.1.4 had issues, but 1.1.3 version is no longer available. This means projects like https://github.com/wunderkraut/wundertools that depend on wundermachina are unusable for any developers who don't have the old 1.1.3 version of the box available to them.

Error "with_dict expects a dict" in database role in centos6 branch with ansible 2.2

TASK [database : Setup databases] **********************************************
fatal: [192.168.10.48]: FAILED! => {"failed": true, "msg": "with_dict expects a dict"}

Related upstream error ansible/ansible#17636

Define and use semantic versioning scheme

We should have controlled releasees using e.g. tags to fix specific releases to make it easier to pin versions in projects to a specific tag. @mikaelkundert

Web server role should create webroot folders

... if not exists

PHP: install / enable APCu module by default

See wunderio/WunderTools#125

Make Nginx client_max_body_size configurable

We have a project where would need to have the Nginx client_max_body_size setting configurable, but I don't think it's currently possible

We should use dhcp to assign box ip

Xdebug role not working

A few developers notice this while provisioning.

Related errors below:

TASK [devtools : PHP | Set up xdebug.ini] ************************************** failed: [default] (item=({u'section': u'XDebug'}, {u'key': u'xdebug.remote_enable', u'val': 1})) => {"failed": true, "item": [{"section": "XDebug"}, {"key": "xdebug.remote_enable", "val": 1}], "msg": "unsupported parameter for module: create"} failed: [default] (item=({u'section': u'XDebug'}, {u'key': u'xdebug.remote_handler', u'val': u'dbgp'})) => {"failed": true, "item": [{"section": "XDebug"}, {"key": "xdebug.remote_handler", "val": "dbgp"}], "msg": "unsupported parameter for module: create"} failed: [default] (item=({u'section': u'XDebug'}, {u'key': u'xdebug.remote_connect_back', u'val': 1})) => {"failed": true, "item": [{"section": "XDebug"}, {"key": "xdebug.remote_connect_back", "val": 1}], "msg": "unsupported parameter for module: create"} failed: [default] (item=({u'section': u'XDebug'}, {u'key': u'xdebug.remote_port', u'val': 9000})) => {"failed": true, "item": [{"section": "XDebug"}, {"key": "xdebug.remote_port", "val": 9000}], "msg": "unsupported parameter for module: create"} failed: [default] (item=({u'section': u'XDebug'}, {u'key': u'xdebug.remote_autostart', u'val': 0})) => {"failed": true, "item": [{"section": "XDebug"}, {"key": "xdebug.remote_autostart", "val": 0}], "msg": "unsupported parameter for module: create"}

Add xdebug support to hhvm

Just need to update some configs

Xhprof for devtools

Please add xhprof to devtools playbook, so that we can debug performance related issues. Test it with devel module, it needs two variables - path to xhprof logs and url to xhprof html page.

Xhprof might save log files in jailed tmp folder, so you have set PrivateTmp to false in php-fpm systemd settings. If there is any way to avoid it, leave the PrivateTmp on.

500 error pages are blank

I took latest master branch and configured Varnish. In my Vagrant environment the error pages aren't rendering anything when I take down memcached service (for testing the error page).

$ curl -v -k https://example.com
* Rebuilt URL to: https://example.com/
*   Trying 192.168.10.172...
* Connected to example.com (192.168.10.172) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate: example.com
> GET / HTTP/1.1
> Host: example.com
> User-Agent: curl/7.43.0
> Accept: */*
> 
< HTTP/1.1 503 Internal Error
< Date: Thu, 30 Jun 2016 14:47:58 GMT
< Server: Varnish
< X-Varnish: 32782
< Content-Length: 0
< Connection: close
< 
* Closing connection 0

In varnishlog

*   << Request  >> 32782     
-   Begin          req 32781 rxreq
-   Timestamp      Start: 1467298078.173739 0.000000 0.000000
-   Timestamp      Req: 1467298078.173739 0.000000 0.000000
-   ReqStart       127.0.0.1 19798
-   ReqMethod      GET
-   ReqURL         /
-   ReqProtocol    HTTP/1.0
-   ReqHeader      Host: example.com
-   ReqHeader      User-Agent: curl/7.43.0
-   ReqHeader      Accept: */*
-   ReqHeader      X-Forwarded-For: 192.168.10.1
-   ReqHeader      X-Forwarded-Host: example.com
-   ReqHeader      X-Forwarded-Server: example.com
-   ReqUnset       X-Forwarded-For: 192.168.10.1
-   ReqHeader      X-Forwarded-For: 192.168.10.1, 127.0.0.1
-   VCL_call       RECV
-   VCL_acl        MATCH upstream_proxy "127.0.0.1"
-   VCL_acl        MATCH upstream_proxy "127.0.0.1"
-   ReqUnset       X-Forwarded-For: 192.168.10.1, 127.0.0.1
-   ReqHeader      X-Forwarded-For: 192.168.10.1, 127.0.0.1
-   ReqHeader      Surrogate-Capability: key=ESI/1.0
-   VCL_return     hash
-   VCL_call       HASH
-   VCL_return     lookup
-   Debug          "XXXX MISS"
-   VCL_call       MISS
-   VCL_return     fetch
-   Link           bereq 32783 fetch
-   Timestamp      Fetch: 1467298078.173944 0.000205 0.000205
-   RespProtocol   HTTP/1.1
-   RespStatus     503
-   RespReason     Backend fetch failed
-   RespHeader     Date: Thu, 30 Jun 2016 14:47:58 GMT
-   RespHeader     Server: Varnish
-   RespHeader     Content-Type: text/html; charset=utf-8
-   RespHeader     Retry-After: 5
-   RespHeader     X-Varnish: 32782
-   RespHeader     Age: 0
-   RespHeader     Via: 1.1 varnish-v4
-   VCL_call       DELIVER
-   RespHeader     X-W-Cache: MISS
-   RespHeader     X-W-Cache-Hits: 0
-   RespUnset      Server: Varnish
-   RespUnset      X-Varnish: 32782
-   RespUnset      Via: 1.1 varnish-v4
-   Debug          "VCL_error(503, Internal Error)"
-   VCL_return     synth
-   Timestamp      Process: 1467298078.173966 0.000227 0.000022
-   Timestamp      Process: 1467298078.173971 0.000232 0.000005
-   RespHeader     Date: Thu, 30 Jun 2016 14:47:58 GMT
-   RespHeader     Server: Varnish
-   RespHeader     X-Varnish: 32782
-   RespProtocol   HTTP/1.1
-   RespStatus     503
-   RespReason     Service Unavailable
-   RespReason     Internal Error
-   VCL_call       SYNTH
-   VCL_return     deliver
-   RespHeader     Content-Length: 0
-   Debug          "RES_MODE 2"
-   RespHeader     Connection: close
-   Timestamp      Resp: 1467298078.174002 0.000263 0.000031
-   ReqAcct        212 0 212 141 0 141
-   End

SOLR role is broken

I'm getting this issue:

TASK [solr : ensure solr is running now and on boot] ***************************
fatal: [default]: FAILED! => {"changed": false, "failed": true, "msg": "Could not find the requested service \"'solr'\": "}

Backup.sh template assumes d7 style tables

Drupal 8 doesn't have cache_form or search_index tables that are excluded by default in backup.sh resulting in error. Excluded tables should be better configurable.

Releases for centos6 and centos7

It would be ideal to provide releases, so projects may specify which version of WunderMachina they want to pull (and therefore knows that is compatible to its project).

It seems that we have two major branches, centos6 and centos7. We could have releases for each branch as following example:

centos6-1.2.3
centos7-1.2.3

We also need to discuss that if this can be done, what type of changes is allowed to change the each version segment. I would prefer to follow semantic version standard here.

Letsencrypt creates chain cert named after first domain given

It should be possible to separate / group those e.g. by the top level domains by running the client multiple times, once for each group.

Papertrail error with vagrant provision

TASK [sslterminator : lineinfile] ********************************************** fatal: [default]: FAILED! => {"failed": true, "msg": "ERROR! The conditional check 'papertrail_enabled == True' failed. The error was: ERROR! error while evaluating conditional (papertrail_enabled == True): ERROR! 'papertrail_enabled' is undefined\n\nThe error appears to have been in '/Users/bernt/Sites/altia-industrial/ansible/playbook/roles/sslterminator/tasks/main.yml': line 63, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n# Make papertrail follow appserver logs.\n- lineinfile:\n ^ here\n"}

Add example how to configure MailHog to work

I've gotten information this might be the right configuration:

      - section: PHP
        options:
          - key: sendmail_path
            val: "/usr/sbin/ssmtp -t"```

Make Elasticsearch 5 the default

Elasticsearch 5 has been around for a while, we are using it on many projects, it makes sense to have it as the default.

Note that the "kopf" plugin is not available anymore (plugins that deliver HTML pages have been deprecated for security reasons). Instead we should provide https://github.com/lmenezes/cerebro as a separate role for development environments.

Change certbot to use webroot method

Change certbot to use webroot method for fetching the certificate too. Now it's only used to renew the certificate. https://certbot.eff.org/docs/using.html#webroot

Varnish is required for base and php-fpm roles

Excluding varnish should not prevent those roles from working independently.
The varnish.control_key checks should also check whether varnish itself exists.

Deprecation warnings: bare variables and become_method

Using: Vagrant 1.8.1 , ansible 2.0.1.0 gets deprecation warnings:
[DEPRECATION WARNING]: Instead of sudo/sudo_user, use become/become_user and make sure become_method is 'sudo' (default). This feature will be removed in a future release. [DEPRECATION WARNING]: Using bare variables is deprecated. Update your playbooks so that the environment value uses the full variable syntax ('{{databases}}'). This feature will be removed in a future release.

NewRelic role doesn't work out of the box

Missing application name variable

Dependency-manager might lead errors if dependencies are not met

vagrant-hostmanager requires libffi to be present on a system. Missing libffi will lead to failure of any vagrant command.

Letsencrypt role doesn't add new domains later on for already deployed servers

Letsencrypt currently only adds new domain names in initial deployment. This hasn't been a problem so far because we usually use new servers for everything.

Today I needed to deploy new site into shared production server and couldn't add new domains to letsencrypt even though I used:

letsencrypt_domains:
  - old-domain.com
  - www.old-domain.com
  - new-domain.com
  - www.new-domain.com

This happens because the ansible will add the certs only in initial deploy:

- name: Run initial certificate request only if port 80 is free
  command: "{{ letsencrypt_command }}"
  when: port_80.stdout.find('nginx') == -1

Source: https://github.com/wunderkraut/WunderMachina/blob/master/playbook/roles/letsencrypt/tasks/main.yml#L35-L37

What we should do is this:

vars:
  # Define a command for servers which are already running
  letsencrypt_webroot_command: "{{ letsencrypt_src }}/letsencrypt/letsencrypt-auto certonly --webroot --agree-tos --text -n --expand --email {{ letsencrypt_email }} {% for d in letsencrypt_domains %}-d {{ d }} {% endfor %}"

tasks:
- name: Run initial certificate request only if port 80 is free
  command: "{{ letsencrypt_webroot_command }}"
  when: port_80.stdout.find('nginx') != -1

This should also check the output of:

$ "{{ letsencrypt_src }}/letsencrypt/letsencrypt-auto certificates"

And only run the webroot command if all of the domains are not already in the list so we don't waste valuable request limits from letsencrypt.

Can't make the shell working:)

Sorry to bother you @tharna :).

i tried to do a cd when vagrant ssh, but can't make it work. I'm pretty sure it has done once by inserting the same command directly in my ~/.bashrc, but putting it into provision.sh it fails and i can't understand why :).
I'm referring to the feature/14-cd_upon_login in git :)

Create Centos7 branch

Moved from Ansibleref
Will need several changes:

Packaged Varnish is v4, so need to update vcl etc.
Packaged Nginx includes SPDY support, so consider updating vhost configs with SPDY
Packaged PHP is 5.4.x, so support for alternatives will need to be added
Mailcatcher install from Centos 6.x Ansibleref doesn't seem to work on 7, so consider MailHog cf. #1
Update Ansibleref my.cnf to work with MariaDB 10.x

Building the latest version failed. I'll still try and get some newer version built.