centerforopenscience / cos-ansible-base

Our base ansible scripts for deployment and orchestration

cos-ansible-base

• Issue/Task tracking: https://huboard.com/CenterForOpenScience/cos-ansible-base

Requirements

• ansible >= 2.1
• virtualbox
• vagrant >= 1.6
• invoke (Python task execution library)
• python >= 2.7 or >= 3.4 with pip
• fwknop >= 2.6.5
• At least 2Gb of memory

Installing Ansible and Vagrant on Mac OSX with homebrew

Virtualbox and Vagrant can be installed with homebrew cask. If you have homebrew installed, run the following from the project directory:

$ brew bundle

Installing python requirements

Invoke can be installed with pip

$ pip install invoke
$ pip install -r requirements.txt

Getting cos-ansible-base

To clone cos-ansible-base locally, run:

$ git clone https://github.com/CenterForOpenScience/cos-ansible-base --recursive

The --recursive option ensures that all submodules will be cloned.

Vagrant setup

Once you have Vagrant and ansible installed, follow these steps:

• Generate your ssh key with ssh-keygen

$ ssh-keygen

• Run vagrant up <machine_to_run>. Then will start the VM provision with invoke vprovision. Use the --limit (or -l) option to limit to a specific group.

# Start the osf-staging server
$ vagrant up osf-staging
# Provision the osf-staging server
$ invoke vprovision --limit osf-staging

SSH

To ssh into your Vagrant box, run vagrant ssh <box-name>:

$ vagrant ssh osf-staging

Generating passwords

To generate a password, run

$ invoke genpass

This crypted password can be used by the generic-users role in a group_vars file.

Running playbooks

Playbooks can be run with the ansible-playbook command. You need to specify which inventory file with the -i option as well as a user with the -u option. Run in sudo mode with -s

$ ansible-playbook security.yml -i vagranthosts -u sloria -s

Or, using invoke for shorthand:

$ invoke play security.yml -i vagranthosts -u sloria

Provisioning

The site.yml playbook is responsible for provisioning all servers in an inventory.

Run it like so:

$ ansible-playbook site.yml -i vagranthosts -u sloria -s

The above command runs the site.yml playbook using the vagrant inventory file with user sloria in sudo mode.

Or, if you prefer to use invoke:

$ invoke provision -i vagranthosts -u sloria

NOTE: You can also provision the vagrant box by running invoke vprovision with no arguments.

Many of the roles use variables defined in their defaults/main.yml file. You can override these on the command line with the -e option:

$ ansible-playbook site.yml -i vagranthosts -u sloria -e "ssh_test=false"

or, equivalently:

$ invoke provision -u sloria -e "ssh_test=false"

The above would temporarily disable SSH configuration testing.

Setting up for Single Packet Authorization

If using encryption and HMAC keys, execute on client:

fwknop -A tcp/22 -a PUBLIC_CLIENT_IP -D TARGET_SERVER_IP --key-gen --use-hmac --save-rc-stanza

Print your newly generated keys:

grep KEY ~/.fwknoprc

And add to Single Packet Authorization Server access configuration /etc/fwknop/access.conf:

SOURCE              ANY
KEY_BASE64          [KEY]
HMAC_KEY_BASE64     [HMAC_KEY]

Setting up for OSF deployment

You will need to set up agent forwarding in order to be able to properly authenticate with Github over SSH in ansible. To do so, add the following to your ~/.ssh/config/ file.

Host staging.osf.io
    HostName 66.228.46.171
    User sloria
    ForwardAgent yes

Host osf.io
    HostName 69.164.210.152
    User sloria
    ForwardAgent yes

Deployment

The deploy.yml script is used to deploy the OSF.

To deploy on staging:

$ invoke deploy_staging -u sloria

You will be prompted for the branch to checkout on staging.

To deploy to production:

$ invoke deploy_production -u sloria

This will deploy to the production server, checking out the master branch from Github.

COS is Hiring!

Want to help save science? Want to get paid to develop free, open source software? Check out our openings!

cos-ansible-base's Issues

[role] uwsgi

Ensure unique cookie names across OSF deployments

Just a quick note for when we get back to developing the OSF role: we should ensure that each OSF deployment has a unique cookie name defined in website.settings. Otherwise we'll get strange behavior in IE9, which allows multiple cookies with the same name.

h/t @jmcarp

[role] elasticsearch

@fabianvf is currently working on this

csf not installing

TASK: [csf | csf | install] ***************************************************
failed: [192.168.111.111] => {"cmd": "cd '/tmp/csf/;' sh install.sh", "failed": true, "item": "", "rc": 2}
msg: [Errno 2] No such file or directory

Testing LFD

I have manually tested LFD like so:

• Temporarily allow password authentication by changing groupvars
• set LF_SSHD_PERM = 30 in the csf configuration file
• ssh sloria@localhost -p 2222 -o PubkeyAuthentication=no
  • do at least 5 bad logins
• Check that log is saved to /var/log/lfd.log

Can anyone think of a better way to test this?

[role] nginx

Test that root login is disabled

[role] rabbitmq

I believe @chrisseto has already made a role for this. Posting this for record-keeping

[role] mongodb

"lookup" function not working for pub keys

For some reason, using the lookup function for public ssh keys doesn't seem to be working.

genericusers_users:
   # ...
  ssh_keys:
     - "{{ lookup('file', 'group_vars/pubkeys/sloria.pub') }}"

When the genericusers role runs, the ssh keys are not copied (the ssh_keys variable appears to be an empty list.

@lyndsysimon is seeing the same thing.

For now, I'm just copying the pubkeys directly.

[role] znc

@asmacdo is working on this

Status?

Hi there,

I'm looking into deploying osf on a test server and tried a few things with this repo but didn't get very far. Before I spend more time looking into it, would you mind letting me know what is the current status of the scripts here, or if there is a particular branch I should use to have some known good configuration?

I've tried to use deploy.yml but it seems that the osf role has been moved into the (legacy) folder on master.

I've tried to use docker-osf.yml but it seems relatively bleeding edge? I'm happy to chase this as ansible/docker would be my preferred approach if that's useful.

Cheers!

Jun