Coder Social home page Coder Social logo

dreadnot's Introduction

Dreadnot - deploy without dread

Build Status npm Inline docs License

Dreadnot is a 'one click' deploy tool written in Node.js.

Dreadnot was heavily inspired by Etsy's Deployinator.

Deploy all the branches

Configuration

Dreadnot loads its configuration from a javascript file:

exports.config = {
  // The name of this Dreadnot instance, used for display
  name: 'Example Dreadnot',

  // Each Dreadnot instance supports one environment such as 'dev', 'staging'
  // or 'production'
  env: 'staging',

  // The data root Dreadnot will use
  data_root: '/var/dreadnot',

  // Base URL to access dreadnot (used in IRC, email, Hipchat)
  default_url: 'http://example.com',

  // Dreadnot uses an htpasswd file (with support for bcrypt, md5 and sha1) for auth
  htpasswd_file: '/etc/dreadnot/htpasswd',

  // Each stack represents a code base that should be deployed to one or more regions
  stacks: {

    // For a stack named 'webapp', there should be a 'webapp.js' file in the
    // stacks directory
    webapp: {
      // What branch to look in for the latest revision of the code base
      tip: 'master',

      // How long to cache the latest revision of the code base
      tip_ttl: 120 * 1000,

      // What regions this stack should be deployed to
      regions: ['ord1'],

      // Stacks should implement dryrun for testing
      dryrun: true
    }
  },

  // The GitHub organization you provide is used to build URLs for your stacks
  github: {
    organization: 'racker'
  },

  // Plugins provide optional functionality such as notifications. Any plugins
  // that are not configured won't be used.
  plugins: {

    // An IRC notification plugin
    irc: {
      nick: 'staging-dreadnot',
      channels: {'irc.freenode.net': ['#public-channel', '#private-channel pass']}
    },

    // An email notification plugin
    email: {
      server: {
        user: '[email protected]',
        password: '',
        host: 'smtp.example.com',
        ssl: true
      },
      to: '[email protected]',
      from: '[email protected]'
    },

    // A Hipchat notification plugin
    hipchat: {
      name: 'Dreadnot',
      apiKey: '123456789abcdefg',
      rooms: [
        1234,
        5678
      ]
    }
  }
};

Stacks

Dreadnot looks in a directory (by default ./stacks, but this can be changed from the command line) for "stack files". A stack file is simply a javascript file that exports

  • A get_deployedRevision function which takes an object containing environment and region fields, and a callback taking (err, deployed_revision).
  • A targets hash that maps target names to lists of task names. Currently, the only supported targets are deploy, which defaults to ['task_preDeploy', 'task_deploy', 'task_postDeploy'], and finally which does not have a default value. You should use the finally target if there are any tasks you would like to run every time, regardless of the success or failure of the tasks in deploy (i.e. re-enable monitoring alerts). The tasks in the finally target itself are each dependent on the success of the last task in the target, so an error in one will prevent the rest from running.
  • One or more "task functions" whose names are prefixed with task_. Each task function takes:
    1. A "stack" object. The most useful fields on the stack are stackConfig which contains the config for this particular stack, and config which contains the global config.
    2. A "baton" object. Each task executed during a run of a given target receives the same baton object. By default, it contains a log field with methods such as debug, info, and error that can be used to log output to deployment log and web view.
    3. An "args" hash with dryrun, environment, region, revision and user, each of which is a string.
    4. A "callback" function that should be called without arguments on completion, or with a single error object if an error occurs.

Tasks

In the configuration used by Rackspace Cloud Monitoring, a deployment looks something like:

  1. Build: verify that the requested revision has been successfully built and that all tests pass.
  2. Prepare: remove the region being deployed from the load balancer rotation, redirecting all traffic to another region.
  3. Execute: use a chef search to locate all servers in the region, then ssh to each in parallel to upgrade the code.
  4. Validate: execute checks against each upgraded service to verify that it is functioning properly.
  5. Restore: restore the region to the load balancer rotation.

Imporantly, Dreadnot knows nothing about the hosts to which it is deploying - if it did, we would have to modify our Dreadnot configuration every time we added or removed a machine from our environment. Instead, we rely on chef (although anything that knows about your servers will work) to give us an up-to-date list of all hosts in a given region. In smaller deployments it might be suitable to hardcode a list of hosts.

FAQ

Does Dreadnot support SVN?

Dreadnot supports Node.js - you can use any technology or topology that suits you, as long as you can find a library for it.

Development

To create a development environment, you'll need Vagrant and Virtualbox. Once installed, run:

    vagrant up

Then visit http://localhost:8000

Log into the VM by running and running common commands:

    vagrant ssh
    sudo cat /var/log/upstart/dreadnot.log

Running Dreadnot

    npm install dreadnot -g

Alternatively, when developing, you can find a compiled dreadnot binary in the bin folder.

Dreadnot takes a number of options on the command line. The defaults are:

  dreadnot -c ./local_settings.js -s ./stacks -p 8000

This will start dreadnot with the specified config file and stack directories, listening on port 8000.

dreadnot's People

Contributors

andris9 avatar bison avatar chriskuehl avatar creationix avatar dispalt avatar dlobue avatar dylanahsmith avatar ebensing avatar fourk avatar gdusbabek avatar harageth avatar jasonthomas avatar jguice avatar jirwin avatar kami avatar kaustavha avatar ksheedlo avatar manobi avatar mburns avatar oremj avatar philips avatar phoebesimon avatar pquerna avatar robert-chiniquy avatar rphillips avatar russellhaering avatar simonvetter avatar stufflebear avatar threetee avatar zzantozz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dreadnot's Issues

Support for knife locations other than /usr/bin

Currently the knife path is hard-coded to /usr/bin/knife. The knife path should probably be taken from an environment variable first, then default to /usr/bin/knife if no other path is specified. Pull request coming, I created this issue just to have something to track it against. :)

support revert

If a deploy goes badly, we need an 'undo' option.

  1. deploy the previously successful version?

Dry-run support

For debugging, it would be useful to have a dry-run, where Dreadnot would output all the tasks and commands it would run, without actually execing anything.

It could grab the shell commands it would run and confirm the file mode was compatible to be executed. (owned by the dreadnot user, etc).

Automate Rolling Rollouts

  1. Create a top level button for a Stack, Deploy all regions. Have a drop down with 'pause times' in it. Initial options are 5 minutes, 30 minutes, 1 hour, 1 day. Later we can make more complex scheduling.

  2. When this button is pressed, save out a json file for this stack.

  3. Start the first region. Keep track of the deployment like normal.

  4. When it finishes, and it is part of a global rolling deployment, start a timer for the next region.

  5. On startup, read the json in, look for an incomplete global rollout, continue it.

recover from timeout

If a deployment job "hangs", there doesn't seem to be a way to recover other than restarting the process.

Deploy warnings can crash the dreadnot irc bot

After setting the below as the deploy warning, the irc bot crashed and dreadnot returned me to /warning with no css.

Be sure to leave the following sha1s in staging until 2/14: 3847282d3274d79eb80bf728ce6eb0f5f16e0295 8bd8efc8301cd7527e4285aea43667582ccacf99 75734eb573a1d12c4c99023b0c9a040233fbe75d 4f991b5a25312d5eecaf34b623de90e93f7df694

Stack names should be independent of the github URL of the repository

We hardcode the way we construct github URls from a stack. Currently, we have ele.js, noit.js and chef.js stacks which map nicely to github.com/racker/{ELE,CHEF,NOIT}.git

We want to be able to have a test.js stack or dev.js stack that relies on a github repository that has nothing to do with test.git or dev.git.

Dreadnot should add an option to specify the repository in the stack config and prefer that over github.com/racker/FOO.git

use bcrypt in addition to md5 scheme

For crypting passwords it should support bcrypt in addition to of the crazy md5 scheme. (The big feature of bcrypt is "option to tell how slow it should be").

There's also an npm package.

(I realized this was to be compatible with other htpasswd implementations that uses the md5crypt thing and updated the ticket).

Deployment log does not show in Safari

When viewing an in-progress deployment in Safari 5.1.1 on Max OS 10.6, the deployment log is not displayed. In my Javascript console, I see the following error:

Error during WebSocket handshake: location mismatch: wss://reach-dreadnot.k1k.me/socket.io/1/websocket/1373491425311499637 != ws://reach-dreadnot.k1k.me/socket.io/1/websocket/1373491425311499637

warning causes UX lockout

Setting a deploy warning does not show a modal window to be accepted/denied. This blocks deploys.

add 'stack' page

You can view a region (that has many deploys) in a stack but you cannot view a stack (that has many regions). ie https://dreadnot.example.com/stacks/chef/regions/dfw

Add a page https://dreadnot.example.com/stacks/chef that shows its regions.

Update knife tool to return FQDNs instead of IPs

Currently, Dreadnot wraps the knife command to query Chef-Server to get values like the array of IPs to use for a given step.

We should tweak Dreadnot to return hostname / FQDNs instead of IP address. This way, when we ssh $(ip) $(cmd), we'll be doing a DNS lookup.

I tried applying #137 manually, but it didn't appear to make a difference. CC @racker/rackspacemonitoringbombardiers

Dreadnot breaks if a stack is missing it's js file

If a stack: {"webapp": {}} is missings its corresponding stacks/webapp.js file, dreadnot crashes and is usuable.

A better failure would be to list all the valid stack configs, and show an error message that 1 or more stacks are misconfigured.

On Error Target for a stack

I think we should add a 'deploy_error' to the targets of a stack.

This would be treated like a normal stack, but only ran if a deployment failed. it would be handed information about the failure (as much as possible).

This would let people rollback or do other processes automatically.

Documentation for dreadnot

Hi
I need some help in using the open source tool "dreadnot". I downloaded the tool . My goal is to use dreadnot for automatic deployment of software, configuration, test.
If that's possible with dreadnot - I need to know the steps to set up the same .
Where do I click to see the documentation for the tool? ( User manual, Installation guide etc )

Thanks
PR

Using node 0.6.7 using npm to install fails

Following your Readme instructions
npm install dreadnot -g

Fails

npm ERR! error installing [email protected]
npm ERR! error rolling back [email protected] Error: UNKNOWN, unknown error '/usr/local/lib/node_modules/dreadnot'
npm ERR! Unsupported
npm ERR! Not compatible with your version of node/npm: [email protected]
npm ERR! Required: {"node":">= 0.4.1 < 0.5.0"}
npm ERR! Actual:   {"npm":"1.1.0-beta-10","node":"0.6.7"}
npm ERR! 
npm ERR! System Linux 3.0.4-linode38
npm ERR! command "node" "/usr/local/bin/npm" "install" "dreadnot" "-g"
npm ERR! cwd /usr/local/src
npm ERR! node -v v0.6.7
npm ERR! npm -v 1.1.0-beta-10
npm ERR! code ENOTSUP
npm ERR! message Unsupported
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /usr/local/src/npm-debug.log
npm not ok

I also have the debug log file posted here: http://66.228.54.189/npm-debug.txt

make individual errors linkable

20:23:48 <jguice> | wish I could link to a particular clickable error-line in dreadnot

It would be nice to be able to have anchor links to errors in a given Dreadnot deploy.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.