arnoo / git-deliver Goto Github PK

Some error checks after git operations might have been broken by the introduction of output indentation (piping to sed causes the exit status of the pipe to be that of sed and not the git operation).

Tests need to be added and the error checks fixed.

Useful if the remote-init scripts have changed

Provide an option for automate gc, could be wanted

There's a bit of code (check any version in the Git repo from April 2013) for signing the delivery commit with PGP, This needs to be finished, tests added and reincorporated into git-deliver.

We should probably send some output to $PAGER instead of directly to stdout/err. (remote-status when reading the full history for example).

The deliver directories on the target server are on a special *nix group. All users that can deliver are on this group.

When a git deliver is done, new objects may (are) created in the "objects" git directory.

The problem is: those files are created using default group, not with the special git deliver group.

git-deliver should be able to autodetect target directory group and chgroup it.

Note: config has sharedrepository = 1

Maybe add a --reinit option to --init ?

The disk-space.sh script does a "git archive" to mesure disk space needed.

But, if the repo has some git submodules to clone, the size taken (possibly far bigger) will not be taken into account.

If we deliver to multiple remotes at once, how do we deal with failure ?

What sort of "atomicity" can we attain ?

What impact would there be on stage scripts ?

Core disk space has a bug in some cases, maybe with some space formats.

Running scripts for stage pre-delivery
   pre-delivery/01-core-disk-space.sh
      Delivery will require 11.3086 MB on remote, 16.6601 GB available
bash: line 47: [[: 1.78886e+10: syntax error in expression (error token is ".78886e+10")
      Not enough disk space abvailable on remote
   Script returned with status 1

First, thanks for this project ! I've hope than could I use it.

My issue: When I run

git deliver --init-remote name user@server:/path/to/dir

I receive

/path/git-deliver/deliver.sh: ligne 514: /tmp/git_deliver_ssh_wrapper_8370.sh: Permission non accordée
/path/git-deliver/deliver.sh: ligne 514: /tmp/git_deliver_ssh_wrapper_8370.sh: Permission non accordée
/path/git-deliver/deliver.sh: ligne 514: /tmp/git_deliver_ssh_wrapper_8370.sh: Permission non accordée
Error creating root directory on remote
/path/git-deliver/deliver.sh: ligne 422: /tmp/git_deliver_ssh_wrapper_8370.sh: Permission non accordée

I've missed a step ?

deliver echoes to stdout or to stderr a bit randomly.

We need to figure out a policy and make the code consistent with it.

I'm evaluating the use of git-deliver and I'm wondering if it could be used with multi host deployement, and if yes, how ?

If the application is deployed on several host, there is some chance that I don't want to run git-deliver to each host. Plus, I guess that some scripts won't be necessary more than once (a database migration should not be made for each host)

Any thoughts about that ?

I run bundle install, assets:precompile and db:migrate using the .remote.sh feature, but mgirate always fails, it tries to rerun migrations and throws "IrreversibleMigration" exception, leaving the production schema inconsistent. If I run this command inside the machine, it runs correctly.

Hi,

Is there an option in git-deliver to change the default ssh port ?

Thanks

Check that the exit happens and that the exit code is right.

Hi,

My question sounds maybe like a troll, I'm not here for that, this is just what will be better with git-deliver for example ?

I haven't tested it yet, but for the compatibility you just have to install git that's all ? (It can be a very good thing on Windows, because when you want to use Capistrano and you don't need a ruby env, install it on a PC it's just a waste of time…)

During humantalks, you said to me that the gc manage 3 folders.

It could be great to add the ability to manage it

[_delivered 98df5dc] Git-deliver automated commit
Switching the 'current' symlink to the newly delivered version.
No scripts for stage post-symlink
/tmp/git-deliver-3728.16515: line 1: Delivery: command not found
/tmp/git-deliver-3728.16515: line 4: Delivery: command not found
/tmp/git-deliver-3728.16515: line 6: running: command not found
/tmp/git-deliver-3728.16515: line 7: running: command not found
/tmp/git-deliver-3728.16515: line 8: running: command not found
/tmp/git-deliver-3728.16515: line 9: running: command not found
/tmp/git-deliver-3728.16515: line 10: running: command not found
/tmp/git-deliver-3728.16515: line 11: running: command not found
/tmp/git-deliver-3728.16515: line 12: running: command not found
/tmp/git-deliver-3728.16515: line 13: running: command not found
/tmp/git-deliver-3728.16515: line 14: [email protected]: No such f
ile or directory
/tmp/git-deliver-3728.16515: line 15: running: command not found
Delivery complete.
You might want to publish tag delivered-toto-2013-03-12_09-47-47 (git push o
rigin delivered-toto-2013-03-12_09-47-47 ?)

$ git deliver --status

Remote toto:
bash: line 5: realpath: command not found

See https://news.ycombinator.com/item?id=7595499

At least :

set -o nounset

#!/usr/bin/env bash

What do you think about an installation via brew (the missing PM for OS X : https://github.com/Homebrew/homebrew) ?

Like :

$ brew install git-deliver

Create a Formulae with Ruby :

$ brew create http://foo.com/bar-1.0.tgz Created /usr/local/Library/Formula/bar.rb
source : http://brew.sh/

We need a way for developers to document their presets and for users to access the documentation for a given preset.

Use of getopt by git-deliver does not work on mac os.

Log:

$ git deliver --status myremote
flags:ERROR short flag required for (source) on this platform
flags:ERROR short flag required for (init) on this platform
flags:ERROR short flag required for (init-remote) on this platform
flags:ERROR short flag required for (list-presets) on this platform
flags:ERROR short flag required for (status) on this platform
flags:ERROR short flag required for (gc) on this platform
flags:ERROR short flag required for (rollback) on this platform
flags:ERROR short flag required for (color) on this platform
Git-deliver: illegal option -- -
illegal option -- s
illegal option -- t
illegal option -- a
illegal option -- t
illegal option -- u
illegal option -- s
-- myremote
Usage :
git deliver
git deliver --rollback [DELIVERY]
git deliver --gc
git deliver --init [PRESETS]
git deliver --init-remote <REMOTE_NAME> <REMOTE_URL>
git deliver --list-presets
git deliver --status [REMOTE]

See what we can do about that, at the very least mention it in the readme.

When doing a git deliver --gc, access is denied when git-deliver wants to delete files from deliveries done by other users.

After a "chmod -R g+w" on those directories, git deliver --gc works fine agains.

At delivery, git-deliver should chmod g+w on its working directory.

We could use some possibilities for configuration : auto-push the delivery tag (and to where), don't open the editor with the delivery log, etc.

This should probably be in the form of a "deliver" section in the git configuration files ?

Hey Guys, not sure if it is really a bug! I think it is because of my stupidity.

1. First of all I ran git deliver --init on my local machine and got this response Setting up core preset.
2. Then I tried to add the remote with git deliver --init-remote production [email protected]:/www/test/public and get this weird error:

ssh: Could not resolve hostname +1246AaCfgKkMNnqsTtVvXxYb:c:D:e:F:i:L:l:m:O:o:p:R:S:w:z: nodename nor servname provided, or not known
ssh: Could not resolve hostname +1246AaCfgKkMNnqsTtVvXxYb:c:D:e:F:i:L:l:m:O:o:p:R:S:w:z: nodename nor servname provided, or not known
ssh: Could not resolve hostname +1246AaCfgKkMNnqsTtVvXxYb:c:D:e:F:i:L:l:m:O:o:p:R:S:w:z: nodename nor servname provided, or not known
\E[31mError creating root directory on remote

No idea what went wrong :(

 $ git rev-parse HEAD
 ef4ec59d9a32ef6513a2f3b1ae5ebb30454840b2

 $ git deliver mymachine master
 Current version on mymachine:
   d9716892bbee41202b7bda215b2c51866df1762e
   * not delivered with git-deliver *
 Running scripts for stage pre-delivery
   pre-delivery/01-core-disk-space.sh
   pre-delivery/01-core-mem-free.sh
 Pushing necessary commits to remote
   To ssh://myurl
    ! [rejected]        master -> master (non-fast-forward)
   error: failed to push some refs to 'ssh://myurl'
   hint: Updates were rejected because the tip of your current branch is behind
   hint: its remote counterpart. Merge the remote changes (e.g. 'git pull')
   hint: before pushing again.
   hint: See the 'Note about fast-forwards' in 'git push --help' for details.
 Creating new delivery clone
   Cloning into '/path/to/deliver/2013-04-30_09-50-30_ef4ec5_master'...
   done.
 
 No scripts for stage post-checkout
 [_delivered 0ec924e] Git-deliver automated commit
 Switching the 'current' symlink to the newly delivered version.
 Running scripts for stage pre-symlink
   pre-symlink/01-stop-puppetmaster.remote.sh
      Puppetmaster is running. Try to stop it...
      Puppetmaster is now stopped.
 Running scripts for stage post-symlink
   post-symlink/01-start-puppetmaster.remote.sh
      Puppetmaster is not running. Try to start it...
      Puppetmaster is now started.
 Tagging delivery commit
 Delivery complete.
 You might want to publish tag delivered-mymachine-2013-04-30_09-50-30 (git push o
rigin delivered-mymachine-2013-04-30_09-50-30 ?)

=> connect to remote

 $ ls -l /path/to/deliver
 current -> 2013-04-30_09-50-30_ef4ec5_master

 $ cd /path/to/deliver/current
 $ git log
 commit 0ec924ea2fd83b2446b99779c0dcc354ba8462a4
 Author: ...
 Date:   Tue Apr 30 11:51:10 2013 +0200

    Git-deliver automated commit

 commit 68d46030c847a744d5a34b65e4d5d0ed2a251d7e
 Author: ...
 Date:   Tue Apr 30 11:29:42 2013 +0200

    My comment
 

As you can see, the directory is correctly called "ef4ec5" (that is the commit I wanted to deliver), BUT if I go inside, it is commit "68d46".

(for the record, it comes from a git commit --amend on the previous commit I thought has not been commited)

The documentation should be accessible via --help, just like it is for other git commands.

In some situations (which ? seen with msys (client) / debian wheezy (server))

check-disk-space crashes at the ssh connection, and does not really check the available disk space on the remote. Additionnaly, the delivery continues, which could indicate a problem in stage-script error checking

We could use a more reliable way to log deliveries. Local tags might get lost, We could at least push them to the base delivery repo ?

After dozens of deliveries, git-deliver begins to slow down.

The problem is that the remote_status function get all deliveries from the remote, even if the whole list is not needed to deliver.

If we run "git deliver gc", git deliver is fast again.

There should be an option for not requiring the user to save the log file, just write it.

There are probably a few dependencies between tests. These should be removed to ensure that each test is accurate.

Intersect between #12 and #5: if a git submodule checkout fails, git-deliver does not stop.

Example (with github down - yes!):

$ git deliver myremote master
Current version on myremote:
   c090395c5a3f60b79332971bb9260511172ab6ef
   delivered Thu, 2 May 2013 10:40:15 +0200
   by myself 
Running scripts for stage pre-delivery
   pre-delivery/01-core-disk-space.sh
   pre-delivery/01-core-mem-free.sh
Pushing necessary commits to remote
   Everything up-to-date
Creating new delivery clone
   Cloning into '/my/path/delivered/2013-05-02_09-15-55_c09039_master'...
   done.
   Checking out files...
   Switched to a new branch '_delivered'
   Submodule 'modules/apache' (https://github.com/soluvas/puppet-apache.git) registered for path 'modules/apache'
   Submodule 'modules/apt' (https://github.com/puppetlabs/puppetlabs-apt.git) registered for path 'modules/apt'
   Submodule 'modules/concat' (https://github.com/ripienaar/puppet-concat.git) registered for path 'modules/concat'
   Submodule 'modules/cron' (https://github.com/torrancew/puppet-cron.git) registered for path 'modules/cron'
   Submodule 'modules/firewall' (https://github.com/puppetlabs/puppetlabs-firewall.git) registered for path 'modules/firewall'
   Submodule 'modules/jenkins' (git://github.com/rtyler/puppet-jenkins.git) registered for path 'modules/jenkins'
   Submodule 'modules/lvm' (https://github.com/puppetlabs/puppetlabs-lvm.git) registered for path 'modules/lvm'
   Submodule 'modules/maven' (https://github.com/maestrodev/puppet-maven.git) registered for path 'modules/maven'
   Submodule 'modules/postgresql' (https://github.com/puppetlabs/puppet-postgresql.git) registered for path 'modules/postgresql'
   Submodule 'modules/sonar' (https://github.com/maestrodev/puppet-sonar.git) registered for path 'modules/sonar'
   Submodule 'modules/stdlib' (https://github.com/puppetlabs/puppetlabs-stdlib.git) registered for path 'modules/stdlib'
   Submodule 'modules/wget' (https://github.com/maestrodev/puppet-wget.git) registered for path 'modules/wget'
   Cloning into 'modules/apache'...
   Submodule path 'modules/apache': checked out '10d2d73907c5a5600c0dd8eb9c870715ef2a58e1'
   Cloning into 'modules/apt'...
   error: The requested URL returned error: 503 while accessing https://github.com/puppetlabs/puppetlabs-apt.git/info/refs
   fatal: HTTP request failed
   Clone of 'https://github.com/puppetlabs/puppetlabs-apt.git' into submodule path 'modules/apt' failed
No scripts for stage post-checkout
[_delivered 89440e4] Git-deliver automated commit
 1 file changed, 1 deletion(-)
 delete mode 160000 modules/apt
Switching the 'current' symlink to the newly delivered version.
Running scripts for stage pre-symlink
   pre-symlink/01-stop-puppetmaster.remote.sh
      Puppetmaster is running. Try to stop it...
      Puppetmaster is now stopped.
Running scripts for stage post-symlink
   post-symlink/01-start-puppetmaster.remote.sh
      Puppetmaster is not running. Try to start it...
      Puppetmaster is now started.
Tagging delivery commit
Delivery complete.
You might want to publish tag delivered-myremote-2013-05-02_09-15-55 (git push o
rigin delivered-myremote-2013-05-02_09-15-55 ?)

A lock should be added to prevent simultaneous deliveries to the same server.

This may be a bit outside the scope of the project. To be discussed.

I am not sure why this error occurs.

I also couldn't get --init and --init-remote to work without first creating and doing a git init bare to get git-deliver to say that the remote is ready for deliveries.
Is there something I am doing wrong?

Let me know if I need to provide more details

More info about multiplexing can be found here : http://en.wikibooks.org/wiki/OpenSSH/Cookbook/Multiplexing

I have a remote origin branch and a remote production branch where I will deploy using git-deliver.

Right now, I'm seeing this error constantly when trying to deploy:

Pushing necessary commits to remote
   To [email protected]:/home/ec2-user/test
    ! [rejected]        master -> master (fetch first)
   error: failed to push some refs to '[email protected]:/home/ec2-user/test'
   hint: Updates were rejected because the remote contains work that you do
   hint: not have locally. This is usually caused by another repository pushing
   hint: to the same ref. You may want to first integrate the remote changes
   hint: (e.g., 'git pull ...') before pushing again.
   hint: See the 'Note about fast-forwards' in 'git push --help' for details.

I've tried running git pull production master and git pull origin master but still gives me this :(

Maybe with an option to see the complete diff through $PAGER since the last delivery, and an option to check this diff against the ones generated by running the same update on other environments ?

On mingw, after git deliver --gc my-remote-repo

[snip]
Removing /my/remote/repo/delivered/2013-04-23_13-53-19_417cfc_master
awk: line 4: syntax error at or near *
29 version(s) removed, freed

Is a separate program a good idea ? What are the alternatives ? RIght now, this aspect is pretty messy.

We need to check the git version on both the client and server to make sure that they are adequate, warn the user and stop work if they aren't.

It could be nice to push only the necessary commits when delivering, instead of always pushing the whole branch.

In some conditions (which ?), --status says "uncommited changes" when there are actually no changes to the delivery.