arnoo / git-deliver Goto Github PK
View Code? Open in Web Editor NEWDelivery system based on git push and ssh
License: GNU General Public License v3.0
Delivery system based on git push and ssh
License: GNU General Public License v3.0
Some error checks after git operations might have been broken by the introduction of output indentation (piping to sed causes the exit status of the pipe to be that of sed and not the git operation).
Tests need to be added and the error checks fixed.
Useful if the remote-init scripts have changed
Provide an option for automate gc, could be wanted
There's a bit of code (check any version in the Git repo from April 2013) for signing the delivery commit with PGP, This needs to be finished, tests added and reincorporated into git-deliver.
We should probably send some output to $PAGER instead of directly to stdout/err. (remote-status when reading the full history for example).
The deliver directories on the target server are on a special *nix group. All users that can deliver are on this group.
When a git deliver is done, new objects may (are) created in the "objects" git directory.
The problem is: those files are created using default group, not with the special git deliver group.
git-deliver should be able to autodetect target directory group and chgroup it.
Note: config has sharedrepository = 1
Maybe add a --reinit option to --init ?
The disk-space.sh script does a "git archive" to mesure disk space needed.
But, if the repo has some git submodules to clone, the size taken (possibly far bigger) will not be taken into account.
If we deliver to multiple remotes at once, how do we deal with failure ?
What sort of "atomicity" can we attain ?
What impact would there be on stage scripts ?
Core disk space has a bug in some cases, maybe with some space formats.
Running scripts for stage pre-delivery
pre-delivery/01-core-disk-space.sh
Delivery will require 11.3086 MB on remote, 16.6601 GB available
bash: line 47: [[: 1.78886e+10: syntax error in expression (error token is ".78886e+10")
Not enough disk space abvailable on remote
Script returned with status 1
First, thanks for this project ! I've hope than could I use it.
My issue: When I run
git deliver --init-remote name user@server:/path/to/dir
I receive
/path/git-deliver/deliver.sh: ligne 514: /tmp/git_deliver_ssh_wrapper_8370.sh: Permission non accordée
/path/git-deliver/deliver.sh: ligne 514: /tmp/git_deliver_ssh_wrapper_8370.sh: Permission non accordée
/path/git-deliver/deliver.sh: ligne 514: /tmp/git_deliver_ssh_wrapper_8370.sh: Permission non accordée
Error creating root directory on remote
/path/git-deliver/deliver.sh: ligne 422: /tmp/git_deliver_ssh_wrapper_8370.sh: Permission non accordée
I've missed a step ?
$ git branch
* (no branch)
master
$ git deliver the_remote master
Current version on the_remote:
[...]
Running scripts for stage pre-delivery
pre-delivery/01-core-disk-space.sh
Pushing necessary commits to remote
bash: -c: line 0: Erreur de syntaxe près du symbole inattendu « ( »
bash: -c: line 0: `git push "the_remote" (nobranch)'
deliver echoes to stdout or to stderr a bit randomly.
We need to figure out a policy and make the code consistent with it.
I'm evaluating the use of git-deliver and I'm wondering if it could be used with multi host deployement, and if yes, how ?
If the application is deployed on several host, there is some chance that I don't want to run git-deliver to each host. Plus, I guess that some scripts won't be necessary more than once (a database migration should not be made for each host)
Any thoughts about that ?
I run bundle install, assets:precompile and db:migrate using the .remote.sh feature, but mgirate always fails, it tries to rerun migrations and throws "IrreversibleMigration" exception, leaving the production schema inconsistent. If I run this command inside the machine, it runs correctly.
Hi,
Is there an option in git-deliver to change the default ssh port ?
Thanks
Check that the exit happens and that the exit code is right.
Hi,
My question sounds maybe like a troll, I'm not here for that, this is just what will be better with git-deliver for example ?
I haven't tested it yet, but for the compatibility you just have to install git that's all ? (It can be a very good thing on Windows, because when you want to use Capistrano and you don't need a ruby env, install it on a PC it's just a waste of time…)
During humantalks, you said to me that the gc manage 3 folders.
It could be great to add the ability to manage it
[_delivered 98df5dc] Git-deliver automated commit
Switching the 'current' symlink to the newly delivered version.
No scripts for stage post-symlink
/tmp/git-deliver-3728.16515: line 1: Delivery: command not found
/tmp/git-deliver-3728.16515: line 4: Delivery: command not found
/tmp/git-deliver-3728.16515: line 6: running: command not found
/tmp/git-deliver-3728.16515: line 7: running: command not found
/tmp/git-deliver-3728.16515: line 8: running: command not found
/tmp/git-deliver-3728.16515: line 9: running: command not found
/tmp/git-deliver-3728.16515: line 10: running: command not found
/tmp/git-deliver-3728.16515: line 11: running: command not found
/tmp/git-deliver-3728.16515: line 12: running: command not found
/tmp/git-deliver-3728.16515: line 13: running: command not found
/tmp/git-deliver-3728.16515: line 14: [email protected]: No such f
ile or directory
/tmp/git-deliver-3728.16515: line 15: running: command not found
Delivery complete.
You might want to publish tag delivered-toto-2013-03-12_09-47-47 (git push o
rigin delivered-toto-2013-03-12_09-47-47 ?)
$ git deliver --status
Remote toto:
bash: line 5: realpath: command not found
What do you think about an installation via brew (the missing PM for OS X : https://github.com/Homebrew/homebrew) ?
Like :
$ brew install git-deliver
Create a Formulae with Ruby :
$ brew create http://foo.com/bar-1.0.tgz Created /usr/local/Library/Formula/bar.rb
source : http://brew.sh/
We need a way for developers to document their presets and for users to access the documentation for a given preset.
Use of getopt by git-deliver does not work on mac os.
Log:
$ git deliver --status myremote
flags:ERROR short flag required for (source) on this platform
flags:ERROR short flag required for (init) on this platform
flags:ERROR short flag required for (init-remote) on this platform
flags:ERROR short flag required for (list-presets) on this platform
flags:ERROR short flag required for (status) on this platform
flags:ERROR short flag required for (gc) on this platform
flags:ERROR short flag required for (rollback) on this platform
flags:ERROR short flag required for (color) on this platform
Git-deliver: illegal option -- -
illegal option -- s
illegal option -- t
illegal option -- a
illegal option -- t
illegal option -- u
illegal option -- s
-- myremote
Usage :
git deliver
git deliver --rollback [DELIVERY]
git deliver --gc
git deliver --init [PRESETS]
git deliver --init-remote <REMOTE_NAME> <REMOTE_URL>
git deliver --list-presets
git deliver --status [REMOTE]
See what we can do about that, at the very least mention it in the readme.
When doing a git deliver --gc, access is denied when git-deliver wants to delete files from deliveries done by other users.
After a "chmod -R g+w" on those directories, git deliver --gc works fine agains.
At delivery, git-deliver should chmod g+w on its working directory.
We could use some possibilities for configuration : auto-push the delivery tag (and to where), don't open the editor with the delivery log, etc.
This should probably be in the form of a "deliver" section in the git configuration files ?
Hey Guys, not sure if it is really a bug! I think it is because of my stupidity.
git deliver --init
on my local machine and got this response Setting up core preset
.git deliver --init-remote production [email protected]:/www/test/public
and get this weird error:ssh: Could not resolve hostname +1246AaCfgKkMNnqsTtVvXxYb:c:D:e:F:i:L:l:m:O:o:p:R:S:w:z: nodename nor servname provided, or not known
ssh: Could not resolve hostname +1246AaCfgKkMNnqsTtVvXxYb:c:D:e:F:i:L:l:m:O:o:p:R:S:w:z: nodename nor servname provided, or not known
ssh: Could not resolve hostname +1246AaCfgKkMNnqsTtVvXxYb:c:D:e:F:i:L:l:m:O:o:p:R:S:w:z: nodename nor servname provided, or not known
\E[31mError creating root directory on remote
No idea what went wrong :(
$ git rev-parse HEAD ef4ec59d9a32ef6513a2f3b1ae5ebb30454840b2 $ git deliver mymachine master Current version on mymachine: d9716892bbee41202b7bda215b2c51866df1762e * not delivered with git-deliver * Running scripts for stage pre-delivery pre-delivery/01-core-disk-space.sh pre-delivery/01-core-mem-free.sh Pushing necessary commits to remote To ssh://myurl ! [rejected] master -> master (non-fast-forward) error: failed to push some refs to 'ssh://myurl' hint: Updates were rejected because the tip of your current branch is behind hint: its remote counterpart. Merge the remote changes (e.g. 'git pull') hint: before pushing again. hint: See the 'Note about fast-forwards' in 'git push --help' for details. Creating new delivery clone Cloning into '/path/to/deliver/2013-04-30_09-50-30_ef4ec5_master'... done. No scripts for stage post-checkout [_delivered 0ec924e] Git-deliver automated commit Switching the 'current' symlink to the newly delivered version. Running scripts for stage pre-symlink pre-symlink/01-stop-puppetmaster.remote.sh Puppetmaster is running. Try to stop it... Puppetmaster is now stopped. Running scripts for stage post-symlink post-symlink/01-start-puppetmaster.remote.sh Puppetmaster is not running. Try to start it... Puppetmaster is now started. Tagging delivery commit Delivery complete. You might want to publish tag delivered-mymachine-2013-04-30_09-50-30 (git push o rigin delivered-mymachine-2013-04-30_09-50-30 ?)
=> connect to remote
$ ls -l /path/to/deliver current -> 2013-04-30_09-50-30_ef4ec5_master $ cd /path/to/deliver/current $ git log commit 0ec924ea2fd83b2446b99779c0dcc354ba8462a4 Author: ... Date: Tue Apr 30 11:51:10 2013 +0200 Git-deliver automated commit commit 68d46030c847a744d5a34b65e4d5d0ed2a251d7e Author: ... Date: Tue Apr 30 11:29:42 2013 +0200 My comment
As you can see, the directory is correctly called "ef4ec5" (that is the commit I wanted to deliver), BUT if I go inside, it is commit "68d46".
(for the record, it comes from a git commit --amend on the previous commit I thought has not been commited)
The documentation should be accessible via --help, just like it is for other git commands.
In some situations (which ? seen with msys (client) / debian wheezy (server))
check-disk-space crashes at the ssh connection, and does not really check the available disk space on the remote. Additionnaly, the delivery continues, which could indicate a problem in stage-script error checking
We could use a more reliable way to log deliveries. Local tags might get lost, We could at least push them to the base delivery repo ?
After dozens of deliveries, git-deliver begins to slow down.
The problem is that the remote_status function get all deliveries from the remote, even if the whole list is not needed to deliver.
If we run "git deliver gc", git deliver is fast again.
There should be an option for not requiring the user to save the log file, just write it.
There are probably a few dependencies between tests. These should be removed to ensure that each test is accurate.
Intersect between #12 and #5: if a git submodule checkout fails, git-deliver does not stop.
Example (with github down - yes!):
$ git deliver myremote master Current version on myremote: c090395c5a3f60b79332971bb9260511172ab6ef delivered Thu, 2 May 2013 10:40:15 +0200 by myself Running scripts for stage pre-delivery pre-delivery/01-core-disk-space.sh pre-delivery/01-core-mem-free.sh Pushing necessary commits to remote Everything up-to-date Creating new delivery clone Cloning into '/my/path/delivered/2013-05-02_09-15-55_c09039_master'... done. Checking out files... Switched to a new branch '_delivered' Submodule 'modules/apache' (https://github.com/soluvas/puppet-apache.git) registered for path 'modules/apache' Submodule 'modules/apt' (https://github.com/puppetlabs/puppetlabs-apt.git) registered for path 'modules/apt' Submodule 'modules/concat' (https://github.com/ripienaar/puppet-concat.git) registered for path 'modules/concat' Submodule 'modules/cron' (https://github.com/torrancew/puppet-cron.git) registered for path 'modules/cron' Submodule 'modules/firewall' (https://github.com/puppetlabs/puppetlabs-firewall.git) registered for path 'modules/firewall' Submodule 'modules/jenkins' (git://github.com/rtyler/puppet-jenkins.git) registered for path 'modules/jenkins' Submodule 'modules/lvm' (https://github.com/puppetlabs/puppetlabs-lvm.git) registered for path 'modules/lvm' Submodule 'modules/maven' (https://github.com/maestrodev/puppet-maven.git) registered for path 'modules/maven' Submodule 'modules/postgresql' (https://github.com/puppetlabs/puppet-postgresql.git) registered for path 'modules/postgresql' Submodule 'modules/sonar' (https://github.com/maestrodev/puppet-sonar.git) registered for path 'modules/sonar' Submodule 'modules/stdlib' (https://github.com/puppetlabs/puppetlabs-stdlib.git) registered for path 'modules/stdlib' Submodule 'modules/wget' (https://github.com/maestrodev/puppet-wget.git) registered for path 'modules/wget' Cloning into 'modules/apache'... Submodule path 'modules/apache': checked out '10d2d73907c5a5600c0dd8eb9c870715ef2a58e1' Cloning into 'modules/apt'... error: The requested URL returned error: 503 while accessing https://github.com/puppetlabs/puppetlabs-apt.git/info/refs fatal: HTTP request failed Clone of 'https://github.com/puppetlabs/puppetlabs-apt.git' into submodule path 'modules/apt' failed No scripts for stage post-checkout [_delivered 89440e4] Git-deliver automated commit 1 file changed, 1 deletion(-) delete mode 160000 modules/apt Switching the 'current' symlink to the newly delivered version. Running scripts for stage pre-symlink pre-symlink/01-stop-puppetmaster.remote.sh Puppetmaster is running. Try to stop it... Puppetmaster is now stopped. Running scripts for stage post-symlink post-symlink/01-start-puppetmaster.remote.sh Puppetmaster is not running. Try to start it... Puppetmaster is now started. Tagging delivery commit Delivery complete. You might want to publish tag delivered-myremote-2013-05-02_09-15-55 (git push o rigin delivered-myremote-2013-05-02_09-15-55 ?)
A lock should be added to prevent simultaneous deliveries to the same server.
This may be a bit outside the scope of the project. To be discussed.
I am not sure why this error occurs.
I also couldn't get --init and --init-remote to work without first creating and doing a git init bare to get git-deliver to say that the remote is ready for deliveries.
Is there something I am doing wrong?
Let me know if I need to provide more details
More info about multiplexing can be found here : http://en.wikibooks.org/wiki/OpenSSH/Cookbook/Multiplexing
I have a remote origin branch and a remote production branch where I will deploy using git-deliver.
Right now, I'm seeing this error constantly when trying to deploy:
Pushing necessary commits to remote
To [email protected]:/home/ec2-user/test
! [rejected] master -> master (fetch first)
error: failed to push some refs to '[email protected]:/home/ec2-user/test'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
I've tried running git pull production master
and git pull origin master
but still gives me this :(
Maybe with an option to see the complete diff through $PAGER since the last delivery, and an option to check this diff against the ones generated by running the same update on other environments ?
On mingw, after git deliver --gc my-remote-repo
[snip]
Removing /my/remote/repo/delivered/2013-04-23_13-53-19_417cfc_master
awk: line 4: syntax error at or near *
29 version(s) removed, freed
Is a separate program a good idea ? What are the alternatives ? RIght now, this aspect is pretty messy.
We need to check the git version on both the client and server to make sure that they are adequate, warn the user and stop work if they aren't.
It could be nice to push only the necessary commits when delivering, instead of always pushing the whole branch.
In some conditions (which ?), --status says "uncommited changes" when there are actually no changes to the delivery.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.