xuwupeng2000 / capistrano-scm-gitcopy

Capsitrano 3 copy scm - tar your git branch and deploy to your server

capistrano-scm-gitcopy

Capistrano 3 :copy

A copy strategy for Capistrano 3, which mimics the :copy scm of Capistrano 2. This Gem is inspired by and based on https://github.com/wercker/capistrano-scm-copy. Thank wercker so much.

Why you should use this gem?

• no need to add id_rsa.pub to all your servers to deploy your remote code
• just add this gem and use your deploy service (your laptop) read from repo and deploy to your servers

This will make Capistrano tar the a specific git branch, upload it to the server(s) and then extract it in the release directory.

Release notes

0.1.5

• local_path is now configurable
• Use Git Revison as the name of tar file, also keep a proper revisions.log
• Add possibility to have tmp_dir_remote

0.1.4

• Tidy up
• New local tmp folder (/tmp/application-name/timestamp) so you can deploy without worry about clearup
• Remove incorrect README content

Requirements

Machine running Capistrano:

• Capistrano 3
• tar

Servers:

• mktemp
• tar
• ruby
• bundler (if you allow cap to run bundle for you)

Installation

First make sure you install the capistrano-scm-gitcopy by adding it to your Gemfile:

gem "capistrano-scm-gitcopy"

Then switch the :scm option to :gitcopy in config/deploy.rb:

set :scm, :gitcopy

Finally, DO NOT ADD require 'capistrano/gitcopy' to Capfile because capistrano/setup already loads the scm module with the :scm value you specified.

There is no default branch, you have to pass the branch you want to deploy as argument in the command line.

Usage

  bundle exec cap staging deploy branch=(your release branch)

Output is quiet by default set gitcopy_verbose to true in order get verbose output

set :gitcopy_verbose, true

capistrano-scm-gitcopy's Issues

git doesn't find a repository to clone in /tmp

I have git version 1.9.1, updated to capistrano 3.5 and am getting the following error:

02 git clone --verbose --mirror  /tmp/stage-repository/1467722316
02 fatal: repository '/tmp/stage-repository/1467722316' does not exist

Any idea what went wrong?

Empty archive, composer.json not found, Operation not allowed by protocol

Our GIT(stack) server is behind a internal firewall which is accessible by the following url:

set :repo_url, 'http://username:[email protected]/semcoweb.git' 

Internally in our network and without gitcopy the correct repo is successfully deployed with Capistrano.

Using gitcopy for production everything starts to work fine. The following message seems to have no further meaning and the deployment continues:

... refs/head/master
... refs/remotes/origin/master
fatal: Operation not allowed by protocoll 

Then the archive is created and uploaded successfully.

If I unzip this archive on my local machine the .tar file is 0 bytes big. It seems like no files have been copied from the repo?

At the end the deployment fails because it cannot find the composer.json file that should have been uploaded with alle the other files from the repo.

As mentioned these git settings work when deploying to staging behind the firewall.
Am I missing any extra setting for deploy.rb?

Possibly related to #26 by @jbaptperez.

Release SHA status information missing

This was fixed in the main capistrano git repo as part of this PR. But, with v0.0.8, it seems to produce
Branch master (at ) deployed as release 20160129183502 by XXX

Testing branch name and making archive

Hello,

I have the following problem : my git branch name is already part of other branch names.
Example :
My branch name is foo, but I have also 2 branches which names are foobar and barfoobar.
So, stdout will show me :

d00c52294e78ce782eb12cfa47249d3523286879    refs/heads/foo
ee5e21f46d9d35ef362ed8df80d95b121a9a1482    refs/heads/foobar
304bd988585c5507198075cdbafebc2de2fd339a    refs/heads/barfoobar
fatal: ambiguous argument 'foo': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'

In gitcopy.rake, I can see this code:

system "git ls-remote #{fetch(:repo_url)} | grep #{fetch(:branch)}"

-> This call produces the first 3 lines above.
As the exit status of this code is then tested to make or not the archive, I can easily produce a bug:
Specifying a branch like fo or ooba. Both don't exists but the test will pass!

Note that this code just bellow:

system "git archive #{no_repo_url ? '' : "--remote #{fetch(:repo_url)}" } --format=tar #{fetch(:branch)}:#{fetch(:sub_directory)} | gzip > #{ archive_name }"

will produce the last three lines: the archive which is being built can be built from another reference!

Idea of resolution :
First, using exact match with a regular expression. Something like:

system "git ls-remote #{fetch(:repo_url)} | grep -P '^.{40}\t.*#{fetch(:branch)}$'"

Then, count results lines (| wc -l), getting stdout in a variable instead of showing it and test the variable's value:

• "0" = error
• "1" = OK
• More than "1" = error message like "Branch name is ambiguous, please type exact ref (ie refs/heads/fooinstead of foo)"

So the last 3 lines will have no chances to be produced and the archive will always be the good one.

I am a noob in ruby language, but if you have no time to fix it, I can try to find the code for that and then send you a pull request.

Thanks.

Installation not work / LoadError: cannot load such file -- capistrano/gitcopy.rb

Hey guys,

I get this error.

LoadError: cannot load such file -- capistrano/gitcopy.rb

If I follow the installation process.

I added gem 'capistrano-scm-gitcopy' in the GemFile and replaced the :scm string with set :scm, :gitcopy.
But then I get this error.

If I add this bad ``to theCapfile it seems to work but then I get this error. I think this is because I'm using windows?

** Execute gitcopy:wrapper
INFO [2f6bed48] Running /usr/bin/env mkdir -p /tmp/app/ as Username@localhost
DEBUG [2f6bed48] Command: ( export SYMFONY_ENV="prod" ; /usr/bin/env mkdir -p /tmp/app/ )
cap aborted!
Errno::ENOENT: No such file or directory - ( export SYMFONY_ENV="prod" ; /usr/bin/env mkdir -p /tmp/app/ )
C:/Ruby22-x64/lib/ruby/2.2.0/open3.rb:193:in `spawn'
C:/Ruby22-x64/lib/ruby/2.2.0/open3.rb:193:in `popen_run'
C:/Ruby22-x64/lib/ruby/2.2.0/open3.rb:93:in `popen3'
C:/Ruby22-x64/lib/ruby/gems/2.2.0/gems/sshkit-1.9.0/lib/sshkit/backends/local.rb:41:in `execute_command'
C:/Ruby22-x64/lib/ruby/gems/2.2.0/gems/sshkit-1.9.0/lib/sshkit/backends/abstract.rb:137:in `block in create_command_and_execute'

Thanks in Advance for any ideas!

LoadError: cannot load such file -- capistrano/gitcopy.rb

Hi,

This still seems to be a problem.

Using Capistrano 3.6.1

All I did was:

1. add "gem 'capistrano-scm-gitcopy', group: :development" to my gemfile
2. add "set :scm, :gitcopy" to my deploy file.

gitcopy doesn't work with capistrano-rvm

This line of code is the issue...

Since this gem is using capture instead of execute :ruby, '--args' capistrano-rvm cannot plug in the rvm:hook task for this command. I spent a lot of time troubleshooting this one and took a lot of wrong turns trying to get Cap3 to work with a login shell, trying to rewrite the gitcopy:deploy task to include the rvm:hook among others... Once I finally figured out how capistrano-rvm was injecting the call to RVM the solution was an easy one.

If you have any questions, please let me know... I'll be submitting a pull request shortly.

Doesn't work without remote

If I don't have any remote specified or if I do but simply can't access it, the deploy won't make it past the git ls-remote check method.

I tried adding a placeholder remote pointing to ${source directory}/.git/ which is valid from the git perspective, but it doesn't feel right and it includes everything in my working copy so I'd need a way to exclude stuff.

Deprecation Warning

When running with capistrano 3.8.1, I receive the warning:

[Deprecation Notice] `set :scm, :gitcopy` is deprecated.
To ensure this custom SCM will work with future versions of Capistrano,
please upgrade it to a version that uses the new SCM plugin mechanism
documented here:

http://capistranorb.com/documentation/advanced-features/custom-scm

Could this project be updated to the more future-proof setup?

Build script

Hi,

Thank you for all your work, I just want to know if your can add a build script or a task to execute some scripts within the local clone before send it to servers.

Thank.

Yann

cd /tmp/... is missing when running gitcopy:set_current_revision

when i'm running cap x deploy from a folder that is not a git repo, it fails on gitcopy:set_current_revision task. the log is below

INFO [f5f06bee] Running /usr/bin/env git remote update as user@localhost
DEBUG [f5f06bee] Command: cd /tmp/project-repository/1464541485 && ( export GIT_ASKPASS="/bin/echo" GIT_SSH="/tmp/project/git-ssh.sh" ; /usr/bin/env git remote update )
DEBUG [f5f06bee] Fetching origin
INFO [f5f06bee] Finished in 0.312 seconds with exit status 0 (successful).
DEBUG [e8eb8b2d] Running /usr/bin/env git rev-list --max-count=1 --abbrev-commit --abbrev=12 master as user@localhost
DEBUG [e8eb8b2d] Command: ( export GIT_ASKPASS="/bin/echo" GIT_SSH="/tmp/project/git-ssh.sh" ; /usr/bin/env git rev-list --max-count=1 --abbrev-commit --abbrev=12 master )
DEBUG [e8eb8b2d] fatal: Not a git repository (or any parent up to mount point /usr)
DEBUG [e8eb8b2d] Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

It looks like the reason is "cd /tmp/project-repository/1464541485 &&" needs to happen before running git rev-list

Rspec tests are broken

After upgrading this to use the plugin approach, it looks like the tests need to be modified,

capistrano-scm-gitcopy/spec/lib/capistrano/gitcopy_spec.rb:18:in `module:Capistrano': uninitialized constant Capistrano::GitCopy::DefaultStrategy (NameError)

Delete archives also on deploy failed

Is there any chance to handle capistrano fail and cleanup archives after?

tar verbosity when extracting on remote

Is it really needed? For large project there are hundred of files.
I'm thinking about switch to configure this behaviour, but can we make assumption that default behaviour is without tar verbosity?

repo_tree

Hi I am trying to use repo_tree however, the entire repo is still being copied. Is this supported?

Error ambiguous argument on executing gitcopy:set_current_revision

I'm running cap on ubuntu 14.04.1, git 2.8.1

The error seems to happen when gitcopy does a git rev-list

** Invoke gitcopy:set_current_revision (first_time)
** Execute gitcopy:set_current_revision
cap aborted!
SSHKit::Command::Failed: git exit status: 32768
git stdout: Nothing written
git stderr: fatal: ambiguous argument 'develop': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'

Tasks: TOP => gitcopy:create_release => gitcopy:create_tarfile => gitcopy:set_current_revision
(See full trace by running task with --trace)
The deploy has failed with an error: git exit status: 32768
git stdout: Nothing written
git stderr: fatal: ambiguous argument 'develop': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'

Wrong usage of application symbol

I'm not sure if this is NOT a Capistrano issue. But since everything used to work fine in staging before implementing scm-gitcopy into production I will post it here.

In my deploy.rb my application symbol is set to:

set :application, 'Semcoglas'
set :deploy_to, '/var/www/semcoweb'

I always regarded application as App name.

When I run cap production deploy everything works fine so far. All paths are used correctly based on the deploy_to symbol.

But then, when Composer is supposed to run, the path suddenly changes to /var/www/Semcoglas <- name of the :application symbol.

Of course I get a _Could not open file: /var/www/Semcoglas/shared/composer.phar_ error.

I tried without defining :application and indeed the wrong path will become /var/www/shared/....

Any idea what is happening here?

Directory already exist and is not empty

Hi
I'm using capistrano 3.5 with Ubuntu 16

Deploy fail with error

DEBUG [7684efec]    Cloning into bare repository '/tmp/capipro-repository/1463328717'...
DEBUG [7684efec]    POST git-upload-pack (gzip 33402 to 16590 bytes)

INFO [7684efec] Finished in 102.995 seconds with exit status 0 (successful).
INFO [c40c4941] Running /usr/bin/env mkdir -p /tmp/capipro-repository/1463328717 as ftpdata@localhost
DEBUG [c40c4941] Command: /usr/bin/env mkdir -p /tmp/capipro-repository/1463328717
INFO [c40c4941] Finished in 0.004 seconds with exit status 0 (successful).
DEBUG [6530b4ab] Running if test ! -d /tmp/capipro-repository/1463328717; then echo "Directory does not exist '/tmp/capipro-repository/1463328717'" 1>&2; false; fi as ftpdata@localhost
DEBUG [6530b4ab] Command: if test ! -d /tmp/capipro-repository/1463328717; then echo "Directory does not exist '/tmp/capipro-repository/1463328717'" 1>&2; false; fi
DEBUG [6530b4ab] Finished in 0.006 seconds with exit status 0 (successful).
INFO [4bf89768] Running /usr/bin/env git clone --verbose --mirror http://pass:[email protected]/scm/cfw/test.git /tmp/capipro-repository/1463328717 as ftpdata@localhost
DEBUG [4bf89768] Command: cd /tmp/capipro-repository/1463328717 && ( export GIT_ASKPASS="/bin/echo" GIT_SSH="/tmp/capipro/git-ssh.sh" ; /usr/bin/env git clone --verbose --mirror http://pass:[email protected]/scm/cfw/test.git /tmp/capipro-repository/1463328717 )
DEBUG [4bf89768]    fatal: destination path '/tmp/capipro-repository/1463328717' already exists and is not an empty directory.
cap aborted!
SSHKit::Command::Failed: git exit status: 32768
git stdout: Nothing written
git stderr: fatal: destination path '/tmp/capipro-repository/1463328717' already exists and is not an empty directory.

I'm using just a basic deployement configuration in deploy.rb

# config valid only for current version of Capistrano
lock '3.5.0'

set :application, 'capipro'
set :repo_url, 'http://pass:[email protected]/scm/cfw/test.git'

# Default branch is :master
 ask :branch, `git rev-parse --abbrev-ref HEAD`.chomp

# Default value for :scm is :git .... :gitcopy
set :scm, :gitcopy

and stagin.rb

# server-based syntax
# ======================
# Defines a single server with a list of roles and multiple properties.
# You can define all roles on a single server, or split them:

server '172.17.0.3', user: 'ftpdata', roles: %w{a`pp}

role :app, %w{[email protected]}

Any idea ?

Olivier

"LoadError: cannot load such file -- capistrano/gitcopy.rb" with capistrano 3.3.3

Hi,

There is a issue with capistrano 3.3.3, it doesn't load gitcopy anymore.

I tried to require 'capistrano/gitcopy' inside the Capfile, but each task are executed twice. Have you got any idea.

Same issue : https://github.com/wercker/capistrano-scm-copy/issues/12

Thank

Brand new release -Test please

I just released version 0.1.2 (master branch)
Can someone plz confirm it is working ?

Thanks

Is github not suported?

I am getting empty tarballs and running deployments that complete, but are empty. I believe that no code is being pulled from my GitHub repo, but the script continues the deployment anyway. I have tried different configurations for the :repo_url like:

set :repo_url, '[email protected]:me/my-repo.git'

With this config I get:

Invalid command: 'git-upload-archive 'me/myrep.git''
  You appear to be using ssh to clone a git:// URL.
  Make sure your core.gitProxy config option and the
  GIT_PROXY_COMMAND environment variable are NOT set.
fatal: The remote end hung up unexpectedly

I have also tried:

set :repo_url, 'https://me:[email protected]/me/my-repo.git'

But then I get this error:

fatal: Operation not supported by protocol.

It looks like GitHub does not support archiving. I guess I am just asking for clarification that this strategy will not work with GitHub?