Coder Social home page Coder Social logo

artifact-cookbook's People

Contributors

aaronfeng avatar andrewgarson avatar awillis avatar barthv avatar capoferro avatar clifferson avatar ephess avatar florentdupont avatar geardley avatar gitfool avatar gregsymons avatar guilhem avatar ivey avatar jhowarth avatar jmickle avatar jonathanq avatar kallan357 avatar passion4code avatar pmaccamp avatar reset avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

artifact-cookbook's Issues

ArtifactNotFound exceptions

Not really sure why i get constant ArtifactNotFound exceptions in the AMS env. The artifact is there, it actually gets deployed, and some chef runs work fine; while subsequent runs fail with ArtifactNotFoundException. You can see 3 runs in mb-plugin: #461-#463, where 461 and 463 succeeded, and 462 failed.

Duplicate extract directory

I'm using the remove_top_level_directory attribute set to true to unpack the archive directly into the releases/<version> directory. This works great but on some subsequent runs I wind up with the archive unpacked again into the releases/<version>/<artifact> directory.

Symlinks can changed in the resource definition without triggering a change in the provider

If you go from:

artifact_deploy "artifact_test" do
  version node[:artifact_test][:version]
  artifact_location node[:artifact_test][:location]
  artifact_checksum node[:artifact_test][:checksum]
  deploy_to "/srv/artifact_test"
  owner "artifact"
  group "artifact"
  symlinks({"directory_that_should_never_exist" => "directory_that_should_never_exist"})

  action :deploy
end

to:

artifact_deploy "artifact_test" do
  version node[:artifact_test][:version]
  artifact_location node[:artifact_test][:location]
  artifact_checksum node[:artifact_test][:checksum]
  deploy_to "/srv/artifact_test"
  owner "artifact"
  group "artifact"
  symlinks({"foo" => "directory_that_should_never_exist"})

  action :deploy
end

The symlink won't be changed unless force is set to true.

Should do some testing on zip / tar extraction

Should gather some metrics on the following, because I just don't know.

When you have a large, compressed artifact...

  • is the tar command on Windows efficient? (tar binary is installed by Chef)
  • windows_zipfile is a pure Ruby implementation for .zip files. Is there a better implementation that Windows has?
  • comare tar command on Unix vs. Windows. Any differences?

Weirdo error about 'undefined method checksum' during symlink

  • In before_symlink I have a Proc which creates some files in the shared directory if they don't exist, or copies them from our pre-Chef-based install (which had a slightly different directory layout.) This part appears to be working, as after it all falls apart you can see the files in question there in the shared directory. One such file is /opt/benbria/nodeloop-deploy/shared/.migrate-migrating.

  • artifact_deploy gets to the "symlink" stage and then dies with:

    [2013-08-01T12:17:20-04:00] FATAL: NoMethodError: artifact_deploy[nodeloop-deploy](nodeloop::install line 141) had an error: NoMethodError: directory[/opt/benbria/nodeloop-deploy/shared/.migrate-migrating](/tmp/loop_install/chef/cookbooks/artifact/providers/deploy.rb line 485) had an error: NoMethodError: undefined method `checksum' for Chef::Resource::Directory

Full output here:

https://gist.github.com/jwalton/da1634758061e190f049

It looks like the "directory" in question is really the one defined on line 485, but I don't see where checksum is coming from here???

This is from artifact-cookbook 1.7.1, BTW.

Current Symlink & Restart == Chicken & Egg

Hi,

We have a slight frustration with the fact that the restart occurs before the "current" symlink is in place. We like to refer to the app for it's service control via that symlink.

Obviously there are ways around it but I wonder if you might be able to offer the ability to have the symlink creation somewhere else?

Thanks

Issues when using artifact-deploy from an S3 source on a Windows Client

  1. credentials data-bags : I couldn't get it to use custom naming for data-bags, I had to use a data-bag called "aws" with item name "_wildcard". I am not sure if it was intentional, if so then I would suggest documenting a little more explicitly instead of giving it as an example.

  2. I could get the file downloaded but wasn't able to unzip in the right directory, I thought this fix for Windows cookbook could help : torresdal/windows@453ef82
    But I am not sure if that helped because of (3)

  3. Even though I used (2), unzip failed with a different error and I couldn't even manually unzip the file downloaded from s3. A little google and realized that when file is being downloaded and written to disk the "w" flag is used, when I changed it to "wb" finally I got everything to work i.e. download from s3 with the credentials and unzip the contents in a desired folder. A similar issue can be found here : http://stackoverflow.com/questions/12352230/using-aws-sdk-to-download-files-from-s3-encoding-not-right
    The part of code I am talking about is on Line 152 of libraries/chef_artifact.rb file

Restart proc does not run when symlink is changing but artifact remains unchanged

For testing purposes, it is nice to see what sequential chef runs will do when deploying the same artifact but incrementing the version number in the artifact_deploy resource. When doing so, the restart proc does not run. It seems that the restart proc will only run when the manifest is changed and not when only a symlink is changing.

Error when trying to download a Snapshot version from nexus

When trying to download a snapshot version from nexus there seems to be a file naming problem. For example, when I try and download library 1.1-SNAPSHOT it does download it but it saves the file using the snapshot timestamp format.

/tmp/kitchen/cache/artifact_deploys/library/1.1-SNAPSHOT/library-1.1-20140210.180346-6.tar.gz

This causes artifact problems and generates this error.

Errno::ENOENT: artifact_deploy[library] (library::default line 43) had an error: 
Errno::ENOENT: artifact_file[/tmp/kitchen/cache/artifact_deploys/library/1.1-SNAPSHOT/library-1.1-SNAPSHOT.tar.gz] (library::none line 582) had an error: 
Errno::ENOENT: No such file or directory - /tmp/kitchen/cache/artifact_deploys/library/1.1-SNAPSHOT/library-1.1-SNAPSHOT.tar.gz

Is there a way to resolve that?

artifact_file 'path' argument is not functionnal

Hi !
This is working :

artifact_file '/tmp/test-1.3.175.jar' do
  location 'com.test:test:jar:1.3.175'
  nexus_configuration nexus_conn
  owner 'root'
  group 'root'
end

This is not :

artifact_file 'test jar' do
  path '/tmp/test-1.3.175.jar'
  location 'com.test:test:jar:1.3.175'
  nexus_configuration nexus_conn
  owner 'root'
  group 'root'
end

artifact_file LWRP seems to totally ignore path option (path doesn't seems to be mentionned once in the provider code).
Tell me if you want me to dive a bit into file.rb provider file, and sumbit a fix.

Speed up LWRP run

Hi,

Currently each time you call one of the artifact's LWRP, it loads a bunch of deps and gems.

For one of my project, I'm working with a large number of artifacts and Chef takes a very long time (10~20 min) only to install/update chef_gem for each lwrp.

Maybe it would be interesting to provides a single recipe which would load all needed gems and dependencies, and then makes repetitives call lighter and faster ?

artifact_package fails to detect the file extension

The artifact_package resource fails to detect the extension of the file that it will be downloading. This causes the following exception:

2014-03-07T16:28:17+00:00] ERROR: artifact_package[package] (cookbook::recipe line 43) had an error: NameError: Cannot find a resource for ext on centos version 6.5

Name of resource could break deployment

The name of the resource is used in the artifact path but white-space is not removed. If the name contains spaces, for example, the shell commands will most likely fail or behave unpredictably

@artifact_root          = ::File.join(@new_resource.artifact_deploy_path, @new_resource.name)

...

def extract_artifact
  execute "extract_artifact" do
    command "tar xzf #{cached_tar_path} -C #{release_path}"
    user new_resource.owner
    group new_resource.group
  end
end

def copy_artifact
  execute "copy artifact" do
    command "cp #{cached_tar_path} #{release_path}"
    user new_resource.owner
    group new_resource.group
  end
end

In the above code, if the name was "Deploying Files for User" then the command being sent to the execute resource would be:

"cp /tmp/artifacts/Deploying Files for User /home/user"

The spaces will break the shell script and the chef-client will fatally error.

We might try to do a gsub on the name when putting it into artifact_root. Something like:

@artifact_root          = ::File.join(@new_resource.artifact_deploy_path, @new_resource.name.gsub(/\W/, '_'))

Undefined method 'chef_environment' (looks like from chef_artifact_nexus_configuration library)

I just upgraded to 1.10.0, however every time I run chef-client it fails.

================================================================================
Recipe Compile Error in /var/chef/cache/cookbooks/artifact/resources/deploy.rb
================================================================================


NoMethodError
-------------
undefined method `chef_environment' for nil:NilClass


Cookbook Trace:
---------------
  /var/chef/cache/cookbooks/artifact/libraries/chef_artifact.rb:235:in `encrypted_data_bag_for'
  /var/chef/cache/cookbooks/artifact/libraries/chef_artifact.rb:71:in `data_bag_config_for'
  /var/chef/cache/cookbooks/artifact/libraries/chef_artifact_nexus_configuration.rb:6:in `from_data_bag'
  /var/chef/cache/cookbooks/artifact/resources/deploy.rb:56:in `class_from_file'


Relevant File Content:
----------------------
/var/chef/cache/cookbooks/artifact/libraries/chef_artifact.rb:

228:        # @return [Chef::Mash] the data bag item in Mash form
229:        def encrypted_data_bag_for(node, data_bag)
230:          @encrypted_data_bags = {} unless @encrypted_data_bags
231:
232:          if encrypted_data_bags[data_bag]
233:            return get_from_data_bags_cache(data_bag)
234:          else
235>>           data_bag_item = encrypted_data_bag_item(data_bag, node.chef_environment)
236:            data_bag_item ||= encrypted_data_bag_item(data_bag, WILDCARD_DATABAG_ITEM)
237:            data_bag_item ||= encrypted_data_bag_item(data_bag, "nexus")
238:            data_bag_item ||= {}
239:            @encrypted_data_bags[data_bag] = data_bag_item
240:            return data_bag_item
241:          end
242:        end
243:
244:        # @return [Hash]

Looking in the code (https://github.com/RiotGames/artifact-cookbook/blob/master/libraries/chef_artifact_nexus_configuration.rb#L6), nil is being passed in for the node object which then tries to call chef_environment on it.

This might be a different code-path as I am using the S3 deploy method in a Vagrant box (so with an encrypted data_bag, running against chef-server).

If this is just specific to the S3 case, perhaps I should convert the configuration to follow the same model as the new Nexus config? Going back to a commit before the chef_artifact_nexus_configuration was added fixes it for me.

Re-evaluate delete_previous_versions method

The delete_previous_versions method leaves some things to be desired. In particular,

  • its position during the deploy -> it comes first when it should possibly execute at the end.
  • it defines two ruby methods, delete_release_path_for and delete_release_path_for. It seems like these could be removed with a recursive :delete action on either the File or Directory resource.
  • there may be some bugs in the logic in terms of "when does it actually delete"

artifact_deploy always notifies

action :deploy always calls

new_resource.updated_by_last_action(true)

This should only be called if something actually gets updated.

Maybe call this only if you need to redploy and leave it up to proc writers to call it when their proc changes something in the system

does having a 'latest' folder matter

If the version you want is provided as latest and the artifact_location is not a Nexus, you will still end up with a latest folder.

The condition in the Library should probably change to:

if version.casecmp("latest") == 0 && is_nexus?(artifact_location)

but the question is, should it?

artifact 1.8.0 dies on Linux with PaxHeader error

[email protected]] out: [2013-08-07T13:42:19-04:00] FATAL: Stacktrace dumped to /tmp/chef-solo/chef-stacktrace.out
[[email protected]] out: [2013-08-07T13:42:19-04:00] FATAL: SyntaxError: /tmp/loop_install/chef/cookbooks/artifact/providers/PaxHeader/deploy.rb:1: syntax error, unexpected tIDENTIFIER, expecting $end
[[email protected]] out: 17 gid=995576803
[[email protected]] out:       ^

Possibly related to http://tickets.opscode.com/browse/CHEF-4107?

Configurable SSL Checking

SSL is important, but perhaps broadly assuming all SSL communications with Nexus should be verified was the wrong path.

artifact_file downloads files when it probably shouldn't

artifact_file downloads a file. On a subsequent chef-run, it will be downloaded again, because the default path is to use the remote_file resource which downloads the file to a tmp dir and checks the checksum before copying, even if the file is already there.

Support for Ubuntu 12.04

So far, the only issue that I've seen with the cookbook on Ubuntu 12.04 is that it doesn't specify the correct package names for S3 support:

--- a/chef/cookbooks/artifact/providers/deploy.rb
+++ b/chef/cookbooks/artifact/providers/deploy.rb
@@ -58,7 +58,7 @@ def load_current_resource
     @artifact_location = [group_id, artifact_id, artifact_version, extension].join(':')
   elsif Chef::Artifact.from_s3?(@new_resource.artifact_location)
     unless Chef::Artifact.windows?
-      %W{gcc make libxml2 libxslt libxml2-devel libxslt-devel}.each do |nokogiri_requirement|
+      %W{gcc make libxml2 libxslt1.1 libxml2-dev libxslt1-dev}.each do |nokogiri_requirement|
         package nokogiri_requirement do
           action :nothing
         end.run_action(:install)

Support Preseeding of Content

No matter what, we currently execute this code:

recipe_eval do
    link new_resource.current_path do
      to release_path
      user new_resource.owner
      group new_resource.group
    end
  end

I'd like to see a way (perhaps an additional attribute) to allow me to pre-seed and install content without switching the current symlink. This would also probably bypass the execution of any of the Procs, which may do any number of things.

It might make sense as an attribute:

artifact_deploy "my-artifact" do
  artifact_location "somewhere"
  version                node[:cookbook][:artifact_version]
  pre_seed           node[:cookbook][:pre_seed]
end

Or an entirely new provider / action

artifact_deploy "my-artifact" do
  ...
  action :pre_seed
end
artifact_pre_seed "my-artifact" do
  ...
end

artifact_deploy "my-artifact" do
  ...
end

Reconcile pre_seed and deploy actions

There are some possible edge cases with both actions in regards to the retrieve_artifact! method.

Probably need to do some checking on whether or not we really need to download a file. There might also be a gap in the extract_artifact! method and idempotency since its just an execute block.

Rollback

Hi,

Handy cookbook, I much prefer the model of using an artifact in a repo to pulling from source control.

As far as I can see, there's no way to roll back to a previous release. If I run with version set to 1.0, then again set to 2.0, then a third time setting the version to 1.0, then version 2.0 is still in place.

I'd expect it to relink to the 1.0 release and restart the app, but instead it seems to consider it as already deployed, and skips (unless force is set to true, of course).

I can hack around it for myself, but would be handy if it did the rollback to a previous version.

Thanks,
Kief

undefined method `to_hash` for nil when missing data bags

Had this error occur when I definitely didn't have a single data bag in my Chef server. Looks like we are missing a check somewhere.

NoMethodError: artifact_deploy[foo] (foo::bar line 28) had an error: NoMethodError: undefined method `to_hash' for nil:NilClass
C:/chef/cache/cookbooks/artifact/libraries/chef_artifact_nexus.rb:15:in `remote'
C:/chef/cache/cookbooks/artifact/libraries/chef_artifact_nexus.rb:32:in `get_actual_version'
C:/chef/cache/cookbooks/artifact/providers/deploy.rb:57:in `load_current_resource'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/provider.rb:97:in `run_action'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/resource.rb:625:in `run_action'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/runner.rb:49:in `run_action'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/runner.rb:81:in `block (2 levels) in converge'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/runner.rb:81:in `each'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/runner.rb:81:in `block in converge'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/resource_collection.rb:98:in `block in execute_each_resource'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/resource_collection.rb:96:in `execute_each_resource'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/runner.rb:80:in `converge'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/client.rb:433:in `converge'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/client.rb:500:in `do_run'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/client.rb:213:in `run'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/application.rb:208:in `run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/application/client.rb:312:in `block in run_application'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/application/client.rb:304:in `loop'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/application/client.rb:304:in `run_application'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/lib/chef/application.rb:66:in `run'
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.2-x86-mingw32/bin/chef-client:26:in `<top (required)>'
C:/opscode/chef/bin/chef-client:23:in `load'
C:/opscode/chef/bin/chef-client:23:in `<main>'

Verify checksum

If an attr with the artifact.s checksum s included, verify successful download before continuing

artifact_deploy procs don't change cwd

From the readme

 migrate Proc.new {
    execute "bundle exec rake db:migrate" do
      environment { 'RAILS_ENV' => 'production' }
      user "riot"
      group "riot"
    end
  }

does not work for me, because it is executed in /srv/pvpnet, and not in /srv/pvpnet/releases/whatever.

Since the procs are just recipe_eval'ed, it seems hard to change the execute block in the proc to add the cwd automatically, but then the example in the readme should be changed to

 migrate Proc.new {
    execute "bundle exec rake db:migrate" do
      cwd release_path
      environment { 'RAILS_ENV' => 'production' }
      user "riot"
      group "riot"
    end
  }

Same thing for before_migrate proc.

Anonymous access doesn't seem to work for me

I see that you recently accepted a pull that makes the cookbook work if anonymous access is disabled, however, I can't figure out how you used anonymous access in the first place. If I omit the username and password fields from the data bag or set them to blank, then nexus_cli complains about them being missing. If I set them to "anonymous" then anonymous_enabled? fails because the anonymous user doesn't have the privileges to check to see if it's enabled. Does anonymous access actually work?

is_tarball assumes content is gzip'ed

extract_artifact passes xzf to tar which will fail if the tarball is not gzipped. Since there are multiple ways to zip and also the ability to leave unzipped, we need to have the ability to specify the type of tarball.

tar xzf 

force does not clear out contents of current release

If the force option is specified, the current release directory is not rm -rfed. The implication is that if my tarball has files removed from it and I redeploy, I expect the files to be removed from the installed directory. Since the files are not removed, this expectation is not met.

My recommendation is to change:

action :deploy do
  next unless new_resource.force or not deployed?

to

action :deploy do
  delete_release_path_for(version.basename) if force
  next unless deployed?

There will probably need to be some other changes, but this is the simplest approach.

new snapshots are not deployed

not sure if i'm doing something wrong or if it's related to #75

i publish a tar.gz artifact to my snapshots repo, containing my project jar and it's dependent jar (assembled with the sbt-pack plugin). when i publish a new artifact with the same snapshot version (but the project jar internally has changed), the artifact is not pulled down again but the cached version is reused.

as an aside (not sure if it has any bearing here), doing a /artifact/maven/redirect&v=LATEST does not work on my snapshots repository for some reason (but does on my public proxied group), so i set the version explicitly.

Files written in a proc

If a file is written into the artifact deployment directory during one of the Proc calls, it will be included in the manifest.

If that file is a configuration file that can/should be changed throughout the lifetime of that version of the artifact, it should not be overwritten because the manifest check caused a redeployment.

Perhaps, we should add an attribute where we can list files to skip during manifest checking. Or skip all the files that end up in a shared_directory and make the documentation very clear.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.