alertlogic / al_agents Goto Github PK
View Code? Open in Web Editor NEWAlert Logic Agent Cookbook
Home Page: http://www.alertlogic.com/
License: Apache License 2.0
Alert Logic Agent Cookbook
Home Page: http://www.alertlogic.com/
License: Apache License 2.0
I've appied the default recipe to a windows node running 2012 R2, Yet the installation always fails. private key redacted.
Recipe: al_agents::_windows
* remote_file[al_agent-LATEST.msi] action create_if_missing
- create new file C:\chef\cache/al_agent-LATEST.msi
- update content in file C:\chef\cache/al_agent-LATEST.msi from none to 0d8fc0
(new content is binary, diff output suppressed)
* windows_package[al_agent-LATEST.msi] action install
================================================================================
Error executing action `install` on resource 'windows_package[al_agent-LATEST.msi]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1603'
---- Begin output of msiexec /qn /i "c:\chef\cache\al_agent-latest.msi" /quiet prov_key=xxxxxxxxxxxxx prov_only=host sensor_host=10.254.90.220 sensor_port=80 ----
STDOUT:
STDERR:
---- End output of msiexec /qn /i "c:\chef\cache\al_agent-latest.msi" /quiet prov_key=xxxxxxxxxxxxx prov_only=host sensor_host=10.254.90.220 sensor_port=80 ----
Ran msiexec /qn /i "c:\chef\cache\al_agent-latest.msi" /quiet prov_key=xxxxxxxxxxxxx prov_only=host sensor_host=10.254.90.220 sensor_port=80 returned 1603
Resource Declaration:
---------------------
# In C:/chef/cache/cookbooks/al_agents/recipes/_windows.rb
16: package basename do
17: source cached_package
18: action :install
19: options windows_options
20: not_if windows_install_guard
21: end
22:
Compiled Resource:
------------------
# Declared in C:/chef/cache/cookbooks/al_agents/recipes/_windows.rb:16:in `from_file'
windows_package("al_agent-LATEST.msi") do
action [:install]
retries 0
retry_delay 2
default_guard_interpreter :default
options "/quiet prov_key=xxxxxxxxxxxxx prov_only=host sensor_host=10.254.90.220 sensor_port=80"
package_name "al_agent-LATEST.msi"
source "c:\\chef\\cache\\al_agent-latest.msi"
version "2.1.2.0"
timeout 600
returns [0]
declared_type :package
cookbook_name "al_agents"
recipe_name "_windows"
not_if "C:\Program Files (x86)\Common Files\AlertLogic\host_key.pem"
end
I can't tell why the msi doesn't work. Can you provide any assistance to get this cookbook working?
Update
Looks like running the following MSI command manually works. (Removed the provision only flag)
msiexec /i "c:\chef\cache\al_agent-latest.msi" prov_key=xxxxxxx sensor_host=vaporator.alertlogic.com sensor_port=443���
There is an issue with the package resource as the "al-agent-LATEST-1.x86_64.rpm" installation fails during the Chef run on RHEL 6x. We need to mention the correct provider for this scenario.
The following fix helps
package basename do
source cached_package
action :install
provider Chef::Provider::Package::Dpkg if node['platform_family'] == 'debian'
provider Chef::Provider::Package::Rpm if node['platform_family'] == 'rhel' && node['platform_version'].to_i == 6
end
At minimum, let's please start tagging in sync with the cookbook metadata version. A github release with release notes included are then the shiny finishing touches.
This was previously issue #12. Two new releases went out without git tags or GitHub releases.
The default of https://vaporator.alertlogic.com:443/
does not work. If you remove the port, it will successfully provision itself.
I'm trying to get the agents to install through AWS OpsWorks (Chef 11.10) and receiving the following:
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of ./al-agent provision --key [KEY REDACTED] --inst-type host ----
STDOUT:
STDERR: May 11 22:40:20 2016 al-agent[24242]: ALC00089I [alc_config_set_api_key] setting api_key
May 11 22:40:20 2016 al-agent[24242]: ALC00083I [unregister] clean up registration artefacts
May 11 22:40:20 2016 al-agent[24242]: ALC00315E [alc_socket_send_request] Response http code is 301
May 11 22:40:20 2016 al-agent[24242]: ALC00074E [alc_config_provision_host] Could not issue provisioning request
May 11 22:40:20 2016 al-agent[24242]: ALC00079E [alc_config_init] Could not provision the host
---- End output of ./al-agent provision --key [KEY REDACTED] --inst-type host ----
Ran ./al-agent provision --key [KEY REDACTED] --inst-type host returned 1
Not sure where the 301 Moved Permanently is coming from....
Any ideas?
When the API Key is changed, the log agent doesn't replace the current key with the new.
The dependent cookbook, selinux_policy, require Chef 12 to run which is not available via OpsWorks. We needed to clone selinux_policy v1.1.1 and then make two further modifications so that the cookbook worked properly:
On Linux systems where rsyslog is installed but not enabled, the installation fails.
During the installation, these recipes are conditionally included:
include_recipe 'al_agents::selinux' if selinux_enabled?
include_recipe 'al_agents::rsyslog' if rsyslog_detected?
include_recipe 'al_agents::syslog_ng' if syslogng_detected?
The rsyslog_detected
helper function only detects whether rsyslog is installed, not whether it is enabled. So the corresponding recipe is runs to configure for rsyslog and restart the service (which fails). It then never gets to the syslog_ng check/installation.
Howdy!
TL;DR: libraries/helpers.rb
needs to be namespaced.
We were attempting to use this cookbook in conjunction with others, and we noticed that helpers.rb
isn't namespaced which causes issues with other cookbooks. I've made a simple example cookbook in the following repo:
https://github.com/brint/libexample
Run kitchen test
to see the trace. I've included a sample trace at the bottom of this issue.
The service_name function was the one that bit us. We started noticing al-agent
showing up as the service_name
with the mysql cookbook (~> 5.0). In the mysql cookbook, service_name
is a variable, and the helper function in this cookbook trumps the variable used in the library in their cookbook. Here's the specific line where the al_agents service_name
function is causing issues within the mysql cookbook:
https://github.com/chef-cookbooks/mysql/blob/v5.6.3/libraries/provider_mysql_service_rhel.rb#L123
Example trace using the libexample cookbook whose recipe only has a line of include_recipe 'mysql::server'
and dependencies of mysql and al_agents:
Recipe: mysql::server
================================================================================
Error executing action `create` on resource 'mysql_service[default]'
================================================================================
Chef::Exceptions::ResourceNotFound
----------------------------------
resource template[/etc/my.cnf] is configured to notify resource service[al-agent] with action restart, but service[al-agent] cannot be found in the resource collection. template[/etc/my.cnf] is defined in /tmp/kitchen/cache/cookbooks/mysql/libraries/provider_mysql_service_rhel.rb:101:in `block in <class:Rhel>'
Resource Declaration:
---------------------
# In /tmp/kitchen/cache/cookbooks/mysql/recipes/server.rb
20: mysql_service node['mysql']['service_name'] do
21: version node['mysql']['version']
22: port node['mysql']['port']
23: data_dir node['mysql']['data_dir']
24: server_root_password node['mysql']['server_root_password']
25: server_debian_password node['mysql']['server_debian_password']
26: server_repl_password node['mysql']['server_repl_password']
27: allow_remote_root node['mysql']['allow_remote_root']
28: remove_anonymous_users node['mysql']['remove_anonymous_users']
29: remove_test_database node['mysql']['remove_test_database']
30: root_network_acl node['mysql']['root_network_acl']
31: package_version node['mysql']['server_package_version']
32: package_action node['mysql']['server_package_action']
33: enable_utf8 node['mysql']['enable_utf8']
34: action :create
35: end
Compiled Resource:
------------------
# Declared in /tmp/kitchen/cache/cookbooks/mysql/recipes/server.rb:20:in `from_file'
mysql_service("default") do
action [:create]
retries 0
retry_delay 2
default_guard_interpreter :default
declared_type :mysql_service
cookbook_name "mysql"
recipe_name "server"
port "3306"
data_dir "/var/lib/mysql"
server_root_password "mysql_test"
server_debian_password "gnuslashlinux4ev4r"
remove_anonymous_users true
remove_test_database true
package_action "install"
end
Running handlers:
[2015-08-12T02:06:23+00:00] ERROR: Running exception handlers
Running handlers complete
[2015-08-12T02:06:23+00:00] ERROR: Exception handlers complete
Chef Client failed. 0 resources updated in 3.364878055 seconds
[2015-08-12T02:06:23+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
[2015-08-12T02:06:23+00:00] ERROR: mysql_service[default] (mysql::server line 20) had an error: Chef::Exceptions::ResourceNotFound: resource template[/etc/my.cnf] is configured to notify resource service[al-agent] with action restart, but service[al-agent] cannot be found in the resource collection. template[/etc/my.cnf] is defined in /tmp/kitchen/cache/cookbooks/mysql/libraries/provider_mysql_service_rhel.rb:101:in `block in <class:Rhel>'
[2015-08-12T02:06:24+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Please lets start, at minimum, start tagging in sync with the cookbook metadata version. A github release with release notes included are then the shiny finish.
The line at https://github.com/alertlogic/al_agents/blob/master/recipes/rsyslog.rb#L14 references a helper in rsyslog that was removed 24 days ago -
sous-chefs/rsyslog@c696b7d#diff-38a4aeddb4e620724d3b85362cb8b939
This throws an exception when using that cookbook - "No resource, method, or local variable named declare_rsyslog_service' for
Chef::Recipe "rsyslog"'"
We use Chef to build our AMIs and only want to install the al_agent package (not to configure or provision). We will handle configuration and provisioning at boot time. We hacked this into a forked repo (al_agents::package
), but it would be nice to have this option available in the main cookbook.
Are we ok with removing deprecated recipes?
Otherwise, If we holding this for a while, all changes must be done everywhere, including #4
Out of curiosity, where all cloud-init
related stuff disappeared?
And what is licence for this project right now?
Since the communication is local, packet loss should not be an issue, what was the though process around the use of UDP? Or even better the use of a socket?
Would it be possible to have an attribute (or some mechanism) whereby we can allow failures for various execute statements? We have a problem when the recipe runs the ./al-agent provision
command we get a non-zero errorcode if our AlertLogic appliance is offline. Although AlertLogic being unavailable is a problem, we do not want it to stop the chef run and prevent our application deployments.
Under normal circumstances, I'd use a wrapper cookbook to target the particular execute statement and add the ignore_failure true
property, however, since the name of the execute statement is based on a variable that's generated from a library, I'm not sure of a way of getting around this.
Any advice welcome.
By default the msi configures to log to the following egress_url
"egress_url": "https://vaporator.alertlogic.com:443"
Is there a way to change that later; maybe with an environment variable or config file? Or are you expected to destroy and recreate your nodes?
Every time chef runs, the cookbook tries to open the .pem file for some reason.
This results in a pop up on my screen
PS C:\chef\cache> chef-client -l debug -L c:\foo.log
Starting Chef Client, version 12.5.1
resolving cookbooks for run list: ["al_agents::default"]
Synchronizing Cookbooks:
- al_agents (1.0.8)
- line (0.6.3)
- rsyslog (2.2.0)
- selinux_policy (0.9.2)
Compiling Cookbooks...
Converging 3 resources
Recipe: al_agents::_windows
* remote_file[al_agent-LATEST.msi] action create_if_missing (up to date)
* windows_package[al_agent-LATEST.msi] action install (skipped due to not_if)
Recipe: al_agents::start
* windows_service[al_agent] action start (up to date)
Running handlers:
Running handlers complete
Chef Client finished, 0/3 resources updated in 14 seconds���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������
As a user of the al_agent cookbook
I want to reference al_agent in metadata.rb
And I do not want to reference the github location in Berksfile
So that it is very easy to use the al_agent cookbook
Today I must have a depends 'al_agent', '~> 0.3'
in my metadata.rb
And I must have cookbook 'al_agents', git: 'https://github.com/alertlogic/al_agents.git', tag: 'v0.3.0'
If al_agent cookbook were loaded into the supermarket
Then only the entry in metadata.rb
would be required.
On every run, the code appears to attempt to add info to syslog-ng, even though it is not installed.
From Direct Supply (Customer):
There seems to be explicit support for the ‘rhel’, ‘fedora’, and ‘debian’ platform families, based on how it gets the package to install for the agent, but not for ‘amazon’ which is what Amazon Linux is classified as. As such, it fails when trying to get the package to install the agent
========================================================================================
Recipe Compile Error in /tmp/kitchen/cache/cookbooks/amazonlinux-base/recipes/default.rb
========================================================================================
URI::InvalidURIError
--------------------
bad URI(is not URI?):
Cookbook Trace:
---------------
/tmp/kitchen/cache/cookbooks/al_agents/libraries/helpers.rb:14:in `agent_file'
/tmp/kitchen/cache/cookbooks/al_agents/libraries/helpers.rb:18:in `agent_basename'
/tmp/kitchen/cache/cookbooks/al_agents/recipes/install.rb:11:in `from_file'
/tmp/kitchen/cache/cookbooks/al_agents/recipes/_linux.rb:7:in `from_file'
/tmp/kitchen/cache/cookbooks/al_agents/recipes/default.rb:11:in `from_file'
/tmp/kitchen/cache/cookbooks/amazonlinux-base/recipes/default.rb:120:in `from_file'
Relevant File Content:
----------------------
/tmp/kitchen/cache/cookbooks/al_agents/libraries/helpers.rb:
7: def al_agent
8: node['al_agents']['package']['name']
9: end
10:
11: def agent_file(uri)
12: require 'pathname'
13: require 'uri'
14>> Pathname.new(URI.parse(uri).path).basename.to_s
15: end
16:
17: def agent_basename
18: agent_file(node['al_agents']['package']['url'])
19: end
20:
21: def al_agent_service
22: node['al_agents']['agent']['al_agent_service']
23: end
If you use the alert logic cookbook on windows, it is easy to get conflicting variables.
For example, if I have a cookbook that uses a variable or method named "service_name", the alert logic cookbook will overwrite this setting, resulting in weird things as discussed in the mailing list:
https://discourse.chef.io/t/how-do-you-avoid-variable-conflicts-in-custom-resources/8296/5
The best solution would probably be to not send the entire helper to all chef resources, instead only send it to the resources needed.
Instead of this:
::Chef::Resource.send(:include, AlAgents::Helpers)
Do this:
::Chef::Resource::Batch.send(:include, AlAgents::Helpers)
That way other cookbooks have a much narrower risk of collisions.
The readme doesn't specify windows as being supported, however there does appear to be a windows recipe.
Is windows supported? If so, which versions are tested?
I'd like to use the non rsyslog lock version
The agent supports SUSE but the cookbook does not.
Can this be added please?
Rsyslog being pinned at such an old version is causing issues in our environment and is forcing us to use 1.0.6 of this cookbook. Re-factoring to support newer versions of rsyslog would be great!
I believe the logic for the for_imaging
flag is wrong. According to the readme, the default value of for_imaging
is false.
al_agents/libraries/helpers.rb
Line 90 in 51b7799
Yet if you run the cookbook without specifying this attribute, then the installer installs in provisioning-only mode.
Here is the command that chef ran.
msiexec /qn /i "c:\chef\cache\al_agent-latest.msi" /quiet prov_key=xxx
prov_only=host sensor_host=xxxxxxxx sensor_port=80
If you run this command without the /qn
option, you see the following. Notice the "provisioning-only mode" at the bottom? That shouldn't be there if prov_only
is actually false
Here is proof that the value is set to false
PS C:\chef\cache> chef-shell -z
chef (12.5.1)> puts node['al_agents']['agent']['for_imaging']
false
=> nil
chef (12.5.1)> puts node['al_agents']['agent']['for_autoscaling']
false
chef (12.5.1)> puts node['al_agents']
{"agent"=>{"registration_key"=>"xxxxxxxxx", "egress_url"=>"xxxxxxxxxx", "proxy_url"=>nil, "for_autoscaling"=>false, "for_imaging"=>false, "service_name"=>"al_agent"}, "package"=>{"name"=>"al-agent", "url"=>"https://scc.alertlogic.net/software/al_agent-LATEST.msi"}, "windows_install_guard"=>"C:\\Program Files (x86)\\Common Files\\AlertLogic\\host_key.pem"}
You must set the value to false in the role. The msi then installs properly. I can reproduce this consistently.
"al_agents": {
"agent": {
"registration_key": "xxxxxxxx",
"egress_url": "https://vaporator.alertlogic.com:443",
"for_imaging": false,
"for_autoscaling": false
}
},
Even though there's an Apache v2 LICENSE, you may want to clean up the boilerplate headers on your files.
CentOS 6+ uses s_all not s_sys for syslog-ng global logger. This will cause syslog-ng to fail to start
The correction needed is in defaults.rb to evaluate what platform_version we are on.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.