saltstack-formulas / salt-formula Goto Github PK
View Code? Open in Web Editor NEWYes, Salt can Salt itself!
Home Page: http://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html
License: Other
Yes, Salt can Salt itself!
Home Page: http://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html
License: Other
I've recently converted a bunch of formulas I cloned and then referenced via file_roots to local git repos referenced from gitfs_remotes. A couple of formulas now break whenever a dictionary item can't be resolved.
I am using CentOS, and this formula fails because it can't resolve salt.pkrepo.centos. Since this was working fine when pulled from file_roots I am not sure what to say, although wrapping the contents of salt/pkgrepo/init.sls in an if checking for os_family == 'Debian'
eliminates the problem for me.
How about salting/upgrading salt minions on windows in a stable way? I cant see support for that in this formula.
I have some work implemented for this but its very "hacky" due to that nssm.exe often hangs and locks files after an uninstall. To come around this I'm rebooting the server and continue the installation afterwards but this is far from optimal.
I can however, share the work if anyone is interested.
Formulas iterating over dictionaries should use the items()
method instead of iteritems()
because the latter method has been removed from Python 3. While the items()
method in Python 2 uses more memory than iteritems()
, these dictionaries are typically small (e.g., sets of settings from Pillar).
When the master is also being configured. The problem is this line in salt/files/minion:
{{ get_config('master', 'salt') }}
If pillars['salt']['minion']['master']
does not exist, it will try pillars['salt']['master']
, which of course is itself a dict.
Ran into an issue today: salt.minion does all of its management in minion.d and doesn't touch the /etc/salt/minion file. salt-cloud does the opposite. Usually the settings in the former override the latter so everything is okay. However, for features that can be configured either from the minion config or the pillar root (such as mine_functions), the cloud-deployed file takes precedence over pillar, and salt.minion has no way of cleaning it up after the fact.
I don't think this is really a bug, but removing those settings after the fact is surprisingly irritating given that salt.minion won't touch that file.
I don't know the correct way to deal with this, but it seems like some option for managing /etc/salt/minion or otherwise neutering it (perhaps off by default) would be a helpful thing to have.
Salting minion fails on all RHEL 6 boxes with python-jinja2-2.2.1-2.el6_5.x86_64. Same run succeeds on RHEL 7 boxes with python-jinja2-2.7.2-2.el7.noarch.
----------
ID: salt-minion
Function: file.recurse
Name: /etc/salt/minion.d
Result: False
Comment: #### /etc/salt/minion.d/f_defaults.conf ####
Unable to manage file: Jinja syntax error: no test named 'mapping'
/var/cache/salt/minion/files/prod/salt/formulas.jinja(14):
---
[...]
{%- macro formulas_git_opt(env, opt) -%}
{%- set value = salt['pillar.get']('salt_formulas:git_opts:{0}:{1}'.format(env, opt),
salt['pillar.get']('salt_formulas:git_opts:default:{0}'.format(opt),
defaults[opt])) -%}
{%- if value is mapping -%} <======================
{{ value|yaml }}
{%- else -%}
{{ value }}
{%- endif -%}
{%- endmacro -%}
[...]
---
Started: 22:55:39.879651
Duration: 133.819 ms
Changes:
as of 2015.8.x, vmware as a provider is available, as such i've manually altered the formula to add that functionality locally:
I'm raising this issue, because this functionality should probably exist in the cloud formula going forward.
if people are interested, i can prep a PR for review.
One can not fork "saltstack/salt" and " saltstack-formulas/salt".
And having a "salt" outside saltstack-formulas is misleading.
This issue applies to saltstack-formulas/*
Maybe prefix with something like salt-formula-* ?
i.e. {% set salt = .... BAD!~!!!
I'm new to Salt. I've read the documentation fairly extensively but without a "here's how to run this" in the README.md
I'm unlikely to have time to figure out how to use this, even though I would really like to.
Can I use salt-ssh
from my local machine to use this to deploy salt onto a remote machine? If so, how? Can it be explained in the README?
Running the salt.formulas state results in the following error:
root@uxeprdbsdsalt01:~ # salt-call state.sls salt.formulas saltenv=production test=True
[INFO ] Loading fresh modules for state activity
[INFO ] Fetching file from saltenv 'production', ** skipped ** latest already in cache 'salt://salt/formulas.sls'
[INFO ] Fetching file from saltenv 'production', ** done ** 'salt/formulas.jinja'
[CRITICAL] Rendering SLS 'production:salt.formulas' failed: mapping values are not allowed here; line 18
---
[...]
# Setup the directory hosting the Git repository
/usr/local/etc/salt/formulas
...: <======================
file.directory:
- makedirs: True
- group: wheel
- user: root
- mode: 755
[...]
---
local:
Data failed to compile:
----------
Rendering SLS 'production:salt.formulas' failed: mapping values are not allowed here; line 18
---
[...]
# Setup the directory hosting the Git repository
/usr/local/etc/salt/formulas
...: <======================
file.directory:
- makedirs: True
- group: wheel
- user: root
- mode: 755
[...]
---
Running the salt.master state results in the following incorrect rendering (truncated for brevity's sake):
root@uxeprdbsdsalt01:~ # salt-call state.sls salt.master saltenv=development test=True
[INFO ] Loading fresh modules for state activity
[INFO ] Fetching file from saltenv 'development', ** skipped ** latest already in cache 'salt://salt/master.sls'
[INFO ] Fetching file from saltenv 'development', ** done ** 'salt/map.jinja'
[INFO ] Fetching file from saltenv 'development', ** done ** 'salt/defaults.yaml'
[INFO ] Executing command '/sbin/zfs -?' in directory '/root'
[INFO ] Running state [py27-salt] at time 14:50:01.749847
[INFO ] Executing state pkg.installed for py27-salt
[INFO ] Executing command 'pkg info -ao' in directory '/root'
[INFO ] Package py27-salt is already installed.
[INFO ] Completed state [py27-salt] at time 14:50:01.963569
[INFO ] Running state [/usr/local/etc/salt/master.d] at time 14:50:01.968229
[INFO ] Executing state file.recurse for /usr/local/etc/salt/master.d
[INFO ] Fetching file from saltenv 'development', ** skipped ** latest already in cache u'salt://salt/files/master.d/f_defaults.conf'
[INFO ] Fetching file from saltenv 'development', ** skipped ** latest already in cache 'salt://salt/formulas.jinja'
[INFO ] Fetching file from saltenv 'development', ** skipped ** latest already in cache u'salt://salt/files/master.d/reactor.conf'
[INFO ] {'/usr/local/etc/salt/master.d/f_defaults.conf': {'diff': '--- \n+++ \n@@ -431,77 +431,146 @@\n - /code/twgs-formula\n - /code/users-formula\n production:\n- - /usr/local/etc/salt/formulas/active-directory-formula\n- - /usr/local/etc/salt/formulas/epel-formula\n- - /usr/local/etc/salt/formulas/fail2ban-formula\n- - /usr/local/etc/salt/formulas/firewalld-formula\n- - /usr/local/etc/salt/formulas/git-formula\n- - /usr/local/etc/salt/formulas/jenkins-formula\n- - /usr/local/etc/salt/formulas/mysql-formula\n- - /usr/local/etc/salt/formulas/ntp-formula\n- - /usr/local/etc/salt/formulas/nux-formula\n- - /usr/local/etc/salt/formulas/openstack-formula\n- - /usr/local/etc/salt/formulas/openssh-formula\n- - /usr
...
----------
ID: salt-master
Function: file.recurse
Name: /usr/local/etc/salt/master.d
Result: None
Comment: #### /usr/local/etc/salt/master.d/f_defaults.conf ####
The file /usr/local/etc/salt/master.d/f_defaults.conf is set to be changed
Started: 14:50:01.968229
Duration: 890.319 ms
Changes:
----------
/usr/local/etc/salt/master.d/f_defaults.conf:
----------
diff:
---
+++
@@ -431,77 +431,146 @@
- /code/twgs-formula
- /code/users-formula
production:
- - /usr/local/etc/salt/formulas/active-directory-formula
- - /usr/local/etc/salt/formulas/epel-formula
- - /usr/local/etc/salt/formulas/fail2ban-formula
- - /usr/local/etc/salt/formulas/firewalld-formula
- - /usr/local/etc/salt/formulas/git-formula
- - /usr/local/etc/salt/formulas/jenkins-formula
- - /usr/local/etc/salt/formulas/mysql-formula
- - /usr/local/etc/salt/formulas/ntp-formula
- - /usr/local/etc/salt/formulas/nux-formula
- - /usr/local/etc/salt/formulas/openstack-formula
- - /usr/local/etc/salt/formulas/openssh-formula
- - /usr/local/etc/salt/formulas/os-hardening-formula
- - /usr/local/etc/salt/formulas/owncloud-formula
- - /usr/local/etc/salt/formulas/postgres-formula
- - /usr/local/etc/salt/formulas/rabbitmq-formula
- - /usr/local/etc/salt/formulas/salt-formula
- - /usr/local/etc/salt/formulas/shibboleth-formula
- - /usr/local/etc/salt/formulas/snmp-formula
- - /usr/local/etc/salt/formulas/spigotmc-formula
- - /usr/local/etc/salt/formulas/sudoers-formula
- - /usr/local/etc/salt/formulas/tomcat-formula
- - /usr/local/etc/salt/formulas/twgs-formula
- - /usr/local/etc/salt/formulas/users-formula
+ - /usr/local/etc/salt/formulas
+.../active-directory-formula
+ - /usr/local/etc/salt/formulas
+.../epel-formula
+ - /usr/local/etc/salt/formulas
+.../fail2ban-formula
+ - /usr/local/etc/salt/formulas
+.../firewalld-formula
+ - /usr/local/etc/salt/formulas
+.../git-formula
+ - /usr/local/etc/salt/formulas
+.../jenkins-formula
+ - /usr/local/etc/salt/formulas
+.../mysql-formula
+ - /usr/local/etc/salt/formulas
+.../ntp-formula
+ - /usr/local/etc/salt/formulas
+.../nux-formula
+ - /usr/local/etc/salt/formulas
+.../openstack-formula
+ - /usr/local/etc/salt/formulas
+.../openssh-formula
+ - /usr/local/etc/salt/formulas
+.../os-hardening-formula
+ - /usr/local/etc/salt/formulas
+.../owncloud-formula
+ - /usr/local/etc/salt/formulas
+.../postgres-formula
+ - /usr/local/etc/salt/formulas
+.../rabbitmq-formula
+ - /usr/local/etc/salt/formulas
+.../salt-formula
+ - /usr/local/etc/salt/formulas
+.../shibboleth-formula
The salt code (and myself) put files in /etc/salt/{master,minion}.d
The "clean: True" is killing all that. It probably shouldn't do that.
salt-minion:
file.recurse:
- name: {{ salt.get('config-path', '/etc/salt') }}/minion.d
- template: jinja
- source: salt://salt/files/minion.d
- clean: True
- context:
standalone: False
Accepting minions after installation with this formula will make provisioning much easier.
In terms of testing the Salt project in general it would be cool if this formula offered a streamlined way to install the Salt codebase from a GitHub Pull Request ref point.
Just a thought. Feel free to close this up if that is too crazy an idea!
The salt-cloud provider configuration files (.../cloud.providers.d/*.conf) may contain keying material (e.g., passwords or secret keys). The permissions of these files should be restricted to some group of authorized users (e.g., root or wheel) in order to prevent unauthorized access.
It would be awesome if the salt-formula could be configured to clone Salt from the Git repo (e.g. giving it a version tag), and then it set up a virtualenv and made sure the minion/master were running from source, simultaneously disabling any distro-wide installations.
I'm attempting to use salt-ssh 2015.5.0 to set up a salt master. The salt.master state is failing.
----------
ID: salt-master
Function: file.recurse
Name: /etc/salt/master.d
Result: False
Comment: #### /etc/salt/master.d/f_defaults.conf ####
Unable to manage file: Jinja error: salt/formulas.jinja
Traceback (most recent call last):
File "/tmp/.root_679d2e__salt/salt/utils/templates.py", line 306, in render_jinja_tmpl
output = template.render(**decoded_context)
File "/tmp/.root_679d2e__salt/jinja2/environment.py", line 969, in render
return self.environment.handle_exception(exc_info, True)
File "/tmp/.root_679d2e__salt/jinja2/environment.py", line 742, in handle_exception
reraise(exc_type, exc_value, tb)
File "<template>", line 15, in top-level template code
File "/tmp/.root_679d2e__salt/salt/utils/jinja.py", line 110, in get_source
raise TemplateNotFound(template)
TemplateNotFound: salt/formulas.jinja
; line 15
---
[...]
{{ configname }}: {{ cfg_salt[configname] }}
{%- else -%}
#{{ configname }}: {{ default_value }}
{%- endif -%}
{%- endmacro -%}
{%- from 'salt/formulas.jinja' import file_roots, formulas with context -%} <======================
##### Primary configuration settings #####
##########################################
# This configuration file is used to manage the behavior of the Salt Master
# Values that are commented out but have no space after the comment are
# defaults that need not be set in the config. If there is a space after the
[...]
---
Started: 04:18:21.410948
Duration: 271.237 ms
Changes:
I cloned the formula manually and have it referenced in my file_roots. Not sure if that would have an impact over bringing it in via gitfs_remotes, but I'm trying to avoid adding the gitfs dependencies.
I removed the salt_formulas: thing from my pillar, but forgot the salt/top.sls with the salt.formulas there and it started to fail.
[INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/master.sls'
[INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja'
[INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml'
[INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/minion.sls'
[INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja'
[INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml'
[INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/formulas.sls'
[INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/formulas.jinja'
[CRITICAL] Rendering SLS 'base:salt.formulas' failed: Jinja variable 'str object' has no attribute 'iteritems'
local:
Data failed to compile:
----------
Rendering SLS 'base:salt.formulas' failed: Jinja variable 'str object' has no attribute 'iteritems'
I rebuilt my salt master today and the salt.cloud state refuses to run, saying that the attempt to pip.install package 'crypto' failed. I tried it manually and it also failed. I looked at the package index located here: https://pypi.python.org/pypi?%3Aaction=index and see no package named 'crypto.'
I am confused, because this worked a month ago. Did the package get renamed in pip recently, or something? Shouldn't the distribution-provided crypto package be getting used anyway, if possible?
As reported on this mailinglist post, I was hoping that salt-master would automatically restart when a config file changes there.
salt-formula/salt/files/master.d/_defaults.conf
Lines 486 to 502 in 0f5f976
is iterable
doesn't seem to be a deep enough check.
My salt.sls pillar looks something like this:
salt:
master:
fileserver_backend:
- git
- roots
gitfs_remotes:
- [email protected]:saltstack-formulas/salt-formula.git
And if the remote item doesn't have a child as the above doesn't (note that there's no data about the remote, just the address) it fails with the following jinja error:
I get the following jinja error when running state.highstate:
ID: salt-master
Function: file.recurse
Name: /etc/salt/master.d
Result: False
Comment: #### /etc/salt/master.d/_defaults.conf ####
Unable to manage file: Jinja variable 'str object' has no attribute 'iteritems'
Started: 03:18:47.515161
Duration: 153.94 ms
Changes:
If I add a remote child, it starts working:
gitfs_remotes:
- [email protected]:saltstack-formulas/salt-formula.git:
- base: master
salt versions-report is:
Salt: 2014.7.0
Python: 2.7.5 (default, Jun 17 2014, 18:11:42)
Jinja2: 2.7.2
M2Crypto: 0.21.1
msgpack-python: 0.4.2
msgpack-pure: Not Installed
pycrypto: 2.6.1
libnacl: Not Installed
PyYAML: 3.10
ioflo: Not Installed
PyZMQ: 14.3.1
RAET: Not Installed
ZMQ: 4.0.4
Mako: Not Installed
In context, see cb3aa80#diff-cc4a14e8ec642fa03a6038c172bd52b0L46.
The original Pillar lookup had nothing to do with the salt variable that was renamed to salt_settings. Instead, it was querying the value of the Pillar key "salt_cloud_certs". This commit changes the Pillar key to "salt:salt_cloud_certs" without also updating https://github.com/saltstack-formulas/salt-formula/blob/master/salt/files/key and https://github.com/saltstack-formulas/salt-formula/blob/master/pillar.example, and it violates POLA in that it breaks working salt-cloud configurations managed by salt-formula.
in the latest salt version 2015.8.0rc1 the source salt://{{ slspath }} doesn't work anymore
file_|-salt-master_|-/etc/salt/master.d_|-recurse:
----------
__run_num__:
1
changes:
----------
comment:
#### /etc/salt/master.d/reactor.conf ####
Unable to manage file: [Errno 2] No such file or directory: u'|/var/cache/salt/minion/files/base/salt/files/master.d/reactor.conf'
#### /etc/salt/master.d/f_defaults.conf ####
Unable to manage file: [Errno 2] No such file or directory: u'|/var/cache/salt/minion/files/base/salt/files/master.d/f_defaults.conf'
duration:
52.152 ms
name:
/etc/salt/master.d
result:
False
start_time:
21:36:23.706337
if I change the {{ slspath }} back to the hardcoded salt://salt/files/ it works
The dev directory (the vagrant files, etc.) gets plopped down at the top level of your salt:// fileserver root. Seems like it could cause a name conflict. Maybe it should be namespaced?
Hi all,
Is there a reason why something like : http://docs.saltstack.com/en/latest/faq.html#what-is-the-best-way-to-restart-a-salt-daemon-using-salt is not used in this formula ?
I changed salt-formula (locally), it's a huge win : I can now spawn a new salt minion (with salt-cloud) in one highstate run, without interruption.
salt/minion.sls :
cmd.wait:
- name: echo salt-call --local service.restart salt-minion | at now + 1 minute
- python_shell: True
- order: last
- watch:
{% if salt_settings.install_packages %}
- pkg: salt-minion
{% endif %}
- file: salt-minion
- file: remove-old-minion-conf-file
I didn't manage "atd" since it's on by default on Debian, but it's definitely doable.
(Warning: I'm a Salt newb)
Running the following from a clean checkout of salt-formula:
vagrant up
vagrant ssh
vagrant@salt:~$ sudo su -
root@salt:~# salt-call state.highstate
I get the following failed states:
----------
ID: salt-minion
Function: file.recurse
Name: /etc/salt/minion.d
Result: False
Comment: Recurse failed: none of the specified sources were found
Started: 23:03:24.860639
Duration: 8292.687 ms
Changes:
----------
ID: salt-minion
Function: service.running
Result: False
Comment: One or more requisite failed: {'salt.minion.salt-minion': 'Recurse failed: none of the specified sources were found'}
Started:
Duration:
Changes:
----------
ID: cloud-cert-aws-pem
Function: file.managed
Name: /etc/salt/cloud.providers.d/key/aws.pem
Result: False
Comment: Unable to manage file: string indices must be integers, not str
Started: 23:04:46.656005
Duration: 8.735 ms
Changes:
----------
ID: cloud-cert-gce-pem
Function: file.managed
Name: /etc/salt/cloud.providers.d/key/gce.pem
Result: False
Comment: Unable to manage file: string indices must be integers, not str
Started: 23:04:46.666232
Duration: 11.163 ms
Changes:
----------
ID: salt-cloud-profiles-ec2
Function: file.managed
Name: /etc/salt/cloud.profiles.d/ec2.conf
Result: False
Comment: Unable to manage file: string indices must be integers, not str
Started: 23:04:46.677963
Duration: 7.45 ms
Changes:
----------
ID: salt-cloud-profiles-gce
Function: file.managed
Name: /etc/salt/cloud.profiles.d/gce.conf
Result: False
Comment: Unable to manage file: string indices must be integers, not str
Started: 23:04:46.686129
Duration: 12.509 ms
Changes:
----------
ID: salt-cloud-providers-ec2
Function: file.managed
Name: /etc/salt/cloud.providers.d/ec2.conf
Result: False
Comment: Unable to manage file: string indices must be integers, not str
Started: 23:04:46.699382
Duration: 3.817 ms
Changes:
----------
ID: salt-cloud-providers-gce
Function: file.managed
Name: /etc/salt/cloud.providers.d/gce.conf
Result: False
Comment: Unable to manage file: string indices must be integers, not str
Started: 23:04:46.703812
Duration: 5.745 ms
Changes:
----------
ID: salt-cloud-maps-ec2
Function: file.managed
Name: /etc/salt/cloud.maps.d/ec2.conf
Result: False
Comment: Unable to manage file: string indices must be integers, not str
Started: 23:04:46.710319
Duration: 6.986 ms
Changes:
----------
ID: salt-cloud-maps-gce
Function: file.managed
Name: /etc/salt/cloud.maps.d/gce.conf
Result: False
Comment: Unable to manage file: string indices must be integers, not str
Started: 23:04:46.722172
Duration: 10.791 ms
Changes:
After this failure if I run:
salt-run fileserver.update
salt-call state.highstate
all states succeed.
What is the ideal set of commands to go from 'vagrant up' to all states passing with no failures? Not sure if I am missing something... for a new person seeing a successful run of this formula from a clean repo would be a good confidence builder :).
For example in this one we can see many default comments.
Although this is important for new users, the comments make much harder to navigate among all this lines.
Moreover if meaning of some parameter changes and we don't syncronize the change with original, our words will be confusing.
What do you think about removing these comments?
i.e. style-short-dec branch
I would like to propose the following change:
if a config file is changed by pillar data put the file in:
/etc/salt/master.d/master.conf
/etc/salt/minion.d/minion.conf
So not to overwrite the defaults salt package provides ?
If i'm not mistaken salt will prefer the files in master.d and minion.d
because all defaults are commented.
Unfortunately, since CentOS 6 only ships with Jinja2 2.2.1, the mapping
test is not supported. Looking through the documentation, I don't see anything else equivalent to it (both iterable
and sequence
test true for dicts and lists), so I'm not sure how to maintian functionality without that test.
Tapping @Sjd-Risca, just in case they have some idea
I am looking to have the state do the steps outlined in this doc out of the box.
apt-get install python-service-identity python-cherrypy
salt-call --local tls.create_self_signed_cert
http://docs.saltstack.com/en/latest/ref/netapi/all/salt.netapi.rest_cherrypy.html
I can not see how to use this has a git submodule.
I'm forced to create a state to override all salt:// declarations with include and extend to get things work.
Not being able to use this as submodule reduce the interest of the project from "plug and play" to "cherry pick". And cherry pick won't bring contributions..
I've found no solution for now. But I'm new to saltstack. I may have missed something.
Due to how the get_config macro
works, if you don't have the master
option set in salt.minion
, and you have a salt.master
config, then the entire salt.master
config is mistakenly used as the value for the master
option for the minion.
The process of installing from source must be completely overhauled.
A binary package named py27-pygit2 is available on FreeBSD.
It looks like mapping test requires Jinja2 v2.6 or higher and I'm on 2.2.1 - which is the best RHEL 6 provides.
ID: salt-minion
Function: file.recurse
Name: /etc/salt/minion.d
Result: False
Comment: #### /etc/salt/minion.d/f_defaults.conf ####
Unable to manage file: Jinja syntax error: no test named 'mapping'
/var/cache/salt/minion/files/base/salt/formulas.jinja(14):
---
[...]
{%- macro formulas_git_opt(env, opt) -%}
{%- set value = salt['pillar.get']('salt_formulas:git_opts:{0}:{1}'.format(env, opt),
salt['pillar.get']('salt_formulas:git_opts:default:{0}'.format(opt),
defaults[opt])) -%}
{%- if value is mapping -%} <======================
{{ value|yaml }}
{%- else -%}
{{ value }}
{%- endif -%}
{%- endmacro -%}
# salt-master --versions
Salt: 2015.5.3
Python: 2.6.6 (r266:84292, Nov 21 2013, 10:50:32)
Jinja2: 2.2.1
M2Crypto: 0.20.2
msgpack-python: 0.4.6
msgpack-pure: Not Installed
pycrypto: 2.0.1
libnacl: Not Installed
PyYAML: 3.10
ioflo: Not Installed
PyZMQ: 14.5.0
RAET: Not Installed
ZMQ: 4.0.5
Mako: 0.3.4
Tornado: Not Installed
gitfs
support makes use of a pip.installed
state in several places.
When pip
isn't installed, applying the salt.gitfs
state will fail like this:
# salt-call state.sls salt.gitfs.pygit2 test=true
[INFO ] Loading fresh modules for state activity
[INFO ] Fetching file from saltenv 'base', ** done ** u'salt/gitfs/pygit2.sls'
[INFO ] Fetching file from saltenv 'base', ** done ** u'salt/map.jinja'
[INFO ] Fetching file from saltenv 'base', ** done ** u'salt/defaults.yaml'
[ERROR ] You should upgrade pyOpenSSL to at least 0.14.1 to enable the use of X509 extensions
[INFO ] Running state [git] at time 11:41:13.008687
[INFO ] Executing state pkg.installed for git
[INFO ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}\n', '-W'] in directory '/root'
[INFO ] Package git is already installed
[INFO ] Completed state [git] at time 11:41:13.499474
[INFO ] Running state [libgit2-22] at time 11:41:13.499758
[INFO ] Executing state pkg.installed for libgit2-22
[INFO ] The following packages are set to be installed/updated: libgit2-22
[INFO ] Completed state [libgit2-22] at time 11:41:13.504759
[INFO ] Running state [pygit2 == 0.22.1] at time 11:41:13.505978
[INFO ] Executing state pip.installed for pygit2 == 0.22.1
[ERROR ] An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
[INFO ] Completed state [pygit2 == 0.22.1] at time 11:41:13.506796
local:
----------
ID: pygit2-git
Function: pkg.installed
Name: git
Result: True
Comment: Package git is already installed
Started: 11:41:13.008687
Duration: 490.787 ms
Changes:
----------
ID: libgit2-22
Function: pkg.installed
Result: None
Comment: The following packages are set to be installed/updated: libgit2-22
Started: 11:41:13.499758
Duration: 5.001 ms
Changes:
----------
ID: install-pygit2
Function: pip.installed
Name: pygit2 == 0.22.1
Result: False
Comment: An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
Started: 11:41:13.505978
Duration: 0.818 ms
Changes:
Summary for local
------------
Succeeded: 2 (unchanged=1)
Failed: 1
------------
Total states run: 3
Total run time: 496.606 ms
77522d6 added the base keyword to filter_by() in map.jinja. This is not available until the next release.
# salt-call state.sls salt.syndic test=True [INFO ] Loading fresh modules for state activity [INFO ] Fetching file from saltenv 'base', ** done ** 'salt/syndic.sls' [INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja' [INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml' [INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/master.sls' [INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja' [INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml' local: Data failed to compile: ---------- A recursive requisite was found, SLS "salt.syndic" ID "salt-syndic" ID "salt-syndic"
Looks like this is most prevalent here: https://github.com/saltstack-formulas/salt-formula/blob/master/salt/ssh.sls (shame on you @timoguin ;) ). We should fix that.
I just posted this issue with SaltStack as I believe the bug exists in salt-ssh
rather than in this formula directly. This is really just for visibility.
Brief summary:
When calling salt.minion via salt-ssh
, the formulas.jinja
file is not included and so the minion.d/f_defaults.conf
file fails to render.
Workaround is to simply modify the following lines in salt/files/minion.d/f_defaults.conf
:
@@ -12,7 +12,6 @@
#{{ configname }}: {{ default_value }}
{%- endif -%}
{%- endmacro -%}
-{%- from 'salt/formulas.jinja' import file_roots, formulas with context -%}
##### Primary configuration settings #####
##########################################
@@ -235,8 +234,6 @@ file_client: local
{{ file_roots(cfg_minion['file_roots']) }}
{%- elif 'file_roots' in cfg_salt -%}
{{ file_roots(cfg_salt['file_roots']) }}
-{%- elif formulas|length -%}
-{{ file_roots({'base': ['/srv/salt']}) }}
{%- endif %}
# limit fileserver traversal
I'm getting an error when trying use this state.
johncc@aws-dev-c-salt-1:~⟫ sudo salt-call -l debug state.sls salt.gitfs.gitpython
[sudo] password for johncc:
[DEBUG ] Reading configuration from /etc/salt/minion
[DEBUG ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG ] Including configuration from '/etc/salt/minion.d/f_defaults.conf'
[DEBUG ] Reading configuration from /etc/salt/minion.d/f_defaults.conf
[DEBUG ] Configuration file path: /etc/salt/minion
[DEBUG ] Reading configuration from /etc/salt/minion
[DEBUG ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG ] Including configuration from '/etc/salt/minion.d/f_defaults.conf'
[DEBUG ] Reading configuration from /etc/salt/minion.d/f_defaults.conf
[DEBUG ] Initializing new SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[DEBUG ] Decrypting the current master AES key
[DEBUG ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG ] LazyLoaded jinja.render
[DEBUG ] LazyLoaded yaml.render
[DEBUG ] LazyLoaded state.sls
[DEBUG ] LazyLoaded saltutil.is_running
[DEBUG ] LazyLoaded grains.get
[DEBUG ] Re-using SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[DEBUG ] Reading configuration from /etc/salt/minion
[DEBUG ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG ] Including configuration from '/etc/salt/minion.d/f_defaults.conf'
[DEBUG ] Reading configuration from /etc/salt/minion.d/f_defaults.conf
[DEBUG ] Re-using SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[DEBUG ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[INFO ] Loading fresh modules for state activity
[DEBUG ] LazyLoaded jinja.render
[DEBUG ] LazyLoaded yaml.render
[INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/gitfs/gitpython.sls'
[DEBUG ] LazyLoaded cmd.run
[DEBUG ] Jinja search path: ['/var/cache/salt/minion/files/base']
[DEBUG ] Re-using SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja'
[INFO ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml'
[DEBUG ] LazyLoaded grains.filter_by
[DEBUG ] LazyLoaded pillar.get
[CRITICAL] Rendering SLS 'base:salt.gitfs.gitpython' failed: Jinja variable 'dict object' has no attribute 'gitpython'
[DEBUG ] Re-using SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[DEBUG ] LazyLoaded highstate.output
local:
Data failed to compile:
----------
Rendering SLS 'base:salt.gitfs.gitpython' failed: Jinja variable 'dict object' has no attribute 'gitpython'
It was suggested in the IRC channel by AndreasLutro that the problem is caused by incorrect merging of the default and OS specific settings dictionaries.
In particular, it was suggested that this line is at fault https://github.com/saltstack-formulas/salt-formula/blob/master/salt/map.jinja#L79 and should be changed to do a deep merge instead of a shallow merge.
This is surprisingly easy to reproduce:
/srv/salt/top.sls
:base:
'*':
- salt.minion
'G@roles:master':
- salt.master
/srv/pillar/master.sls
:salt:
master:
fileserver_backend:
- roots
- git
file_roots:
base:
- /srv/salt
pillar_roots:
base:
- /srv/pillar
gitfs_provider: pygit2
gitfs_remotes:
- git://github.com/saltstack-formulas/salt-formula
/srv/pillar/minion.sls
:salt:
minion:
- master: 'salt'
/srv/pillar/top.sls
:
base:
'*':
- minion
'G@roles:master':
- master
Given those values, I end up with the following in /etc/salt/minion.d/f_defaults.conf
:
[root@pluto ~]# grep -v '^#' /etc/salt/minion.d/f_defaults.conf | uniq
master:
- fileserver_backend
- gitfs_remotes
- pillar_roots
- file_roots
- gitfs_provider
[root@pluto ~]#
Currently, the following distributions/releases are covered by https://repo.saltstack.com:
salt.pkgrepo
should make use of repo.saltstack.com
where appropriate/possible.
I might take care of providing a PR for Ubuntu. Can't promise yet to add support for other distributions as well.
Am I missing something, or should the configuration files be /etc/salt/minion.d/minion.conf
rather than the current /etc/salt/minion.d/minion
?
http://docs.saltstack.com/en/latest/ref/configuration/master.html#default-include
Per default the master will automatically include all config files from master.d/*.conf where master.d is relative to the directory of the master configuration file.
The following error occurs:
ID: salt-minion
Function: file.recurse
Name: /etc/salt/minion.d
Result: False
Comment: #### /etc/salt/minion.d/_defaults.conf ####
Unable to manage file: Jinja syntax error: no test named 'mapping'
/var/cache/salt/minion/files/base/salt/formulas.jinja(14):
---
[...]
{%- macro formulas_git_opt(env, opt) -%}
{%- set value = salt['pillar.get']('salt_formulas:git_opts:{}:{}'.format(env, opt),
salt['pillar.get']('salt_formulas:git_opts:default:{}'.format(opt),
defaults[opt])) -%}
{%- if value is mapping -%} <======================
{{ value|yaml }}
{%- else -%}
{{ value }}
{%- endif -%}
{%- endmacro -%}
[...]
---
Which appears to come from line 14 in ../salt/formulas.jinja
Not sure what the right value should be.
2015-07-29 15:09:16,376 [salt.state ][ERROR ][8517] #### /etc/salt/minion.d/f_defaults.conf ####
Unable to manage file: Jinja syntax error: no test named 'mapping'
/var/cache/salt/minion/files/itg/salt/formulas.jinja(14):
---
[...]
{%- macro formulas_git_opt(env, opt) -%}
{%- set value = salt['pillar.get']('salt_formulas:git_opts:{0}:{1}'.format(env, opt),
salt['pillar.get']('salt_formulas:git_opts:default:{0}'.format(opt),
defaults[opt])) -%}
{%- if value is mapping -%} <======================
{{ value|yaml }}
{%- else -%}
{{ value }}
{%- endif -%}
{%- endmacro -%}
[...]
Mapping test is available from jinja version >= 2.6. But for centos5 available is only 2.5 version.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.