Python library to interact with the Chef server API
License: Other
A Python API for interacting with a Chef server.
from chef import autoconfigure, Node
api = autoconfigure()
n = Node('web1')
print n['fqdn']
n['myapp']['version'] = '1.0'
n.save()
For more information check out http://pychef.readthedocs.org/en/latest/index.html
We are trying to fetch details from chef "reports API" call using pychef, i am using the below code for API call.
import chef
with chef.ChefAPI('https://chef-server/organizations/devopsorg', 'devopsuser.pem', 'devopsuser', ssl_verify=False):
api= chef.ChefAPI.get_global()
print api
print api["/reports/nodes/chefnode1/runs"]
I could able to fetch other chef server API form organization end but not "/organizations/ORG_NAME/reports/nodes/NODE/runs".
Please suggest us any way for handling this
Hi folks.
This is just a suggestion motivated for a bug reported against pychef server in Debian.
https://bugs.debian.org/751058
In Debian, we enabled unit tests during build time to test the library but when automated tests are ran on pbuilder nodes without network access, they obviously fail.
Thanks,
Looks like searching with [ ] for ranges fails. Assuming it's an escaping issue. Works if I manually escape.
PyChef 0.3.0
Python 2.7.13 on CentOS
Working:
>>> len(chef.Search('node', q="ohai_time:\[0 to 1488474332\]"))
DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): owichef.usgs.chs.ead
/var/lib/rundeck/var/tmp/project/Chef/venv/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
DEBUG:requests.packages.urllib3.connectionpool:https://cheftest.url.org:443 "GET /organizations/test/search/node?q=ohai_time%3A%5C%5B0+to+1488474332%5C%5D&start=0&rows=1000 HTTP/1.1" 200 None
79
Not working:
>>> len(chef.Search('node', q="ohai_time:[0 to 1488474332]"))
DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): owichef.usgs.chs.ead
/var/lib/rundeck/var/tmp/project/Chef/venv/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
DEBUG:requests.packages.urllib3.connectionpool:https://cheftest.url.org:443 "GET /organizations/test/search/node?q=ohai_time%3A%5B0+to+1488474332%5D&start=0&rows=1000 HTTP/1.1" 400 65
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/var/lib/rundeck/var/tmp/project/Chef/venv/lib/python2.7/site-packages/chef/search.py", line 83, in __len__
return len(self.data['rows'])
File "/var/lib/rundeck/var/tmp/project/Chef/venv/lib/python2.7/site-packages/chef/search.py", line 60, in data
self._data = self.api[self.url]
File "/var/lib/rundeck/var/tmp/project/Chef/venv/lib/python2.7/site-packages/chef/api.py", line 221, in __getitem__
return self.api_request('GET', path)
File "/var/lib/rundeck/var/tmp/project/Chef/venv/lib/python2.7/site-packages/chef/api.py", line 217, in api_request
response = self.request(method, path, headers, data)
File "/var/lib/rundeck/var/tmp/project/Chef/venv/lib/python2.7/site-packages/chef/api.py", line 207, in request
raise ChefServerError.from_error(response.reason, code=response.status_code)
chef.exceptions.ChefServerError: Bad Request
I just realized I've not seen a LICENSE.md or similar file around, and I can't find any license terms in the source code, this github page, or pychef.readthedocs.org .
https://help.github.com/articles/open-source-licensing#what-happens-if-i-dont-choose-a-license
chef/search.py
33:class Search(collections.Sequence):
chef/node.py
7:class NodeAttributes(collections.MutableMapping):
22: if not isinstance(search_path, collections.Sequence):
chef/base.py
10:class ChefQuery(collections.Mapping):
chef/data_bag.py
35:class DataBagItem(six.with_metaclass(DataBagMeta, ChefObject, collections.MutableMapping)):
Just stumbled across pychef today and loving it!!.
Sorry that this is more of a question, than an issue, but I couldn't figure out from the documentation what the best place to ask a question would be.
Is it possible to do partial search? From what I can see, I don't believe it is, but was hoping maybe there is some hidden magic that might make this work?
Thanks
There is no ApiClient(ChefObject), so calls to .object on client search results fail:
>>> import chef
>>> chef.autoconfigure()
<chef.api.ChefAPI object at 0x1037d7290>
>>> list(chef.Search('client'))[0].object
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/hugh3869/.virtualenvs/default27/lib/python2.7/site-packages/PyChef-0.2.1-py2.7.egg/chef/search.py", line 25, in object
raise ValueError('Unknown class %s'%chef_class)
ValueError: Unknown class ApiClient.object works on node results:
>>> list(chef.Search('node'))[0].object
<Node test-foo>
>>>Workaround is to use [] instead of .object in client searches, but it would be nice if .object worked for clients.
Running on MacOS 10.10.5
Installed pychef via pip
Python 2.7.13:
$ python
Python 2.7.13 (default, Dec 18 2016, 07:03:34)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import chef
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/site-packages/chef/__init__.py", line 5, in <module>
from chef.api import ChefAPI, autoconfigure
File "/usr/local/lib/python2.7/site-packages/chef/api.py", line 11, in <module>
import pkg_resources
File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 74, in <module>
__import__('pkg_resources.extern.packaging.requirements')
File "/usr/local/lib/python2.7/site-packages/pkg_resources/extern/__init__.py", line 61, in load_module
"distribution.".format(**locals())
ImportError: The 'packaging.requirements' package is required; normally this is bundled with this package so if you get this warning, consult the packager of your distribution.
Even when using a Chef endpoint that is on version 12, Acl is unsupported when ChefAPI is instantiated with autoconfigure().
An instance of the Acl class doesn't seem to get initialized (via the reload() function), due to is_supported() returning False:
https://github.com/coderanger/pychef/blob/master/chef/acl.py#L113
is_supported() returns True, even when my Chef server is on version 12. It seems like it's because the value returned from version_parsed on my ChefAPI object is returning a version much lesser than 12; it is returning 0.10.8. This is the default value for version:
https://github.com/coderanger/pychef/blob/master/chef/api.py#L59
My Chef client is instantiated with autoconfigure(), so the default version of 0.10.8 can't be overwritten.
In the end, I think that there is a conflation of "chef server version" and "chef api version" in the Acl class.
When passing a relative path to api.autoconfigure() (e.g. autoconfigure('../chef-repo')) it will wrongly change convert the relative path, of the client-key, to a absolute path.
For instance: When calling autoconfigure('../chef-repo') where chef-repo contains .chef/client.pem file and configuration file .chef/knife.rb containing the line:
client_key "#{current_dir}/client.pem"
the path of the client key will be changed from the correct relative path: ../chef-repo/.chef/client.pem to the wrong /some/abs/path/chef-repo/chef-repo/.chef/client.pem which should be /some/abs/path/chef-repo/.chef/client.pem.
This happens because of a wrong assumption at https://github.com/coderanger/pychef/blob/master/chef/api.py#L130
Tested on version 0.2.3.
Please let me know if I should create a pull-request.
Right now if I'm groking pychef / docs correctly it looks like you can only set node attributes at the "normal" level. https://github.com/coderanger/pychef/blob/master/chef/node.py#L195
I'd like to be able to set them arbitrarily to any level of precedence. Is this do-able in the existing pychef?
This is installing into a "standalone" Python install (as contrasted with Cygwin).
>scripts\pip install pychef
...
>python -m chef
Traceback (most recent call last):
File "C:\Python35\lib\runpy.py", line 151, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name)
File "C:\Python35\lib\runpy.py", line 118, in _get_module_details
return _get_module_details(pkg_main_name)
File "C:\Python35\lib\runpy.py", line 104, in _get_module_details
spec = importlib.util.find_spec(mod_name)
File "C:\Python35\lib\importlib\util.py", line 88, in find_spec
parent = __import__(parent_name, fromlist=['__path__'])
File "C:\Python35\lib\site-packages\chef\__init__.py", line 5, in
<module>
from chef.api import ChefAPI, autoconfigure
File "C:\Python35\lib\site-packages\chef\api.py", line 17, in <mo
dule>
from chef.rsa import Key
File "C:\Python35\lib\site-packages\chef\rsa.py", line 6, in <mod
ule>
_eay = CDLL('libeay32.dll')
File "C:\Python35\lib\ctypes\__init__.py", line 347, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found
I am using the following pattern for identifying nodes and it works a charm:
result_set = chef.Search('node', q="name:test*")
for result in result_set:
node = chef.Node(result["name"])
print node
Executing this works as expected. I get an object for each result.
I then attempt to apply this to clients:
result_set = chef.Search('client', q="name:test*")
for result in result_set:
client = chef.Client(result["name"])
print client
no result is shown on the screen, however I can verify there are three clients that should be present.
I will also mention here that there is nothing in the documentation about the Client object, I sort of guessed it would be there then read the code.
I am not handy enough with Python to find the bug, though I have tried.
Thanks
Boyd Hemphill
Anyone using a self-signed cert on their Chef server is unable to use PyChef
Would support for knife job API be on the roadmap? Thanks
Very nice tool. Thanks for supporting this.
I am using pychef to delete terminated 'nodes'. Sample code below.
Does pychef allow deleting 'clients' as well ?
def cleanup(svr,org):
chefURL="https://%s/organizations/%s" % (chefserver[svr]['name'],org)
with chef.ChefAPI(chefURL, key, uid) :
for node in chef.Node.list():
cNode = chef.Node(node)
ipaddress=cNode.get('ipaddress')
if not (ipaddress) and (str(cNode).lower().startswith("ip")) : # ip-10-04-19-19.ec2.internal
ipaddress='.'.join(str(cNode).split('.')[0].split('-')[1:5])
if (ipaddress):
instance=find_instance_with(ipaddress)
print("Found node: {} IP: {} AWS Instance: {}".format(node, ipaddress, instance)),
if not (instance):
if (DRYRUN):
print("Will remove Node.."),
else:
print("Removing Node !!"),
cNode.delete()
else:
print("Node: {} IP: {} AWS Instance: {}".format(cNode, 'No IP Yet', None )),
print
Hey all,
I've been using pychef for a little while in conjunction with Fabric and I've found it quite useful. However, I have run into a bit of trouble when I added some additional things to my knife.rb file.
As it turns out pychef makes some assumptions about the layout of the knife.rb file that may not always be true.
The offending knife.rb file looks like:
knife_override = '/other_knife_file.rb'
Chef::Config.from_file(knife_override) if File.exist?(knife_override)
url "chef.local.vm"
user_email = `git config --get user.email`
home_dir = ENV['HOME'] || ENV['HOMEDRIVE']
org = ENV['chef_org'] || 'My Stuff'
knife[:openstack_username] = "openstack login name"
knife[:openstack_tenant] = "project name or tenant name"
knife[:openstack_auth_url] = "http://openstack/auth/url"The offending lines of code can be found here: https://github.com/coderanger/pychef/blob/master/chef/api.py#l97-130
It seems that pychef is attempting to do some manipulation of the knife.rb file to as a bit of an optimization to avoid shelling out. However, this tends to be fragile. Removal of this block of code means that pychef will always shell out to ruby but as a result it will handle the knife config properly.
Hello,
I am trying to use pychef on redhat 6.4. I have Python 3.4.5.
The first line I have it chef.autoconfigure("/etc/chef/client.rb") and it errors with this
Traceback (most recent call last):
File "restore_metadata.py", line 7, in
chef.autoconfigure("/etc/chef/client.rb")
File "/usr/lib/python3.4/site-packages/chef/api.py", line 253, in autoconfigure
api = ChefAPI.from_config_file(config_path)
File "/usr/lib/python3.4/site-packages/chef/api.py", line 139, in from_config_file
out, err = proc.communicate(script)
File "/usr/lib64/python3.4/subprocess.py", line 959, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/usr/lib64/python3.4/subprocess.py", line 1605, in _communicate
input_view = memoryview(self._input)
TypeError: memoryview: str object does not have the buffer interface
I want to do something like "Knife environment from file environments/env.json". Basically, I want to upload environments/env.json from my disk to my ChefServer. Is it possible to do so with pyChef? I only see Create and Delete for the environment Class. The Save doesn't seems would do the job either.
Thanks!
Can you please help me on this error
Python 3.4.3 (default, Feb 23 2017, 16:35:53)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import chef
Traceback (most recent call last):
File "", line 1, in
File "/Users/svellore/workspace/python/flask-code/virtualenvs/py3.4/lib/python3.4/site-packages/PyChef-0.3.0-py3.4.egg/chef/init.py", line 6, in
from chef.client import Client
File "/Users/svellore/workspace/python/flask-code/virtualenvs/py3.4/lib/python3.4/site-packages/PyChef-0.3.0-py3.4.egg/chef/client.py", line 2, in
from chef.base import ChefObject
File "/Users/svellore/workspace/python/flask-code/virtualenvs/py3.4/lib/python3.4/site-packages/PyChef-0.3.0-py3.4.egg/chef/base.py", line 39, in
class ChefObject(six.with_metaclass(ChefObjectMeta, object)):
File "/Users/svellore/workspace/python/flask-code/virtualenvs/py3.4/lib/python3.4/site-packages/six-1.6.0-py3.4.egg/six.py", line 627, in with_metaclass
return meta("NewBase", bases, {})
File "/Users/svellore/workspace/python/flask-code/virtualenvs/py3.4/lib/python3.4/site-packages/PyChef-0.3.0-py3.4.egg/chef/base.py", line 35, in init
ChefObject.types[name.lower()] = cls
NameError: name 'ChefObject' is not defined
Also tried with the save on databag item after changing contents of the item.
File chef/api.py line 216 in request
raise e
urllib2.HTTPError 301 Moved Permanently
Chef server might return 301 on malformed URL
The last release is ~2 years old, and there are some interesting fixes in the git repo. Could you please do a new release ? (I could get the code from git, but I'd like to simply to pip install).
Per https://docs.chef.io/config_rb.html, config.rb should be used in autoconfigure and then fallback to knife.rb
When i try to use chef library on centos 6, i get:
import chef
File "/usr/lib64/python2.7/site-packages/chef/__init__.py", line 5, in <module>
from chef.api import ChefAPI, autoconfigure
File "/usr/lib64/python2.7/site-packages/chef/api.py", line 18, in <module>
from chef.rsa import Key
File "/usr/lib64/python2.7/site-packages/chef/rsa.py", line 9, in <module>
_eay = CDLL('libcrypto.so')
File "/usr/lib64/python2.7/ctypes/__init__.py", line 360, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libcrypto.so: cannot open shared object file: No such file or directorypackage openssl098e has only these libs:
# rpm -ql openssl098e-0.9.8e-18.el6_5.2.x86_64| grep libcrypto.so
/usr/lib64/.libcrypto.so.0.9.8e.hmac
/usr/lib64/.libcrypto.so.6.hmac
/usr/lib64/libcrypto.so.0.9.8e
/usr/lib64/libcrypto.so.6chef library installed from pip2.7
PyChef-0.2.3
It looks like these were dropped in 0.3.0, but that breaks lots of things -- e.g. chef_environment no longer works (in fabfiles that define their own non-legacy tasks) because there's a try/except on from fabric.api import env, task, roles, output, which will raise an ImportError because fabric will be chef.fabric, and then the except block then has task being a passthrough, but that doesn't work, fabric will ignore legacy tasks if a fabfile defines its own. I'd guess the behavior of that try/except might be worth changing on its own too, but apologies if I've missed anything.
experiencing problems when trying to delete clients or nodes from a hosted chef account using the .delete() method.
the following pastebin outlines this, has anyone seen this, or can shed any light on it?
http://pastebin.com/AW1qZc1M
cheers
I might have missed this in the documentation, but is there an equivalent of knife cookbook list with pychef? Thanks
I can run:
>>> import chef
>>> chef.autoconfigure()
<chef.api.ChefAPI object at 0xafd290>
>>> for node in chef.Node.list():
... if "auto" in node.lower():
... print "deleting node " + node
... nodeObj = chef.Node(node)
... nodeObj.delete()
...
deleting node auto-test2.cto.example.net
in the console without issue. But if I put it into a script and run the script, I get:
>python2.7 test.py
Traceback (most recent call last):
File "test.py", line 6, in <module>
for node in chef.Node.list():
File "/usr/local/lib/python2.7/site-packages/chef/base.py", line 86, in list
names = [name for name, url in api[cls.url].iteritems()]
TypeError: 'NoneType' object has no attribute '__getitem__'
Loading the unversioned libcrypto library will crash the Python process on macOS 10.15 Catalina:
Lines 6 to 9 in 216cbb2
The crash dialog shows:
Application Specific Information:
/usr/lib/libcrypto.dylib
abort() called
Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI.
Hit a bug trying to create and ChefAPI with and empty string as the RSA key.
This was a bug in our code for even trying to do such a thing, but the
resulting segmenation fault was not very helpful.
gdb:
Starting program: /home/cameron/.venvs/monster/bin/python ./compute.py destroy -n qa-gate-precise-nova-net
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff4087654 in RSA_size () from /usr/lib/libcrypto.so.1.0.0
backtrace:
#0 0x00007ffff4087654 in RSA_size () from /usr/lib/libcrypto.so.1.0.0
#1 0x00007ffff2b2cd8c in ffi_call_unix64 () from /usr/lib/libffi.so.6
#2 0x00007ffff2b2c6bc in ffi_call () from /usr/lib/libffi.so.6
#3 0x00007ffff2d3f9fb in _ctypes_callproc () from /home/cameron/.venvs/monster/lib/python2.7/lib-dynload/_ctypes.so
#4 0x00007ffff2d39a65 in ?? () from /home/cameron/.venvs/monster/lib/python2.7/lib-dynload/_ctypes.so
#5 0x00007ffff7a65c43 in PyObject_Call () from /usr/lib/libpython2.7.so.1.0
#6 0x00007ffff7af59f1 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#7 0x00007ffff7afa2a0 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#8 0x00007ffff7af9499 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#9 0x00007ffff7afa2a0 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#10 0x00007ffff7af9499 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#11 0x00007ffff7afa2a0 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#12 0x00007ffff7af9499 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#13 0x00007ffff7afa2a0 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#14 0x00007ffff7af9499 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#15 0x00007ffff7afa2a0 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#16 0x00007ffff7a89b30 in function_call () from /usr/lib/libpython2.7.so.1.0
#17 0x00007ffff7a65c43 in PyObject_Call () from /usr/lib/libpython2.7.so.1.0
#18 0x00007ffff7a7444d in instancemethod_call () from /usr/lib/libpython2.7.so.1.0
#19 0x00007ffff7a65c43 in PyObject_Call () from /usr/lib/libpython2.7.so.1.0
#20 0x00007ffff7ab9333 in call_method () from /usr/lib/libpython2.7.so.1.0
#21 0x00007ffff7af56a7 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#22 0x00007ffff7afa2a0 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#23 0x00007ffff7a89c0d in function_call () from /usr/lib/libpython2.7.so.1.0
#24 0x00007ffff7a65c43 in PyObject_Call () from /usr/lib/libpython2.7.so.1.0
#25 0x00007ffff7a7444d in instancemethod_call () from /usr/lib/libpython2.7.so.1.0
#26 0x00007ffff7a65c43 in PyObject_Call () from /usr/lib/libpython2.7.so.1.0
#27 0x00007ffff7ab9e57 in slot_tp_init () from /usr/lib/libpython2.7.so.1.0
#28 0x00007ffff7ab8bcf in type_call () from /usr/lib/libpython2.7.so.1.0
#29 0x00007ffff7a65c43 in PyObject_Call () from /usr/lib/libpython2.7.so.1.0
#30 0x00007ffff7af59f1 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#31 0x00007ffff7afa2a0 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#32 0x00007ffff7af9499 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#33 0x00007ffff7afa2a0 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#34 0x00007ffff7af9499 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#35 0x00007ffff7afa2a0 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#36 0x00007ffff7a89c0d in function_call () from /usr/lib/libpython2.7.so.1.0
#37 0x00007ffff7a65c43 in PyObject_Call () from /usr/lib/libpython2.7.so.1.0
#38 0x00007ffff7af5d70 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#39 0x00007ffff7a7f8d8 in gen_send_ex.isra.0 () from /usr/lib/libpython2.7.so.1.0
#40 0x00007ffff7af65c6 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#41 0x00007ffff7a7f8d8 in gen_send_ex.isra.0 () from /usr/lib/libpython2.7.so.1.0
#42 0x00007ffff7af65c6 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#43 0x00007ffff7afa2a0 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#44 0x00007ffff7a89c0d in function_call () from /usr/lib/libpython2.7.so.1.0
#45 0x00007ffff7a65c43 in PyObject_Call () from /usr/lib/libpython2.7.so.1.0
#46 0x00007ffff7af5d70 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#47 0x00007ffff7afa2a0 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#48 0x00007ffff7af9499 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#49 0x00007ffff7afa2a0 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#50 0x00007ffff7afa3a2 in PyEval_EvalCode () from /usr/lib/libpython2.7.so.1.0
#51 0x00007ffff7b1309f in run_mod () from /usr/lib/libpython2.7.so.1.0
#52 0x00007ffff7b141be in PyRun_FileExFlags () from /usr/lib/libpython2.7.so.1.0
#53 0x00007ffff7b15329 in PyRun_SimpleFileExFlags () from /usr/lib/libpython2.7.so.1.0
#54 0x00007ffff7b25ccf in Py_Main () from /usr/lib/libpython2.7.so.1.0
#55 0x00007ffff7476b05 in __libc_start_main () from /usr/lib/libc.so.6
#56 0x0000000000400741 in _start ()
strace:
open("/home/cameron/work/monster/monster/../config.yaml", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=13309, ...}) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=13309, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f16aeed7000
read(3, "chef:\n server:\n install_scri"..., 4096) = 4096
brk(0x1a7e000) = 0x1a7e000
brk(0x1a9f000) = 0x1a9f000
brk(0x1ac0000) = 0x1ac0000
brk(0x1ae1000) = 0x1ae1000
brk(0x1b27000) = 0x1b27000
read(3, "101.133.0/24\n nova: 198.1"..., 4096) = 4096
brk(0x1b48000) = 0x1b48000
brk(0x1b69000) = 0x1b69000
brk(0x1b8a000) = 0x1b8a000
brk(0x1bab000) = 0x1bab000
brk(0x1bcc000) = 0x1bcc000
read(3, "198.101.133.159\n hori"..., 4096) = 4096
brk(0x1bed000) = 0x1bed000
brk(0x1c0e000) = 0x1c0e000
brk(0x1c2f000) = 0x1c2f000
brk(0x1c50000) = 0x1c50000
brk(0x1c71000) = 0x1c71000
read(3, ":\n precise: Ubuntu 12.04 LTS "..., 4096) = 1021
read(3, "", 4096) = 0
brk(0x1c92000) = 0x1c92000
read(3, "", 4096) = 0
brk(0x1cb3000) = 0x1cb3000
open("/home/cameron/work/monster/monster/../secret.yaml", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=228, ...}) = 0
fstat(4, {st_mode=S_IFREG|0644, st_size=228, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f16aeed6000
read(4, "cloudfiles:\n user: cameron.lope"..., 4096) = 228
read(4, "", 4096) = 0
read(4, "", 4096) = 0
close(3) = 0
munmap(0x7f16aeed7000, 4096) = 0
close(4) = 0
munmap(0x7f16aeed6000, 4096) = 0
getcwd("/home/cameron/work/monster", 1024) = 27
stat("/home/cameron/work/monster/.chef/knife.rb", 0x7fff3b500310) = -1 ENOENT (No such file or directory)
stat("/home/cameron/work/.chef/knife.rb", 0x7fff3b500310) = -1 ENOENT (No such file or directory)
stat("/home/cameron/.chef/knife.rb", {st_mode=S_IFREG|0644, st_size=384, ...}) = 0
access("/home/cameron/.chef/knife.rb", R_OK) = 0
open("/home/cameron/.chef/knife.rb", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=384, ...}) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=384, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f16aeed7000
read(3, "log_level :info\nl"..., 8192) = 384
read(3, "", 4096) = 0
read(3, "", 8192) = 0
close(3) = 0
munmap(0x7f16aeed7000, 4096) = 0
stat("/home/cameron/.chef/cameron.pem", {st_mode=S_IFREG|0644, st_size=1675, ...}) = 0
access("/home/cameron/.chef/cameron.pem", R_OK) = 0
open("/home/cameron/.chef/cameron.pem", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=1675, ...}) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=1675, ...}) = 0
lseek(3, 0, SEEK_CUR) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=1675, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f16aeed7000
lseek(3, 0, SEEK_CUR) = 0
read(3, "-----BEGIN RSA PRIVATE KEY-----\n"..., 4096) = 1675
read(3, "", 4096) = 0
close(3) = 0
munmap(0x7f16aeed7000, 4096) = 0
open("/dev/urandom", O_RDONLY|O_NOCTTY|O_NONBLOCK) = 3
fstat(3, {st_mode=S_IFCHR|0666, st_rdev=makedev(1, 9), ...}) = 0
poll([{fd=3, events=POLLIN}], 1, 10) = 1 ([{fd=3, revents=POLLIN}])
read(3, "\2623\261\376\314\350\32\372\221o\2252L)\7\373,~\206c\336\241>\231\21\215~Lq\305V4", 32) = 32
close(3) = 0
getuid() = 1000
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("198.101.133.3")}, 16) = 0
getpeername(3, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("198.101.133.3")}, [16]) = 0
write(3, "\26\3\1\1-\1\0\1)\3\3R\3148}\rzf\334]\300\207\343%\\\25\272\343\1\3331\240"..., 306) = 306
read(3, "\26\3\1\0:\2\0", 7) = 7
read(3, "\0006\3\1R\3148\372\314\234E0\234*\250z\36\260X{\215\236\332\231\371ZkX\276\246\f\4"..., 56) = 56
read(3, "\26\3\1\3\316", 5) = 5
read(3, "\v\0\3\312\0\3\307\0\3\3040\202\3\3000\202\2\250\2\t\0\213\213:\303\33\224\273\3330\r\6"..., 974) = 974
read(3, "\26\3\1\0\4", 5) = 5
read(3, "\16\0\0\0", 4) = 4
write(3, "\26\3\1\1\6\20\0\1\2\1\0\26c\236(V\317v\260\34\355\237\230\26x\363\355\2533%s\311"..., 314) = 314
read(3, "\26\3\1\0\252", 5) = 5
read(3, "\4\0\0\246\0\0\1,\0\240\360\303v\323\362\354\252\223\310\237\271\373\254\320T\362\264\345\3311;\t"..., 170) = 170
read(3, "\24\3\1\0\1", 5) = 5
read(3, "\1", 1) = 1
read(3, "\26\3\1\0$", 5) = 5
read(3, "\374\336\177\267\26\265\16\375eU\312\3217\366%/v\37\203>SLK\32$\245\271<Y\233A\337"..., 36) = 36
write(3, "\27\3\1\3Z\2\357x\357\242\311HI\352\26\234\343\307\226\310n\270\23B\322\211\6i\363G\254\360"..., 863) = 863
read(3, "\27\3\1\10\222", 5) = 5
read(3, ":\245\34\3517\251\32\225P\321\233\216\237\t~\n\360#\257\275S\227\257E\344\236\27E\320\27\363\324"..., 2194) = 2194
close(3) = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x20} ---
+++ killed by SIGSEGV (core dumped) +++
zsh: segmentation fault (core dumped) strace python ./compute.py destroy -n qa-gate-precise-nova-net
Tried to upgrade and fresh install PyChef today via PIP.
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/Users/dstegelman/Documents/code/work/.virtualenvs/k-state-cba-mentor/build/pychef/setup.py", line 34, in <module>
test_suite = 'unittest2.collector',
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py", line 152, in setup
dist.run_commands()
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py", line 975, in run_commands
self.run_command(cmd)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py", line 994, in run_command
cmd_obj.ensure_finalized()
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/cmd.py", line 117, in ensure_finalized
self.finalize_options()
File "/Users//Documents/code/work/.virtualenvs//lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/egg_info.py", line 86, in finalize_options
File "/Users//Documents/code/work/.virtualenvs//lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/egg_info.py", line 163, in tagged_version
File "versiontools_support.py", line 96, in get_version
return (self.__get_live_version() or self.__get_frozen_version()
File "versiontools_support.py", line 117, in __get_frozen_version
return self.__base("PKG-INFO").version
TypeError: __init__() takes exactly 1 argument (2 given)
If a chef node has an attribute set on multiple levels to dictionary values, then the dictionary produced by NodeAttributes.to_dict() only contains the items in the dictionary set at the level with higher precedence, omitting the items that only exist in the dictionary set at the level with lower precedence. For example, if "postgresql" is set on the default level to {"version": "9.5"} and on the normal level to {"enable_pgdg_apt": true}, then the "postgresql" key in the output of to_dict() would have the value of {"enable_pgdg_apt": True} instead of {"enable_pgdg_apt": True, "version": "9.5"}.
Hey,
Discovered a nasty bug that only seems to manifest on the latest version of OSX, Mavericks, due to some additional security features they enabled.
The issue arises when you try to spawn a subprocess, or use os calls like execp which replace your running process. It seems that when there are open remote connections, the python process has a file descriptor pointing a mac kernel queue so that it can receive events from the OS. This shows up in lsof -p $PID as a KQUEUE.
This behavior is normal, but before running any of the fun OS process commands, the documentation warns us to flush STDOUT and close any open file handles as these would normally be inherited by the child process.
Unfortunately, it seems that something about the way PyChef builds it's connections, is that it cannot clean them up properly in this use case. In the code where I discovered the bug, I used boto extensively but was unable to replicate the issue like I can below.
When you trigger the bug, your process will exit, only stating Killed: 9 in the terminal. In the Mac Console App you can see that the system log has ... ssh: guarded fd exception: ... and a link to a kernel dump.
If you have a Mac with a fresh install of OSX Mavericks, you can replicate the bug with the following code:
import os
from chef import ChefAPI, Search
ChefAPI(server_url, key_path, username)
Search('node', 'role:*')[0]
os.execvp('ssh', ['ssh', '[email protected]'])I haven't had a change to dive in to the PyChef code to figure out where it is leaking connections (or perhaps some other event that it is subscribing to from the queue).
OS X Yosemite 10.10.5 (14F1713)
I have same python version 2.7.10 for /usr/local/bin/python and /usr/bin/python, but import fails for the latter:
mac:[~]: /usr/bin/python
Python 2.7.10 (default, Jul 14 2015, 19:46:27)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import chef
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Python/2.7/site-packages/chef/__init__.py", line 6, in <module>
from chef.client import Client
File "/Library/Python/2.7/site-packages/chef/client.py", line 2, in <module>
from chef.base import ChefObject
File "/Library/Python/2.7/site-packages/chef/base.py", line 39, in <module>
class ChefObject(six.with_metaclass(ChefObjectMeta, object)):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six.py", line 566, in with_metaclass
return meta("NewBase", bases, {})
File "/Library/Python/2.7/site-packages/chef/base.py", line 35, in __init__
ChefObject.types[name.lower()] = cls
NameError: global name 'ChefObject' is not defined
mac:[~]: cat /Library/Python/2.7/site-packages/chef/__init__.py
# Copyright (c) 2010 Noah Kantrowitz <[email protected]>
__version__ = (0, 3, 0)
from chef.api import ChefAPI, autoconfigure
from chef.client import Client
from chef.data_bag import DataBag, DataBagItem
from chef.exceptions import ChefError
from chef.node import Node
from chef.role import Role
from chef.environment import Environment
from chef.search import Search
from chef.acl import Acl
But /usr/local/bin/python is fine:
mac:[~]: /usr/local/bin/python
Python 2.7.10 (default, Sep 23 2015, 04:34:14)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import chef
>>> chef.__version__
(0, 3, 0)
>>>
NOTE: python 2.7.10 build ver/dates are different for /usr/local/bin/python and /usr/bin/python
I ran `sudo python setup.py install' under pychef 0.3.0 folder and installed it.
all the files were installed to /Library/Python/2.7/site-packages/PyChef-0.3.0-py2.7.egg
but when I tried to use
from chef.api import autoconfigure
from chef.node import Node
api = autoconfigure()
I always got errors:
/usr/bin/python /Users/alexya/Source/chef/chef.py
Traceback (most recent call last):
File "/Users/alexya/Source/chef/chef.py", line 3, in <module>
from chef.api import autoconfigure
File "/Users/alexya/Source/chef/chef.py", line 3, in <module>
from chef.api import autoconfigure
ImportError: No module named api
I don't know how to fix it. even if I tried by the sample in readme.rst
from chef import autoconfigure, Node
api = autoconfigure()
n = Node('web1')
print n['fqdn']
n['myapp']['version'] = '1.0'
n.save()
I still got the similar errors.
my colleague suggested me to install pychef by pip. so, I want to know, how can I uninstall pychef from my OSX system? and then I will try to create a virtualenv and use pip to install it.
thx a lot.
This is raised from api.py:request line 207 (or 204 in 0.3 release)
Since ConnectionError has no 'message' attribute it causes an exception inside the except clause.
Python3.6, pychef==0.3.
Here is the relevant code:
try:
response = self._request(method, self.url + path, data, dict(
(k.capitalize(), v) for k, v in six.iteritems(request_headers)))
except requests.ConnectionError as e:
raise ChefServerError(e.message)
Hi,
I just wanted to know the license of this software since I am interesting in packaging and making your software available in some Linux distributions.
Thanks,
Is there a way to interrogate an existing role to see what it contains?
#!/usr/bin/env python3
import chef
api = chef.autoconfigure()
print('Role Attributes:')
role = chef.Role('some_role')
print(role.override_attributes)
I've run the above and it actually returns data but it seems to be a pretty dirty/not truly supported method. Is there a way to get an object that would contain similar data to what knife role show <some_role> would return?
I'm sure this is some kind of operating system bug and totally not PyChef's fault, but I just tried to use PyChef in an AWS Lambda, and I ran into snags using libcrypto.so, because it doesn't exist:
__| __|_ )
_| ( / Amazon Linux AMI
___|\___|___|
https://aws.amazon.com/amazon-linux-ami/2015.09-release-notes/
No packages needed for security; 14 packages available
Run "sudo yum update" to apply all updates.
[ec2-user@ip-172-10-12-87 ~]$ ls /lib64/crypto
/lib64/libcrypto.so.10 /lib64/libcrypto.so.1.0.1k /lib64/libk5crypto.so.3 /lib64/libk5crypto.so.3.1
I "fixed" it, for my own use, by changing line 9 of rsa.py to use libcrypto.so.10, instead.
My Environment:
Issue:
Erroneous "unauthorized" error message if the node hostname is in the wrong format.
Good Example (No erroneous space in front of hostname):
hostname = "hostname"
api = chef.ChefAPI.from_config_file("E:\\path\\to\\knife\\file")
node = chef.Node(hostname)
Good Result:
Comes back with node information if the node exists in Chef.
Bad Example (Erroneous space in front of hostname):
hostname = " hostname"
api = chef.ChefAPI.from_config_file("E:\\path\\to\\knife\\file")
node = chef.Node(hostname)
Bad Result:
Traceback (most recent call last):
File "e:/chef/validate_host.py", line 33, in <module>
node = chef.Node(hostname)
File "C:\Program Files\Python36\lib\site-packages\chef\base.py", line 58, in __init__
data = self.api[self.url]
File "C:\Program Files\Python36\lib\site-packages\chef\api.py", line 221, in __getitem__
return self.api_request('GET', path)
File "C:\Program Files\Python36\lib\site-packages\chef\api.py", line 217, in api_request
response = self.request(method, path, headers, data)
File "C:\Program Files\Python36\lib\site-packages\chef\api.py", line 207, in request
raise ChefServerError.from_error(response.reason, code=response.status_code)
chef.exceptions.ChefServerError: Unauthorized
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.