gunnery / gunnery Goto Github PK
View Code? Open in Web Editor NEWRemote task execution tool
Home Page: http://gunnery.io/
License: Apache License 2.0
Remote task execution tool
Home Page: http://gunnery.io/
License: Apache License 2.0
It would be awesome if task execution could be either scheduled from the web interface or executed from console and scheduled with cron.
I ran into the issue that when I ran vagrant up with a fresh git clone nothing would provision. I would get a ton of errors with ubuntu links that would 404 in terminal. I am assuming that the vm Box that is currently in the Vagrantfile is outdated which is causing the issues.
I decided to change the box that I would run on to ubuntu/trusty64 and everything seems to be fine. Would updating the box to a newer version of ubuntu be something considered to help fix the vagrant?
The setup directions at http://gunnery.readthedocs.org/en/latest/install.html write gunnery.ini to /etc/uwsgi/apps-enabled
but the init.d script for uwsgi points to sites-enabled
.
The dir needs to be renamed in the docs.
There should be an option to choose between password and key authentication.
Even after I setup google SMTP or AWS SMS i got this error:
Traceback (most recent call last):
File "/root/.virtualenvs/dev_webdeploy2/local/lib/python2.7/site-packages/celery/app/trace.py", line 238, in trace_task
R = retval = fun(*args, **kwargs)
File "/root/.virtualenvs/dev_webdeploy2/local/lib/python2.7/site-packages/celery/app/trace.py", line 416, in __protected_call__
return self.run(*args, **kwargs)
File "/data/repos/webdeploy2/backend/tasks.py", line 274, in run
message.send()
File "/root/.virtualenvs/dev_webdeploy2/local/lib/python2.7/site-packages/django/core/mail/message.py", line 276, in send
return self.get_connection(fail_silently).send_messages([self])
File "/root/.virtualenvs/dev_webdeploy2/local/lib/python2.7/site-packages/django/core/mail/backends/smtp.py", line 87, in send_messages
new_conn_created = self.open()
File "/root/.virtualenvs/dev_webdeploy2/local/lib/python2.7/site-packages/django/core/mail/backends/smtp.py", line 51, in open
self.connection.starttls()
File "/usr/lib/python2.7/smtplib.py", line 644, in starttls
raise SMTPException("STARTTLS extension not supported by server.")
SMTPException: STARTTLS extension not supported by server.
TestConnectionTask server_id=5: <type 'exceptions.TypeError'> cannot concatenate 'str' and 'long' objects
Traceback (most recent call last):
File "backend/tasks.py", line 240, in run
transport = self.create_transport(server_id)
File "backend/tasks.py", line 259, in create_transport
transport = ssh.SSHTransport(server)
File "backend/ssh.py", line 26, in __init__
self.secure_files = SecureFileStorage(self.server.environment_id)
File "backend/securefile.py", line 12, in __init__
'private_key': PrivateKey(uid),
File "backend/securefile.py", line 33, in __init__
name_hash = md5(settings.SECRET_KEY + self.prefix + uid).hexdigest()
TypeError: cannot concatenate 'str' and 'long' objects
[2018-01-22 09:16:55,656: INFO/MainProcess] Task backend.tasks.TestConnectionTask[5afd2b89-afee-45ac-bb9e-66ab800ef0b8] succeeded in 0.183372716001s: (False, u'Unknown error')
Using Gunnery+Vagrant setup.
Login, Click Settings -> Applications tab has no options to add anything new.
Other than the Settings page info it just says "no applications yet.".
URL on Vagrant box:
http://localhost:8080/settings/applications/
i also like to give this a try, but like other mentioned, the doc does not give me any hint, howto i.e. use phing with it. it would be nice to have a complete tutorial from a clean setup (without vagrant/puppet) to a simple basic project and some tasks. i.e. how do i say: if this task failes, stop and don´t do any further tasks... ?
Hello,
I am following http://gunnery.readthedocs.org/en/latest/install.html
documentation to get server up and running. I am able to get it running.
But, css, jquery, bootstrap files are not found http://localhost:8000/static/css/bootstrap.css
Also, nothing happens if I click on "Create first".
what could be the reason / what do i need to change ?
thanks,
Manjiri
This tool is great, but it would be great to have capistrano recipes administration on Gunnery, something like Webistrano. Right now, is not clear for me on how to deploy recipes to servers and deploy.
Cheers!
http://gunnery.readthedocs.org/en/latest/install.html when following the non-puppet version of install steps do not create a virtualenv
directory.
I needed to apt-get install python-virtualenv and then virtualenv env
to create it. Then my pip install -r requirements/production.txt
became sudo -u gunnery env/bin/pip install -r requirements/production.txt
.
Lists of users, servers and executions should have pagination and search feature.
Specifically, the erb for the nginx configuration is:
<% if @static_only_collected %>
try_files /gunnery$uri /404;
<% else %>
try_files $uri /core$uri /static$uri /<%= @app_name %>$uri /404;
<% end %>
<% if @gzip_enable %>
gzip on;
gzip_min_length 1100;
gzip_buffers 4 32k;
gzip_types text/plain application/x-javascript text/xml text/css;
gzip_vary on;
<% end %>
<% if @cache_headers_enable %>
expires 30d;
add_header Cache-Control "public";
<% end %>
It'd be good to know which of these sections to leave :)
Add hidden-xs class to FA icons in h1 elements on all pages.
I would really like to test this as it seems a useful app but following the steps (either shortcut or longer installation) both leave me at the same spot with seemingly not reading the settings properly. It complains about SECRET_KEY being empty and DATABASE/ENGINE settings etc. so seems to be reading from /var/gunnery/virtualenv/production/local/lib/python2.7/site-packages/django/conf/ when I believe it should read from /var/gunnery/gunnery/gunnery/settings?
Basically, following the steps always fails at the "python manage.py syncdb" step.
The instructions I personally find really poor, unless this is only meant for people who have a deep understanding of all the various apps used for it to work? In a previous issue about poor documentation one reply is "For now, I recommend looking at puppet rules, they will get you running application in minutes.". I don't think anyone could get a running app in minutes on this, especially without knowing puppet really well also.
For now I have to give up as I have spent almost a whole day on this which is a shame, but until following the instructions works, I cannot spend more time on it. I hope you can tell me something really obvious I am doing wrong but if I am, it is not in the docs.
There is a small typo in docs/develop.rst.
Should read specific
rather than specyfic
.
Hi! The documentation does not describe how to set email notifications. I am far from python, so there are some difficulties in adjustment. You can add this in the documentation?
My mistake
Clicking on plus sign causes this request to spawn and never finish:
http://10.211.55.7:8080/modal_form/a:/environment/1/server/
It halts whole UWSGI process, no requests can be made, until UWSGI is killed (SIGTERM does nothing). All requests timeout with 504.
Nothing new appears in logs when this happens. Not even nginx request.
I've tried this in development environment, i.e. by running python manage.py runserver and issuing curl request, which causes it to hang too and Python must be killed. Note that by using curl I'm not even logged in. The request doesn't even shop up on the screen.
root@vmdebian7:/var/gunnery/gunnery# python manage.py runserver
Validating models...
0 errors found
June 10, 2014 - 12:43:11
Django version 1.6.2, using settings 'gunnery.settings.production'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
[10/Jun/2014 12:43:16] "HEAD / HTTP/1.1" 302 0
Second terminal window:
root@vmdebian7:/var/gunnery/log# curl localhost:8000/ -I
HTTP/1.0 302 FOUND
Date: Tue, 10 Jun 2014 10:43:16 GMT
Server: WSGIServer/0.1 Python/2.7.3
Vary: Cookie
X-Frame-Options: SAMEORIGIN
Content-Type: text/html; charset=utf-8
Location: http://localhost:8000/account/login/?next=/
root@vmdebian7:/var/gunnery/log# curl localhost:8000/modal_form/a:/environment/1/server/ -I
# hangs....
It is a clear installation. I just created Department, Server roles, Application, Environment and clicking on + in chosen Environment causes this.
I'm not really familiar with Django or Python, is there something I can do to nail down this issue?
Also, I don't know if it can be related - issuing service celeryd stop does nothing, it waits. I must ctrl+c the stopping process and issue it again. Workers stop properly on this second try. Always. Doesn't matter if I have one worker or 3 of them. I don't know if clicking on Add server causes some background task to fire... so that's why I'm mentioning this.
root@vmdebian7:/var/gunnery/log# /etc/init.d/celeryd start
celery multi v3.1.7 (Cipater)
> Starting nodes...
> worker1@vmdebian7: OK
root@vmdebian7:/var/gunnery/log# /etc/init.d/celeryd stop
celery multi v3.1.7 (Cipater)
> Stopping nodes...
> worker1@vmdebian7: TERM -> 5685
> Waiting for 1 node -> 5685....................................................................................^C
Session terminated, terminating shell... ...terminated.
root@vmdebian7:/var/gunnery/log# /etc/init.d/celeryd stop
celery multi v3.1.7 (Cipater)
> Stopping nodes...
> worker1@vmdebian7: TERM -> 5685
> Waiting for 1 node -> 5685.....
> worker1@vmdebian7: OK
VM:
Debian 3.2.0-4-amd64
Python 2.7.3 (default, Mar 13 2014, 11:03:55)
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [async-threads:30] [kernel-poll:true]
RabbitMQ 2.8.4
If you need anything more, let me know.
User needs to have an option to abort running task.
hey,
just ran the install on a ubuntu 12.04 and a 13.10 box ( aws ec2 instance )
and i am getting a 400 bad request.
does it run on 8080 or 80 ?
Hi,
I am getting this error while cloning the repo
"fatal: repository 'https://github.com/puppetmodules/puppet-module-python.git/' not found
Clone of 'https://github.com/puppetmodules/puppet-module-python.git' into submodule path 'puppet/modules/python' failed"
Turns out https://github.com/puppetmodules/puppet-module-python has been deleted or made private, is there an alternative to above module?
I think adding webhook support for automated triggering of deployments will be a great addition to this already awesome tool. This way CI services like travis, jenkins etc. can easily be integrated creating a fully automated development to deployment chain .
For a webhook per task, sample Workflow could be:
[Code commit to a VCS e.g. github] => [github makes webhook call to CI] => [CI runs the tests and validates the build[ => [CI makes webhook call to gunnery ]=> [Gunnery verifies the call and triggers respective task]
Similar use cases can be made for automated back-ups etc.
I will be more than happy to work on it. Your thoughts??
Hi @Eyjafjallajokull
I know you want to do some re factoring, are you ok to do some code clean-ups first?
My propositions:
If you ok with that can prepare PR's to all of them.
It's not rocket science, I just want to use some PyCharm features(code inspections report).
Please let me know what do you think.
Great Work!
It will be great if there are guidelines on what to put in <% ... %>
for the configs.
Even better if a docker image can be created! 👍
Default username on Vagrant setup when running 'python manage.py syncdb' is 'vagrant'. If you hit enter and take that default you can't login to the web interface since the login page forces [email protected] format.
Event that Readme states "Supports capistrano..." there are no guides how to deploy using capistrano. It would be nice to have some at least notes how to deploy with Gunnery.
Thanks
Special characters in the output of commands (such as the checkmarks used by Capifony) result in the task stopping with error (code 1024).
Thanks for this great solution.
Due to our org system security policy.
Our org need to regenerate all these ssh certificate after one month.
So If there can add our own ssh cert that will be great.
Hope that can consider this advice. thanks.
When I spin up the VM and go to the localhost I am immediately greeted with this error
relation "django_content_type" does not exist
LINE 1: ..."."app_label", "django_content_type"."model" FROM "django_co...
Environment:
Request Method: GET
Request URL: http://localhost:8080/account/login/?next=/
Django Version: 1.6.7
Python Version: 2.7.6
Installed Applications:
('django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.humanize',
'django_extensions',
'timezone_field',
'guardian',
'crispy_forms',
'djcelery',
'south',
'core',
'task',
'backend',
'account',
'event',
'debug_toolbar')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.contrib.auth.backends.ModelBackend',
'guardian.backends.ObjectPermissionBackend',
'core.middleware.CurrentDepartment',
'debug_toolbar.middleware.DebugToolbarMiddleware')
Traceback:
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
137. response = response.render()
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/template/response.py" in render
105. self.content = self.rendered_content
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/template/response.py" in rendered_content
81. context = self.resolve_context(self.context_data)
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/template/response.py" in resolve_context
159. return RequestContext(self._request, context, current_app=self._current_app)
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/debug_toolbar/panels/templates/panel.py" in _request_context__init__
55. context = processor(request)
File "./core/context_processors.py" in sidebar
8. departments = get_objects_for_user(request.user, 'core.view_department')
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/guardian/shortcuts.py" in get_objects_for_user
354. permission__codename=codename)
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/db/models/manager.py" in get
151. return self.get_queryset().get(*args, **kwargs)
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/db/models/query.py" in get
304. num = len(clone)
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/db/models/query.py" in __len__
77. self._fetch_all()
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/db/models/query.py" in _fetch_all
857. self._result_cache = list(self.iterator())
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/db/models/query.py" in iterator
220. for row in compiler.results_iter():
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in results_iter
713. for rows in self.execute_sql(MULTI):
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in execute_sql
786. cursor.execute(sql, params)
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/debug_toolbar/panels/sql/tracking.py" in execute
174. return self._record(self.cursor.execute, sql, params)
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/debug_toolbar/panels/sql/tracking.py" in _record
104. return method(sql, params)
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/db/backends/util.py" in execute
69. return super(CursorDebugWrapper, self).execute(sql, params)
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/db/backends/util.py" in execute
53. return self.cursor.execute(sql, params)
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/db/utils.py" in __exit__
99. six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/var/gunnery/virtualenv/development/local/lib/python2.7/site-packages/django/db/backends/util.py" in execute
53. return self.cursor.execute(sql, params)
Exception Type: ProgrammingError at /account/login/
Exception Value: relation "django_content_type" does not exist
LINE 1: ..."."app_label", "django_content_type"."model" FROM "django_co...
^
Not sure exactly why this is occurring on a fresh git clone
Thanks for the help
We have 80 machines that need to execute the command in a task.
And we found out that there only 8 concurrency worker will run in a same time.
I know celery is the key of gunnery task and I found some config in puppet file
https://github.com/gunnery/gunnery/blob/master/puppet/manifests/hieradata/common.yaml
celery::workers : 1
celery::concurrency : 8
We are trying to increase those number after already installed.
Any hint will be great. Thanks.
I'd really like to see some better documentation about setting this up and getting it running. If you're not familiar with the db layout, or any python apps, it's a real nightmare navigating how to install it.
I'd like to contribute but your develop branch is out of sync with master.
In docs you say we need to create branch from develop but it's old.
Please @Eyjafjallajokull update develop so I can add some PR's .:D
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.