Coder Social home page Coder Social logo

kafka's Introduction

SAnsible

This repo is a set of tools to aid with the development of roles in the SAnsible organisation.

Role style guide

All roles in SAnsible follow a similiar pattern, however some roles are slightly older than others so their Makefiles, tests and travis.yml files are somewhat out of date.

In general though the roles follow a standardised pattern.

  • Tests are conducted using Vagrant locally and travis for PRs and branches
  • Proper YAML syntax should be used everywhere
  • Each role has an editorconfig file to ensure proper formating
  • The Makefile contains useful targets for running the tests
  • Two tags are present in each role: build and configure
  • The test.yml playbook uses an additional tag called assert
  • The build tag handles installation and things that do not vary per environment
  • The configure tag handles config files and other things that vary per environment
  • The additional assert tag covers tests and is generally used for local tests and travis tests
  • Ansible-lint is used as part of the tests (pip install ansible-lint)
  • Variables all have a parent hash, this has the advantage of making it easier to read the vars, but has the disadvantage of requiring hash behaviour to be set to 'merge'
  • The parent hash should have the prefix sansible_ eg. sansible_ansible

Contributing

If you wish to contribute to one of our roles please open an issue in the roles repo first, then create a branch named with the issue number (eg. GITHUB-1) making sure that you create your branch from develop.

When you are ready to open a PR please squash all of your commits into one and ensure that your commit message contains the Github ID in it's title surrounded by square braces.

Please try to include a descriptive title followed by more detail in the body, eg:

[GITHUB-1] Fixes something

A more detailed description of how this PR will close the issue at hand.

A good guide on commit messages is Tim Pope's guide.

Once your PR is opened Travis will test it, if all the tests passed your PR will be considered for merging.

Creating new roles

Here are the steps for adding a new Sansible role:

  • Create new repo in Sansible org, starting with the default README option
  • Create a develop branch from master, should have master and develop after this
  • Set develop to be the default branch
  • Set branch protection on both master and develop branches
  • Set develop to require reviews before merges
  • Add Trusted Collaborators with admin permissions
  • Add Collaborators with write permissions
  • Login to Travis, re-sync repos and add new repo
  • Create first issue with the title 'Add initial role'
  • Copy skeleton for role from the scaffold branch in the Sansible repo
  • Add role contents following standards in the style guide above
  • Open first PR referencing initial role issue
  • Once Travis tests start running set these as a merge requirement for develop
  • Once first PR is merged login to Ansible Galaxy and add new repo
  • Follow the merge and release guide for pushing from develop to master
  • Check Ansible Galaxy to ensure the new role is in place

Merges and releases

Once a PR is merged into the develop branch Travis will run tests against it, once finished the develop branch needs to be merged into the master, at the moment this process is manual and can be done like so:

  • Checkout this repo
  • Cd into your local checkout of the role that your PR was for
  • Run the sansible release script: bash [PATH TO THIS REPO]/bin/release.sh

The release script will handle the merge and cut a new release tag for you, note that you must have write permissions on the roles repo to do this.

Once master is updated Travis will run tests on the master and update the role in Galaxy.

You can run ./bin/report.sh to check the status of all sansible role repos.

Versioning

The sansible release script will automatically increment the patch version by one when it is run, it will check what the last version is and simply increment it.

In addition to the semver versions we always have a major and minor version, eg:

v1.0.1
v1.0.2
v1.0.3
v1.0

The major.minor tag always points to the same commit as the most recent patch version, so in this case v1.0 will be at the same commit as v1.0.3. In your Galaxy requirements file it is recommended that you point to the major.minor version so you automatically get the latest patch.

If you are introducing new features or breaking changes you should bump the major or minor version accordingly, to do this you need to create or edit the version file in the root of the roles repo. The file should be named .version and simply contain the major and minor version like so:

#.version
v1.2

When the Sansible release script is executed it will check this file for the major and minor version and base the release that it cuts on it.

Troubleshooting

Symptom

Despite of running sansible/bin/release.sh successfully without errors, the master branch points to the wrong commit.

Description

On one particular laptop we once observed that the execution of the sansible/bin/release.sh script succeeded but in actual fact the remote master branch was not updated by moving the HEAD. While attempting to go through this process manually, git showed the following warning.

$ git push
warning: push.default is unset; its implicit value has changed in
Git 2.0 from 'matching' to 'simple'. To squelch this message
and maintain the traditional behaviour, use:

  git config --global push.default matching

To squelch this message and adopt the new behaviour now, use:

  git config --global push.default simple

This laptop was only recently setup and it had already been used to push commits to remote repositories, however the target branch was usually specified. The weired thing is that with git 2.0 or higher, simple is the default push.default option. Therefore it should normally not be required to be configured.

Solution

Configure the push.default option as the warning outlines. If in doubt, use git config --global push.default simple.

kafka's People

Contributors

dennisconrad avatar dm avatar js-kingkong-bot avatar lobsterdore avatar mehonoshin avatar msahota14 avatar quater avatar racer1680 avatar rsgoodall avatar timgrist avatar wojtek-oledzki avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kafka's Issues

Directory paths in defaults are ignored

Variables like sansible_kafka_data_dir: /home/kafka/data in defaults are ignored because the paths are based on the username, e.g.: "/home/{{ sansible_kafka_user }}/data".

Workaround ID clashes in autodiscover script

When starting a fresh cluster in AWS there are often ID conflicts due to machines starting at the same time, a quick fix will be to add a random wait to the beginning of the autodiscovery script.

Allow more flexible server.properties

As a developer
I want to control properties in server.properties file
And not be limited to current template
So that I can easily configure any current or new options.

Should have an option to disable java instalation

Java role should be removed from dependencies and the user should include it in the roles of his playbook if requires to install it, similar to the zookeeper. This would prevent installation of java.

Cant see authorization denied logs

As a platform alert system
I would like to be informed when an authorization is denied
So that I can detect users with authorization issues

hi! I have a problem and it is in "Wait for Kafka port".

image
it is always waiting, i sorry. i don't kown why is problem

'- name: Wait for Kafka port
wait_for:
host: "{{ sansible_kafka_listen_address }}"
port: "{{ sansible_kafka_port }}"
state: started
timeout: "{{ sansible_kafka_wait_for_kafka_port }}"
loop:
- Initial check
- Second check
loop_control:
pause: "{{ sansible_kafka_wait_for_delay }}"
'

Fix status and restarts in sys v script

Status detection is broken in the sysv init script if another Java process is running on the same instance, switch to using a more detailed status of proc check.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.