Coder Social home page Coder Social logo

jenkinsci / remoting-kafka-plugin Goto Github PK

View Code? Open in Web Editor NEW
41.0 12.0 12.0 403 KB

Jenkins Remoting over Apache Kafka

Home Page: https://plugins.jenkins.io/remoting-kafka/

License: MIT License

Shell 1.38% Java 96.32% Makefile 0.28% Dockerfile 0.78% Smarty 1.18% HTML 0.07%
jenkins kafka remoting remote-execution java plugin docker docker-compose

remoting-kafka-plugin's Introduction

Remoting Kafka Plugin

Build Status Join the chat at https://gitter.im/jenkinsci/remoting

This plugin can be found in Jenkins Plugin.

Plugin

Slides for DevOps World | Jenkins World 2018 presentation available here.

Overview

Current versions of Jenkins Remoting are based on the TCP protocol. If it fails, the agent connection and the build fails as well. There are also issues with traffic prioritization and multi-agent communications, which impact Jenkins stability and scalability.

This project aims an update of Remoting and Jenkins in order to add support of a popular message queue/bus technology (Kafka) as a fault-tolerant communication layer in Jenkins.

More information about this project can be found at: https://jenkins.io/projects/gsoc/2018/remoting-over-message-bus/

Release Notes

See the CHANGELOG.

Developer documentation

How to run demo of the plugin

Using Docker

  1. Requirements: docker, docker-compose installed.

  2. Set environment variable DOCKERHOST

  • Windows: set DOCKERHOST=host.docker.internal
  • macOS: export DOCKERHOST=host.docker.internal
  • Linux: export DOCKERHOST=$(ifconfig docker0 | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}')
  1. Build the demo: make all.

  2. Run the demo: make run (without security feature make run-no-auth).

  3. Features in the demo:

  • Docker Compose starts preconfigured Master and agent instance, they connect automatically using Kafka launcher.
  • Kafka is secured and encrypted with SSL.
  • There few demo jobs in the instance so that a user can launch a job on the agent.
  • Kakfa Manager supported in localhost:9000 to support monitoring of Kafka cluster.
  1. Stop the demo: make clean.

Using Kubernetes

  1. Requirements: A Kubernetes cluster (e.g. minikube), Helm 2

  2. Run the demo:

helm dep build helm/jenkins-remoting-kafka && \
  helm install helm/jenkins-remoting-kafka -n demo -f demo/helm/values-override.yaml
  1. Stop the demo:
helm delete --purge demo

Links

remoting-kafka-plugin's People

Contributors

afalko avatar fnaum avatar jeffret-b avatar longngn avatar oleg-nenashev avatar pvtuan10 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.