Coder Social home page Coder Social logo

docker-elasticsearch's Introduction

ElasticSearch on Kubernetes

Build Status Docker Repository on Quay

ElasticSearch 6.8.1.

Configuration

Configuration is done via environment variables.

The following configuration defaults may not necessarily be set to the same values in kube/ example files.

Plugins

Deployment

By default if you start the docker container, ElasticSearch will start in standalone mode.

Deploying onto a Kubernetes cluster is fairly easy. There are example kubernetes controller and service files in kube/ directory.

Deploy Master Node

First of all we need to deploy master service for ES master nodes to find each other and other communications between nodes. Then we can create the master replication controller.

$ kubectl create -f kube/es-master-svc.yaml
$ kubectl create -f kube/es-master-rc.yaml

Wait a few seconds and verify whether it is up and running. You can also scale the master nodes to 3.

$ kubectl logs -f es-master-fdfw -c elasticsearch
$ kubectl scale --replicas=3 rc/es-master

Deploy Client and Data Nodes

Once the master node is up and running, you can start deploying the rest of the cluster.

$ kubectl create -f kube/es-svc.yaml
$ kubectl create -f kube/es-client-rc.yaml
$ kubectl create -f kube/es-data-rc.yaml

docker-elasticsearch's People

Contributors

devlinmr avatar joseph-irving avatar kashifsaadat avatar lewismarshall avatar marcinc avatar osterzel avatar purplebooth avatar vaijab avatar vinn946 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

docker-elasticsearch's Issues

the readme needs fixing up

docker run --name es_thing --rm=true -e 'PROXY_SERVICE_HOST=google.com' -e 'PROXY_SERVICE_PORT=80' -p 9200:9200 -p 9300:9300 quay.io/ukhomeofficedigital/elasticsearch:v0.1.0

isn't the right options ;-)

[1]: memory locking requested for elasticsearch process but memory is not locked

Created elasticsearch keystore in /elasticsearch/config
/run.sh: line 88: ulimit: max locked memory: cannot modify limit: Operation not permitted
Empty values sent to add_to_keystore s3.client.default.access_key
Empty values sent to add_to_keystore s3.client.default.secret_key
[2019-11-26T11:54:06,533][WARN ][o.e.b.JNANatives ] [es-master-dvx5z]Unable to lock JVM Memory: error=12, reason=Cannot allocate memory
[2019-11-26T11:54:06,535][WARN ][o.e.b.JNANatives ] [es-master-dvx5z]This can result in part of the JVM being swapped out.
[2019-11-26T11:54:06,535][WARN ][o.e.b.JNANatives ] [es-master-dvx5z]Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536
[2019-11-26T11:54:06,536][WARN ][o.e.b.JNANatives ] [es-master-dvx5z]These can be adjusted by modifying /etc/security/limits.conf, for example:
# allow user 'elasticsearch' mlockall
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
[2019-11-26T11:54:06,536][WARN ][o.e.b.JNANatives ] [es-master-dvx5z]If you are logged in interactively, you will have to re-login for the new limits to take effect.
[2019-11-26T11:54:06,750][INFO ][o.e.e.NodeEnvironment ] [es-master-dvx5z]using [1] data paths, mounts [[/data (/dev/mapper/centos-root)]], net usable_space [382.6gb], net total_space [432.5gb], types [xfs]
[2019-11-26T11:54:06,751][INFO ][o.e.e.NodeEnvironment ] [es-master-dvx5z]heap size [441.6mb], compressed ordinary object pointers [true]
[2019-11-26T11:54:06,752][INFO ][o.e.n.Node ] [es-master-dvx5z]node name [es-master-dvx5z], node ID [lKI3MzYiREK7LOMdovLgig]
[2019-11-26T11:54:06,752][INFO ][o.e.n.Node ] [es-master-dvx5z]version[6.6.1], pid[129], build[default/tar/1fd8f69/2019-02-13T17:10:04.160291Z], OS[Linux/3.10.0-862.el7.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_232/25.232-b09]
[2019-11-26T11:54:06,752][INFO ][o.e.n.Node ] [es-master-dvx5z]JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-3204802986171057500, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Xms450m, -Xmx450m, -Dlog4j2.disable.jmx=true, -Des.path.home=/elasticsearch, -Des.path.conf=/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[2019-11-26T11:54:08,877][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [aggs-matrix-stats]
[2019-11-26T11:54:08,877][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [analysis-common]
[2019-11-26T11:54:08,877][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [ingest-common]
[2019-11-26T11:54:08,877][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [lang-expression]
[2019-11-26T11:54:08,877][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [lang-mustache]
[2019-11-26T11:54:08,877][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [lang-painless]
[2019-11-26T11:54:08,877][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [mapper-extras]
[2019-11-26T11:54:08,878][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [parent-join]
[2019-11-26T11:54:08,878][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [percolator]
[2019-11-26T11:54:08,878][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [rank-eval]
[2019-11-26T11:54:08,878][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [reindex]
[2019-11-26T11:54:08,878][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [repository-url]
[2019-11-26T11:54:08,878][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [transport-netty4]
[2019-11-26T11:54:08,878][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [tribe]
[2019-11-26T11:54:08,878][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [x-pack-ccr]
[2019-11-26T11:54:08,878][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [x-pack-core]
[2019-11-26T11:54:08,878][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [x-pack-deprecation]
[2019-11-26T11:54:08,879][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [x-pack-graph]
[2019-11-26T11:54:08,879][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [x-pack-ilm]
[2019-11-26T11:54:08,879][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [x-pack-logstash]
[2019-11-26T11:54:08,879][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [x-pack-ml]
[2019-11-26T11:54:08,879][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [x-pack-monitoring]
[2019-11-26T11:54:08,879][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [x-pack-rollup]
[2019-11-26T11:54:08,879][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [x-pack-security]
[2019-11-26T11:54:08,879][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [x-pack-sql]
[2019-11-26T11:54:08,879][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [x-pack-upgrade]
[2019-11-26T11:54:08,880][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded module [x-pack-watcher]
[2019-11-26T11:54:08,880][INFO ][o.e.p.PluginsService ] [es-master-dvx5z]loaded plugin [repository-s3]
[2019-11-26T11:54:12,047][INFO ][o.e.d.DiscoveryModule ] [es-master-dvx5z]using discovery type [zen] and host providers [settings]
[2019-11-26T11:54:12,486][INFO ][o.e.n.Node ] [es-master-dvx5z]initialized
[2019-11-26T11:54:12,486][INFO ][o.e.n.Node ] [es-master-dvx5z]starting ...
[2019-11-26T11:54:12,576][INFO ][o.e.t.TransportService ] [es-master-dvx5z]publish_address {10.244.2.45:9300}, bound_addresses {0.0.0.0:9300}
[2019-11-26T11:54:12,600][INFO ][o.e.b.BootstrapChecks ] [es-master-dvx5z]bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [1] bootstrap checks failed
[1]: memory locking requested for elasticsearch process but memory is not locked
[2019-11-26T11:54:12,608][INFO ][o.e.n.Node ] [es-master-dvx5z]stopping ...
[2019-11-26T11:54:12,622][INFO ][o.e.n.Node ] [es-master-dvx5z]stopped
[2019-11-26T11:54:12,622][INFO ][o.e.n.Node ] [es-master-dvx5z]closing ...
[2019-11-26T11:54:12,631][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [es-master-dvx5z]uncaught exception in thread [Thread-2]
java.lang.NullPointerException: null
at org.elasticsearch.xpack.watcher.Watcher.close(Watcher.java:700) ~[?:?]
at org.elasticsearch.core.internal.io.IOUtils.close(IOUtils.java:103) ~[elasticsearch-core-6.6.1.jar:6.6.1]
at org.elasticsearch.core.internal.io.IOUtils.close(IOUtils.java:85) ~[elasticsearch-core-6.6.1.jar:6.6.1]
at org.elasticsearch.node.Node.close(Node.java:919) ~[elasticsearch-6.6.1.jar:6.6.1]
at org.elasticsearch.core.internal.io.IOUtils.close(IOUtils.java:103) ~[elasticsearch-core-6.6.1.jar:6.6.1]
at org.elasticsearch.core.internal.io.IOUtils.close(IOUtils.java:61) ~[elasticsearch-core-6.6.1.jar:6.6.1]
at org.elasticsearch.bootstrap.Bootstrap$4.run(Bootstrap.java:184) ~[elasticsearch-6.6.1.jar:6.6.1]
[root@master docker-elasticsearch-master]#

I am use k8s 1.14.1 docker 18.09

but failed to running

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.