Official Elasticsearch Docker image
License: Apache License 2.0
This repository is no longer used to generate the official Elasticsearch Docker image from Elastic.
The image is now built from the Elasticsearch repository.
To build the Elasticsearch docker image for versions before 6.6.0, checkout the corresponding release branch in this repository.
Update ES version to 5.1.2. The current docker image is still pointing to ES version 5.0.1.
Hi,
is it possible to also include a :latest tag (like it was in the old dockerhub repo here https://hub.docker.com/_/elasticsearch/) for the newest container build and published? Of course this would also be great for the other containers published, but i am not sure if i should open an issue for all of them?
Cheers,
Jan
$docker run -p 9200:9200 -e "http.host=0.0.0.0" -e "transport.host=127.0.0.1" docker.elastic.co/elasticsearch/elasticsearch:5.3.0
[2017-03-29T21:00:20,116][INFO ][o.e.n.Node ] [] initializing ...
[2017-03-29T21:00:20,182][INFO ][o.e.e.NodeEnvironment ] [5QcqJ9x] using [1] data paths, mounts [[/ (none)]], net usable_space [193.6gb], net total_space [933.1gb], spins? [possibly], types [aufs]
[2017-03-29T21:00:20,182][INFO ][o.e.e.NodeEnvironment ] [5QcqJ9x] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-03-29T21:00:20,183][INFO ][o.e.n.Node ] node name [5QcqJ9x] derived from node ID [5QcqJ9xjRJKcyxaxqxv6HQ]; set [node.name] to override
[2017-03-29T21:00:20,184][INFO ][o.e.n.Node ] version[5.3.0], pid[1], build[3adb13b/2017-03-23T03:31:50.652Z], OS[Linux/4.4.0-62-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_92-internal/25.92-b14]
**[2017-03-29T21:00:21,776][ERROR][o.e.x.s.SSLService ] [5QcqJ9x] unsupported ciphers [[TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA]] were requested but cannot be used in this JVM, however there are supported ciphers that will be used [[TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA]]. If you are trying to use ciphers with a key length greater than 128 bits on an Oracle JVM, you will need to install the unlimited strength JCE policy files.**
[2017-03-29T21:00:21,853][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failed to load plugin class [org.elasticsearch.xpack.XPackPlugin]]; nested: InvocationTargetException; nested: ElasticsearchException[failed to generate key]; nested: NoSuchAlgorithmException[HmacSHA512 KeyGenerator not available];
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:127) ~[elasticsearch-5.3.0.jar:5.3.0]
$docker info
Containers: 36
Running: 0
Paused: 0
Stopped: 36
Images: 328
Server Version: 1.13.1
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 314
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.4.0-62-generic
Operating System: Linux Mint 18
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 31.3 GiB
Name: swartjie
ID: OBAE:PF6U:XIE3:IOVE:H7LT:RFXR:YJMI:2DXA:KIF5:DV4K:IOLY:I6VR
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Temporary Workaround:
Dockerfile changes:
FROM **frolvlad/alpine-oraclejdk8:latest**
MAINTAINER Elastic Docker Team <[email protected]>
ARG ELASTIC_VERSION
ARG ES_DOWNLOAD_URL
ARG ES_JAVA_OPTS
ENV PATH /usr/share/elasticsearch/bin:$PATH
#ENV JAVA_HOME /usr/lib/jvm/java-1.8-openjdk
**ENV JAVA_HOME /usr/lib/jvm/default-jvm**
WORKDIR /usr/share/elasticsearch
**RUN apk update && \
apk upgrade && \
apk add bash curl openssl && \
rm -rf /var/cache/apk/***
**# Install elasticsearch user
RUN adduser -D -u 1000 -h /usr/share/elasticsearch elasticsearch**[edited by @jarpy for clarity]
Starting elasticsearch:5.2.2 results in Failed to created node environment. This has been covered in #21, the recommendation was to set user permissions on the host system.
docker run --rm -v /tmp/elastic:/usr/share/elasticsearch/data docker.elastic.co/elasticsearch/elasticsearch:5.2.2
# breaks
I prefer simple fire & forget docker containers which don't need any preparation on the host system. Would it be an option to check the Elasticsearch image from Docker, that runs fine, without setting any permissions before, and replicate that behaviour in new elasticsearch-docker?
docker run --rm -v /tmp/elastic:/usr/share/elasticsearch/data elasticsearch:5.2.2
# runs
# sudo docker run --rm -v /tmp/data:/usr/share/elasticsearch/data docker.elastic.co/elasticsearch/elasticsearch:5.2.2
[2017-03-04T14:44:14,972][INFO ][o.e.n.Node ] [] initializing ...
[2017-03-04T14:44:14,989][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Failed to created node environment
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) ~[elasticsearch-5.2.2.jar:5.2.2]
Caused by: java.lang.IllegalStateException: Failed to created node environment
at org.elasticsearch.node.Node.<init>(Node.java:255) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.node.Node.<init>(Node.java:232) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.2.2.jar:5.2.2]
... 6 more
Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]
at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:?]
at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_92-internal]
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_92-internal]
at java.nio.file.Files.createDirectories(Files.java:767) ~[?:1.8.0_92-internal]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:221) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.node.Node.<init>(Node.java:232) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.2.2.jar:5.2.2]
... 6 more
# uname -a
Linux yukka1 4.4.0-59-generic #80-Ubuntu SMP Fri Jan 6 17:47:47 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
# docker info
Containers: 20
Running: 16
Paused: 0
Stopped: 4
Images: 62
Server Version: 1.13.1
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 311
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.4.0-59-generic
Operating System: Ubuntu 16.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 12
Total Memory: 62.35 GiB
Name: yukka1
ID: XAGB:VGNL:EAJW:GXJ2:USZP:HOOK:KKQL:J2HB:YNU5:ZMAG:SA3X:RVOG
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Labels:
provider=generic
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false```
S3 repository plugin is very popular plugin used to snapshot the data , specially in the cloud
I think it will be nice to have it installed by default in the image
docker info output:{
"error": {
"root_cause": [
{
"type": "security_exception",
"reason": "failed to authenticate user [elastic]",
"header": {
"WWW-Authenticate": "Basic realm=\"security\" charset=\"UTF-8\""
}
}
],
"type": "security_exception",
"reason": "failed to authenticate user [elastic]",
"header": {
"WWW-Authenticate": "Basic realm=\"security\" charset=\"UTF-8\""
}
},
"status": 401
}
docker-compose version:
2
Host OS and version:
osx
Then I run the docker container using
docker run -p 9200:9200 -e "http.host=0.0.0.0" -e "transport.host=127.0.0.1" docker.elastic.co/elasticsearch/elasticsearch:5.2.2
I've also tried with my compose file but get the same results
version: "2"
services:
elastic:
image: docker.elastic.co/elasticsearch/elasticsearch:5.2.2
ports:
- "9200:9200"
logstash:
image: docker.elastic.co/logstash/logstash:5.2.2
volumes:
- ./logstash/pipeline:/usr/share/logstash/pipeline/
- ./logstash/config:/usr/share/logstash/config/
- ./logstash/csv:/usr/share/logstash/csv/
links:
- elastic
I can curl
curl -u elastic http://127.0.0.1:9200/_cat/health
and get an expect response.
If I use basic auth from postman or more importantly from logstash
elasticsearch {
hosts => ["http://elastic:[email protected]:9200/test"]
index => "logstash-test"
user => "elastic"
password => "changeme"
}
I get
{
"error": {
"root_cause": [
{
"type": "security_exception",
"reason": "failed to authenticate user [elastic]",
"header": {
"WWW-Authenticate": "Basic realm=\"security\" charset=\"UTF-8\""
}
}
],
"type": "security_exception",
"reason": "failed to authenticate user [elastic]",
"header": {
"WWW-Authenticate": "Basic realm=\"security\" charset=\"UTF-8\""
}
},
"status": 401
}
I want to use Docker for testing the elastic on GitLab CI.
In the example here it's stated that the following command could be used:
docker run -p 9200:9200 -e "http.host=0.0.0.0" -e "transport.host=127.0.0.1" docker.elastic.co/elasticsearch/elasticsearch:5.3.0However, if I just run the following it binds elastic to a random address.
docker run docker.elastic.co/elasticsearch/elasticsearch:5.3.0I need to have it set to 127.0.0.1:9200 by default (with no arguments provided).
Changing yml file to the following did not help:
network.host: 0.0.0.0
http.port: 9200
http.cors.enabled: true
http.cors.allow-origin: "/.*/"
http.host: 0.0.0.0
transport.host: 127.0.0.1
I also tried other settings such as:
network.publish_host: 127.0.0.1
network.bind_host: 127.0.0.1
All attempts failed. Any advice on this?
I don't know if this is a docker problem or a general ES bug, but it looks like when I run the container locally and run a delete_by_query, it doesn't wait for it to complete if I add the parameter wait_for_completion=True
This also happens for the tasks API.
Happy to give a workable test if you'd like, but was just wondering if this was a known bug off the top of your head.
How do we enable LDAP/AD authentication for shield and add custom role mapping?
$make
Step 12/20 : RUN eval ${ES_JAVA_OPTS:-} elasticsearch-plugin install --batch x-pack
---> Running in 07d8088bc9ee
-> Downloading x-pack from elastic
Exception in thread "main" java.nio.file.AccessDeniedException: /tmp/6291781730439625928.zip
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
Kernel Version: 4.4.0-62-generic
Operating System: Linux Mint 18
$ docker info
Containers: 36
Running: 0
Paused: 0
Stopped: 36
Images: 331
Server Version: 17.03.1-ce
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 316
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 4ab9917febca54791c5f071a9d1f404867857fcc
runc version: 54296cf40ad8143b62dbcaa1d90e520a2136ddfe
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.4.0-62-generic
Operating System: Linux Mint 18
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 31.3 GiB
Name: swartjie
ID: OBAE:PF6U:XIE3:IOVE:H7LT:RFXR:YJMI:2DXA:KIF5:DV4K:IOLY:I6VR
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Minor doc issue:
Use the env var ES_JAVA_OPTS to set heap size, e.g. to use 16GB use -e ES_JAVA_OPTS="-Xms16g -Xmx=16g" with docker run. It is also recommended to set a memory limit for the container.
The above has an extra = which makes the Xmx value invalid for the JVM, should be just -Xmx16g :)
While we are on the topic of heap size, I wonder if it makes sense to call out #8 in the doc. I expect more folks to run into this when developing on small server/laptop with limited memory, for the boot2docker can often default to a little less than 1G - which means that it will always throw the # Native memory allocation (mmap) failed to map 2060255232 bytes for committing reserved memory. error in these environments since the default ES_HEAP_SIZE of Elasticsearch has been increased to 2G starting in ES 5.0.
docker info output:Containers: 13
Running: 0
Paused: 0
Stopped: 13
Images: 117
Server Version: 17.03.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 977c511eda0925a723debdc94d09459af49d082a
runc version: a01dafd48bc1c7cc12bdb01206f9fea7dd6feb70
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.12-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 11.71 GiB
Name: moby
ID: FX7I:NJNA:YJZN:LC7Z:6XA4:XGEJ:HUO5:MC24:FTIQ:D2WM:H3LZ:56Z6
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 17
Goroutines: 28
System Time: 2017-03-29T18:26:49.377743743Z
EventsListeners: 1
No Proxy: *.local, 169.254/16
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
docker-compose version:
docker-compose version 1.11.2, build dfed245
Host OS and version:
MacOS 10.10.5
ElasticSearch containers pulled from docker.elastic.co fail to start and throw the following error: docker: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "exec: \"-Etransport.host=0.0.0.0\": executable file not found in $PATH". I've tried all the 5.x variants of the image and all exhibit the same problem.
Here is the command used: docker run -d -p 9118:9200 -p 9119:9300 --name tester_es docker.elastic.co/elasticsearch/elasticsearch:5.3.0 -Etransport.host=0.0.0.0 -Ehttp.host=0.0.0.0
Interestingly, containers pulled from the soon-to-be-deprecated docker hub collection start up and run just fine: docker run -d -p 9118:9200 -p 9119:9300 --name tester_es elasticsearch:5.3.0 -Etransport.host=0.0.0.0 -Ehttp.host=0.0.0.0
Hi,
How to configure LDAP integration with search guard in elastic search.
i followed below link
https://github.com/floragunncom/search-guard-docs/blob/master/ldap.md
please assist me.
Hi,
I tried to run docker compose based on the file attached here:
https://gist.githubusercontent.com/dliappis/52de1f4bb10fd23f5b91ea5fcb5d2560/raw/260723cf8d496ed15a5dd4801a6e9cb616c9f3c3/docker-compose.yml
Error Im getting is:
Pulling elasticsearch (docker.elastic.co/elasticsearch/elasticsearch:latest)... ERROR: Get https://docker.elastic.co/v2/elasticsearch/elasticsearch/manifests/latest: Get https://container-registry.elastic.co:7000/auth?scope=repository%3Aelasticsearch%2Felasticsearch%3Apull&service=token-service: EOF
Versions:
Docker version 1.12.1, build 6f9534c, experimental
docker-compose version 1.8.0, build f3628c7
docker info output:Containers: 3
Running: 1
Paused: 0
Stopped: 2
Images: 39
Server Version: 17.03.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 977c511eda0925a723debdc94d09459af49d082a
runc version: a01dafd48bc1c7cc12bdb01206f9fea7dd6feb70
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.12-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 2.933 GiB
Name: moby
ID: ELNG:I7QS:OVYM:D26P:MXM7:3UUJ:GAWR:NZWV:2DEP:T7TX:NGRE:TYWD
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 30
Goroutines: 45
System Time: 2017-03-15T13:54:03.450167757Z
EventsListeners: 3
No Proxy: *.local, 169.254/16
Username: jeantil
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
docker-compose version:docker-compose version 1.11.2, build dfed245I tried migrating an existing docker-compose which uses a named volume to store data. It would be sad if I had to delete the volume and recreate it
I running the docker-compose file with the docker.elastic.co image uncommented as in :
version: '2'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.2.2
#image: elasticsearch:5.2.2
environment:
- cluster.name=rzv2
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
mem_limit: 1g
expose:
- "9200"
- "9300"
ports:
- "9301:9300"
- "9201:9200"
volumes:
- "./es:/usr/share/elasticsearch/config"
kibana:
image: docker.elastic.co/kibana/kibana:5.2.2
#image: kibana:5.2.2
links:
- elasticsearch:elasticsearch
ports:
- "5601:5601"
yields the following output
elasticsearch_1 | 2017-03-15 13:53:26,149 main ERROR Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
elasticsearch_1 | at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
elasticsearch_1 | at java.lang.SecurityManager.checkPermission(SecurityManager.java:585)
elasticsearch_1 | at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanTrustPermission(DefaultMBeanServerInterceptor.java:1848)
elasticsearch_1 | at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:322)
elasticsearch_1 | at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
elasticsearch_1 | at org.apache.logging.log4j.core.jmx.Server.register(Server.java:389)
elasticsearch_1 | at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:167)
elasticsearch_1 | at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:140)
elasticsearch_1 | at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:541)
elasticsearch_1 | at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:258)
elasticsearch_1 | at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:206)
elasticsearch_1 | at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:220)
elasticsearch_1 | at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:197)
elasticsearch_1 | at org.elasticsearch.common.logging.LogConfigurator.configureStatusLogger(LogConfigurator.java:125)
elasticsearch_1 | at org.elasticsearch.common.logging.LogConfigurator.configureWithoutConfig(LogConfigurator.java:67)
elasticsearch_1 | at org.elasticsearch.cli.Command.main(Command.java:85)
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89)
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82)
elasticsearch_1 |
elasticsearch_1 | 2017-03-15 13:53:26,448 main ERROR Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
elasticsearch_1 | at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
elasticsearch_1 | at java.lang.SecurityManager.checkPermission(SecurityManager.java:585)
elasticsearch_1 | at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanTrustPermission(DefaultMBeanServerInterceptor.java:1848)
elasticsearch_1 | at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:322)
elasticsearch_1 | at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
elasticsearch_1 | at org.apache.logging.log4j.core.jmx.Server.register(Server.java:389)
elasticsearch_1 | at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:167)
elasticsearch_1 | at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:140)
elasticsearch_1 | at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:541)
elasticsearch_1 | at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:258)
elasticsearch_1 | at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:116)
elasticsearch_1 | at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:83)
elasticsearch_1 | at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:299)
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121)
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112)
elasticsearch_1 | at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54)
elasticsearch_1 | at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122)
elasticsearch_1 | at org.elasticsearch.cli.Command.main(Command.java:88)
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89)
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82)
elasticsearch_1 |
elasticsearch_1 | [2017-03-15T13:53:26,456][WARN ][o.e.c.l.LogConfigurator ] ignoring unsupported logging configuration file [/usr/share/elasticsearch/config/logging.yml], logging is configured via [/usr/share/elasticsearch/config/log4j2.properties]
elasticsearch_1 | [2017-03-15T13:53:26,694][INFO ][o.e.n.Node ] [] initializing ...
elasticsearch_1 | [2017-03-15T13:53:26,721][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
elasticsearch_1 | org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/elasticsearch/data/rzv2]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/elasticsearch/data/rzv2]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
elasticsearch_1 | at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:260) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.node.Node.<init>(Node.java:232) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | ... 6 more
elasticsearch_1 | Caused by: java.io.IOException: failed to obtain lock on /usr/share/elasticsearch/data/nodes/0
elasticsearch_1 | at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:239) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.node.Node.<init>(Node.java:232) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | ... 6 more
elasticsearch_1 | Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes/0/node.lock
elasticsearch_1 | at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:?]
elasticsearch_1 | at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]
elasticsearch_1 | at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]
elasticsearch_1 | at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177) ~[?:?]
elasticsearch_1 | at java.nio.channels.FileChannel.open(FileChannel.java:287) ~[?:1.8.0_92-internal]
elasticsearch_1 | at java.nio.channels.FileChannel.open(FileChannel.java:335) ~[?:1.8.0_92-internal]
elasticsearch_1 | at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:113) ~[lucene-core-6.4.1.jar:6.4.1 72f75b2503fa0aa4f0aff76d439874feb923bb0e - jpountz - 2017-02-01 14:43:32]
elasticsearch_1 | at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) ~[lucene-core-6.4.1.jar:6.4.1 72f75b2503fa0aa4f0aff76d439874feb923bb0e - jpountz - 2017-02-01 14:43:32]
elasticsearch_1 | at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) ~[lucene-core-6.4.1.jar:6.4.1 72f75b2503fa0aa4f0aff76d439874feb923bb0e - jpountz - 2017-02-01 14:43:32]
elasticsearch_1 | at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:226) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.node.Node.<init>(Node.java:232) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.2.2.jar:5.2.2]
elasticsearch_1 | ... 6 more
elasticsearch_1 | 2017-03-15 13:53:26,729 pool-1-thread-1 ERROR Unable to unregister MBeans java.security.AccessControlException: access denied ("javax.management.MBeanServerPermission" "createMBeanServer")
elasticsearch_1 | at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
elasticsearch_1 | at java.security.AccessController.checkPermission(AccessController.java:884)
elasticsearch_1 | at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
elasticsearch_1 | at java.lang.management.ManagementFactory.getPlatformMBeanServer(ManagementFactory.java:465)
elasticsearch_1 | at org.apache.logging.log4j.core.jmx.Server.unregisterLoggerContext(Server.java:246)
elasticsearch_1 | at org.apache.logging.log4j.core.LoggerContext.stop(LoggerContext.java:317)
elasticsearch_1 | at org.apache.logging.log4j.core.AbstractLifeCycle.stop(AbstractLifeCycle.java:127)
elasticsearch_1 | at org.apache.logging.log4j.core.LoggerContext$1.run(LoggerContext.java:275)
elasticsearch_1 | at org.apache.logging.log4j.core.util.DefaultShutdownCallbackRegistry$RegisteredCancellable.run(DefaultShutdownCallbackRegistry.java:104)
elasticsearch_1 | at org.apache.logging.log4j.core.util.DefaultShutdownCallbackRegistry.run(DefaultShutdownCallbackRegistry.java:74)
elasticsearch_1 | at java.lang.Thread.run(Thread.java:745)
elasticsearch_1 |
docker_elasticsearch_1 exited with code 1
running the same docker-compose file with the previous image uncommented works just fine.
When I try to remove x-pack in the official image, I get the following error.
Dockerfile
FROM docker.elastic.co/elasticsearch/elasticsearch:5.2.2
.....
Step 5 : RUN /usr/share/elasticsearch/bin/elasticsearch-plugin remove x-pack
---> Running in 3c08ad904d0f
-> Removing x-pack...
[91mException in thread "main" java.nio.file.AtomicMoveNotSupportedException: /usr/share/elasticsearch/plugins/x-pack -> /usr/share/elasticsearch/plugins/.removing-x-pack: Cross-device link
at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:394)
at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
at java.nio.file.Files.move(Files.java:1395)
at org.elasticsearch.plugins.RemovePluginCommand.execute(RemovePluginCommand.java:88)
at org.elasticsearch.plugins.RemovePluginCommand.execute(RemovePluginCommand.java:59)
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122)
at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:69)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122)
at org.elasticsearch.cli.Command.main(Command.java:88)
at org.elasticsearch.plugins.PluginCli.main(PluginCli.java:47)
[0m
Removing intermediate container 3c08ad904d0f
I created a repository to built into docker hub to demonstrate this problem.
GitHub Repo: https://github.com/frekele/docker-elasticsearch
Docker Hub: https://hub.docker.com/r/frekele/elasticsearch/
You can see inside the build log the error:
Related Issues with deprecated repository:
docker info output:docker-compose version: 1.2.0I am using the following docker-compose.yml file to instanciate one Elasticsearch node:
version: '2'
services:
elasticsearch-master-1:
image: docker.elastic.co/elasticsearch/elasticsearch:5.1.1
container_name: master-1
labels:
type: master-node
environment:
- cluster.name=${cluster_name}
- node.name=master-1
- bootstrap.memory_lock=true
- discovery.zen.ping.unicast.hosts=elasticsearch-master-2
- discovery.zen.minimum_master_nodes=2
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- node.master=true
- node.data=false
- node.ingest=false
- xpack.security.enabled=false
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
mem_limit: 1000000000
cap_add:
- IPC_LOCK
My expectation is to have a Xms and Xmx with a value of 512m (overloading. However, it seems that the elasticsearch node is started with the default value defined in the jvm.options file. The container shows 2 different Xms and Xmx values:
ps -aef | grep elasticsearch
1 elastics 257:21 /usr/lib/jvm/java-1.8-openjdk/bin/java -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Xms512m -Xmx512m -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-5.1.1.jar:/usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -Ecluster.name=docker-cluster -Ediscovery.zen.ping.unicast.hosts=master-1,master-2 -Enode.data=true -Enode.master=false -Enode.name=data-3 -Ediscovery.zen.minimum_master_nodes=2 -Ebootstrap.memory_lock=true -Expack.security.enabled=false -Enode.ingest=true
Indeed, the bin/es-docker entrypoint only appends the Xmx and Xms in the ES_JAVA_OPTS variable.
Official images was running elasticsearch with user elasticsearch, and Kubernetes doesn't support ulimit, I failed to start container with below errors:
[2017-03-20T05:33:50,204][WARN ][o.e.b.JNANatives ] Unable to lock JVM Memory: error=12, reason=Out of memory
[2017-03-20T05:33:50,206][WARN ][o.e.b.JNANatives ] This can result in part of the JVM being swapped out.
[2017-03-20T05:33:50,206][WARN ][o.e.b.JNANatives ] Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536
[2017-03-20T05:33:50,206][WARN ][o.e.b.JNANatives ] These can be adjusted by modifying /etc/security/limits.conf, for example:
# allow user 'elasticsearch' mlockall
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
I tried to use a custom Dockerfile and ADD the limits.conf, that doesn't work also, anyone has the idea how to get it running on Kubernetes?
elasticsearch.yml:
cluster.name: es-cluster
node.name: es-${NODE_NAME}
network.host: 0.0.0.0
action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"
NODE_NAME is defined in a .env file.
Dockerfile:
FROM docker.elastic.co/elasticsearch/elasticsearch:5.3.0
ADD config/elasticsearch.yml /usr/share/elasticsearch/config/
RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install --batch cloud-aws
When starting the container, I get:
Exception in thread "main" java.lang.IllegalArgumentException: Could not resolve placeholder 'NODE_NAME'
at org.elasticsearch.common.settings.PropertyPlaceholder.parseStringValue(PropertyPlaceholder.java:116)
at org.elasticsearch.common.settings.PropertyPlaceholder.replacePlaceholders(PropertyPlaceholder.java:69)
at org.elasticsearch.common.settings.Settings$Builder.replacePropertyPlaceholders(Settings.java:1083)
at org.elasticsearch.common.settings.Settings$Builder.replacePropertyPlaceholders(Settings.java:1043)
at org.elasticsearch.node.InternalSettingsPreparer.initializeSettings(InternalSettingsPreparer.java:140)
at org.elasticsearch.node.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:120)
at org.elasticsearch.cli.EnvironmentAwareCommand.createEnv(EnvironmentAwareCommand.java:63)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:58)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122)
at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:69)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122)
at org.elasticsearch.cli.Command.main(Command.java:88)
at org.elasticsearch.plugins.PluginCli.main(PluginCli.java:47)
If I replace ${NODE_NAME} by a string, the container starts.
You have elasticsearch-docker and kibana-docker images, could you also provide logstash-docker images? I think is great to have full stack docker images for testing things.
I found out that if my child Docker image has 4 or more file layers then Elasticsearch terminates with the following error:
# docker run elasticsearch-test:5.2.2
[2017-03-26T21:46:54,902][INFO ][o.e.n.Node ] [] initializing ...
[2017-03-26T21:46:54,948][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/elasticsearch/data/docker-cluster]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) ~[elasticsearch-5.2.2.jar:5.2.2]
Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/elasticsearch/data/docker-cluster]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:260) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.node.Node.<init>(Node.java:232) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.2.2.jar:5.2.2]
... 6 more
Caused by: java.io.IOException: failed to obtain lock on /usr/share/elasticsearch/data/nodes/0
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:239) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.node.Node.<init>(Node.java:232) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.2.2.jar:5.2.2]
... 6 more
Caused by: java.io.IOException: Mount point not found
at sun.nio.fs.LinuxFileStore.findMountEntry(LinuxFileStore.java:91) ~[?:?]
at sun.nio.fs.UnixFileStore.<init>(UnixFileStore.java:65) ~[?:?]
at sun.nio.fs.LinuxFileStore.<init>(LinuxFileStore.java:44) ~[?:?]
at sun.nio.fs.LinuxFileSystemProvider.getFileStore(LinuxFileSystemProvider.java:51) ~[?:?]
at sun.nio.fs.LinuxFileSystemProvider.getFileStore(LinuxFileSystemProvider.java:39) ~[?:?]
at sun.nio.fs.UnixFileSystemProvider.getFileStore(UnixFileSystemProvider.java:368) ~[?:?]
at java.nio.file.Files.getFileStore(Files.java:1461) ~[?:1.8.0_92-internal]
at org.elasticsearch.env.ESFileStore.getMatchingFileStore(ESFileStore.java:107) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.env.Environment.getFileStore(Environment.java:340) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.env.NodeEnvironment$NodePath.<init>(NodeEnvironment.java:108) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:227) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.node.Node.<init>(Node.java:232) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:241) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.2.2.jar:5.2.2]
... 6 more
There is my Dockerfile for elasticsearch-test image:
# Failed on 4 or more layers
FROM docker.elastic.co/elasticsearch/elasticsearch:5.2.2
# Failed on 4 or more layers
# FROM elasticsearch:5.2.2-alpine
# Never failed
# FROM elasticsearch:5.2.2
RUN echo test 0 > ftest0
RUN echo test 1 > ftest1
# RUN echo test 2 > ftest2
# RUN echo test 3 > ftest3
# RUN echo test 4 > ftest4
# RUN echo test 5 > ftest5
# RUN echo test 6 > ftest6
COPY test0 dtest0/
COPY test1 dtest1/
# COPY test2 dtest2/
# COPY test3 dtest3/
# COPY test4 dtest4/
# COPY test5 dtest5/
# COPY test6 dtest6/
Elasticsearch failed on any arbitrary combination of 4 or more RUN or COPY layers.
All folders testN contains file with the same name. File contains approximately 15 random characters.
Out of curiosity I have tried images from Docker Hub - Alpine based image exhibits the same behavior but Debian based image seems that it is not affected.
I'm running on Docker for macOS 1.13.1 (15353) and OS X 10.11.6:
# docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 158
Server Version: 1.13.1
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Swarm: active
NodeID: srwwkecxby41p6pv153ywy6y0
Is Manager: true
ClusterID: fdmz6u2uni02t5ozsxnkgdrkg
Managers: 1
Nodes: 1
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 3
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Node Address: 192.168.65.2
Manager Addresses:
192.168.65.2:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.8-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.787 GiB
Name: moby
ID: 3ODG:GSIF:YOIL:LAXH:DB32:YYZZ:PIIX:A5XB:WXR2:QZ3C:EVZA:NZ6A
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 41
Goroutines: 136
System Time: 2017-03-26T21:35:01.318890699Z
EventsListeners: 1
No Proxy: *.local, 169.254/16
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
# uname -a
Darwin Marvin.local 15.6.0 Darwin Kernel Version 15.6.0: Mon Jan 9 23:07:29 PST 2017; root:xnu-3248.60.11.2.1~1/RELEASE_X86_64 x86_64
It would be nice to add the HEALTCHECK feature, see: https://docs.docker.com/engine/reference/builder/ to deduct ES is up and running in a "healthy" state. Healthy in this context should probably mean available, and not cluster state yellow etc
docker info output:
Containers: 85 Running: 0 Paused: 0 Stopped: 85 Images: 336 Server Version: 17.04.0-ce Storage Driver: aufs Root Dir: /var/lib/docker/1000.1000/aufs Backing Filesystem: extfs Dirs: 372 Dirperm1 Supported: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: containerd version: 422e31ce907fd9c3833a38d7b8fdd023e5a76e73 runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228 init version: 949e6fa Security Options: apparmor seccomp Profile: default userns Kernel Version: 4.8.0-46-generic Operating System: Ubuntu 16.04.2 LTS OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 31.33GiB Name: workBuntu ID: 5N25:YF6I:3CKB:YMIE:G4QQ:QYDD:PTVZ:TXP4:7YMH:IKR2:Q4XX:K4FO Docker Root Dir: /var/lib/docker/1000.1000 Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false WARNING: No swap limit support
docker-compose version:
docker-compose version 1.12.0, build b31ff33
Host OS and version:
Ubuntu 16,04
I am running elasticsearch with user remapping for docker as such:
https://blog.yadutaf.fr/2016/04/14/docker-for-your-users-introducing-user-namespace/
Running the image with docker-compose gives me the following error:
/bin/bash: bin/es-docker: Permission denied
elasticsearch exited with code 126
docker info output:alec@Alecs-MBP / $ docker info
Containers: 10
Running: 0
Paused: 0
Stopped: 10
Images: 451
Server Version: 17.03.0-ce
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 515
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 977c511eda0925a723debdc94d09459af49d082a
runc version: a01dafd48bc1c7cc12bdb01206f9fea7dd6feb70
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.12-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 1.952 GiB
Name: moby
ID: XXEV:R3TK:Q4Z7:5Z34:M625:5RH7:2K3U:3EIS:44UK:VIU3:KHRM:4DHC
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 17
Goroutines: 28
System Time: 2017-03-18T03:58:26.757918501Z
EventsListeners: 1
No Proxy: *.local, 169.254/16
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
alec@Alecs-MBP / $
docker-compose version:alec@Alecs-MBP / $ docker-compose -v
docker-compose version 1.11.2, build dfed245
documentation here: https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
says to run the following:
docker run -p 9200:9200 -e "http.host=0.0.0.0" -e "transport.host=127.0.0.1" docker.elastic.co/elasticsearch/elasticsearch:5.2.2
This fails on my system as follows:
alec@Alecs-MBP / $ docker --version
Docker version 17.03.0-ce, build 60ccb22
alec@Alecs-MBP / $ docker run -p 9200:9200 -e "http.host=0.0.0.0" -e "transport.host=127.0.0.1" docker.elastic.co/elasticsearch/elasticsearch:5.2.2
OpenJDK 64-Bit Server VM warning: Insufficient space for shared memory file:
1
Try using the -Djava.io.tmpdir= option to select an alternate temp location.
[2017-03-18T03:57:03,945][WARN ][o.e.b.Natives ] unable to load JNA native support library, native methods will be disabled.
java.lang.UnsatisfiedLinkError: Failed to create temporary file for /com/sun/jna/linux-x86-64/libjnidispatch.so library: No space left on device
at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:866) ~[jna-4.2.2.jar:4.2.2 (b0)]
at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826) ~[jna-4.2.2.jar:4.2.2 (b0)]
at com.sun.jna.Native.<clinit>(Native.java:140) ~[jna-4.2.2.jar:4.2.2 (b0)]
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_92-internal]
at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_92-internal]
at org.elasticsearch.bootstrap.Natives.<clinit>(Natives.java:45) [elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:104) [elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:203) [elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) [elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) [elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) [elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) [elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) [elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.cli.Command.main(Command.java:88) [elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) [elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) [elasticsearch-5.2.2.jar:5.2.2]
[2017-03-18T03:57:03,957][WARN ][o.e.b.Natives ] cannot check if running as root because JNA is not available
[2017-03-18T03:57:03,958][WARN ][o.e.b.Natives ] cannot install system call filter because JNA is not available
[2017-03-18T03:57:03,959][WARN ][o.e.b.Natives ] cannot register console handler because JNA is not available
[2017-03-18T03:57:03,967][WARN ][o.e.b.Natives ] cannot getrlimit RLIMIT_NPROC because JNA is not available
[2017-03-18T03:57:03,967][WARN ][o.e.b.Natives ] cannot getrlimit RLIMIT_AS beacuse JNA is not available
[2017-03-18T03:57:04,280][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Unable to access 'path.scripts' (/usr/share/elasticsearch/config/scripts)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) ~[elasticsearch-5.2.2.jar:5.2.2]
Caused by: java.lang.IllegalStateException: Unable to access 'path.scripts' (/usr/share/elasticsearch/config/scripts)
at org.elasticsearch.bootstrap.Security.addPath(Security.java:379) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:251) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:215) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Security.configure(Security.java:121) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:236) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.2.2.jar:5.2.2]
... 6 more
Caused by: java.nio.file.FileSystemException: /usr/share/elasticsearch/config/scripts: No space left on device
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) ~[?:1.8.0_92-internal]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_92-internal]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_92-internal]
at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:1.8.0_92-internal]
at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_92-internal]
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_92-internal]
at java.nio.file.Files.createDirectories(Files.java:767) ~[?:1.8.0_92-internal]
at org.elasticsearch.bootstrap.Security.ensureDirectoryExists(Security.java:421) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Security.addPath(Security.java:377) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:251) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:215) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Security.configure(Security.java:121) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:236) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.2.2.jar:5.2.2]
... 6 more
alec@Alecs-MBP / $
As Filebeat modules require the Elasticsearch ingest-geoip and ingest-user-agent, it would help the getting starting experience quite a bit if these two plugins would be included in the default Docker image.
I'm NOT able to PULL this image
docker.elastic.co/elasticsearch/elasticsearch
on official elastic hosting
Why we have leave DockerHUB??? Why??? Mirroring is the best approch!!
How do we install discovery plugins(AWS, Auzure etc..) in to the docker image?
Please provide real world examples in the style of the docker-compose.yml at Install Elasticsearch with Docker.
A cluster of 2 nodes on the same host are a starting point, I would love to see how to "officially" setup and run a "production" cluster on 3 bare-metal servers.
Any chance there will be an alpine image created for elastic search?
How do we create Hot-warm architecture with dedicated Master, Client and Data Nodes using official docker image? Any instructions will be helpful.
Config options are passed in as environmental variables like -e "cluster.name=mynewclustername" but this causes issues with Kubernetes and many other places as they don't accept environmental variables with . in the name. This should be changed to match standards and use _ instead.
As far as I can tell the only thing that accepts environmental variables with . in the name is docker.
I am trying to map my local instance volume as data drive with docker image. When i run the docker image its throwing java.nio.file.AccessDeniedException
Docker command:
docker run --rm -p 9200:9200 -v /elasticsearchdb/:/usr/share/elasticsearch/data docker.elastic.co/elasticsearch/elasticsearch:5.0.1
ERROR from logs:
[2016-11-27T16:51:28,964][INFO ][o.e.n.Node ] [] initializing ...
[2016-11-27T16:51:28,992][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Failed to created node environment
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:116) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:103) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:96) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.cli.Command.main(Command.java:62) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:73) ~[elasticsearch-5.0.1.jar:5.0.1]
Caused by: java.lang.IllegalStateException: Failed to created node environment
at org.elasticsearch.node.Node.(Node.java:243) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.node.Node.(Node.java:220) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:191) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:191) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:286) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:112) ~[elasticsearch-5.0.1.jar:5.0.1]
... 6 more
Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]
at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:?]
at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_92-internal]
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_92-internal]
at java.nio.file.Files.createDirectories(Files.java:767) ~[?:1.8.0_92-internal]
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:220) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.node.Node.(Node.java:240) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.node.Node.(Node.java:220) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:191) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:191) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:286) ~[elasticsearch-5.0.1.jar:5.0.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:112) ~[elasticsearch-5.0.1.jar:5.0.1]
... 6 more
Getting this error on Docker-
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000085330000, 2060255232, 0) failed; error='Cannot allocate memory' (errno=12)
Running Command-docker run --name elasticsearch -d -p 9200:9200 elasticsearch -e ES_JAVA_OPTS="-Xms2g -Xmx2g"
I am getting the following error when trying to run Elasticsearch with the --user option
main ERROR Could not register mbeans java.security.AccessControlException: access denied
Docker run command:
docker run -d --name ES --user ES:ES --log-opt max-size=1g -m 3G --cpus="1.0" -v "$PWD/esdata":/usr/share/elasticsearch/data -p 9200:9200 elasticsearch
if i just remove the --user option the error don't show up.
it's running on a Debian host and everything is updated to the latest version.
There a few Kubernetes configs out there and associated lifecycle hook script (ie. wait until all shards move away from container on shutdown). Is there any appetite for including them here, where they would benefit from more review?
I'm thinking things like:
kubernetes/kubernetes#3595 (comment)
https://github.com/jetstack/elasticsearch-pet/blob/master/pre-stop-hook.sh
When enabling audit logging in x-pack, this is written to /usr/share/elasticsearch/logs/elasticsearch_access.log inside the container.
Proposing to adjust config/x-pack/log4j2.properties to write to stdout as we do with the normal elasticsearch logs.
For those of us that can't upgrade to ES 5.x (yet), it would be super helpful if there was an official 2.X version of this Docker image that we could build upon. Any plans to support versions prior to 5.x?
I try to install unzip and zip with RUN yum install -y zip unzip
but it throw /bin/sh: yum: not found,
and in v5.4.0, it works.
It would be awesome if we could get a docker image that is based on windows server images instead of CentOS.
See here for a PR for the deprecated docker-library es repo: docker-library/elasticsearch#151
Will there be an official elasticsearch-docker container based on the windows nano server 2016 image?
My instance expired and i have installed a free basic license but i cant login in kibana with message: Your "Basic license does not support Security. Please upgrade your license or disable Security in Elasticsearch."
I read i have to disable security using xpack.security.enabled = false on elasticsearch.yml and kibana.yml.
How can i run the elasticksearch and kibana instances disabling security? is it possible to pass it as a param?
Hello to everyone!
I'm trying to change the current configuration of ELK cluster from docker host mode into docker bridge and faced with some problems.
According to the documentation: https://www.elastic.co/blog/docker-networking when I run elasticsearch in bridge mode, I need to provide the following configuration:
"When running Elasticsearch, you will need to ensure it publishes to an IP address that is reachable from outside the container; this can be configured via the setting network.publish_host.
For the discovery between the nodes you have to configure Zen Discovery via the settings discovery.zen.ping.unicast.hosts and discovery.zen.minimum_master_nodes."
Example:
docker run -d -p 9200:9200 -p 9300:9300 elasticsearch:2
elasticsearch
-Des.discovery.zen.ping.unicast.hosts=192.168.99.100,192.168.99.101
-Des.discovery.zen.minimum_master_nodes=2
-Des.network.publish_host=192.168.99.100
docker run -d -p 9200:9200 -p 9300:9300 elasticsearch:2
elasticsearch
-Des.discovery.zen.ping.unicast.hosts=192.168.99.100,192.168.99.101
-Des.discovery.zen.minimum_master_nodes=2
-Des.network.publish_host=192.168.99.101
We need to run elasticsearch on each node. So when I start a cluster with 3 nodes, it looks ok, I can provide needed zen parameters. But if the cluster scale up, how will old nodes know about a new one? New ip-addresses should be added to the Des.discovery.zen.ping.unicast.hosts of currently running dockers with elasticsearch.
Could you please clarify to me how the configuration should be set up? All docker containers are under Marathon control.
Thanks,
Olga
When I run this locally I don't get this but now I am mounting a drive in a Kubernetes cluster and I am getting the following error:
[2017-06-06T21:32:38,666][INFO ][o.e.n.Node ] [eazy-e-0] initializing ...
[2017-06-06T21:32:38,701][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [eazy-e-0] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/elasticsearch/data/icx-es-data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:96) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.cli.Command.main(Command.java:62) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) ~[elasticsearch-5.1.1.jar:5.1.1]
Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/elasticsearch/data/icx-es-data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:259) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.node.Node.<init>(Node.java:249) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.node.Node.<init>(Node.java:229) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:214) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:214) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:306) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.1.1.jar:5.1.1]
... 6 more
Caused by: java.io.IOException: failed to obtain lock on /usr/share/elasticsearch/data/nodes/0
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:238) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.node.Node.<init>(Node.java:249) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.node.Node.<init>(Node.java:229) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:214) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:214) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:306) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.1.1.jar:5.1.1]
... 6 more
Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes/0/node.lock
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]
at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177) ~[?:?]
at java.nio.channels.FileChannel.open(FileChannel.java:287) ~[?:1.8.0_92-internal]
at java.nio.channels.FileChannel.open(FileChannel.java:335) ~[?:1.8.0_92-internal]
at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:113) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:225) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.node.Node.<init>(Node.java:249) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.node.Node.<init>(Node.java:229) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:214) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:214) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:306) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.1.1.jar:5.1.1]
... 6 more
I noticed that the cluster name is now in the mount path.
I am utilizing the following image:
FROM docker.elastic.co/elasticsearch/elasticsearch:5.1.1
Maybe this is a docker version issue as well? Thanks!
I'm trying to create a new image (extending this one) which just removes x-pack.
Here is my Dockerfile:
FROM docker.elastic.co/elasticsearch/elasticsearch:5.2.2
# Uninstall xpack
RUN eval ${ES_JAVA_OPTS:-} elasticsearch-plugin remove x-packWhen I try to build it.
docker build -t mynamespace/elasticsearch:5.2.2 .
I'm getting the following error:
Sending build context to Docker daemon 2.048 kB
Step 1/2 : FROM docker.elastic.co/elasticsearch/elasticsearch:5.2.2
---> 89315294a341
Step 2/2 : RUN eval ${ES_JAVA_OPTS:-} elasticsearch-plugin remove x-pack
---> Running in b17c4028966a
-> Removing x-pack...
Exception in thread "main" java.nio.file.AtomicMoveNotSupportedException: /usr/share/elasticsearch/plugins/x-pack -> /usr/share/elasticsearch/plugins/.removing-x-pack: Cross-device link
at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:394)
at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
at java.nio.file.Files.move(Files.java:1395)
at org.elasticsearch.plugins.RemovePluginCommand.execute(RemovePluginCommand.java:88)
at org.elasticsearch.plugins.RemovePluginCommand.execute(RemovePluginCommand.java:59)
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122)
at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:69)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122)
at org.elasticsearch.cli.Command.main(Command.java:88)
at org.elasticsearch.plugins.PluginCli.main(PluginCli.java:47)
The command '/bin/sh -c eval ${ES_JAVA_OPTS:-} elasticsearch-plugin remove x-pack' returned a non-zero code: 1
docker info outputContainers: 8
Running: 0
Paused: 0
Stopped: 8
Images: 105
Server Version: 17.03.0-ce
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 307
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 977c511eda0925a723debdc94d09459af49d082a
runc version: a01dafd48bc1c7cc12bdb01206f9fea7dd6feb70
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.4.0-65-generic
Operating System: Ubuntu 16.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 7.701 GiB
Name: cit009992
ID: JYLV:KNHK:MMQR:NSB3:3J5F:SBH3:I3K4:T7KC:JKKQ:UN5J:PEGI:3N3W
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Username: fabito
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
I setup a standalone elasticsearch using elasticsearch:alpine.
Try to enable cors setting in /usr/share/elasticsearch/config/elasticsearch.yml
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-methods: OPTIONS, HEAD, GET, POST, PUT, DELETE
http.cors.allow-headers: "X-Requested-With,X-Auth-Token,Content-Type, Content-Length, Authorization"
then restart this container.
curl --head http://localhost:9200, the result is
HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 327
No CORS header, is it right?
Hi,
Some env variable are not passed to Elasticsearch. For example, if I run:
$ docker run -it -p 9300:9200 \n
-e "thread_pool.bulk.queue_size=500" \n
-e "cluster.name=hey" \n
-e "http.host=0.0.0.0" \n
-e "transport.host=127.0.0.1" \n
docker.elastic.co/elasticsearch/elasticsearch:5.4.1the variable thread_pool.bulk.queue_size is not set to 500 while cluster.name is correctly set to "hey".
$ curl -s 'localhost:9300/_cluster/health?pretty'
{
"cluster_name" : "hey",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 3,
"active_shards" : 3,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 3,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 50.0
}$ $ curl -s 'localhost:9300/_cat/thread_pool/bulk?v&h=queue_size'
queue_size
200Do I miss something?
Without digging deeper, it seems to me that https://github.com/elastic/elasticsearch-docker/blob/master/build/elasticsearch/bin/es-docker#L16 may not match variable with an '_' in their name.
Benoît
docker info output:docker-compose version:docker-compose version
docker-compose version 1.8.0, build unknown
docker-py version: 1.9.0
CPython version: 2.7.10
OpenSSL version: OpenSSL 0.9.8zh 14 Jan 2016
OS X EI Capitan Version 10.11.6
trying to run the docker image on Mac Osx and seeing the following error.
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000085330000, 2060255232, 0) failed; error='Out of memory' (errno=12)
The link referenced in the README ("Elastic web site") points to a page which does not exist.
docker info outputwhen running docker pull docker.elastic.co/elasticsearch/elasticsearch:5.4.2
the image cannot be found
Tcp service binds to 127.0.0.1 and makes it inaccessible from outside of the container. Http service binds to 0.0.0.0
Elastic search log:
elasticsearch | [2017-03-10T15:35:32,418][WARN ][i.n.u.i.MacAddressUtil ] Failed to find a usable hardware address from the network interfaces; using random bytes: 17:75:0b:37:b3:dd:79:c6
elasticsearch | [2017-03-10T15:35:32,532][INFO ][o.e.t.TransportService ] [fye6pJB] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
elasticsearch | [2017-03-10T15:35:35,630][INFO ][o.e.c.s.ClusterService ] [fye6pJB] new_master {fye6pJB}{fye6pJB4SgO2oz4NQ0UKvg}{kF0EEapzQaWTIZARDM5CDA}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
elasticsearch | [2017-03-10T15:35:35,656][INFO ][o.e.h.HttpServer ] [fye6pJB] publish_address {172.18.0.2:9200}, bound_addresses {[::]:9200}
elasticsearch | [2017-03-10T15:35:35,657][INFO ][o.e.n.Node ] [fye6pJB] started
elasticsearch | [2017-03-10T15:35:35,704][INFO ][o.e.g.GatewayService ] [fye6pJB] recovered [0] indices into cluster_state
docker-info:
Containers: 4
Running: 3
Paused: 0
Stopped: 1
Images: 321
Server Version: 17.03.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 977c511eda0925a723debdc94d09459af49d082a
runc version: a01dafd48bc1c7cc12bdb01206f9fea7dd6feb70
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.12-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 15.65 GiB
Name: moby
ID: 2XJR:H7L7:UVXN:VAWH:2EHG:GPUZ:GQSY:I55R:XJUA:3LUX:4VH5:6UMC
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 45
Goroutines: 64
System Time: 2017-03-10T15:37:54.026016483Z
EventsListeners: 2
No Proxy: *.local, 169.254/16
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
docker-compose version:
docker-compose version 1.11.2, build dfed245
docker-py version: 2.1.0
CPython version: 2.7.12
OpenSSL version: OpenSSL 1.0.2j 26 Sep 2016
uname -a:
Darwin 16.4.0 Darwin Kernel Version 16.4.0: Thu Dec 22 22:53:21 PST 2016; root:xnu-3789.41.3~3/RELEASE_X86_64 x86_64
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.