kubernetes-retired / application-images Goto Github PK
View Code? Open in Web Editor NEW[EOL] Image build contents for Kubernetes applications.
License: Apache License 2.0
[EOL] Image build contents for Kubernetes applications.
License: Apache License 2.0
As per the email sent to kubernetes-dev[1], please create a SECURITY_CONTACTS
file.
The template for the file can be found in the kubernetes-template repository[2].
A description for the file is in the steering-committee docs[3], you might need
to search that page for "Security Contacts".
Please feel free to ping me on the PR when you make it, otherwise I will see when
you close this issue. :)
Thanks so much, let me know if you have any questions.
(This issue was generated from a tool, apologies for any weirdness.)
[1] https://groups.google.com/forum/#!topic/kubernetes-dev/codeiIoQ6QE
[2] https://github.com/kubernetes/kubernetes-template-project/blob/master/SECURITY_CONTACTS
[3] https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance-template-short.md
Currently we're running Spark on K8s in AWS with Flannel.
Spark Version: 1.6.2
Spark Deployment Mode: Standalone
K8s Version: 1.3.7
The cluster is up, and all the components connect successfully. The issues appear when we submit a job to Spark.
In the Spark UI, when I go to the executors tab, I see wrong IP address for executors, which doesn't match the POD IP addresse.
POD IP Addresses from kubectl
$kubectl get po
Name | IP |
---|---|
spark-master-7v08x | 10.70.20.5 |
spark-worker-1soa1 | 10.70.19.6 |
spark-worker-mg7q5 | 10.70.28.8 |
spark-worker-vvr4n | 10.70.20.4 |
The executor IP Addresses from the Spark UI
Executor ID | Address |
---|---|
0 | 10.70.20.4:39955 |
1 | 10.70.20.4:48970 |
2 | 10.70.19.0:38067 |
If I check the executor logs, I can see exceptions because the executors are trying to connect to the wrong IP:
16/10/07 16:44:04 INFO RetryingBlockFetcher: Retrying fetch (2/3) for 1 outstanding blocks after 5000 ms
16/10/07 16:45:12 ERROR RetryingBlockFetcher: Exception while beginning fetch of 1 outstanding blocks (after 2 retries)
java.io.IOException: Failed to connect to /10.70.19.0:38067
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:216)
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:167)
at org.apache.spark.network.netty.NettyBlockTransferService$$anon$1.createAndStart(NettyBlockTransferService.scala:90)
at org.apache.spark.network.shuffle.RetryingBlockFetcher.fetchAllOutstanding(RetryingBlockFetcher.java:140)
at org.apache.spark.network.shuffle.RetryingBlockFetcher.access$200(RetryingBlockFetcher.java:43)
at org.apache.spark.network.shuffle.RetryingBlockFetcher$1.run(RetryingBlockFetcher.java:170)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection timed out: /10.70.19.0:38067
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
... 1 more
I've found some related issues on the Spark Jira:
I've seen that @mattf and @zmerlynn did a lot of work around running spark in docker and on Kubernetes so I was hoping we can come up with some configuration workaround ( probably by using Kubernetes features) for launching the cluster which will prevent this from happening.
The k8s-spark-native docker files can live here till we find a better location for them.
This is single point that can be used, rather than having multiple custom images as we currently do.
I've committed here with some modifications, the files that were previously in https://github.com/erikerlandson/openshift-spark/tree/k8s-native.
Do let me know if I've missed something.
TODO:
Let's not use a separate repo for images and configs.
cc @kubernetes/examples
I have a branch on my fork where I update to spark 2.0.1 and Hadoop 2.7.3 and increase the ram sizes including ZEPPELIN_MEM so it can be used for a real workload. Curious if you are interested in getting these changes as a PR.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.