ertis-research / kafka-ml Goto Github PK

For example we create a new MINST deployment however it could not be running as error
"/proc/driver/nvidia/version does not exist"
2022-12-09 15:51:57.845894: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: UNKNOWN ERROR (34)
2022-12-09 15:51:57.845959: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (tfexecutor-7bc4bbdbf4-tw7sl): /proc/driver/nvidia/version does not exist

We install vanilla K8s based on redhat8 then deploy kafka-ml on this k8s cluster, so we don't have GPU.

Please advise how we can disable GPU or fix the issue.

Hi,

I tried to use Build_Kafka_ML.sh in Ubuntu but it game some problems because the file seems to be in Windows format.

Solution: I used dos2unix utility to convert it to Linux format.

Hi there 👋

I open this issue because I think it could be a great idea to start writing guides that make working with Kafka-ML easier. For example, how to allow Kafka-ML to access your Kubernetes cluster on Google Cloud. That would require additional knowledge in Kubernetes, so we could provide the necessary information to get started with this framework using a cloud cluster (e.g., in Google Cloud).

I would like to write a short guide including the following links to explain the above example, how should it be done? We should think in an organized way to write them before the README files are too long.

Links

• https://kubernetes.io/docs/reference/access-authn-authz/rbac/
• kubernetes-client/python#1297
• https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/
• kubernetes-client/python#1410
• https://github.com/kubernetes-client/python
• https://stackoverflow.com/questions/62726057/accessing-kubernetes-apis-from-local-machine
• https://stackoverflow.com/questions/62589073/kubernetes-api-cannot-list-resource-pods-in-api-group

Hello,
I am trying to run kafka-ml on a k8s cluster and execute the mnist example code. However, I am facing the following error and having difficulty connecting to Kafka.

This is the result after cloning the repository, creating pods using 'apply kustomize/local', and modifying the bootstrap_servers in mnist_dataset_training_example.py to the IP and NodePort of the node where the Kafka pod is running.

I appreciate your assistance, even though it might be inconvenient. Thank you.

hi, do you involve kafka-ml incremental training?