Deploying with mesos about pachyderm

[jdoliner]

The above is also true for Kubernetes as far as I know. We'll aim to eventually have support for both of those platforms.

[brndnmtthws]

Pachyderm is likely a great candidate to become a first-class Mesos framework. It can be run in containers (via Docker or App Container) on CoreOS, much like you're already doing. For service discovery, you can use Mesos-DNS.

[kunthar]

@brndnmtthws 👍 as a reminder :)

[chicagobuss]

Any updates here? Really looking forward to trying this but we're definitely moving towards mesos.

[JoeyZwicker]

We've been in discussions with Mesosphere discussing a joint project to get this working, but there hasn't been significant progress in that direction yet. One of the things we're looking for is a customer who wants to use Mesos and Pachyderm to help us sponsor the development and has a specific use case in mind.

[chicagobuss]

Our problem with Kubernetes is how slow it is to schedule containers. We tested it pretty heavily and were left feeling it was nowhere near production ready.

I feel like regardless of whether you get it sponsored or not, it'd be very good to make Pachyderm work on platforms other than Kubernetes.

[jdoliner]

Our problem with Kubernetes is how slow it is to provision new VMs.

I'm a little confused about what you mean by this, in my experience with Kubernetes it's always run on VMs (or bare metal) that are provisioned by something external like an AWS scaling group or a GCE instance group. I wasn't aware that it was possible for K8s to provision VMs directly.

From what I understand it's not uncommon for people to run Kubernetes as a framework on Mesos which allows scheduling things with the Kubernetes API without requiring a separate cluster. I realize that running k8s on mesos is kind of lame but would something like that be an option for you guys?

It would definitely be good to get Pachyderm running on other platforms, but unfortunately as a small team with very limited resources right now we need to focus on the needs of existing users and don't have a lot of resources to spend on projects that might bring in new users (but also might not). That being said I think we'll likely have more resources available for things like this soon and this particular issue is among the highest priority issues of that type.

[sjezewski]

This is a great ask, but is stale. As we see more requests for this, we'll re-evaluate