Static routes from frontend to "isolated" workload networks about vmware-haproxy

[brakthehack]

Hey Bill, thanks for filing the issue.

It's essentially an air gapped management network

This by itself shouldn't be an issue.

two additional "isolated" Workload networks.

Can you elaborate a bit what you mean by isolated here. Is it that these networks are not routable to each other, but they are routable to the primary WL network? We only support a configuration in which workload networks are L3 routable to each other. Apologies if that is the case here and I'm simply misinterpreting your comment.

Do you mind sharing which routes you added as workarounds and why this was sufficient for your needs?

Could you also clarify a bit more what you are asking? Is it support for such a network topology in general or automation around the route configuration?

[souhradab]

The workload networks are routable to each other.

Perhaps this will help:

[souhradab]

Its more or less the section in the VMware documentation on Tanzu with vSphere Networking Topologies document called "Topology with Multiple Isolated Workload Networks":

https://docs.vmware.com/en/VMware-vSphere/7.0/vmware-vsphere-with-tanzu/GUID-C86B9028-2701-40FE-BA05-519486E010F4.html

And the HA Proxy 3 NIC configuration

[brakthehack]

Thank you for the excellent diagram. It is very helpful in understanding what you're accomplishing.

We have two options.

1. Make the workload network the default gateway instead of the management network. This means that anything that is routable via the management network will no longer be accessible. At present this is only expected to be vCenter, but if that changes in the future there could be an issue. At present though it's not an issue.
2. Add logic to issue a NIC per workload network. This is useful because it's clear how workload networks would be routed and would not change existing behavior. However, it comes with downside as the user must enter all of these into the configuration screen. If a user has a large number of workload networks, they could make mistakes. Terminology would also likely have to change which could cause some confusion for users.

@souhradab I want to get your opinion whether you prefer one option or another. Thanks!

[brakthehack]

@daniel-corbett we would welcome your advice if you're interested as well.

[souhradab]

Regarding (1), without binding your options maybe just giving the option for custom (static) routes will help solve this for people with special setups like mine. You could even till give the option of setting any interface as default GW.

Regarding (2), NIC per workload network will likely run into VM virtual NIC limits in larger environments.

[brakthehack]

Regarding (1), without binding your options maybe just giving the option for custom (static) routes will help solve this for people with special setups like mine. You could even till give the option of setting any interface as default GW.

This is true, but we also want to keep things simple. By default, workload networks must be routable to each other, which means defaulting to the workload gateway would be good enough for the majority of users. Allowing users to add specific routes by default may also encourage complex configurations which may increase the cost to debug for both users and VMware/HAProxy.

For more complex configurations, users are free to make edits to route configurations as they wish once the appliance is stood up. In this setup, they are the owners of the appliance. It's unclear if adding passthrough code for routes as a tradeoff for an even more complex configuration is a net benefit over a default gateway.

I agree with your point on NIC limits. If we expect a large number of networks for some users, this is probably not a viable option if we want to trouble that user segment.

[brakthehack]

After more thought, making workload the default gateway may make it harder for some users to manage the appliance over something like SSH. Routes, as you suggested, might be the way to go here.

[souhradab]

maybe some sort of menu item: default GW on management or workload network? And then, based on the choice: static routes needed, if necessary, on the interface that is NOT the default GW assigned interface.