Skip to main content

Additional External Network Configuration

This guide explains how to add additional external networks to Kube-DC alongside the default cloud network.

Overview​

The configuration demonstrates how to add a second external network (public) to an existing Kube-DC setup that already has a cloud external network, using multiple VLANs on a single physical interface per node.

Network Types Explained by Example​

Cloud Network (egressNetworkType: cloud)​

  • Purpose: Default external network for most workloads
  • Subnet: ext-cloud (100.65.0.0/16) on VLAN 4013
  • Use Cases:
    • General internet access for applications
    • Standard egress traffic from project workloads
    • Cost-effective external connectivity
  • IP Pool: Large address space (65,000+ IPs available)

Public Network (egressNetworkType: public)​

  • Purpose: Premium external network for specialized workloads

  • Subnet: ext-public (168.119.17.48/28) on VLAN 4011

  • Use Cases:

    • Production services requiring dedicated public IPs
    • Load balancers and ingress controllers
    • Services needing specific public IP ranges or routing

  • IP Pool: Limited address space with real ipv4 addresses (16 IPs total)

Architecture​

Physical Interface (enp0s31f6)
├── VLAN 4013 (Cloud Network) - 100.65.0.0/16 (ext-cloud)
└── VLAN 4011 (Public Network) - 168.119.17.48/28 (ext-public)

Example Cluster Usage​

  • shalb-demo project: Uses egressNetworkType: cloud → EIP: 100.65.0.102 (development/testing)
  • shalb-dev project: Uses egressNetworkType: public → EIP: 168.119.17.51 (development with public access)
  • shalb-envoy project: Uses egressNetworkType: public → EIPs: 168.119.17.52, 168.119.17.54 (production load balancer)

Choosing the Right Network​

Use Cloud Network when:

  • Need basic internet connectivity
  • Don't require specific public IP ranges

Use Public Network when:

  • Need dedicated public IP addresses
  • Have specific routing or compliance requirements
  • Running load balancers or ingress controllers

OVS/OVN Modifications Applied​

1. OVS Bridge Configuration​

The system automatically creates the necessary OVS infrastructure:

Bridge: br-ext-cloud

  • Physical interface enp0s31f6 attached with VLAN trunking
  • Trunk VLANs: [0, 4011, 4013]
  • Patch ports for both external networks:
    • patch-localnet.ext-cloud-to-br-int ↔ patch-br-int-to-localnet.ext-cloud
    • patch-localnet.ext-public-to-br-int ↔ patch-br-int-to-localnet.ext-public

2. OVN Logical Switches​

Two logical switches are created automatically:

  • ext-cloud (for VLAN 4013)
  • ext-public (for VLAN 4011)

3. ProviderNetwork Status​

The existing ProviderNetwork ext-cloud is updated to include both VLANs:

status:
  vlans: ["vlan4013", "vlan4011"]
  ready: true
  readyNodes:
  - kube-dc-master-1
  - kube-dc-worker-1

Configuration Steps​

1. Apply VLAN Configuration​

kubectl apply -f examples/networking/additional-external-network.yaml

2. Verify Configuration​

# Check ProviderNetwork VLANs
kubectl get provider-network ext-cloud -o jsonpath='{.status.vlans}'
# Expected output: ["vlan4013","vlan4011"]

# Check external subnets
kubectl get subnets ext-cloud ext-public
# Expected: ext-cloud (100.65.0.0/16) and ext-public (168.119.17.48/28)

# Check EIP assignments
kubectl get eips -A
# Shows which projects are using which external networks

# Check OVS bridge configuration
kubectl exec -n kube-system [ovs-pod] -- ovs-vsctl show | grep -A 10 "br-ext-cloud"

# Check OVN logical switches
kubectl exec -n kube-system [ovn-central-pod] -- ovn-nbctl ls-list | grep ext

3. Test with Project​

Create projects to test both network types:

Project using Cloud Network:

apiVersion: kube-dc.com/v1
kind: Project
metadata:
  name: test-project-cloud
  namespace: test-org
spec:
  cidrBlock: 10.200.0.0/24
  egressNetworkType: cloud  # Uses ext-cloud subnet (100.65.0.0/16)

Project using Public Network:

apiVersion: kube-dc.com/v1
kind: Project
metadata:
  name: test-project-public
  namespace: test-org
spec:
  cidrBlock: 10.201.0.0/24
  egressNetworkType: public  # Uses ext-public subnet (168.119.17.48/28)

Key Points​

  1. Single ProviderNetwork: Use one ProviderNetwork per physical interface with multiple VLANs attached
  2. Automatic Configuration: OVS bridges, patch ports, and OVN logical switches are created automatically
  3. VLAN Trunking: The physical interface supports multiple VLANs simultaneously
  4. No Manual OVS Changes: All OVS/OVN modifications are handled by Kube-DC controllers

Prerequisites​

  • Physical network infrastructure supporting VLAN trunking
  • vSwitch configured with appropriate VLAN IDs

Troubleshooting​

Check VLAN Interface on Nodes​

# On cluster nodes
ip link show enp0s31f6.4011
ip addr show enp0s31f6.4011

Check OVN Resources​

# Check OVN-EIP resources
kubectl get ovn-eip | grep ext-public

# Check subnet status
kubectl get subnet ext-public -o yaml

Test Connectivity​

# Test from pod
kubectl exec -n [namespace] [pod] -- wget -qO- http://httpbin.org/ip