Skip to main content
Version: v25.1.5

Network Control Node Setup

Provisioning the control node and installing SONiC.

This guide walks you through provisioning the control node that manages the network fabric, then installing SONiC on the switches.

What You're Doing​

You'll deploy a VM on the bastion node that acts as the control plane for the HedgeHog network fabric. This control node will then manage the installation and configuration of SONiC on your network switches. The fabric configuration and wiring are pre-baked into the installer ISO.

Before You Start​

Prerequisites Checklist​

  • Router0 Access - SSH access to router0-host, the physical router machine → Details
  • Control Node ISO - Download URL for HedgeHog Control node ISO → Details
  • Switch Access - Serial console access to network switches → Details

Router0 Access​

  • What it is: SSH access to router0-host, the physical machine that hosts the router VM and the USB-to-serial adapters
  • Purpose: Access the USB-to-serial adapters connected to the network switches
  • What you need:
    • SSH key and credentials for router0-host

Control Node ISO​

  • What it is: Pre-built installer ISO for the network control node
  • Purpose: Contains the HedgeHog control plane with pre-configured fabric settings and wiring
  • What you need: Download (Azure Blob Storage) URL previously provided

Switch Access​

  • What it is: Serial console access to SONiC-compatible network switches
  • Purpose: Boot switches into ONIE for SONiC installation
  • How to access: 
    sudo minicom -D /dev/ttyUSB1 -b 115200

If a person is already connected, another person can connect with:

sudo socat -d -d /dev/ttyUSB1,raw,echo=0 stdout
  • What you need: Physical serial connection to switches

Installation Steps​

Step 1: Download Control Node ISO​

On the bastion node, download the pre-built ISO:

# Use the SAS URL from your materials PDF
wget -O hedgehog-installer.iso "https://your-storage-account.blob.core.windows.net/path/to/hedgehog-installer.iso?sp=r&st=..."

Verify the download:

ls -lh hedgehog-installer.iso

Step 2: Create Control VM on Bastion​

Create and start the control node VM using the ISO. The control node requires:

  • vCPU: 8-12 cores
  • RAM: 16 GiB
  • Disk: 35 GB

Set up variables for your environment:

# Adjust these paths and names for your environment
IMG_DIR=/var/lib/libvirt/images
VM_NAME=hedgehog-control
UBNT_REL_YEAR="24.04"
VM_VCPU=8
VM_RAM=16384
ROOTFS_SIZE=35G

# Adjust these for your network setup
VM_MAC=52:54:00:a1:ac:ff  # Must be unique
VM_BRIDGE=br-mgmt          # Your management network bridge

Prepare the blank OS image:

sudo qemu-img create -f qcow2 ${IMG_DIR}/${VM_NAME}.qcow2 ${ROOTFS_SIZE}
sudo chown -v libvirt-qemu:kvm ${IMG_DIR}/${VM_NAME}.qcow2
sudo chmod -v 664 ${IMG_DIR}/${VM_NAME}.qcow2

Set permissions on the installer ISO:

# Assuming hedgehog-installer.iso is in the same directory
sudo chown -v libvirt-qemu:kvm ${IMG_DIR}/hedgehog-installer.iso
sudo chmod -v 664 ${IMG_DIR}/hedgehog-installer.iso

Create and start the VM:

virt-install --name=${VM_NAME} \
  --os-variant=ubuntu${UBNT_REL_YEAR} \
  --machine q35 \
  --boot firmware=efi,firmware.feature0.name=secure-boot,firmware.feature0.enabled=no \
  --tpm none \
  --vcpu=${VM_VCPU} \
  --ram=${VM_RAM} \
  --disk path=${IMG_DIR}/${VM_NAME}.qcow2,bus=virtio \
  --cdrom=${IMG_DIR}/hedgehog-installer.iso \
  --network bridge=${VM_BRIDGE},mac=${VM_MAC} \
  --noautoconsole
note

The exact VM creation method depends on your bastion's virtualization setup (KVM/libvirt, VMware, etc.). If you need PCI passthrough for additional network devices, add --host-device=pci_XXXX_XX_XX_X to the command.

The installation process is automated. Monitor progress via console:

virsh console ${VM_NAME}

The VM will shut itself down when installation is complete. Once it's done, start it:

virsh start ${VM_NAME}

Step 3: Verify Control Node​

Check that the control node VM is running and accessible:

# Check VM status
sudo virsh list --all

# Console into the VM to get IP addresses
virsh console ${VM_NAME}

Once logged in via console, list all IP addresses:

# List all network interfaces and their IPs
ip -br addr show

# Or for more detail
ip addr show

Note the IP address(es) and use them to SSH from your workstation if needed:

ssh core@<hedgehog_control_node_ip>

Verify HedgeHog services are running:

kubectl get pods -A

All pods should be in Running state.

Step 3.1: Apply the fabric configuration to HedgeHog Control Plane​

On the HedgeHog control node VM, download the pre-configured network fabric configuration:

# Use the SAS URL from your materials PDF
wget -O hedgehog-fabric-configuration.yaml "https://your-storage-account.blob.core.windows.net/path/to/hedgehog-fabric-configuration.yaml?sp=r&st=..."

Apply the configuration:

kubectl apply -f hedgehog-fabric-configuration.yaml

Step 4: Prepare Switches for SONiC Installation​

For each switch, you'll set up ONIE to pull the SONiC installer from a temporary HTTP server.

4.1: Setup HTTP Server for ONIE​

On a machine reachable from the switch management network (can be the bastion):

DHCP subnet reachability

ONIE receives its IP address from DHCP on boot and uses it to reach the firmware HTTP server. The machine running the HTTP server must be in a subnet that is routable from the DHCP-assigned ONIE IP. If the DHCP pool and the firmware server are on different subnets with no route between them, onie-self-update will fail silently.

Verify that the DHCP range configured for the switch management interface assigns addresses that can reach the bastion (or whichever host runs the HTTP server). See ROUTER_BOX_CONFIGURATION — DHCP Server for pool configuration.

Create the directory for ONIE files:

mkdir -p /home/ubuntu/boot-http-server

Start nginx container to serve ONIE files:

docker run -d --name onie-server \
  -p 8080:80 \
  -v /home/ubuntu/boot-http-server:/usr/share/nginx/html \
  nginx

4.2: Download ONIE Firmware​

Download the ONIE image appropriate for your switch model directly to the HTTP server directory:

# Download URL will be provided in your materials PDF
# Or obtain from your switch vendor's support site
cd /home/ubuntu/boot-http-server
wget -O onie-updater.bin "https://vendor-url/onie-updater.bin"
note

The exact ONIE firmware filename and version will vary based on your switch hardware. Check your materials PDF for the correct download URL.

4.3: Access Switch Console​

From router0-host (where the USB-to-serial adapters are connected), connect to each switch via serial console:

sudo minicom -D /dev/ttyUSB1 -b 115200
# (adjust device path for each switch)

4.4: Update ONIE Firmware (only if not in the latest version)​

From the ONIE prompt, update the ONIE firmware using the HTTP server on the OpenWRT router:

# In ONIE prompt
onie-stop
onie-self-update http://172.20.0.1/boot/firmware/dell-onie/onie-update-full-x86_64-dellemc_s5200_c3538-r0.3.40.5.1-26.bin

The firmware file is served by uhttpd on the OpenWRT router machine (172.20.0.1).

4.5: Boot into ONIE Mode​

After ONIE firmware is updated, reboot the switch to boot into ONIE:

From the switch console:

  1. Reboot the switch: 
    sudo reboot
  2. Wait till GRUB selection appears
  3. Quickly select "ONIE"
  4. Select "ONIE: Install OS"

Step 5: Install SONiC via HedgeHog​

Once switches are in ONIE mode, HedgeHog will automatically detect them on the management network and begin SONiC installation via DHCP.

warning

During the SONiC first-boot sequence the switch console may display a prompt asking you to change the default password. Do not change the password. HedgeHog uses the default credentials to log in and apply the fabric configuration — if the password is changed at this point, HedgeHog will be unable to connect and the switch will not join the fabric.

Monitor agent heartbeats and configuration from the hedgehog control node:

# SSH to control node
ssh core@<hedgehog_control_node_ip>

# Watch for agent heartbeats and configuration generation
kubectl get agent -o wide -w

Wait for agents to send heartbeats and show current/applied configuration generation.

Step 6: Verification​

List switches and SSH to them:

# List switches to get switch names
kubectl get switches

# SSH to a switch using its name
kubectl fabric switch ssh --name <switch-name>

Use the default HedgeHog admin password when prompted.

Once switches are confirmed reachable, recover the router's LACP bond. The bond (bond0) is configured during bootstrap before the switch PortChannels exist; it always enters a churned state and does not self-recover even after the PortChannels come up.

Run the post-switch setup step to detect and fix the churned bond automatically:

./platform-setup.sh --post-switch-setup

This checks /proc/net/bonding/bond0 on router-0 for a churned state and, if found, cycles the bond members to force a clean LACP renegotiation — without restarting the router.

To inspect the bond manually before or after:

cat /proc/net/bonding/bond0   # look for "Churn State: none" when healthy

Troubleshooting: If the agent doesn't send heartbeats, check the agent logs via serial console:

# From switch serial console
cat /var/log/agent.log

Troubleshooting​

Control VM Won't Boot​

  • Verify ISO downloaded completely: sha256sum hedgehog-installer.iso
  • Check VM resources are sufficient (8-12 vCPU, 16 GiB RAM, 35 GB disk)
  • Review VM console logs: sudo virsh console ${VM_NAME}

Switch Not Getting ONIE Firmware​

  • Verify HTTP server is accessible from the switch: curl http://172.20.0.1/boot/firmware/dell-onie/onie-update-full-x86_64-dellemc_s5200_c3538-r0.3.40.5.1-26.bin
  • Check switch can reach the OpenWRT router (172.20.0.1) from the management network
  • Verify the ONIE firmware file exists on the OpenWRT router under /boot/firmware/dell-onie/

ONIE Cannot Reach Firmware Server (DHCP subnet mismatch)​

Symptom: onie-self-update times out or reports a connection error even though the HTTP server is running.

Root cause: ONIE receives its IP via DHCP on boot. If the DHCP pool assigns an address in a subnet with no route to the machine hosting the firmware files, the update cannot proceed. This is an environment-specific DHCP configuration issue; it is not enforced or detected by the product.

Workaround (immediate): Add a secondary IP on the firmware server host (e.g., the bastion) within the ONIE DHCP range so it becomes reachable:

sudo ip addr add 172.30.0.250/16 dev enp1s0

Remove the secondary address after the firmware update completes:

sudo ip addr del 172.30.0.250/16 dev enp1s0

Permanent fix: Adjust the DHCP pool for the switch management interface so that IPs it assigns have a route to the firmware server, or configure the firmware server to listen on an address within the ONIE DHCP range. See ROUTER_BOX_CONFIGURATION — DHCP Server.

Switches Not Appearing in Fabric​

  • Verify control node has connectivity to management network
  • Check DHCP is working on management network
  • Review the HedgeHog installation log:
    • tail -f /var/log/install.log
  • Review fabric controller logs:
    • kubectl logs -n fab -l app.kubernetes.io/name=fabricator
    • kubectl logs -n fab -l app.kubernetes.io/name=fabric
  • Ensure switches completed ONIE update and rebooted

PXE Not Working After Post-Switch Setup​

If PXE still fails after running --post-switch-setup, inspect the bond state manually from router-0:

cat /proc/net/bonding/bond0

Look for Partner Churn State and Actor Churn State — both should be none when healthy. If either shows churned, the bond members are still dead and the automatic reset did not take effect. In that case, restart the router as a last resort:

sudo reboot

After the router comes back up, PXE should recover automatically.

Can't SSH to Control Node​

  • Console into VM and check IP addresses: virsh console ${VM_NAME} then ip -br addr show
  • Verify VM got IP via DHCP on management network
  • Check network bridge configuration on bastion
  • Verify VM network is connected to management network

References​