After an installation, an automatic one especially, a host needs proper network settings before being provisioned like any other hosts via Ansible common rule.
The goal is to ensure:
- defining the host’s hostname (most needed after an automatic installation using a generic name)
- setup of the network, taking advantage of the switches redundancy if possible (bonding using active-backup ; balance-alb was tested but did not work), with VLAN and bridging support
Blades are in a specific network environment with the two microblade internal switches, so a bonding configuration is needed for redundancy.
The playbook fetches the Blade’s hardware information to get the MAC address of the first interface, search it in the DHCP server’s ARP table to get the dynamic IP, and create a temporary Ansible host to connect to. Then it computes the settings and apply them.
Limitations and Todolist
Interface order can be problematic, especially when not using Network Manager (as it renames interfaces to match the configured MAC address).
Automated Generation Method / Red Hat Systems Only
nmcli Ansible module is plagued by multiple bugs and is unusable. We thought about issuing the command directly but it is fragile. Also when this playbook was started we needed support for Fedora Atomic, and at that time (version 26) there was no Network manager support. So we decided to generate configuration files directly for the needed OSes.
This means the playbook can currently only provision OSes from the RedHat family. Using different templates it should not be difficult to add support for Debian-based OSes in the future. The manual procedure is described below.
Starting from CentOS 8 (to get a more recent version of systemd) we might reconsider using systemd-networkd. As support for Fedora Atomic 26 was dropped, using Network Manager might also be an option.
Host IP definition
A few settings are needed to guide the Ansible playbook to compute the settings.
Here is an example you would store in
host_vars/<host>/net.yml (adding to the list of interfaces):
--- net: ifaces: 0: untagged_vlan: 'OSCI-Public' tagged_vlans: ['OSCI-Management', 'OSCI-Internal'] 1: untagged_vlan: 'OSCI-Public' tagged_vlans: ['OSCI-Management', 'OSCI-Internal'] 2: untagged_vlan: ['OSCI-example'] ips: 'OSCI-Internal': 172.24.32.6 'OSCI-Management': 172.24.31.6 'OSCI-Public': 18.104.22.168 'OSCI-example': 10.0.1.3 bonding: 0: [2, 3]
host declared in the inventory is used to define the hostname.
ips defines the host’s IP for each VLAN it is part of. There is no need to define a netmask/gateway/… as these parameters are part of the VLAN global definitions in
bonding defines the list of bonding interfaces. The key defines the interface name (
bond<key>) and the value is the interface pair.
Interfaces not part of a bonding will also be configured.
Generating Network Configuration Automatically
On Debian systems please refer to the next chapter.
When settings are ready, just fire the playbook:
ansible-playbook --diff -l <host> playbooks/network_setup.yml
If the playbook says:
Could not find MAC address: either the blade position is wrong and there is no blade inserted, or there is a bug in the playbook
Could not find IP address (not in ARP table): most probably the ARP entry expired, so you would need to trigger some network communication with Catton/Speedy (console login + DHCP client restart, reboot, ping dynamic IP from Catton/Speedy…)
IPv6 Suffix Discovery
On the host, list the interface you would like to add IPv6 to:
# ip a s eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:64:ab:5a brd ff:ff:ff:ff:ff:ff inet 22.214.171.124/24 brd 126.96.36.199 scope global noprefixroute eth0 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe64:ab5a/64 scope link noprefixroute valid_lft forever preferred_lft forever
If the interface is a bonding one, then you will need to configure it first for IPv4 before looking for the IPv6 suffix.
Look at the link-local IPv6 IP
fe80::5054:ff:fe64:ab5a/64. Remove the link-local prefix
fe80:: and the CIDR range
/64 and you get the <ipv6-suffix>
It is later combined with the network prefix and CIDR range of our infrastructure.
Network Configuration on Debian Systems
If the host has redundant links, add the following paragraph in
auto bond0 iface bond0 inet static address <ipv4>/24 gateway 188.8.131.52 bond-slaves <eth-iface-1> <eth-iface-2> bond-mode active-backup bond-arp-ip-target 184.108.40.206 bond-arp-interval 100 bond-arp-validate filter_active iface bond0 inet6 static address 2620:52:3:1:<ipv6-suffix>/64 gateway 2620:52:3:1:ffff:ffff:ffff:fffe
If the host has only one link, add the following paragraph in
auto <eth-iface> iface <eth-iface> inet static address <ipv4>/24 gateway 220.127.116.11 iface <eth-iface> inet6 static address 2620:52:3:1:<ipv6-suffix>/64 gateway 2620:52:3:1:ffff:ffff:ffff:fffe