Lesson 03 Policy-Based Routing
Theme:

Solutions — Lesson 03

Worked answers for the homework tasks

Try the homework yourself first. PBR makes much more sense when you've seen it not work and figured out why.

Task 1 — Source-based routing

# 1. Namespaces
for ns in client-a client-b shared gw-a gw-b; do sudo ip netns add $ns; done

# 2. Veth pairs (4 pairs: each client↔shared, shared↔each gw)
sudo ip link add veth-ca type veth peer name veth-sa
sudo ip link add veth-cb type veth peer name veth-sb
sudo ip link add veth-sga type veth peer name veth-gua
sudo ip link add veth-sgb type veth peer name veth-gub

# Move ends into namespaces
sudo ip link set veth-ca  netns client-a
sudo ip link set veth-cb  netns client-b
sudo ip link set veth-sa  netns shared
sudo ip link set veth-sb  netns shared
sudo ip link set veth-sga netns shared
sudo ip link set veth-sgb netns shared
sudo ip link set veth-gua netns gw-a
sudo ip link set veth-gub netns gw-b

# 3. IPs and link up
sudo ip -n client-a addr add 10.10.1.5/24  dev veth-ca
sudo ip -n shared   addr add 10.10.1.1/24  dev veth-sa
sudo ip -n client-b addr add 10.10.2.5/24  dev veth-cb
sudo ip -n shared   addr add 10.10.2.1/24  dev veth-sb
sudo ip -n shared   addr add 10.99.0.2/24  dev veth-sga
sudo ip -n gw-a     addr add 10.99.0.1/24  dev veth-gua
sudo ip -n shared   addr add 10.99.1.2/24  dev veth-sgb
sudo ip -n gw-b     addr add 10.99.1.1/24  dev veth-gub
for ns in client-a client-b shared gw-a gw-b; do sudo ip -n $ns link set lo up; done
for v in veth-ca veth-sa veth-cb veth-sb veth-sga veth-gua veth-sgb veth-gub; do
  for ns in client-a client-b shared gw-a gw-b; do sudo ip -n $ns link set $v up 2>/dev/null; done
done

# 4. Forwarding on shared
sudo ip netns exec shared sysctl -w net.ipv4.ip_forward=1

# 5. Default routes on clients (via shared)
sudo ip -n client-a route add default via 10.10.1.1
sudo ip -n client-b route add default via 10.10.2.1

# 6. THE POLICY: source-based rules on shared
sudo ip -n shared rule add pref 100 from 10.10.1.0/24 lookup 100
sudo ip -n shared rule add pref 200 from 10.10.2.0/24 lookup 200
sudo ip -n shared route add default via 10.99.0.1 dev veth-sga table 100
sudo ip -n shared route add default via 10.99.1.1 dev veth-sgb table 200

# 7. Verify
sudo ip netns exec shared ip route get 8.8.8.8 from 10.10.1.5  # → via 10.99.0.1
sudo ip netns exec shared ip route get 8.8.8.8 from 10.10.2.5  # → via 10.99.1.1

Task 2 — fwmark-based routing

sudo iptables -t mangle -A OUTPUT -p tcp --dport 22 -j MARK --set-mark 0x42
sudo ip rule add pref 90 fwmark 0x42 lookup 300
sudo ip route add default via 10.99.1.1 dev eth0 table 300

ip rule show           # pref 90 fwmark 0x42 lookup 300 present
ip route show table 300
ip route get 8.8.8.8 mark 0x42   # → via 10.99.1.1

Task 3 — CONNMARK persistence

# Order matters: restore BEFORE the conditional --set-mark.
# Otherwise the first packet works (NEW state matches), but you've
# already overwritten the mark before save can capture it.
sudo iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
sudo iptables -t mangle -A PREROUTING -m conntrack --ctstate NEW -j MARK --set-mark 0x42
sudo iptables -t mangle -A PREROUTING -j CONNMARK --save-mark

# Long-lived flow (run in two terminals or two namespaces):
#   server: nc -lk 9999
#   client: nc 10.x.x.x 9999

# Inspect:
sudo conntrack -L | grep mark
# Expected output line includes "mark=66"  (0x42 = 66 decimal)

Why the ordering matters

Common mistake: putting --save-mark before --set-mark saves a zero mark to conntrack, then every "restore" pulls back zero. Symptom: only the first packet of every new flow takes the policy path.

Common mistakes across all three tasks