⏺ pf route-to silently drops DF packets exceeding target interface MTU without generating ICMP Fragmentation Needed

[brendanbank]

Important notices

• I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
• I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Note: This is closely related to #4094 ("ICMP fragmentation required not returned to NAT origin"), filed in 2020 and closed without resolution. That issue described the same root cause (ICMP Fragmentation Needed not reaching LAN clients when traffic exits via WireGuard) but focused on NAT interaction. This report provides additional evidence that the issue is specifically in pf's route-to code path, includes reproducible tcpdump captures, and documents the MSS clamping workaround. Also related: #9225 (PMTUD broken with PPPoE, same pf mechanism), #9398 (MSS clamping not auto-derived from interface MTU).

Describe the bug

When using pf route-to to policy-route traffic from a VLAN interface through a WireGuard tunnel interface (MTU 1420), packets that exceed the tunnel MTU with the DF (Don't Fragment) bit set are silently dropped. OPNsense/pf does not generate ICMP Type 3 Code 4 (Destination Unreachable, Fragmentation Needed) back to the source host. This breaks Path MTU Discovery (RFC 1191) for all source-routed traffic through tunnel interfaces.

In normal IP forwarding (without route-to), the kernel correctly generates ICMP Fragmentation Needed when a DF packet exceeds the outgoing interface MTU. The bug is specific to the route-to code path.

To Reproduce

Setup:

• OPNsense firewall with a WireGuard interface (e.g. wg0, MTU 1420)
• A LAN/VLAN interface (e.g. vlan0.100, MTU 1500)
• A pf pass rule with route-to directing traffic from the VLAN through the WireGuard interface:

  pass in quick on vlan0.100 route-to (wg0 10.0.0.1) inet from any to ! <LOCAL_NET> flags S/SA keep state
  

• NAT rule on the WireGuard interface
• A remote WireGuard peer that provides internet access (e.g. WAN breakout)

Steps to reproduce:

1. Connect a client to the VLAN (client gets MTU 1500 from the network)
2. Client establishes TCP connections to internet hosts
3. Client SYN advertises MSS 1460 (standard for MTU 1500)
4. Remote servers send TCP data packets of 1500 bytes with DF bit set
5. These packets arrive at OPNsense on the WireGuard interface (inbound return traffic) — this direction works fine via normal forwarding
6. For the outbound direction: client sends TCP data packets > 1420 bytes with DF set on the VLAN interface
7. pf matches the route-to rule and attempts to forward via the WireGuard interface (MTU 1420)
8. Bug: The packet is silently dropped. No ICMP Fragmentation Needed is sent back to the client.
9. Client retransmits the same oversized packet repeatedly (observed via tcpdump as identical seq retransmissions every 300-500ms)

Expected behavior

When a packet with DF set exceeds the MTU of the route-to target interface, OPNsense should generate an ICMP Type 3 Code 4 (Destination Unreachable, Fragmentation Needed, Next-Hop MTU = 1420) message back to the source host, just as it does for normal IP forwarding without route-to.

This is required by RFC 1191 (Path MTU Discovery) and RFC 791 (IP specification, DF bit handling).

Describe alternatives you considered

Workaround: TCP MSS Clamping

Adding a normalization rule on the WireGuard interface (Firewall → Settings → Normalization) with max-mss 1368 clamps the TCP MSS in SYN/SYN-ACK packets. This prevents TCP sessions from ever sending packets larger than the tunnel MTU, avoiding the need for PMTUD.

MSS 1368 = 1420 (tunnel MTU) − 20 (IP header) − 20 (TCP header) − 12 (TCP timestamps option)

This workaround is effective for TCP but does not fix the underlying issue for:

• UDP traffic that exceeds the tunnel MTU with DF set
• Any protocol relying on PMTUD through a route-to path
• Users who are unaware of this interaction and expect standard PMTUD behavior

Screenshots

See Mermaid diagram below showing the traffic flow and where the bug occurs.

Relevant log files

Evidence captured via tcpdump on the OPNsense firewall:

1. Client sends oversized DF packets on the VLAN interface (inbound to pf):

tcpdump -i vlan0.100 "src 10.1.80.50 and ip[2:2] > 1420 and ip[6] & 0x40 != 0"

10.1.80.50.60238 > 198.51.100.22.443: Flags [.], flags [DF], length 1388  (IP total ~1440)
10.1.80.50.60238 > 198.51.100.22.443: Flags [.], flags [DF], seq 0:1388   (retransmit)
10.1.80.50.60238 > 198.51.100.22.443: Flags [.], flags [DF], seq 0:1388   (retransmit)

2. These packets never appear on the WireGuard interface:

tcpdump -i wg0 "host 198.51.100.22"
# 0 packets captured (10 seconds, while client actively retransmitting)

3. No ICMP Fragmentation Needed sent to the client:

tcpdump -i vlan0.100 "icmp[icmptype] == 3 and icmp[icmpcode] == 4 and dst 10.1.80.50"
# 0 packets captured (12 seconds, 29781 packets observed on interface)

4. Contrast: the remote WireGuard peer (Linux) correctly generates ICMP for inbound traffic hitting the same MTU constraint:

tcpdump -i eth0 "icmp[icmptype] == 3 and icmp[icmpcode] == 4"

192.168.1.120 > 203.0.113.89: ICMP 192.168.1.120 unreachable - need to frag (mtu 1420)
192.168.1.120 > 203.0.113.50: ICMP 192.168.1.120 unreachable - need to frag (mtu 1420)
# Hundreds of these generated — Linux kernel correctly handles PMTUD

This confirms the issue is specific to OPNsense/pf's route-to code path, not a general WireGuard or MTU issue.

Additional context

flowchart LR
    subgraph LAN["LAN / VLAN (MTU 1500)"]
        Client["Client<br/>10.1.80.50<br/>MSS 1460"]
    end

    subgraph FW["OPNsense Firewall"]
        PF["pf input<br/>(vlan0.100)"]
        RT{{"route\-to<br/>(wg0 10.0.0.1)"}}
        WG["WireGuard wg0<br/>MTU 1420"]
    end

    DROP((("⛔ DROP")))

    subgraph Remote["Remote WireGuard Peer"]
        PEER["WireGuard<br/>10.0.0.1"]
        NAT["NAT/Masquerade"]
    end

    Internet["Internet"]

    Client -- "1440\-byte packet<br/>DF bit set" --> PF --> RT

    RT -- "1440 > MTU 1420, DF set" --> DROP
    DROP -. "Expected: ICMP Frag Needed<br/>(Type 3 Code 4, MTU=1420)<br/>❌ NEVER GENERATED" .-> Client

    RT -. "packets ≤ 1420<br/>(these work)" .-> WG
    WG -- "WireGuard tunnel" --> PEER --> NAT --> Internet

    style DROP fill:#dc3545,stroke:#a71d2a,color:#fff
    style RT fill:#fd7e14,stroke:#e8590c,color:#fff

Loading

The diagram shows the outbound path from client through pf policy routing. At the route-to decision point (orange hexagon), oversized packets (1440 > MTU 1420) with DF set are silently dropped (red) — pf never generates the ICMP Fragmentation Needed that RFC 1191 requires. Small packets (≤ 1420) pass through normally (dotted path).

For comparison, normal IP forwarding (without route-to) through the same WireGuard interface correctly triggers ICMP generation by the FreeBSD kernel.

This issue was debugged with the help of Claude Code (Anthropic's AI coding assistant), which performed the systematic tcpdump analysis across both ends of the tunnel to isolate the route-to code path as the root cause. If further debugging or packet captures are needed to help resolve this, I'm happy to assist.

Environment

OPNsense 26.1.1 (amd64)
FreeBSD 14.3-RELEASE-p8
Protectli VP2420 (Intel Celeron J6412 @ 2.00GHz)
Network: Intel I225-V (igc driver)
WireGuard via os-wireguard plugin

[brendanbank]

Possible Root cause analysis: NAT order-of-operations in pf_route()

I traced through the FreeBSD 14.2 kernel source (sys/netpfil/pf/pf.c) and found that the ICMP generation code does exist in pf_route() — the problem is that it fires after outbound NAT has already been applied to the packet, and the NAT-undo mechanism fails to reverse it. The ICMP Fragmentation Needed is generated, but sent to the firewall itself instead of the LAN client.

The sequence in pf_route() (pf.c lines 7521-7733)

Step 1 — Outbound rules applied first (line 7632-7643):

For inbound route-to packets (pd->dir == PF_IN), pf_route() runs the outbound firewall rules on the target interface before checking MTU:

if (pd->dir == PF_IN) {
    if (pf_test(PF_OUT, 0, ifp, &m0, inp, &pd->act) != PF_PASS)
        goto bad;            // silent drop if outbound rules reject — no ICMP
    ...
    ip = mtod(m0, struct ip *);   // re-read IP header — now NAT'd
}

This pf_test(PF_OUT) call matches the outbound NAT/Masquerade rule on wg0, changing the packet source from 10.1.80.50 (client) to the firewall's WireGuard address (e.g. 10.0.0.2). The packet headers in the mbuf are modified in-place.

Step 2 — MTU check on the NAT'd packet (line 7666-7697):

if (ip_len <= ifp->if_mtu) {   // 1440 <= 1420? NO — fall through
    ...
}

if ((ip_off & IP_DF) || ...) {  // DF set? YES
    error = EMSGSIZE;
    if (r_rt != PF_DUPTO) {
        if (s && pd->nat_rule != NULL)       // ← THIS CONDITION FAILS
            PACKET_UNDO_NAT(m0, pd, ...);    // ← NEVER EXECUTES

        icmp_error(m0, ICMP_UNREACH, ICMP_UNREACH_NEEDFRAG, 0,
            ifp->if_mtu);                     // ← ICMP sent to wrong address
        goto done;
    }
}

Why PACKET_UNDO_NAT never fires

The condition at line 7687 is s && pd->nat_rule != NULL:

• s is non-NULL — state exists from the inbound match on vlan0.100
• pd->nat_rule is NULL — pd (packet descriptor) was populated during the inbound processing on vlan0.100, where there is no NAT rule. The NAT was applied during the pf_test(PF_OUT) call at line 7633, but this does not update pd->nat_rule — it only reflects the inbound context.

So PACKET_UNDO_NAT is skipped. The packet still has source = firewall's WG address.

Where the ICMP actually goes

icmp_error() (ip_icmp.c:212) calls icmp_reflect() (ip_icmp.c:778), which at line 800:

ip->ip_dst = ip->ip_src;   // ICMP destination = source of offending packet

Since the source is the firewall's own NAT'd address (10.0.0.2), icmp_reflect routes the ICMP to the firewall itself via the local stack. It never reaches vlan0.100 or the client. The local TCP stack discards it because no local socket matches the inner packet headers.

This explains the tcpdump evidence: no ICMP on vlan0.100 (it was never sent there), no packets on wg0 (the oversized packet was dropped before transmission).

Contrast with normal kernel forwarding

In ip_fastfwd.c (normal forwarding without route-to), the MTU check at line 474-491 happens in the kernel's IP forwarding path. When route-to is active, the packet is diverted into pf's internal pf_route() function during the input pfil hook, bypassing the kernel's normal forwarding entirely. The ordering difference:

	Normal forwarding (ip_fastfwd.c)	pf_route()
MTU check	In kernel forwarding path	At line 7666, after pf_test(PF_OUT)
NAT timing	Applied by pfil output hook	Applied at line 7633, before MTU check
ICMP destination	Pre-NAT source (client)	Post-NAT source (firewall itself)

Possible fixes

1. Fix PACKET_UNDO_NAT condition: The check pd->nat_rule != NULL only reflects inbound NAT. It should also detect NAT applied during the pf_test(PF_OUT) call. The state s already has both key[PF_SK_WIRE] (NAT'd addresses) and key[PF_SK_STACK] (original addresses) — these could be used to undo NAT regardless of which processing stage applied it.

2. Reorder: MTU check before pf_test(PF_OUT): Move the MTU/DF check (lines 7662-7697) to before the pf_test(PF_OUT) call (line 7633), matching the kernel's normal forwarding order. This way ICMP is generated with the original (pre-NAT) addresses. This would require restructuring the function since ifp->if_mtu is available before pf_test(PF_OUT).

Verification

To confirm this analysis, on the OPNsense firewall run:

tcpdump -i lo0 "icmp[icmptype] == 3 and icmp[icmpcode] == 4"

If this analysis is correct, ICMP Fragmentation Needed messages should appear on lo0 (loopback), addressed to the firewall's own WireGuard interface address — confirming that the ICMP is being generated but routed to the wrong destination.

---

This analysis was performed against the FreeBSD 14.2 source tree (sys/netpfil/pf/pf.c) with Claude Code.

[brendanbank]

Updated Bug Report Analysis — OPNsense #9775

Root Cause Confirmed: NAT Order-of-Operations in pf_route()

The original code analysis from the FreeBSD 14.2 kernel source is correct. When outbound NAT is applied on the route-to target interface, pf_route() runs pf_test(PF_OUT) (which applies NAT) before the MTU/DF check. The ICMP Fragmentation Needed message is sent to the post-NAT source address — typically the firewall itself — instead of the LAN client.

Verification (Feb 11, 2026)

With NAT on wg0 disabled (current config after workaround):

• Sent DF-flagged 1421-byte ICMP packets via scapy to 8.8.8.8
• ips_cantfrag counter incremented exactly +3 for 3 packets
• tcpdump on the VLAN interface captured: <gateway> > <client>: ICMP 8.8.8.8 unreachable - need to frag (mtu 1420)
• tcpdump on client confirmed ICMP Frag Needed arrived
• PMTUD works correctly without NAT

With NAT on wg0 enabled (original config, verified by code analysis):

• pf_test(PF_OUT) at pf.c:7633 applies NAT, changing source from <client LAN IP> → <firewall's WG address>
• PACKET_UNDO_NAT at pf.c:7687 requires pd->nat_rule != NULL — but pd->nat_rule reflects the inbound context, not the outbound NAT → condition fails
• icmp_error() sends ICMP Frag Needed to the firewall's own WG address
• ICMP routed to local stack via loopback, discarded — PMTUD black hole

Additional Finding: ISP Fragment Dropping

When DF is NOT set, pf_route() correctly fragments packets via ip_fragment(). Fragments traverse the WireGuard tunnel and are forwarded by the remote peer with MASQUERADE NAT. However, the ISP at the remote peer drops all IP-fragmented packets:

• ping -M do -s 1472 (1500 bytes, no fragmentation): works
• ping -M want -s 1473 (1501 bytes, fragmented): 100% loss

This is an ISP issue, not a pf bug, but it means fragmentation is not a viable fallback when PMTUD fails.

Workaround Applied

1. Disabled NAT on the WireGuard interface — removes the NAT that causes ICMP misdirection
2. Extended remote peer's AllowedIPs — added source-routed LAN subnets so WireGuard accepts/routes traffic with client source IPs directly
3. Retained MSS clamping — max-mss 1368 on both ends as defense-in-depth

Testing Notes

• macOS ping -D does NOT set the IP DF flag (sets SO_DEBUG instead). Use scapy: IP(dst='target', flags='DF')/ICMP()/Raw(b'\x00'*1393) with sudo send(pkt)
• macOS raw sockets with IPPROTO_RAW silently discard packets. Must use IPPROTO_ICMP with IP_HDRINCL, or use scapy.