Cisco BFD Detect Timer Expired: Investigating a Brief Standby Route Flap
A Catalyst 9300 stack lost its Standby BFD session for about one second. The log reason was DETECT TIMER EXPIRED, the session came back on its own, and the Main path carried traffic the whole time. The cause is still unknown. This post records what was checked, what each check could and could not show, and what to capture if it happens again.
A different incident on the same platform family, with the reason ECHO FAILURE, is covered in Cisco Catalyst 9300 BFD Echo Failure: Troubleshooting Session Flaps and Restoring Stability. The two are separate incidents, and nothing from that case is applied here.
Network environment
The environment is de-identified and simplified. Two sites are joined by a Main and a Standby carrier circuit. Static routes depend on BFD, so a BFD session that goes Down can pull its route out of the routing table. For configuration order and verification of this design, see Cisco Catalyst 9300 HQoS and Static BFD: Configuration Order and Verification.
| Item | Value |
|---|---|
| Platform | Cisco Catalyst 9300 family, stack |
| Redundancy | Main and Standby carrier circuits |
| Routing | Static routes tied to BFD |
| Affected path | Standby only |
| IOS XE release | Not listed in this post |
| BFD timers | Not listed in this post; Detection Time depends on the negotiated values |
| Addresses and hostnames | Omitted |
Incident overview
On October 26, 2026, at about 00:18 KST (time confirmed by the operator, not read from a device clock), the Standby BFD session went Down with the reason DETECT TIMER EXPIRED and came back Up about one second later. The Main path stayed normal and no service impact was reported. The session has stayed Up since.
| When | Event | Basis |
|---|---|---|
| About 00:18 KST | Standby BFD session Down, DETECT TIMER EXPIRED | Observed |
| About 1 second later | Session Up again | Observed |
| During and after | Main path normal, no service impact | Observed |
| Later review | DETECT TIMER EXPIRED appears in the BFD history of both devices | Observed; see the clock note below |
The two devices showed different clock values, and the time basis of the BFD history was not verified. Cisco's sample history lines carry a month, day and time with milliseconds, but no year or time zone. Two matching entries therefore cannot be placed on one timeline, and this post does not claim that they happened at the same moment.
A log line in this format is what to look for (generalized; the numbers are placeholders):
%BFDFSM-6-BFD_SESS_DOWN: BFD-SYSLOG: BFD session ld:<LD> handle:<N>,is going Down Reason: DETECT TIMER EXPIREDWhat DETECT TIMER EXPIRED means
In asynchronous mode each side sends BFD Control packets at the negotiated rate. RFC 5880 section 6.8.4 defines the Detection Time as the Detect Mult received from the remote system multiplied by the agreed transmit interval of the remote system. That interval is the greater of the local required minimum receive interval and the last received desired minimum transmit interval. Each direction is calculated independently by the receiver. If that time passes without a Control packet, a session in Init or Up goes Down and the local diagnostic is set to 1, Control Detection Time Expired. A packet that was discarded under the RFC's receive rules does not count as received.
Cisco's IOS XE BFD troubleshooting guide describes this reason as the router no longer receiving BFD keepalive traffic. It lists ECHO FAILURE, RX DOWN and RX ADMINDOWN as the other common reasons. A DETECT TIMER EXPIRED entry says that Control packets were not received, or not accepted, in time. It does not say where they were lost. For single-hop IPv4 and IPv6, RFC 5881 sets UDP destination port 3784 for Control packets and requires TTL 255, and packets that fail the TTL check are discarded.
Link, BFD session and static route are three different states
Physical link state, BFD session state and static route state are separate. A port can stay up while packets are lost beyond it. A BFD session can go Down while the port stays up. With static BFD configured, Cisco states that the gateway is unreachable unless the BFD session is Up, and the affected routes are not installed in the RIB.
For this incident, nothing in the material reviewed shows that the physical interface went down, and this post does not say it did. The Standby route was probably out of the routing table for about a second, but that was not captured. Because the Main path carried traffic, the loss of the Standby route had no visible effect.
Troubleshooting process
Each step lists what was checked, what it showed and what it could not show.
1. BFD neighbor state
The session was Up again when it was checked. The neighbor table shows the present state, so it cannot show what happened during that one second. It is still where to read the negotiated timers, the Echo state and whether the session is hosted in software or hardware.
2. BFD history on both devices
Both histories contained DETECT TIMER EXPIRED. This is the strongest evidence in the case, and also the one with the weakest timing, because of the clock issue above.
One point is worth recording as inference, not fact. If only one direction had been interrupted, the receiving side would time out first and signal Down, and the other side would normally log a Down received from the neighbor, not its own timer expiry. RFC 5880 defines diagnostic code 3, Neighbor Signaled Session Down, for that case. Two independent timer expiries fit an interruption that stopped Control packets in both directions, or a stall affecting both ends. They do not fit a one-way problem very well. This reasoning depends on the two entries belonging to the same event, which was not verified.
3. Interface counters
CRC and input error counters were 0. That lowers the likelihood of a local physical problem, such as a bad cable or optic. It cannot show loss that happens inside the carrier network, because that traffic never reaches the local interface counters.
4. BFD drop counters
No separate drop was identified in the BFD drop statistics. Cisco notes that local BFD drops, listed by reason such as invalid TTL, can make sessions flap, so a clean result weakens that explanation. It does not rule out loss before the packet reaches that counter.
5. CPU utilization
CPU utilization history was reviewed. A history graph is averaged over fixed buckets, so a stall of about a second can disappear in it. No CPU reading is treated as the cause here, and a CPU reading would not prove a BFD processing delay at that moment either.
6. QoS statistics
The HQoS statistics show cumulative drops. A cumulative counter has no timestamp, so it cannot be assigned to a one-second event. It does show that drops occur on this device at some times. Whether any of them involved BFD Control packets is not known.
7. BFD interval statistics
Some BFD session statistics reported a maximum Control Rx interval of 4,235 ms and a maximum Tx interval of 6,420 ms. Cisco's sample show bfd neighbors details output includes Rx and Tx interval min/max/avg fields next to the session uptime and packet counts. Statistics of this kind appear to accumulate over the life of the session, but their reset behavior was not verified. There is no evidence that they were recorded during this event, so they are not used to explain it.
They do not help the carrier hypothesis either. A long gap between transmitted packets is measured on the local device. If a gap like that did occur around the event, it would point more toward local scheduling than toward the circuit.
The checks above map to these read-only commands:
show bfd neighbors details
show bfd neighbors history
show bfd drops
show bfd summary session
show interfaces <interface>
show processes cpu history
show policy-map interface <interface>Findings and possible causes
| Established | Not established |
|---|---|
Standby BFD session went Down with DETECT TIMER EXPIRED and recovered after about one second | The cause |
| Main path stayed normal, no service impact | Whether the two history entries are the same moment |
| CRC and input errors were 0 | Whether any Control packet was lost, and where |
| No separate BFD drop identified | Whether CPU load, BFD process delay or CoPP played a part |
| Cumulative HQoS drops exist | Whether those drops include BFD packets |
Three hypotheses remain open.
A. A brief packet-delivery problem inside the carrier circuit
This is the operating suspicion, and it is not a verified cause. It rests on four observations. Both devices' histories contain a timer expiry, which is consistent with an interruption between them. The local error counters stayed at 0, which is consistent with loss beyond the local interface. The session recovered by itself in about a second. And the Main path stayed normal. None of this looks inside the carrier network, and no carrier confirmation exists. I also do not know whether the Main path has its own BFD session. If it does, its stability through the event is a useful data point. If it does not, it says little.
B. Delayed BFD packet handling on the device
A control-plane stall, a BFD process scheduling delay or a short queue overload could delay sending or receiving. Cisco's troubleshooting guide lists CPU or forwarding failures on one device among possible causes. Its Catalyst 9000 control-plane guide lists BFD in the low-latency CPU queue, and Cisco notes that some platforms offload BFD to hardware to prevent instability from CPU congestion. The large cumulative Tx interval maximum keeps this hypothesis open. Nothing observed confirms it.
C. QoS and packet drops
The HQoS counters show drops, and BFD packets could in principle be affected by queueing, by CoPP or by how self-generated packets are classified. Cisco's 17.18.x BFD guide lists a restriction that the QoS policy for self-generated packets does not match BFD packets. The release actually running on this stack was not confirmed against that guide. The reviewed documents do not settle whether HQoS drops can hit BFD Control packets on this platform. Without counter deltas around an event, this stays unproven. Cisco's output-drops guide explains how to read the queue statistics.
Current status and future investigation
- The BFD session recovered and has stayed Up since.
- The Main path was normal, so there was no service impact.
- The cause has not been identified.
- A brief packet-delivery problem inside the carrier is suspected and unverified.
- If the event recurs, further analysis follows the list below.
What to do if it recurs
- Synchronize time and correlate logs. Compare both devices against one NTP source and record the BFD history time basis before the next event.
- Capture BFD Control packets at both ends at once. Use a filter for UDP 3784, and 3785 if Echo is on. A packet sent at one end and missing at the other shows the direction of the loss.
- Ask the carrier to check loss and delay on the circuit for the event window.
- Check control-plane queues and policer drops before and after, and confirm that the counters increment while the event is happening.
- Compare CPU utilization and the BFD Rx/Tx intervals from the moment of the event, not from cumulative statistics.
Read-only commands for those steps:
show clock detail
show ntp status
show bfd neighbors details
show bfd neighbors history
show bfd drops
show processes cpu sorted
show platform hardware fed switch active qos queue stats internal cpu policer
show policy-map interface <interface>
show loggingRecord the session host (software or hardware), the Echo state and the negotiated timers from show bfd neighbors details. Also search the log for link-state messages in the same window to settle the link question.
Capture setup follows Cisco's embedded capture example. Creating the access list is a configuration change, so schedule it like any other change:
configure terminal
ip access-list extended <ACL-NAME>
permit udp any any eq 3784
permit udp any any eq 3785
end
monitor capture <name> interface <interface> both
monitor capture <name> access-list <ACL-NAME>
monitor capture <name> start
monitor capture <name> stopCheck the packet capture guide for your release before running it on a production stack. Capture buffer and CPU limits depend on the platform and release.
Conclusion
This was a one-second BFD interruption on a Standby path, with no service impact and no confirmed cause. Local errors and BFD drops were clean, but the two devices' clocks disagree, so the timing between the two ends is unverified. A carrier problem is the leading suspicion, and a device-side delay has not been ruled out. If it happens again, synchronized clocks, captures at both ends and counters taken during the event are what will settle it.
Official references
Documentation reviewed November 3, 2026. Check the release-specific documentation for the software version actually running.
- IETF, RFC 5880: Bidirectional Forwarding Detection (BFD), sections 4.1 and 6.8.4.
- IETF, RFC 5881: Bidirectional Forwarding Detection (BFD) for IPv4 and IPv6 (Single Hop).
- Cisco, Troubleshoot Bidirectional Forwarding Detection in Cisco IOS XE.
- Cisco, IP Routing Configuration Guide, Cisco IOS XE 17.18.x (Catalyst 9300 Switches): Configuring Bidirectional Forwarding Detection.
- Cisco, Troubleshoot Control Plane Operations on Catalyst 9000 Switches.
- Cisco, Troubleshoot Output Drops on Catalyst 9000 Switches.
Comments
Post a Comment