09 Jul 2026
Your Watchdog Margin Is Thinner Than You Think: A Boot-Time-vs-Timeout Math Check
#embedded-linux#watchdog#boot-time#u-boot#bsp#reliability
A board that resets itself during boot, over and over, in a tight loop — no crash dump, no kernel panic, just silence and then the power LED blinking again — is one of the more demoralizing bugs in embedded Linux. Community threads about STM32MP1 boards hitting exactly this pattern eventually converge on the same diagnosis: the watchdog timeout was shorter than the boot actually took. Engineers had to shrink the timeout on purpose just to reproduce it reliably, because in normal conditions the margin was just wide enough to hide the bug.
My claim: this isn't a one-off STM32MP1 quirk. It's the predictable outcome of lining up two numbers that almost nobody puts side by side — the watchdog's timeout ceiling, and the boot time it's supposed to be gating. When you actually cross-reference published boot-time benchmarks against common watchdog defaults, the "comfortable margin" engineers assume shrinks to a handful of seconds, and sometimes disappears entirely.
The boot chain has more silent gaps than people think
Before the numbers, it helps to see where the time actually goes. A typical embedded boot isn't one program starting — it's a relay race with several independent loaders, each one blind to whatever watchdog configuration the next stage assumes.
A few of these stages deserve a second look because they're where boot time quietly balloons:
- SPL/TPL exist specifically to be minimal — SPL is a stripped-down loader, and TPL is smaller still. Neither typically services a watchdog; they're too small to carry the driver.
- U-Boot's autoboot/bootdelay mechanism — the "Press SPACE to abort autoboot in 2 seconds" prompt — is meant for developers, but if it's still set for production, that's dead time nobody accounted for. Worse, the interactive shell (
md,mm,nmmemory commands) can leave the board parked at a prompt indefinitely — the textbook idle window where a running watchdog starves. - fitImage/uImage verification adds checksum and signature time before Linux even starts executing.
- Module loading (
insmod/modprobe, withdepmoddependency resolution) happens serially during boot, and its duration varies with how many drivers your BSP decided to compile as modules instead of built-ins.
None of this is exotic — it's just normal boot-chain anatomy. But every one of these stages is a place where "how long boot takes" and "when the watchdog was configured to expect it" can drift apart.
The numbers, side by side
Here's the comparison that matters — watchdog ceilings and defaults against boot times that people have actually measured and published.
| Watchdog (ceiling / default) | Value | Boot time measured | Value | Margin |
|---|---|---|---|---|
| Raspberry Pi hardware WDT | ~16 s hard ceiling (20-bit counter, 16 µs ticks) | — | — | fixed by silicon, not configurable |
Generic Linux watchdog kernel module | 30 s or 60 s (common driver defaults) | i.MX93 board, before optimization | 13.2 s | looks generous until modules/UBI/network get added |
| Same board, SD card vs eMMC | n/a | 17 s (SD) vs 15 s (eMMC) | 15% swing | swing alone can eat a soft margin |
| i.MX / Vybrid BSPs | CONFIG_WATCHDOG_TIMEOUT_MSECS default 128000 ms | — | — | huge on paper, unvalidated against your image |
| SoCFPGA BSPs | 30000 ms default (NXP Kconfig) | — | — | same problem, smaller number |
The Raspberry Pi's watchdog is the cleanest illustration because the ceiling isn't a software default someone can bump up — it's a hardware limit. The register is 20 bits wide, counting down every 16 microseconds, which caps out at just under 16 seconds, confirmed independently by the driver source and repeated across community threads. Looks like watchdog timeout register has 20 bits, and counts down every 16 microseconds, so limit is just under 16 seconds — and this is a hardware limit. That's not a config knob you can raise; it's the ceiling, full stop.
Meanwhile, published boot-time work shows how little margin that leaves once real-world variance shows up. On one board, booting from an SD card was 15% slower than from eMMC on the same board, taking 17 seconds instead of 15 seconds — a swing driven purely by storage media, with zero application code involved. On an i.MX93 board before any optimization pass, initial boot time was 13.2 seconds — far too long for the target, and part of that overhead was simply that U-Boot proper introduces a 2-second delay to allow user interruption, useful during development but unnecessary in production. Stack a slower SD card on top of an unoptimized boot and you're already flirting with a 16-second ceiling that was never adjustable in the first place.
Why the margin evaporates
The RPi case makes the mechanism obvious: a 16-second hardware ceiling against a boot that lands in the low-to-mid teens leaves two, maybe three seconds of slack. Any of the following can eat that whole:
- A slower SD card batch from a different supplier (the 15% swing above wasn't even the worst case measured).
- A first-boot penalty — filesystem checks, first-boot random-seed generation, or growing a partition — which published benchmarking guidance explicitly warns about: the initial boot may take longer due to extra steps like initializing random seeds or disk cache.
- Leaving verbose serial logging on. ST's own boot-time optimization documentation puts a number on this: Linux boot traces have a size of about 2 kilobytes, so they take about 2 seconds to transfer on a serial link configured at 115200 baud. Two seconds is not a rounding error against a three-second margin — it's most of it, spent purely moving text over a wire nobody's reading in production.
This is the actual mechanism behind the class of bug that STM32MP1 users had to reverse-engineer on community forums: nothing was "broken." The watchdog was doing exactly what it was configured to do. It just turned out the configured timeout and the real-world boot duration were closer together than anyone had checked — close enough that a routine, boring variance (slightly slower storage, first-boot overhead, debug logging left on) was enough to flip a healthy boot into a reset loop. ST's engineers reportedly had to shrink the timeout artificially to reproduce it on demand, which tells you how narrow the natural window already was.
The counter-argument, and why it's a trap
The obvious objection: "our BSP default is huge — 128 seconds for i.MX/Vybrid, 30 seconds for SoCFPGA — so we're nowhere near the edge."
That's the false-safety trap, and it's worth stating plainly: a big default doesn't mean the vendor validated it against your boot. It means the vendor picked a number that wouldn't trip on their reference demo image, with their reference rootfs, on their reference storage. A 128-second ceiling looks bulletproof right up until your product boot picture changes — you add a UBI attach step, a network-dependent mount (NFS root, DHCP timeout before falling back), a handful of out-of-tree modules loaded serially via depmod/modprobe, or a first-boot provisioning script. Boot-time engineers routinely find that once these get added, a boot that used to be an easy 5 seconds balloons to well past 30 or 60 — comfortably inside a "huge" 128-second window on paper, but far outside whatever assumption justified leaving the watchdog at a shorter, more product-appropriate value if someone had tightened it down for power or safety reasons without re-measuring.
In other words: the size of the default tells you about the vendor's comfort zone, not yours. "The default is huge, I'm safe" is exactly the sentence that precedes someone spending a week bisecting a boot loop that only reproduces on the slow batch of eMMC chips.
How to actually size it
The fix isn't a smarter default — it's replacing assumption with measurement.
Concretely:
- Measure worst case, not typical case. Use
grabserial,systemd-analyze, orbootgraph-style tooling — the same tools recommended in published boot-optimization workflows: use tools like oscilloscopes, chronometers, and software tools such as grabserial, bootgraph, systemd-analyze. Run several cold boots, not one warm re-run. - Include first-boot and cold conditions explicitly. First boot after flashing is often the slowest boot you'll ever see in the field — measure that one on purpose, don't average it away.
- Vary the storage/media you actually ship, not the fast dev-kit card sitting on your desk. The SD-vs-eMMC swing above was 15% on identical hardware otherwise — assume your worst supplier batch, not your best.
- Multiply by a safety factor, not by hope — 1.5–2x over measured worst case is a reasonable starting point, adjusted for how much your boot path can still change (adding modules, network mounts, UBI attach).
- Set the timeout to that number — not the vendor default, not the typical-case number you saw once on your bench.
Who actually needs to care about this
This is squarely aimed at engineers doing board bring-up or BSP customization who inherited a reference watchdog configuration — a Yocto meta-layer default, a vendor BSP Kconfig, a demo .dts fragment — and assumed someone upstream had already validated it against a real boot. Nobody did. The vendor validated it against their demo image. Once you change storage media, add drivers, extend the rootfs, or leave debug logging on for field diagnostics, that inherited number stops describing your product and starts describing theirs.
The math isn't complicated. The discipline of actually doing it — measuring worst case across real cold boots before trusting any number, vendor-supplied or otherwise — is the entire fix.
— Source: janiele.dev