← All articles
FirmwareOTAEmbedded

Over-the-air updates without bricking the fleet

A/B partitions, rollback on failure, and how we test an update before it ever leaves the bench.

Bricking a device in the field is expensive. On Aurora we ship updates to several hundred nodes — some in locations that take hours to reach. Getting OTA wrong once would cost more than the entire firmware development budget.

The basic guarantee

Every OTA implementation needs one property above everything else: if power fails at any point during an update, the device must boot a working image.

This means you cannot overwrite the running image. You need a second partition.

A/B partitioning on STM32

The STM32L4 has 512 kB of flash. We split it into three regions:

  • Bootloader: first 32 kB (read-protected, never updated over the air)
  • Slot A: 240 kB
  • Slot B: 240 kB

The bootloader reads a small header at the start of each slot: version number, CRC32, and a "boot me" flag. At startup it finds the slot with the valid CRC and the highest version number and jumps to it. If neither slot has a valid CRC, it enters a known-safe recovery mode and requests a full re-flash over LoRaWAN.

Updates arrive as 240-byte chunks — sized to fit in a single LoRaWAN payload at SF9. The firmware reassembles them into the inactive slot, verifies the CRC, sets the "boot me" flag, and reboots. On the next boot the bootloader sees the higher version in the new slot and jumps to it.

The rollback window

After booting a new image, we run a 90-second health check. If the node receives a successful uplink acknowledgement from the gateway within that window, it writes a "confirmed" flag to the new slot header. If it doesn't — because the new firmware broke the radio configuration, for instance — the bootloader on the next reset sees an unconfirmed slot and falls back to the previous one.

This handles the most common OTA failure mode: a firmware bug that breaks the radio.

Before it leaves the bench

We test every release candidate on a physical node in our rig before pushing it. The rig runs three passes:

  1. Clean install from the bootloader.
  2. OTA update from the previous release.
  3. Simulated power failure during OTA (we cut power randomly during the transfer, then restore and verify recovery).

Pass three is the one that catches bugs. In two years of Aurora deployments, we have never had an OTA failure in the field.