MicrOBU/TODO.md

# TODO

Engineering to-do list. The reviewer-facing open items live in
`docs/01-requirements-traceability.md` ("Open items"); this file is the working list behind them.

## Waiting on hardware

### Confirm the RX queue drop counter explains the bench-session frame drops / map flicker (added 2026-09-22)

Investigated the user's report of "OBU mode keeps dropping a few frames" and "v2x screen comes
and goes" while bench-testing against `obu-cam-transmistter`. Found a real, previously invisible
drop path: `obu-firmware/main/main.c`'s `wifi_promisc_rx_cb()` calls `xQueueSend(s_rx_queue, ...,
0)` (queue depth 8) without checking the return value, so a burst of promiscuously-captured
frames arriving faster than `rx_forward_task` can drain them (each drain can legitimately block up
to ~400ms under USB/UART contention) silently vanishes. None of the existing `EspLinkStatus`
counters (`oversizeDrops`/`txFailures`/`rxCrcErrors`) caught this class of drop.

This plausibly also explains the map symptom: `UseCaseDetectionEngine.pruneStale()` drops a remote
station's marker after `staleRemoteMs` (3 s) with no CAM update. Measured 2026-09-22 via
`tools/cit_one_rx_watch.py --host 192.168.40.201` against `obu-cam-transmistter`'s bench beacon
(stationID 195936478 / 0x0BADC0DE): **75 CAMs in 25 s, ~3 Hz**, not the 1 Hz this note assumed
earlier — faster than assumed means more promiscuous captures per second and a shorter fuse on
`staleRemoteMs`, both of which make the queue-overflow theory more likely, not less.

Fixed to be **visible**, not yet fixed to **not drop**: added a `rxQueueDrops` counter, checked
`xQueueSend`'s return value (`main.c`), wired it through the STATUS heartbeat as a new trailing
`uint16` field (`serial_link.c/.h`, `SerialFrame.kt`'s `EspLinkStatus`), and surfaced it on the
CAM Pinger card (`MqttTopicViewerScreen.kt`, string `mqtt_cam_pinger_fw_counters`). Host build
untouched (serial_link.c/main.c aren't in the host test's standard-headers-only set); IDF build
verification is the remaining pre-flash check. Deliberately did NOT bump `s_rx_queue`'s depth from
8 — no real burst-size data yet, and guessing a bigger number against an unmeasured memory budget
is exactly the kind of assumption [[microbu-hw-review]] flags as needing verification first, not
capacity that's cheap to reason your way into.

Needs: a phone attached to the production OBU's native USB port, watching the CAM Pinger card,
while `obu-cam-transmistter` (or real traffic) beacons.

- [x] `idf.py build` succeeds (obu-firmware, IDF 6.1) — clean, both changed files compiled with no
      warnings, 17% flash free.
- [x] Reflashed the production OBU on **COM3** 2026-09-22 (hash verified). Boot log confirms the
      new build (`21e0149-dirty`, compiled Sep 22 2026 14:14:09), clean boot, OCB @ 5900 MHz
      TX/RX armed, `serial_link up ... 1 Hz heartbeat`, no panic. Incidentally answers part of the
      "measure the OBU's actual transmit power" item below: this boot logged
      `tx power: 72 quarter-dBm = 18.00 dBm (20.00 requested)` — the driver **is** clamping below
      the requested 20 dBm at 5900 MHz, as that item suspected but had not measured.
- [x] 25 s of steady-state console (no phone attached, `obu-cam-transmistter` beaconing nearby):
      silent — no crash, no `oversize`/`rx queue full`/`crc` warnings. Inconclusive on its own
      (successful forwards aren't logged, and nothing was attached to trigger the ~400 ms UART
      stalls the theory needs), but at least rules out a crash-on-boot regression.
- [x] Confirmed the wider bench RF path independently via the CiT One OBU broker
      (`py -3.11 tools/cit_one_rx_watch.py --host 192.168.40.201`): heard `obu-cam-transmistter`'s
      beacon cleanly, 75/25 s, GN source `14:00:02:00:00:00:00:01`, position in the expected
      St. Georg route area. This is a *different* receiver from the production OBU though — it
      shows the beacon is genuinely on air, not that COM3 forwards every one of it without drops.
- [x] **Confirmed on real hardware, 2026-09-22.** Installed the updated debug APK (previous build
      on the phone was from 2026-09-15, predating this fix entirely) on the Pixel 9 Pro (adb over
      Wi-Fi), relaunched against the freshly-reflashed COM3, and read `rx queue drop` via `adb
      logcat -s UsbSerialTransport`. The counter mechanism works end-to-end and **the bug is
      real**: `rxQueueDrops` was 0 at the last flash (14:22), read as 89 at first reconnect
      (14:48, ~26 min later), and 90 at a second reconnect (14:52). No `oversizeDrops`,
      `txFailures`, or `rxCrcErrors` moved at all, and zero `decode FAILED` lines — this queue is
      the only place frames are going missing.
      Nuance: over a clean ~4.5 min window in between (14:48→14:52) with `obu-cam-transmistter`
      actively beaconing at a measured **~3.33 Hz** (matches the CiT One's 75/25 s independently)
      and 490+ CAMs decoding cleanly with steady cadence and no gaps, the counter did **not**
      move — it only ticked at connect/reconnect moments. So this is a low-rate, bursty drop (matches
      the user's own "a few frames" framing), not a continuous overflow under steady single-station
      traffic; it may be specific to WiFi/PHY activity around association or reconnect rather than
      raw beacon rate. Worth a longer, quieter-boot capture before sizing a `s_rx_queue` bump.
      Did **not** independently confirm the map-flicker connection this session — that needs eyes
      on the app's V2X screen while watching this same counter live, not just logcat.

### On-device check of the full-screen V2X live map (added 2026-09-15)

The live map moved out of the V2X Monitor's view-mode row into its own full-screen destination
(`V2xMapScreen`, route `v2x_map`), reached from the map button in that screen's header. Markers are
now cached and reused across updates instead of being rebuilt on every incoming message, and
SPATEM intersections are drawn as traffic lights at the position of the RSU's own CAM. All of that
compiles and the unit tests pass, but none of it has been seen with live traffic.

Needs: the phone with the app, plus a CAM/DENM/SPATEM source - either the CiT One, or the OBU
ESP32-C5 with a second board or a real RSU transmitting.

- [ ] Both hardware modes: tap the map button, confirm the map fills the screen (no status bar, no
      bottom nav) and the back button returns to the V2X Monitor.
- [ ] Panning stays smooth while CAMs are arriving - this is what the marker reuse is for. Compare
      against the old behaviour if it still judders.
- [ ] Touching the map stops it recentring; the location FAB resumes follow and lights up.
- [ ] A DENM shows the warning triangle, and a SPATEM intersection shows a traffic light with the
      lamp matching the Dashboard's SignalCard for the same intersection.
- [ ] Near a real RSU: confirm the RSU is drawn once, as a traffic light, not as a CAM pin with a
      light on top of it. If the RSU sends SPATEM but no CAM, the "signals not shown" note should
      appear instead - worth knowing which of the two the HAW RSUs actually do.


### Over-the-air check of the GN lifetime fix (added 2026-09-11)

`geonet.c` now writes GN lifetime `0x05` (1 s) instead of `0x83`, which decoded to 3200 s. Changed
in both `obu-firmware` and `obu-cam-transmistter`. Both still build (IDF 6.1 / 5.5.4), and the
compiled `geonet_wrap_shb` stores the new byte, but it has not been seen on air yet. Nothing else
reads this byte (`gn_unwrap.c` ignores it, the app never sees GN headers), so the app does not
need updating alongside the firmware.

Needs: the phone with the app, the OBU ESP32-C5, and a **second** ESP32-C5 running
`its-g5-receiver-firmware` to capture with.

- [ ] Flash `obu-firmware` (see `obu-firmware/FLASHING.md`).
- [ ] Connect the phone, let it send CAMs, and confirm the CAM Pinger's `tx fail` counter stays 0.
- [ ] Capture with the receiver into `its-g5-receiver-firmware/recordings/`.
- [ ] Run `python obu-firmware/test/pcap_gn_tally.py its-g5-receiver-firmware/recordings/<capture>.pcap`.
      The rows for the phone's pseudonym MACs must show SHB, port 2001, lifetime `0x05`, exactly
      like every other station's CAMs.
- [ ] While the phone is connected: real-station CAMs/DENMs still reach the app (RX path unchanged).

Partial check possible with one board and no phone: flash it, `idf.py -p COMx monitor`, and look
for `OCB @ 5900 MHz - TX/RX armed`. That proves the new build boots and brings the radio up, not
that it transmits correctly.

### Measure the OBU's actual transmit power (added 2026-09-14)

Nothing in this project has ever measured it. `main.c` asks for 20 dBm
(`esp_wifi_set_max_tx_power(80)`, 0.25 dBm units) and the build's ceiling is the same
(`CONFIG_ESP_PHY_MAX_TX_POWER=20`), but a request is a ceiling, not a guarantee: the driver clamps
it to its own calibrated table, and 5900 MHz is above the range this chip is rated for, so the
table actually in use is channel 177's. The firmware now reads the value back and logs it at boot,
which records what the driver admits to, not what leaves the antenna.

- [ ] Flash and `idf.py -p COM3 monitor`, then note the `tx power:` line. A value below 80 means
      the driver clamped the request, which the code alone cannot tell you.
- [ ] Relative check with the second ESP32-C5 on `its-g5-receiver-firmware`: capture at a measured
      distance in a straight line, read the RSSI the receive path already reports, and record
      distance and RSSI together. This gives a comparable number between builds and antennas,
      which is what matters for range work, without any lab equipment.
- [ ] Only a spectrum analyser or a calibrated reference receiver gives real radiated power. Worth
      it only if the range result looks wrong, or if the thesis needs an absolute figure.

For context: ETSI allows up to 33 dBm EIRP on the ITS band, and production OBUs sit around
20 to 23 dBm, so the requested figure is in the right region if the PA really keys it there.

### obu-cam-transmistter yawRateConfidence fix (added 2026-09-11)

Its `cam.c` (compiled into that firmware) wrote `yawRateConfidence` as 3 bits / 7 instead of
4 bits / unavailable(8), the bug the app fixed on 2026-08-20. Fixed in it and in obu-firmware's
reference copy; asn1tools now decodes the CAM and re-encodes it byte-identically, and it builds on
IDF 5.5.4. No board runs this firmware right now (the production OBU runs obu-firmware), so this
only matters if it is flashed again:

- [ ] After flashing it: capture, run `pcap_gn_tally.py`, and decode the CAM payload with
      asn1tools (`py -3.11`, modules in `asn1/`).

### Signed-message reception and exact payloads (added 2026-09-11)

obu-firmware's `gn_unwrap.c` now unwraps TS 103 097 signed packets (signature not verified,
reported as V2X_RX flags bit1) and cuts every message to the length its header declares, dropping
the 8 bytes the chip's RX appends to each frame, which were forwarded to the phone until now.
Verified on the host (`obu-firmware/test/host`: chain, replay of all recordings against asn1tools,
50M-iteration fuzz) and built on IDF 6.1, but not flashed: the production OBU still runs the
2026-09-10 build. Needs the OBU with this build, the phone, and signed traffic - real vehicles or
RSUs, since the bench CiT One sends unsigned. A second ESP32 running the receiver firmware is
optional, but shows what was on air at the time.

- [ ] Flash obu-firmware (this also carries the GN lifetime fix above).
- [ ] Near signed traffic: signed CAMs/DENMs appear in the app, and a simultaneous capture shows
      them on air (`pcap_gn_tally.py` lists them as `secured`).
- [ ] Unsigned bench traffic still decodes in the app as before (messages now arrive 8 bytes
      shorter).
- [ ] The heartbeat's oversize counter still counts over-long messages (e.g. road SPATEMs).

### CiT One custom CAM injection over `v2x/tx/v2/cam` (added 2026-09-14)

The haw-002 unit now runs the special firmware: Cohda's own CAM transmission disabled, and a
V2X-Gateway build that accepts a `SendV2XMessage` (schemas.consider-innovation.de/its-s/
v2x_interface.proto) carrying a UPER CAM on `v2x/tx/v2/cam`. `tools/cit_one_cam_tx.py` builds
and publishes those from a PC; its `--self-test` passes offline, proving only that the bytes
match `CamEncodeGoldenTest.kt` and that the protobuf wrapper round-trips. Nothing about what
the OBU does with them is established.

Reach the broker over Wi-Fi or Ethernet for now - the USB-peripheral-mode link needs the phone
to be USB host on a `172.25.1.0/24` interface with no DHCP server, which Android cannot
configure from inside an app.

Needs: the CiT One haw-002 on the same network as a PC, and a second ESP32-C5 running
`its-g5-receiver-firmware` sniffing G5CC (`-c 5900`) to capture with.

Bench run 2026-09-14, PC -> haw-002 (192.168.3.201), captured on the RSU (192.168.3.202,
**not** .2.202 - that address does not route). `tools/cit_one_rx_watch.py` decodes what a unit
hears. Result: the injection path works end to end, with one blocker found.

- [x] Publishes without the broker refusing the topic. 1.00 Hz, confirmed by subscribing to
      `v2x/tx/v2/cam` on the OBU itself.
- [x] The RSU hears our CAMs on air, 1.00 Hz, matching what we publish.
- [x] `ItsPduHeader` **is** expected in the payload - we send it included and it decodes.
- [x] BTP destination port 2001. GN source address `08:00:26:93:92:01:91:dc`, the OBU's.
- [x] **Our CAM content goes out intact**: position, speed (417), heading (639), width (7) and
      length (18) arrive byte-exact. The gateway does not touch the content.
- [x] **The gateway overwrites `stationID`** with the OBU's own (999999 -> 4033890855, which
      matches `own_info.stationID` on `v2x/rx/obu_gnss`). This is what the "OBU owns identity"
      decision wants, so `--follow-obu-identity` is not needed on this unit.
- [x] ~~BLOCKER: Cohda's own CAM is still transmitting.~~ Fixed 2026-09-14 by disabling CAM in
      a second conf file: the RSU now hears only our stream, 0 CAMs with the stack's
      unavailable dimensions over 30 s. Note the stack restart gave the unit a new identity
      (stationID 4033890855 -> 2553426533, GN source `08:00:26:...` -> `08:00:a2:...`), which is
      expected under `ItsGnLocalAddrConfMethod = 2` (anonymous, random at boot).
- [x] Re-checked: 41 published / 41 heard over 40 s, 1.02 Hz both ends, inter-arrival a steady
      1.0 s. 100% delivery, no gateway rate limiting. An earlier 0.40 Hz sample was the stack
      still settling after the restart and did not persist.
Two topics the v6 API does not document, found by subscribing to `#` on haw-002:

- `v2x/loopback/cam` - a `RecvV2XMessage` (btpHeader.type=2) carrying each CAM the unit
  transmits, 1:1 with what we publish and **after** the gateway's stationID rewrite. This is the
  TX confirmation we were going to ask consider it for: it makes "did my CAM go out, and under
  which identity" answerable on the transmitting unit alone, without an RSU or a second ESP32.
- `v2x/rx/obuinfo` at 10 Hz - the protobuf `OwnStationInfo` (binary twin of `obu_gnss`;
  field 2 decodes to the same stationID, field 10 to the same heading). Output only, so it is
  not the content-feed input we speculated about.

- [ ] Wire `v2x/loopback/cam` into `cit_one_rx_watch.py` as a local TX check.
- [ ] Sanity-check the rate: `--rate 4` should produce 4 CAMs/s on air, since `ItsDCCEnabled = 0`
      on this unit.

## Set up host testing

- [x] Install MSYS2 UCRT64 gcc (done 2026-09-11: gcc 16.2.0, GNU Make 4.4.1; chosen over WSL,
      vanetza is not going to be built). Setup and the PATH gotcha: `obu-firmware/test/host/README.md`.
- [x] Host round-trip test `obu-firmware/test/host/test_chain.c` (`geonet_wrap_shb` ->
      `dot11p_build_frame` -> `gn_unwrap_its`, byte-checked against the standard). Done
      2026-09-11: 491 checks, 0 failed. Run `make` in that folder before flashing any firmware fix.
- [x] Replay of the recorded captures (`test_replay.c` + `check_replay.py`, independent asn1tools
      check). Done 2026-09-11: C and Python agree on all 15 145 records.
- [x] Mutation fuzzer `fuzz_gn_unwrap.c`, inputs against a no-access guard page. Done 2026-09-11:
      50 000 000 iterations, no crash. `make` runs a 2 000 000-iteration pass every time.

## Firmware ideas from the vanetza review (2026-09-11, not started)

Suggested order after the host tests exist:

- [x] **Read secured packets (GN NextHeader=2) without verifying them.** Done 2026-09-11 in
      `gn_unwrap.c`, host-verified; flagged to the phone as V2X_RX flags bit1. On-air check under
      "Waiting on hardware".
- [ ] **Forward the full GeoBroadcast area**: shape (circle/rectangle/ellipse), DistanceB, angle,
      appended to the V2X_RX prefix behind a capability bit. Port vanetza's `geonet/areas.cpp`
      `inside_or_at_border` to the app, which currently treats every area as a circle.
- [ ] **RX filtering before the serial link**: duplicate detection for GBC (last 8 sequence numbers
      per source, as vanetza does), drop our own frames, reject GN version != 1.
- [ ] **Read the DCC-MCO field** (the 4 "reserved" bytes of an SHB header): neighbours' channel
      busy ratio for free.
- [ ] **Minimum TX gap in firmware** as a DCC safety net (vanetza reactive table: 60 ms relaxed ...
      460 ms restrictive), with a CBR estimate in the heartbeat.
- [ ] **Generic V2X_TX message** (BTP port, SHB/GBC, traffic class, lifetime, area) so the phone can
      send DENM and VAM without reflashing. Consider QoS Data frames: vanetza's Cohda receive path
      drops non-QoS ones.

Dropped: building vanetza as a GN/BTP oracle. Real captures (`pcap_gn_tally.py`), the host
round-trip test and `asn1tools` for UPER cover what it would have checked.

## Follow-ups found 2026-09-11

- [ ] **App: show the signed flag.** `V2xRxFrame.parse` in `SerialFrame.kt` only reads bit0 of
      the flags byte; read bit1 (signed, not verified) and show it where messages are listed.
- [ ] **Messages that do not decode with asn1tools.** In the recordings, 56 from the CiT One
      (`aa:f8:76:7d:bd:ad`: 54 CAMs of 245 bytes, 2 DENMs of 402 bytes) and one 218-byte CAM from
      `6e:94:03:1b:05:26` fail against `cam_1_4_1`/`denm_1_3_1` + `cdd_1_3_1_1`, with or without the
      old trailing bytes. A newer module version on the sender, or a sender bug; check what the
      app's decoders make of them (`check_replay.py` lists the records).