MicrOBU/TODO.md

TODO

Engineering to-do list. The reviewer-facing open items live in docs/01-requirements-traceability.md ("Open items"); this file is the working list behind them.

Waiting on hardware

Confirm the RX queue drop counter explains the bench-session frame drops / map flicker (added 2026-09-22)

Investigated the user's report of "OBU mode keeps dropping a few frames" and "v2x screen comes and goes" while bench-testing against obu-cam-transmistter. Found a real, previously invisible drop path: obu-firmware/main/main.c's wifi_promisc_rx_cb() calls xQueueSend(s_rx_queue, ..., 0) (queue depth 8) without checking the return value, so a burst of promiscuously-captured frames arriving faster than rx_forward_task can drain them (each drain can legitimately block up to ~400ms under USB/UART contention) silently vanishes. None of the existing EspLinkStatus counters (oversizeDrops/txFailures/rxCrcErrors) caught this class of drop.

This plausibly also explains the map symptom: UseCaseDetectionEngine.pruneStale() drops a remote station's marker after staleRemoteMs (3 s) with no CAM update. Measured 2026-09-22 via tools/cit_one_rx_watch.py --host 192.168.40.201 against obu-cam-transmistter's bench beacon (stationID 195936478 / 0x0BADC0DE): 75 CAMs in 25 s, ~3 Hz, not the 1 Hz this note assumed earlier — faster than assumed means more promiscuous captures per second and a shorter fuse on staleRemoteMs, both of which make the queue-overflow theory more likely, not less.

Fixed to be visible, not yet fixed to not drop: added a rxQueueDrops counter, checked xQueueSend's return value (main.c), wired it through the STATUS heartbeat as a new trailing uint16 field (serial_link.c/.h, SerialFrame.kt's EspLinkStatus), and surfaced it on the CAM Pinger card (MqttTopicViewerScreen.kt, string mqtt_cam_pinger_fw_counters). Host build untouched (serial_link.c/main.c aren't in the host test's standard-headers-only set); IDF build verification is the remaining pre-flash check. Deliberately did NOT bump s_rx_queue's depth from 8 — no real burst-size data yet, and guessing a bigger number against an unmeasured memory budget is exactly the kind of assumption microbu-hw-review flags as needing verification first, not capacity that's cheap to reason your way into.

Needs: a phone attached to the production OBU's native USB port, watching the CAM Pinger card, while obu-cam-transmistter (or real traffic) beacons.

• idf.py build succeeds (obu-firmware, IDF 6.1) — clean, both changed files compiled with no warnings, 17% flash free.
• Reflashed the production OBU on COM3 2026-09-22 (hash verified). Boot log confirms the new build (21e0149-dirty, compiled Sep 22 2026 14:14:09), clean boot, OCB @ 5900 MHz TX/RX armed, serial_link up ... 1 Hz heartbeat, no panic. Incidentally answers part of the "measure the OBU's actual transmit power" item below: this boot logged tx power: 72 quarter-dBm = 18.00 dBm (20.00 requested) — the driver is clamping below the requested 20 dBm at 5900 MHz, as that item suspected but had not measured.
• 25 s of steady-state console (no phone attached, obu-cam-transmistter beaconing nearby): silent — no crash, no oversize/rx queue full/crc warnings. Inconclusive on its own (successful forwards aren't logged, and nothing was attached to trigger the ~400 ms UART stalls the theory needs), but at least rules out a crash-on-boot regression.
• Confirmed the wider bench RF path independently via the CiT One OBU broker (py -3.11 tools/cit_one_rx_watch.py --host 192.168.40.201): heard obu-cam-transmistter's beacon cleanly, 75/25 s, GN source 14:00:02:00:00:00:00:01, position in the expected St. Georg route area. This is a different receiver from the production OBU though — it shows the beacon is genuinely on air, not that COM3 forwards every one of it without drops.
• Confirmed on real hardware, 2026-09-22. Installed the updated debug APK (previous build on the phone was from 2026-09-15, predating this fix entirely) on the Pixel 9 Pro (adb over Wi-Fi), relaunched against the freshly-reflashed COM3, and read rx queue drop via adb logcat -s UsbSerialTransport. The counter mechanism works end-to-end and the bug is real: rxQueueDrops was 0 at the last flash (14:22), read as 89 at first reconnect (14:48, ~26 min later), and 90 at a second reconnect (14:52). No oversizeDrops, txFailures, or rxCrcErrors moved at all, and zero decode FAILED lines — this queue is the only place frames are going missing. Nuance: over a clean ~4.5 min window in between (14:48→14:52) with obu-cam-transmistter actively beaconing at a measured ~3.33 Hz (matches the CiT One's 75/25 s independently) and 490+ CAMs decoding cleanly with steady cadence and no gaps, the counter did not move — it only ticked at connect/reconnect moments. So this is a low-rate, bursty drop (matches the user's own "a few frames" framing), not a continuous overflow under steady single-station traffic; it may be specific to WiFi/PHY activity around association or reconnect rather than raw beacon rate. Worth a longer, quieter-boot capture before sizing a s_rx_queue bump. Did not independently confirm the map-flicker connection this session — that needs eyes on the app's V2X screen while watching this same counter live, not just logcat.

On-device check of the full-screen V2X live map (added 2026-09-15)

The live map moved out of the V2X Monitor's view-mode row into its own full-screen destination (V2xMapScreen, route v2x_map), reached from the map button in that screen's header. Markers are now cached and reused across updates instead of being rebuilt on every incoming message, and SPATEM intersections are drawn as traffic lights at the position of the RSU's own CAM. All of that compiles and the unit tests pass, but none of it has been seen with live traffic.

Needs: the phone with the app, plus a CAM/DENM/SPATEM source - either the CiT One, or the OBU ESP32-C5 with a second board or a real RSU transmitting.

• Both hardware modes: tap the map button, confirm the map fills the screen (no status bar, no bottom nav) and the back button returns to the V2X Monitor.
• Panning stays smooth while CAMs are arriving - this is what the marker reuse is for. Compare against the old behaviour if it still judders.
• Touching the map stops it recentring; the location FAB resumes follow and lights up.
• A DENM shows the warning triangle, and a SPATEM intersection shows a traffic light with the lamp matching the Dashboard's SignalCard for the same intersection.
• Near a real RSU: confirm the RSU is drawn once, as a traffic light, not as a CAM pin with a light on top of it. If the RSU sends SPATEM but no CAM, the "signals not shown" note should appear instead - worth knowing which of the two the HAW RSUs actually do.

Over-the-air check of the GN lifetime fix (added 2026-09-11)

geonet.c now writes GN lifetime 0x05 (1 s) instead of 0x83, which decoded to 3200 s. Changed in both obu-firmware and obu-cam-transmistter. Both still build (IDF 6.1 / 5.5.4), and the compiled geonet_wrap_shb stores the new byte, but it has not been seen on air yet. Nothing else reads this byte (gn_unwrap.c ignores it, the app never sees GN headers), so the app does not need updating alongside the firmware.

Needs: the phone with the app, the OBU ESP32-C5, and a second ESP32-C5 running its-g5-receiver-firmware to capture with.

• Flash obu-firmware (see obu-firmware/FLASHING.md).
• Connect the phone, let it send CAMs, and confirm the CAM Pinger's tx fail counter stays 0.
• Capture with the receiver into its-g5-receiver-firmware/recordings/.
• Run python obu-firmware/test/pcap_gn_tally.py its-g5-receiver-firmware/recordings/<capture>.pcap. The rows for the phone's pseudonym MACs must show SHB, port 2001, lifetime 0x05, exactly like every other station's CAMs.
• While the phone is connected: real-station CAMs/DENMs still reach the app (RX path unchanged).

Partial check possible with one board and no phone: flash it, idf.py -p COMx monitor, and look for OCB @ 5900 MHz - TX/RX armed. That proves the new build boots and brings the radio up, not that it transmits correctly.

Measure the OBU's actual transmit power (added 2026-09-14)

Nothing in this project has ever measured it. main.c asks for 20 dBm (esp_wifi_set_max_tx_power(80), 0.25 dBm units) and the build's ceiling is the same (CONFIG_ESP_PHY_MAX_TX_POWER=20), but a request is a ceiling, not a guarantee: the driver clamps it to its own calibrated table, and 5900 MHz is above the range this chip is rated for, so the table actually in use is channel 177's. The firmware now reads the value back and logs it at boot, which records what the driver admits to, not what leaves the antenna.

• Flash and idf.py -p COM3 monitor, then note the tx power: line. A value below 80 means the driver clamped the request, which the code alone cannot tell you.
• Relative check with the second ESP32-C5 on its-g5-receiver-firmware: capture at a measured distance in a straight line, read the RSSI the receive path already reports, and record distance and RSSI together. This gives a comparable number between builds and antennas, which is what matters for range work, without any lab equipment.
• Only a spectrum analyser or a calibrated reference receiver gives real radiated power. Worth it only if the range result looks wrong, or if the thesis needs an absolute figure.

For context: ETSI allows up to 33 dBm EIRP on the ITS band, and production OBUs sit around 20 to 23 dBm, so the requested figure is in the right region if the PA really keys it there.

obu-cam-transmistter yawRateConfidence fix (added 2026-09-11)

Its cam.c (compiled into that firmware) wrote yawRateConfidence as 3 bits / 7 instead of 4 bits / unavailable(8), the bug the app fixed on 2026-08-20. Fixed in it and in obu-firmware's reference copy; asn1tools now decodes the CAM and re-encodes it byte-identically, and it builds on IDF 5.5.4. No board runs this firmware right now (the production OBU runs obu-firmware), so this only matters if it is flashed again:

• After flashing it: capture, run pcap_gn_tally.py, and decode the CAM payload with asn1tools (py -3.11, modules in asn1/).

Signed-message reception and exact payloads (added 2026-09-11)

obu-firmware's gn_unwrap.c now unwraps TS 103 097 signed packets (signature not verified, reported as V2X_RX flags bit1) and cuts every message to the length its header declares, dropping the 8 bytes the chip's RX appends to each frame, which were forwarded to the phone until now. Verified on the host (obu-firmware/test/host: chain, replay of all recordings against asn1tools, 50M-iteration fuzz) and built on IDF 6.1, but not flashed: the production OBU still runs the 2026-09-10 build. Needs the OBU with this build, the phone, and signed traffic - real vehicles or RSUs, since the bench CiT One sends unsigned. A second ESP32 running the receiver firmware is optional, but shows what was on air at the time.

• Flash obu-firmware (this also carries the GN lifetime fix above).
• Near signed traffic: signed CAMs/DENMs appear in the app, and a simultaneous capture shows them on air (pcap_gn_tally.py lists them as secured).
• Unsigned bench traffic still decodes in the app as before (messages now arrive 8 bytes shorter).
• The heartbeat's oversize counter still counts over-long messages (e.g. road SPATEMs).

CiT One custom CAM injection over v2x/tx/v2/cam (added 2026-09-14)

The haw-002 unit now runs the special firmware: Cohda's own CAM transmission disabled, and a V2X-Gateway build that accepts a SendV2XMessage (schemas.consider-innovation.de/its-s/ v2x_interface.proto) carrying a UPER CAM on v2x/tx/v2/cam. tools/cit_one_cam_tx.py builds and publishes those from a PC; its --self-test passes offline, proving only that the bytes match CamEncodeGoldenTest.kt and that the protobuf wrapper round-trips. Nothing about what the OBU does with them is established.

Reach the broker over Wi-Fi or Ethernet for now - the USB-peripheral-mode link needs the phone to be USB host on a 172.25.1.0/24 interface with no DHCP server, which Android cannot configure from inside an app.

Needs: the CiT One haw-002 on the same network as a PC, and a second ESP32-C5 running its-g5-receiver-firmware sniffing G5CC (-c 5900) to capture with.

Bench run 2026-09-14, PC -> haw-002 (192.168.3.201), captured on the RSU (192.168.3.202, not .2.202 - that address does not route). tools/cit_one_rx_watch.py decodes what a unit hears. Result: the injection path works end to end, with one blocker found.

• Publishes without the broker refusing the topic. 1.00 Hz, confirmed by subscribing to v2x/tx/v2/cam on the OBU itself.

• The RSU hears our CAMs on air, 1.00 Hz, matching what we publish.

• ItsPduHeader is expected in the payload - we send it included and it decodes.

• BTP destination port 2001. GN source address 08:00:26:93:92:01:91:dc, the OBU's.

• Our CAM content goes out intact: position, speed (417), heading (639), width (7) and length (18) arrive byte-exact. The gateway does not touch the content.

• The gateway overwrites stationID with the OBU's own (999999 -> 4033890855, which matches own_info.stationID on v2x/rx/obu_gnss). This is what the "OBU owns identity" decision wants, so --follow-obu-identity is not needed on this unit.

• BLOCKER: Cohda's own CAM is still transmitting. Fixed 2026-09-14 by disabling CAM in a second conf file: the RSU now hears only our stream, 0 CAMs with the stack's unavailable dimensions over 30 s. Note the stack restart gave the unit a new identity (stationID 4033890855 -> 2553426533, GN source 08:00:26:... -> 08:00:a2:...), which is expected under ItsGnLocalAddrConfMethod = 2 (anonymous, random at boot).

• Re-checked: 41 published / 41 heard over 40 s, 1.02 Hz both ends, inter-arrival a steady 1.0 s. 100% delivery, no gateway rate limiting. An earlier 0.40 Hz sample was the stack still settling after the restart and did not persist. Two topics the v6 API does not document, found by subscribing to # on haw-002:

• v2x/loopback/cam - a RecvV2XMessage (btpHeader.type=2) carrying each CAM the unit transmits, 1:1 with what we publish and after the gateway's stationID rewrite. This is the TX confirmation we were going to ask consider it for: it makes "did my CAM go out, and under which identity" answerable on the transmitting unit alone, without an RSU or a second ESP32.

• v2x/rx/obuinfo at 10 Hz - the protobuf OwnStationInfo (binary twin of obu_gnss; field 2 decodes to the same stationID, field 10 to the same heading). Output only, so it is not the content-feed input we speculated about.

• Wire v2x/loopback/cam into cit_one_rx_watch.py as a local TX check.

• Sanity-check the rate: --rate 4 should produce 4 CAMs/s on air, since ItsDCCEnabled = 0 on this unit.

Set up host testing

• Install MSYS2 UCRT64 gcc (done 2026-09-11: gcc 16.2.0, GNU Make 4.4.1; chosen over WSL, vanetza is not going to be built). Setup and the PATH gotcha: obu-firmware/test/host/README.md.
• Host round-trip test obu-firmware/test/host/test_chain.c (geonet_wrap_shb -> dot11p_build_frame -> gn_unwrap_its, byte-checked against the standard). Done 2026-09-11: 491 checks, 0 failed. Run make in that folder before flashing any firmware fix.
• Replay of the recorded captures (test_replay.c + check_replay.py, independent asn1tools check). Done 2026-09-11: C and Python agree on all 15 145 records.
• Mutation fuzzer fuzz_gn_unwrap.c, inputs against a no-access guard page. Done 2026-09-11: 50 000 000 iterations, no crash. make runs a 2 000 000-iteration pass every time.

Firmware ideas from the vanetza review (2026-09-11, not started)

Suggested order after the host tests exist:

• Read secured packets (GN NextHeader=2) without verifying them. Done 2026-09-11 in gn_unwrap.c, host-verified; flagged to the phone as V2X_RX flags bit1. On-air check under "Waiting on hardware".
• Forward the full GeoBroadcast area: shape (circle/rectangle/ellipse), DistanceB, angle, appended to the V2X_RX prefix behind a capability bit. Port vanetza's geonet/areas.cpp inside_or_at_border to the app, which currently treats every area as a circle.
• RX filtering before the serial link: duplicate detection for GBC (last 8 sequence numbers per source, as vanetza does), drop our own frames, reject GN version != 1.
• Read the DCC-MCO field (the 4 "reserved" bytes of an SHB header): neighbours' channel busy ratio for free.
• Minimum TX gap in firmware as a DCC safety net (vanetza reactive table: 60 ms relaxed ... 460 ms restrictive), with a CBR estimate in the heartbeat.
• Generic V2X_TX message (BTP port, SHB/GBC, traffic class, lifetime, area) so the phone can send DENM and VAM without reflashing. Consider QoS Data frames: vanetza's Cohda receive path drops non-QoS ones.

Dropped: building vanetza as a GN/BTP oracle. Real captures (pcap_gn_tally.py), the host round-trip test and asn1tools for UPER cover what it would have checked.

Follow-ups found 2026-09-11

• App: show the signed flag. V2xRxFrame.parse in SerialFrame.kt only reads bit0 of the flags byte; read bit1 (signed, not verified) and show it where messages are listed.
• Messages that do not decode with asn1tools. In the recordings, 56 from the CiT One (aa:f8:76:7d:bd:ad: 54 CAMs of 245 bytes, 2 DENMs of 402 bytes) and one 218-byte CAM from 6e:94:03:1b:05:26 fail against cam_1_4_1/denm_1_3_1 + cdd_1_3_1_1, with or without the old trailing bytes. A newer module version on the sender, or a sender bug; check what the app's decoders make of them (check_replay.py lists the records).