MicrOBU

33 Commits 2 Branches 0 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
Ashin Walpola	a5ad3dcc5d	SPATEM receive, RSU CAM decode, and two ASN.1 encoding fixes SPATEM over the air - gn_unwrap.c accepts BTP-B port 2004 alongside 2001/2002. The serial protocol already carries the port in its V2X_RX prefix, so nothing else changed there. Note the crossover that makes this easy to get wrong: SPATEM is port 2004 but messageID 4, while MAPEM is port 2003 and messageID 5. - SpatemUperCodec decodes SPAT down to per-signal-group phase and timing. The bit layout was validated by replaying 79,042 real SPATEMs - the whole 2026-03-18 drive across 7+ RSUs plus the bench trigger - against asn1tools using the ETSI modules. All 79,042 matched on every field, none hit an unsupported branch. Two traps are pinned by tests: TimeChangeDetails is the one SEQUENCE here that is NOT extensible (5 optional bits, no extension bit), and maneuverAssistList cannot be skipped when present - it is variable-length, so it has to be walked to find where the next movement starts. - The V2X list shows one row per intersection with each signal group coloured by phase and a countdown where the RSU supplies timing. TimeMark wraps hourly, so the countdown corrects for it; without that it reads hugely negative once an hour, precisely when someone is watching it. - Entries expire after 15 s, much shorter than DENM's window: a traffic light that stopped updating is not "still green". Size caveat, deliberately deferred: SERIAL_LINK_MAX_PAYLOAD is still 512, so a SPATEM over ~498 bytes is counted as an oversize drop. The bench RSU sends 58 bytes and is unaffected, but real road RSUs measured 555 median / 1243 max, so roughly 70% would not arrive. Raising the cap also requires enlarging RX_FRAME_MAX_LEN and moving rx_item_t off the WiFi driver's callback stack, where it would otherwise overflow. RSU CAM decode - HighFrequencyContainer is a CHOICE, and a roadside unit picks rsuContainerHighFrequency, which carries no kinematics at all. The decoder bailed on that branch, so every RSU CAM was dropped - including the bench RSU, which sends CAM and SPATEM from the same station id. It now decodes for position and stationType. - RSU CAMs are kept out of UseCaseDetectionEngine. They arrive as a permanently stationary station at a fixed point, which is exactly the shape the stopped-vehicle and intersection-movement use cases match, and would raise a standing false alert for as long as the RSU was in range. CAM transmit: yawRateConfidence - YawRateConfidence has nine enumerands (0..8), so UPER needs 4 bits and "unavailable" is 8. The encoder wrote 3 bits with value 7 - one bit short and the wrong symbol - shifting every field after yawRate for any standards-strict receiver. The decoder read 3 bits too, so phone and ESP32 agreed with each other and with nothing else. - This is the third instance of that exact failure mode in this project, after CurvatureCalculationMode and the GeoNetworking reserved bytes. A round-trip test through our own decoder structurally cannot catch it, so CamEncodeGolden Test asserts the bytes asn1tools produces instead: it decoded this encoder's output and re-encoded it byte-identically. Confirmed on air afterwards - 26 of our own CAMs captured back off the OBU's receiver, all 26 accepted, where the same decoder rejected them before. DENM - Hazards now expire 60 s after their last repetition. This needs a clock, not just a filter: both source flows only emit when a DENM arrives, so a sender that drives away or loses power would never trigger a recompute and its hazard would stay on screen indefinitely. - The MQTT path was dropping every DENM for two independent reasons, both found by checking the payload against CI-CiT-MQTT_API_Documentation-v6 listing 2.6 rather than guessing: the station id key is originatingStationId, and eventPosition IS a GeoJSON Point rather than an object containing one. Also parses termination (presence is the signal), sequenceNumber, stationType and the RFC3339 detectionTime. Note roadSideUnit is 15, not 12 - the enumeration has a gap after tram(11). V2X screen - The decoded CAM/DENM list now renders on the CiT One path too; it was gated to the ESP32-C5 path and CiT One fell through to the raw MQTT topic list. Those topics move to their own tab, hidden on the ESP32-C5 path where there is no broker. Testing - Adds org.json as a test-only dependency: the android.jar stub throws "not mocked" on every JSONObject call, which made the MQTT payload parsers untestable off-device. - 23 V2X tests pass. EventDetectorTest's 4 failures are pre-existing and untouched by this change.	2026-08-20 15:47:14 +02:00
Ashin Walpola	0ccb867228	DENM over-the-air receive on the ESP32-C5 path The firmware forwarded CAM only: gn_unwrap_cam accepted single-hop broadcast (HT=5) and BTP port 2001, so every DENM was dropped before it reached the phone. Real OBUs disseminate DENM by GeoBroadcast (HT=4), whose 44-byte extended header also carries the hazard's relevance area - materially more useful on a map than the sender's own position, since a sender may be relaying for someone else. Firmware - gn_unwrap_cam -> gn_unwrap_its: accepts GeoBroadcast alongside TSB/SHB, and BTP ports 2001 and 2002, extracting the GeoBroadcast destination area. Both extended-header lengths were measured against live air capture rather than read off a spec table. Secured packets (Basic Header NextHeader=2) are rejected rather than misparsed. - SERIAL_MSG_CAM_RX (0x02) superseded by SERIAL_MSG_V2X_RX (0x04): a 14-byte prefix carrying BTP port, RSSI and the destination area. Adding MAPEM later needs a decoder on the phone but no protocol change. 0x02 stays reserved so the numbering is not silently reused. - Promiscuous RX capture buffer 400 -> 800 bytes. A real GeoBroadcast DENM is around 500 bytes on air and was being truncated mid-payload, which no amount of correct unwrapping downstream could have recovered from. - geonet_wrap_shb, both firmwares: the SHB extended header is 28 bytes, not 24. The Source Position Vector is followed by a 4-byte reserved field; without it a standards-strict receiver reads the CAM payload's first two bytes as the BTP destination port. App - DenmUperCodec: UPER decoder for the ManagementContainer and the SituationContainer's eventType. ValidityDuration is 17 bits, not 16, and ManagementContainer, SituationContainer and CauseCode each carry their own extension bit - a single wrong bit made a real frame read causeCode 47 instead of 94. - DenmEvent gains actionID (originatingStationID + sequenceNumber), stationType, termination, detectionTime, relevance radius and RSSI. Dedup keys on actionID where available, so a termination lands on the event it ends instead of creating a second pin. - denmEvents merges the MQTT and over-the-air sources and drops terminated events. The V2X list view now shows hazards above the CAM stations; it previously took no DENM parameter at all, so hazards reached the map but never the list. - DenmParser: the Use Case API sends causeCode as a string enum, so reading it as an Int always yielded null. Testing - DenmAirReceiveTest covers the V2X_RX prefix and the decoder using real frames from a live capture as fixtures. Expected values were cross-checked against the ETSI ASN.1 modules via asn1tools, which agreed on all 1885 decodable DENMs across the capture set, every field including detectionTime. - Verified on hardware: a CiT One HLN-SV DENM decodes as cause 94/0 with a 1000 m relevance radius at 1 Hz alongside CAM, with no decode failures and no unexpected BTP ports. Also replaces em dashes with hyphens throughout the user-facing strings, including the German translation.	2026-08-17 18:42:48 +02:00
Ashin Walpola	33c4ec5998	Phase 03: real CAM UPER codec + ESP32-C5 TX/RX serial link - Firmware: rewrite obu-firmware TX loop to be serial-driven (no on-chip timer), add promiscuous RX + GeoNetworking/BTP unwrap (gn_unwrap.c), add binary UART framing to the phone (serial_link.c/.h). Drop local cam_encode() - CAM is now built on the phone. - Kotlin: byte-exact UPER CAM encoder/decoder ported from cam.c (BitWriter/BitReader/CamUperCodec), matching SerialFrame codec, real UsbSerialTransport (usb-serial-for-android), CamTransmitLoop (1Hz base rate, event/geofence boost, ESP32-C5-only), wired into CamUseCaseRepository for RX and TripRecordingService for TX. - Add V2X message retention: persist all CAM (own+remote) to Room while recording, drop otherwise (DB v2 -> v3 migration). - Add jitpack repo + usb-serial-for-android dependency. Fixes: UsbSerialTransport now uses SerialInputOutputManager.start()/stop() (this lib version manages its own thread internally) instead of manual Runnable/Thread submission, which didn't compile.	2026-08-04 17:58:07 +02:00

Author

SHA1

Message

Date

Ashin Walpola

a5ad3dcc5d

SPATEM receive, RSU CAM decode, and two ASN.1 encoding fixes

SPATEM over the air
- gn_unwrap.c accepts BTP-B port 2004 alongside 2001/2002. The serial protocol
  already carries the port in its V2X_RX prefix, so nothing else changed there.
  Note the crossover that makes this easy to get wrong: SPATEM is port 2004 but
  messageID 4, while MAPEM is port 2003 and messageID 5.
- SpatemUperCodec decodes SPAT down to per-signal-group phase and timing. The
  bit layout was validated by replaying 79,042 real SPATEMs - the whole
  2026-03-18 drive across 7+ RSUs plus the bench trigger - against asn1tools
  using the ETSI modules. All 79,042 matched on every field, none hit an
  unsupported branch. Two traps are pinned by tests: TimeChangeDetails is the
  one SEQUENCE here that is NOT extensible (5 optional bits, no extension bit),
  and maneuverAssistList cannot be skipped when present - it is variable-length,
  so it has to be walked to find where the next movement starts.
- The V2X list shows one row per intersection with each signal group coloured by
  phase and a countdown where the RSU supplies timing. TimeMark wraps hourly, so
  the countdown corrects for it; without that it reads hugely negative once an
  hour, precisely when someone is watching it.
- Entries expire after 15 s, much shorter than DENM's window: a traffic light
  that stopped updating is not "still green".

  Size caveat, deliberately deferred: SERIAL_LINK_MAX_PAYLOAD is still 512, so a
  SPATEM over ~498 bytes is counted as an oversize drop. The bench RSU sends 58
  bytes and is unaffected, but real road RSUs measured 555 median / 1243 max, so
  roughly 70% would not arrive. Raising the cap also requires enlarging
  RX_FRAME_MAX_LEN and moving rx_item_t off the WiFi driver's callback stack,
  where it would otherwise overflow.

RSU CAM decode
- HighFrequencyContainer is a CHOICE, and a roadside unit picks
  rsuContainerHighFrequency, which carries no kinematics at all. The decoder
  bailed on that branch, so every RSU CAM was dropped - including the bench RSU,
  which sends CAM and SPATEM from the same station id. It now decodes for
  position and stationType.
- RSU CAMs are kept out of UseCaseDetectionEngine. They arrive as a permanently
  stationary station at a fixed point, which is exactly the shape the
  stopped-vehicle and intersection-movement use cases match, and would raise a
  standing false alert for as long as the RSU was in range.

CAM transmit: yawRateConfidence
- YawRateConfidence has nine enumerands (0..8), so UPER needs 4 bits and
  "unavailable" is 8. The encoder wrote 3 bits with value 7 - one bit short and
  the wrong symbol - shifting every field after yawRate for any standards-strict
  receiver. The decoder read 3 bits too, so phone and ESP32 agreed with each
  other and with nothing else.
- This is the third instance of that exact failure mode in this project, after
  CurvatureCalculationMode and the GeoNetworking reserved bytes. A round-trip
  test through our own decoder structurally cannot catch it, so CamEncodeGolden
  Test asserts the bytes asn1tools produces instead: it decoded this encoder's
  output and re-encoded it byte-identically. Confirmed on air afterwards - 26 of
  our own CAMs captured back off the OBU's receiver, all 26 accepted, where the
  same decoder rejected them before.

DENM
- Hazards now expire 60 s after their last repetition. This needs a clock, not
  just a filter: both source flows only emit when a DENM arrives, so a sender
  that drives away or loses power would never trigger a recompute and its hazard
  would stay on screen indefinitely.
- The MQTT path was dropping every DENM for two independent reasons, both found
  by checking the payload against CI-CiT-MQTT_API_Documentation-v6 listing 2.6
  rather than guessing: the station id key is originatingStationId, and
  eventPosition IS a GeoJSON Point rather than an object containing one. Also
  parses termination (presence is the signal), sequenceNumber, stationType and
  the RFC3339 detectionTime. Note roadSideUnit is 15, not 12 - the enumeration
  has a gap after tram(11).

V2X screen
- The decoded CAM/DENM list now renders on the CiT One path too; it was gated to
  the ESP32-C5 path and CiT One fell through to the raw MQTT topic list. Those
  topics move to their own tab, hidden on the ESP32-C5 path where there is no
  broker.

Testing
- Adds org.json as a test-only dependency: the android.jar stub throws
  "not mocked" on every JSONObject call, which made the MQTT payload parsers
  untestable off-device.
- 23 V2X tests pass. EventDetectorTest's 4 failures are pre-existing and
  untouched by this change.

2026-08-20 15:47:14 +02:00

Ashin Walpola

0ccb867228

DENM over-the-air receive on the ESP32-C5 path

The firmware forwarded CAM only: gn_unwrap_cam accepted single-hop broadcast
(HT=5) and BTP port 2001, so every DENM was dropped before it reached the phone.
Real OBUs disseminate DENM by GeoBroadcast (HT=4), whose 44-byte extended header
also carries the hazard's relevance area - materially more useful on a map than
the sender's own position, since a sender may be relaying for someone else.

Firmware
- gn_unwrap_cam -> gn_unwrap_its: accepts GeoBroadcast alongside TSB/SHB, and
  BTP ports 2001 and 2002, extracting the GeoBroadcast destination area. Both
  extended-header lengths were measured against live air capture rather than
  read off a spec table. Secured packets (Basic Header NextHeader=2) are
  rejected rather than misparsed.
- SERIAL_MSG_CAM_RX (0x02) superseded by SERIAL_MSG_V2X_RX (0x04): a 14-byte
  prefix carrying BTP port, RSSI and the destination area. Adding MAPEM later
  needs a decoder on the phone but no protocol change. 0x02 stays reserved so
  the numbering is not silently reused.
- Promiscuous RX capture buffer 400 -> 800 bytes. A real GeoBroadcast DENM is
  around 500 bytes on air and was being truncated mid-payload, which no amount
  of correct unwrapping downstream could have recovered from.
- geonet_wrap_shb, both firmwares: the SHB extended header is 28 bytes, not 24.
  The Source Position Vector is followed by a 4-byte reserved field; without it
  a standards-strict receiver reads the CAM payload's first two bytes as the BTP
  destination port.

App
- DenmUperCodec: UPER decoder for the ManagementContainer and the
  SituationContainer's eventType. ValidityDuration is 17 bits, not 16, and
  ManagementContainer, SituationContainer and CauseCode each carry their own
  extension bit - a single wrong bit made a real frame read causeCode 47
  instead of 94.
- DenmEvent gains actionID (originatingStationID + sequenceNumber), stationType,
  termination, detectionTime, relevance radius and RSSI. Dedup keys on actionID
  where available, so a termination lands on the event it ends instead of
  creating a second pin.
- denmEvents merges the MQTT and over-the-air sources and drops terminated
  events. The V2X list view now shows hazards above the CAM stations; it
  previously took no DENM parameter at all, so hazards reached the map but never
  the list.
- DenmParser: the Use Case API sends causeCode as a string enum, so reading it
  as an Int always yielded null.

Testing
- DenmAirReceiveTest covers the V2X_RX prefix and the decoder using real frames
  from a live capture as fixtures. Expected values were cross-checked against
  the ETSI ASN.1 modules via asn1tools, which agreed on all 1885 decodable
  DENMs across the capture set, every field including detectionTime.
- Verified on hardware: a CiT One HLN-SV DENM decodes as cause 94/0 with a
  1000 m relevance radius at 1 Hz alongside CAM, with no decode failures and no
  unexpected BTP ports.

Also replaces em dashes with hyphens throughout the user-facing strings,
including the German translation.

2026-08-17 18:42:48 +02:00

Ashin Walpola

33c4ec5998

Phase 03: real CAM UPER codec + ESP32-C5 TX/RX serial link

- Firmware: rewrite obu-firmware TX loop to be serial-driven (no on-chip timer), add promiscuous RX + GeoNetworking/BTP unwrap (gn_unwrap.c), add binary UART framing to the phone (serial_link.c/.h). Drop local cam_encode() - CAM is now built on the phone.
- Kotlin: byte-exact UPER CAM encoder/decoder ported from cam.c (BitWriter/BitReader/CamUperCodec), matching SerialFrame codec, real UsbSerialTransport (usb-serial-for-android), CamTransmitLoop (1Hz base rate, event/geofence boost, ESP32-C5-only), wired into CamUseCaseRepository for RX and TripRecordingService for TX.
- Add V2X message retention: persist all CAM (own+remote) to Room while recording, drop otherwise (DB v2 -> v3 migration).
- Add jitpack repo + usb-serial-for-android dependency.

Fixes: UsbSerialTransport now uses SerialInputOutputManager.start()/stop() (this lib version manages its own thread internally) instead of manual Runnable/Thread submission, which didn't compile.

2026-08-04 17:58:07 +02:00