DENM over-the-air receive on the ESP32-C5 path
The firmware forwarded CAM only: gn_unwrap_cam accepted single-hop broadcast (HT=5) and BTP port 2001, so every DENM was dropped before it reached the phone. Real OBUs disseminate DENM by GeoBroadcast (HT=4), whose 44-byte extended header also carries the hazard's relevance area - materially more useful on a map than the sender's own position, since a sender may be relaying for someone else. Firmware - gn_unwrap_cam -> gn_unwrap_its: accepts GeoBroadcast alongside TSB/SHB, and BTP ports 2001 and 2002, extracting the GeoBroadcast destination area. Both extended-header lengths were measured against live air capture rather than read off a spec table. Secured packets (Basic Header NextHeader=2) are rejected rather than misparsed. - SERIAL_MSG_CAM_RX (0x02) superseded by SERIAL_MSG_V2X_RX (0x04): a 14-byte prefix carrying BTP port, RSSI and the destination area. Adding MAPEM later needs a decoder on the phone but no protocol change. 0x02 stays reserved so the numbering is not silently reused. - Promiscuous RX capture buffer 400 -> 800 bytes. A real GeoBroadcast DENM is around 500 bytes on air and was being truncated mid-payload, which no amount of correct unwrapping downstream could have recovered from. - geonet_wrap_shb, both firmwares: the SHB extended header is 28 bytes, not 24. The Source Position Vector is followed by a 4-byte reserved field; without it a standards-strict receiver reads the CAM payload's first two bytes as the BTP destination port. App - DenmUperCodec: UPER decoder for the ManagementContainer and the SituationContainer's eventType. ValidityDuration is 17 bits, not 16, and ManagementContainer, SituationContainer and CauseCode each carry their own extension bit - a single wrong bit made a real frame read causeCode 47 instead of 94. - DenmEvent gains actionID (originatingStationID + sequenceNumber), stationType, termination, detectionTime, relevance radius and RSSI. Dedup keys on actionID where available, so a termination lands on the event it ends instead of creating a second pin. - denmEvents merges the MQTT and over-the-air sources and drops terminated events. The V2X list view now shows hazards above the CAM stations; it previously took no DENM parameter at all, so hazards reached the map but never the list. - DenmParser: the Use Case API sends causeCode as a string enum, so reading it as an Int always yielded null. Testing - DenmAirReceiveTest covers the V2X_RX prefix and the decoder using real frames from a live capture as fixtures. Expected values were cross-checked against the ETSI ASN.1 modules via asn1tools, which agreed on all 1885 decodable DENMs across the capture set, every field including detectionTime. - Verified on hardware: a CiT One HLN-SV DENM decodes as cause 94/0 with a 1000 m relevance radius at 1 Hz alongside CAM, with no decode failures and no unexpected BTP ports. Also replaces em dashes with hyphens throughout the user-facing strings, including the German translation.
This commit is contained in:
Ashin Walpola
@@ -0,0 +1,199 @@
|
||||
package com.hawhamburg.micr0bu
|
||||
|
||||
import com.hawhamburg.micr0bu.data.transport.BtpPort
|
||||
import com.hawhamburg.micr0bu.data.transport.V2xRxFrame
|
||||
import com.hawhamburg.micr0bu.domain.asn1.DenmUperCodec
|
||||
import org.junit.Assert.assertEquals
|
||||
import org.junit.Assert.assertFalse
|
||||
import org.junit.Assert.assertNotNull
|
||||
import org.junit.Assert.assertNull
|
||||
import org.junit.Assert.assertTrue
|
||||
import org.junit.Test
|
||||
|
||||
/**
|
||||
* Regression tests for the ESP32-C5 over-the-air receive path: the `V2X_RX` serial payload layout
|
||||
* and [DenmUperCodec].
|
||||
*
|
||||
* ## Where the fixtures come from
|
||||
* These are **real frames**, not hand-built ones. They were taken from
|
||||
* `its-g5-receiver-firmware/recordings/capture_20260817_171055.pcap` — a live capture of a CiT One
|
||||
* OBU running the HLN-SV use case — by replaying the capture through a port of the firmware's
|
||||
* `gn_unwrap_its()` and `serial_link_send_v2x_rx()`, so each fixture is byte-for-byte what the
|
||||
* ESP32-C5 hands the phone over USB. The capture's 4-byte 802.11 FCS is trimmed, because the WiFi
|
||||
* driver strips it before the promiscuous callback ever sees the frame.
|
||||
*
|
||||
* ## Why the expected values can be trusted
|
||||
* Every asserted field was cross-checked against `asn1tools` decoding the same bytes with the real
|
||||
* ETSI modules from the `C-ITS-Parser` checkout (`denm_1_3_1.asn` + `cdd_1_3_1_1.asn`) — an
|
||||
* independent implementation, not this codebase's own arithmetic. Across the full capture set that
|
||||
* cross-check agreed on all 1885 decodable DENMs, on every field below including `detectionTime`.
|
||||
*
|
||||
* That matters because this project has twice shipped a UPER bug that was invisible until measured
|
||||
* against real traffic (a one-bit `CurvatureCalculationMode` in CAM, and a 16-vs-17-bit
|
||||
* `ValidityDuration` here that made a real frame read causeCode 47 instead of 94). Hand-built
|
||||
* fixtures would have happily reproduced both. If a field width is ever "cleaned up", these tests
|
||||
* are what should fail.
|
||||
*/
|
||||
class DenmAirReceiveTest {
|
||||
|
||||
// ---- real captured V2X_RX serial payloads ----------------------------------------------
|
||||
|
||||
/**
|
||||
* An active DENM: GeoBroadcast, BTP port 2002, stationaryVehicle (94/0), 1000 m relevance
|
||||
* radius. 406-byte UPER message — most of it the AlacarteContainer this decoder deliberately
|
||||
* stops before reading.
|
||||
*/
|
||||
private val ACTIVE_V2X_RX =
|
||||
"d207c10102bfeb1f7c53f905e8030201fa012bd0e77d0095e8000314c8317dba65320c5f6ff5590a8027143257c1dd1d" +
|
||||
"d0001970898000781432f0030008b9f1be8a2fe943f9e6d390895181e3603696f542543bf04d0052201c02d9df83d7f6" +
|
||||
"e159a88c4f016c402b2d548063f814fa02d66d24044bc0f2d018fb8cb0275e07d480681e550595eff16bfc5cc4e0040f" +
|
||||
"80989ffe970e40d0bbffceffdcb1e20045dffe5802e184200bdefd313f9f4b3b008977e571fb0c58c00d1fc0ed301a7b" +
|
||||
"5840121e04a380d518ce008beffa8c0044c2dc018f7f80a00775bf401dfbf39500fda4ea038de000d800a18970036f02" +
|
||||
"7fc01fee0b006b781ee6007a7e2c026bbfc0eff7e2f6c012bdf86b7f4953a1018af014b4004cd890031f8088200fb67c" +
|
||||
"0018fc00e4ffe4328100efe024e7ff41afe0090f00bc3fff8d1880207802e1ffcd666a00c7c082aff763c1e017bdffe1" +
|
||||
"7fee59230167efd073fb70abe005ef7f31dffaf5a3c05b3bff6effecb16a0063e0036804dd8380077efe833ff62b7c00" +
|
||||
"4d77ee4200a249c010dfc0f84fecfc3f808d3dfbb47fa795fe007cc0e1178000f9010000"
|
||||
|
||||
/**
|
||||
* The termination of that same event, sent once when the hazard ended. Note it carries **no
|
||||
* SituationContainer** — a terminating DENM says "event over", not what the event was — so
|
||||
* `causeCode` is legitimately null here and code must not treat that as a decode failure.
|
||||
*/
|
||||
private val TERMINATION_V2X_RX =
|
||||
"d207c101b6c6eb1f2158f905e8030201fa012bd00f7d0095e8000314c8329f58e5320ca7d792ac857db38a1951098498" +
|
||||
"48000ccd7d40003c0a00000019"
|
||||
|
||||
/** A real CAM from the same capture, for checking the port routing rejects non-DENM cleanly. */
|
||||
private val CAM_V2X_RX =
|
||||
"d107c100000000000000000000000202fa012bd0f8e8605ab214f9ae2864b2415015000032c950487c1fa0010ebfe9ea" +
|
||||
"7b33ff01fffa0028331400fbfab8fe6eb5a222ebe078d80da5bd50950efc134014880980b677e0f5fdb856542313c05b" +
|
||||
"100acb552018fe053f80b59b490112f03cb4063ee33009d781f5201a079541657bfc5aff1731380103e02627ffa5c390" +
|
||||
"342efff3bff72c7b001177ff9600b8610802f7bf4c4fe7d2ce20225df95c7ec316300347f03b4c069ed6100486900000" +
|
||||
"0801"
|
||||
|
||||
/** Later than every fixture's detectionTime, so the decoder's sanity window accepts them. */
|
||||
private val receivedAt = 1_787_100_000_000L
|
||||
|
||||
private fun String.hexToBytes(): ByteArray =
|
||||
chunked(2).map { it.toInt(16).toByte() }.toByteArray()
|
||||
|
||||
// ---- the V2X_RX prefix contract --------------------------------------------------------
|
||||
|
||||
@Test
|
||||
fun `active DENM frame parses its metadata prefix`() {
|
||||
val frame = V2xRxFrame.parse(ACTIVE_V2X_RX.hexToBytes())
|
||||
assertNotNull(frame)
|
||||
frame!!
|
||||
|
||||
assertEquals(BtpPort.DENM, frame.btpPort)
|
||||
assertEquals(-63, frame.rssiDbm) // int8: must survive as negative, not 193
|
||||
|
||||
// GeoBroadcast destination area, converted from GeoNetworking's big-endian 1/10 microdegree
|
||||
// to the little-endian prefix and back out again.
|
||||
val area = frame.geoArea
|
||||
assertNotNull(area)
|
||||
assertEquals(53.5543554, area!!.latitude, 1e-7)
|
||||
assertEquals(10.0225916, area.longitude, 1e-7)
|
||||
assertEquals(1000, area.radiusMeters)
|
||||
|
||||
assertEquals(406, frame.uper.size)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `frame no larger than the firmware's serial payload cap`() {
|
||||
// SERIAL_LINK_MAX_PAYLOAD is 512 on both sides; the firmware counts an oversize drop rather
|
||||
// than truncating. A real GeoBroadcast DENM is the largest thing this path carries today.
|
||||
assertTrue(
|
||||
"real DENM V2X_RX payload must fit SERIAL_LINK_MAX_PAYLOAD",
|
||||
ACTIVE_V2X_RX.hexToBytes().size <= 512,
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `parse rejects a payload with no room for a message`() {
|
||||
assertNull(V2xRxFrame.parse(ByteArray(V2xRxFrame.PREFIX_SIZE)))
|
||||
assertNull(V2xRxFrame.parse(ByteArray(3)))
|
||||
}
|
||||
|
||||
// ---- DENM decode ----------------------------------------------------------------------
|
||||
|
||||
@Test
|
||||
fun `decodes a real stationaryVehicle DENM`() {
|
||||
val frame = V2xRxFrame.parse(ACTIVE_V2X_RX.hexToBytes())!!
|
||||
val denm = DenmUperCodec.decode(
|
||||
bytes = frame.uper,
|
||||
receivedAtEpochMs = receivedAt,
|
||||
rssiDbm = frame.rssiDbm,
|
||||
relevanceRadiusM = frame.geoArea?.radiusMeters,
|
||||
)
|
||||
assertNotNull("real captured DENM must decode", denm)
|
||||
denm!!
|
||||
|
||||
// actionID - the ETSI event identity, cross-checked against asn1tools.
|
||||
assertEquals(4_194_380_752L, denm.stationId)
|
||||
assertEquals(6, denm.sequenceNumber)
|
||||
|
||||
assertEquals(53.5543554, denm.latitude, 1e-7)
|
||||
assertEquals(10.0225916, denm.longitude, 1e-7)
|
||||
|
||||
assertEquals(94, denm.causeCode) // stationaryVehicle
|
||||
assertEquals(0, denm.subCauseCode)
|
||||
assertEquals(5, denm.stationType) // passengerCar
|
||||
|
||||
assertFalse(denm.isTermination)
|
||||
|
||||
// detectionTime is a 42-bit TimestampIts counted from the 2004 ITS epoch. Getting either
|
||||
// the width or the epoch wrong lands the hazard decades away, so the absolute value is
|
||||
// asserted rather than a range.
|
||||
assertEquals(1_786_979_460_563L, denm.detectionTimeMs)
|
||||
|
||||
// Carried through from the GeoNetworking header and the serial prefix, not the payload.
|
||||
assertEquals(1000, denm.relevanceRadiusM)
|
||||
assertEquals(-63, denm.rssiDbm)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `decodes a termination DENM and keeps the same event identity`() {
|
||||
val active = V2xRxFrame.parse(ACTIVE_V2X_RX.hexToBytes())!!
|
||||
val term = V2xRxFrame.parse(TERMINATION_V2X_RX.hexToBytes())!!
|
||||
|
||||
val activeDenm = DenmUperCodec.decode(active.uper, receivedAt)!!
|
||||
val termDenm = DenmUperCodec.decode(term.uper, receivedAt)!!
|
||||
|
||||
assertTrue(termDenm.isTermination)
|
||||
assertNull("a terminating DENM carries no SituationContainer", termDenm.causeCode)
|
||||
assertEquals(4_194_380_752L, termDenm.stationId)
|
||||
assertEquals(6, termDenm.sequenceNumber)
|
||||
assertEquals(1_786_980_053_703L, termDenm.detectionTimeMs)
|
||||
|
||||
// The whole point of keying dedup on actionID: the termination must land on the same key as
|
||||
// the event it ends, so filtering terminations actually removes that hazard from the map
|
||||
// instead of leaving the active pin behind next to a hidden one.
|
||||
assertEquals(activeDenm.dedupKey, termDenm.dedupKey)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `does not decode a CAM as a DENM`() {
|
||||
val cam = V2xRxFrame.parse(CAM_V2X_RX.hexToBytes())!!
|
||||
assertEquals(BtpPort.CAM, cam.btpPort)
|
||||
assertNull("CAM must not decode as DENM - messageID guards this", DenmUperCodec.decode(cam.uper, receivedAt))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `returns null for a truncated DENM rather than a misplaced hazard`() {
|
||||
val frame = V2xRxFrame.parse(ACTIVE_V2X_RX.hexToBytes())!!
|
||||
// Cut inside the ManagementContainer: the BitReader runs out mid-field.
|
||||
assertNull(DenmUperCodec.decode(frame.uper.copyOfRange(0, 12), receivedAt))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `future detection time beyond the sanity window is dropped, not surfaced`() {
|
||||
val frame = V2xRxFrame.parse(ACTIVE_V2X_RX.hexToBytes())!!
|
||||
// A phone whose clock is more than a day behind the sender: the event still decodes, but
|
||||
// the implausible timestamp is reported as unknown instead of being shown.
|
||||
val denm = DenmUperCodec.decode(frame.uper, receivedAtEpochMs = 1_700_000_000_000L)
|
||||
assertNotNull(denm)
|
||||
assertEquals(94, denm!!.causeCode)
|
||||
assertNull(denm.detectionTimeMs)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user