Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Ashin Walpola
|
73d4477f1e | ||
|
Ashin Walpola
|
277c6b20f4 |
@@ -46,3 +46,6 @@ sdkconfig.old
|
||||
# Office lock files. Word/Excel create these beside a document while it is open
|
||||
# and remove them on close, so they are transient and machine-local.
|
||||
~$*
|
||||
|
||||
# Captures are large data files, not source (see capture/README.md).
|
||||
/capture/recordings/
|
||||
|
||||
@@ -9,20 +9,24 @@ Engineering to-do list. The reviewer-facing open items live in
|
||||
|
||||
`geonet.c` now writes GN lifetime `0x05` (1 s) instead of `0x83`, which decoded to 3200 s. Changed
|
||||
in both `obu-firmware` and `obu-cam-transmistter`. Both still build (IDF 6.1 / 5.5.4), and the
|
||||
compiled `geonet_wrap_shb` stores the new byte, but it has not been seen on air yet. Nothing else
|
||||
compiled `geonet_wrap_shb` stores the new byte. Confirmed on air 2026-09-14. Nothing else
|
||||
reads this byte (`gn_unwrap.c` ignores it, the app never sees GN headers), so the app does not
|
||||
need updating alongside the firmware.
|
||||
|
||||
Needs: the phone with the app, the OBU ESP32-C5, and a **second** ESP32-C5 running
|
||||
`its-g5-receiver-firmware` to capture with.
|
||||
|
||||
- [ ] Flash `obu-firmware` (see `obu-firmware/FLASHING.md`).
|
||||
- [ ] Connect the phone, let it send CAMs, and confirm the CAM Pinger's `tx fail` counter stays 0.
|
||||
- [ ] Capture with the receiver into `its-g5-receiver-firmware/recordings/`.
|
||||
- [ ] Run `python obu-firmware/test/pcap_gn_tally.py its-g5-receiver-firmware/recordings/<capture>.pcap`.
|
||||
The rows for the phone's pseudonym MACs must show SHB, port 2001, lifetime `0x05`, exactly
|
||||
like every other station's CAMs.
|
||||
- [ ] While the phone is connected: real-station CAMs/DENMs still reach the app (RX path unchanged).
|
||||
- [x] Flash `obu-firmware` (done 2026-09-14 on COM3; flash backed up first to
|
||||
`Documents/micrOBU_workspace/firmware-backups/COM3-2026-09-14-before-secured-rx.bin`).
|
||||
- [x] Capture with the receiver (COM8) into `its-g5-receiver-firmware/recordings/`.
|
||||
- [x] `pcap_gn_tally.py` on capture_20260914_132126.pcap: our station sends SHB, port 2001,
|
||||
lifetime `0x05`, same as both bench stations. It was `0x83` in the August captures.
|
||||
- [x] Real-station CAMs/DENMs/SPATEM still reach the app (logcat: `handleCamUper`,
|
||||
`handleDenmUper`, `handleSpatUper` all decoding, 2026-09-14).
|
||||
- [x] Our own CAMs decode on air: 397 frames from station 999999 decode with asn1tools and
|
||||
re-encode byte-identically.
|
||||
- [ ] Confirm the CAM Pinger card's `tx fail` / oversize / CRC counters are 0 (needs a look at the
|
||||
phone; not readable from the PC).
|
||||
|
||||
Partial check possible with one board and no phone: flash it, `idf.py -p COMx monitor`, and look
|
||||
for `OCB @ 5900 MHz - TX/RX armed`. That proves the new build boots and brings the radio up, not
|
||||
@@ -33,11 +37,12 @@ that it transmits correctly.
|
||||
Its `cam.c` (compiled into that firmware) wrote `yawRateConfidence` as 3 bits / 7 instead of
|
||||
4 bits / unavailable(8), the bug the app fixed on 2026-08-20. Fixed in it and in obu-firmware's
|
||||
reference copy; asn1tools now decodes the CAM and re-encodes it byte-identically, and it builds on
|
||||
IDF 5.5.4. No board runs this firmware right now (the production OBU runs obu-firmware), so this
|
||||
only matters if it is flashed again:
|
||||
IDF 5.5.4. Since 2026-09-14 the spare board on COM10 runs it as a bench beacon:
|
||||
|
||||
- [ ] After flashing it: capture, run `pcap_gn_tally.py`, and decode the CAM payload with
|
||||
asn1tools (`py -3.11`, modules in `asn1/`).
|
||||
- [x] Done 2026-09-14: flashed on COM10 and captured on COM8. All 72 CAMs from station
|
||||
195936478 (0x0BADC0DE) decode with asn1tools and re-encode byte-identically, so the
|
||||
4-bit yawRateConfidence is right on air. COM10 now runs this beacon rather than
|
||||
obu-firmware - reflash it if the spare is needed as an OBU again.
|
||||
|
||||
### Signed-message reception and exact payloads (added 2026-09-11)
|
||||
|
||||
@@ -45,17 +50,21 @@ obu-firmware's `gn_unwrap.c` now unwraps TS 103 097 signed packets (signature no
|
||||
reported as V2X_RX flags bit1) and cuts every message to the length its header declares, dropping
|
||||
the 8 bytes the chip's RX appends to each frame, which were forwarded to the phone until now.
|
||||
Verified on the host (`obu-firmware/test/host`: chain, replay of all recordings against asn1tools,
|
||||
50M-iteration fuzz) and built on IDF 6.1, but not flashed: the production OBU still runs the
|
||||
2026-09-10 build. Needs the OBU with this build, the phone, and signed traffic - real vehicles or
|
||||
50M-iteration fuzz) and flashed to the production OBU on 2026-09-14. The remaining gap is signed
|
||||
traffic to receive: real vehicles or
|
||||
RSUs, since the bench CiT One sends unsigned. A second ESP32 running the receiver firmware is
|
||||
optional, but shows what was on air at the time.
|
||||
|
||||
- [ ] Flash obu-firmware (this also carries the GN lifetime fix above).
|
||||
- [x] Flash obu-firmware (done 2026-09-14, COM3).
|
||||
- [x] Unsigned bench traffic still decodes in the app, with messages now cut to their declared
|
||||
length (CAM, DENM and SPATEM all decoding in logcat after the flash).
|
||||
- [ ] Near signed traffic: signed CAMs/DENMs appear in the app, and a simultaneous capture shows
|
||||
them on air (`pcap_gn_tally.py` lists them as `secured`).
|
||||
- [ ] Unsigned bench traffic still decodes in the app as before (messages now arrive 8 bytes
|
||||
shorter).
|
||||
- [ ] The heartbeat's oversize counter still counts over-long messages (e.g. road SPATEMs).
|
||||
them on air (`pcap_gn_tally.py` lists them as `secured`). NOT possible at this bench: the
|
||||
CiT One transmits unsigned (`ItsGnSecurity = 0`) and nothing else here signs. Needs a drive
|
||||
past real RSUs, the CiT One switched to signed mode if its API allows, or a replay firmware
|
||||
on a spare board that re-transmits the recorded signed frames.
|
||||
- [ ] The heartbeat's oversize counter still counts over-long messages. Not exercised at the
|
||||
bench: the SPATEMs here are ~340 bytes on air, far below the cap.
|
||||
|
||||
## Set up host testing
|
||||
|
||||
@@ -92,6 +101,19 @@ Suggested order after the host tests exist:
|
||||
Dropped: building vanetza as a GN/BTP oracle. Real captures (`pcap_gn_tally.py`), the host
|
||||
round-trip test and `asn1tools` for UPER cover what it would have checked.
|
||||
|
||||
## Follow-ups found 2026-09-14
|
||||
|
||||
- [x] **Capture tooling moved into this repo** (`capture/`), with the CR-insertion fix. The
|
||||
sniffer's console inserts a CR before every LF, which also hits every 0x0a byte of the binary
|
||||
pcap stream, shifting pcap record headers and frames. A 787 KB capture parsed cleanly for
|
||||
only 82 of ~2000 records, and DENMs showed up on nonsense BTP ports. `undo_crlf()` reverses
|
||||
it on the raw stream before framing; afterwards a capture parsed to EOF and DENMs read as
|
||||
port 2002. **Every capture taken before 2026-09-14 is truncated at its first corrupted
|
||||
record** - re-measure anything derived from them.
|
||||
- [ ] `capture/dump_pcap.py` reads the same console and still needs the same treatment.
|
||||
- [ ] **Do not open COM3's console while the phone is attached.** Opening it toggles DTR/RTS on the
|
||||
CH343 and resets the OBU, which drops the phone's USB link and needs a manual Connect.
|
||||
|
||||
## Follow-ups found 2026-09-11
|
||||
|
||||
- [ ] **App: show the signed flag.** `V2xRxFrame.parse` in `SerialFrame.kt` only reads bit0 of
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
# Sniffer board and capture tooling
|
||||
|
||||
How to put an ESP32-C5 on the ITS-G5 channel as a passive sniffer, pull its captures onto this
|
||||
PC, and check what is on air. The sniffer firmware itself is the third-party
|
||||
`its-g5-receiver-firmware` checkout beside this repo; only the tooling and these notes are ours.
|
||||
|
||||
| File | What it does |
|
||||
|---|---|
|
||||
| `live_capture.py` | Streams the device's captures into a growing `.pcap` while it runs. The usual choice. |
|
||||
| `dump_pcap.py` | Pulls one capture out of the device's in-memory buffer after the fact. |
|
||||
| `../obu-firmware/test/pcap_gn_tally.py` | Tallies GeoNetworking headers per station over a `.pcap`. |
|
||||
|
||||
One-time: `pip install pyserial` (present in Python 3.11 on the bench PC, so `py -3.11` works).
|
||||
|
||||
## Which port
|
||||
|
||||
The sniffer firmware's console, and with it the pcap stream, goes out **UART0** - the board's
|
||||
USB-bridge port (a CH343, its own COM number), not the native USB-C port. A board with only one
|
||||
USB-C port cannot be used as a sniffer for this reason. On the bench this has been COM5 and, after
|
||||
a re-enumeration, COM8.
|
||||
|
||||
## Live capture (preferred)
|
||||
|
||||
```powershell
|
||||
cd capture
|
||||
py -3.11 live_capture.py COM8
|
||||
```
|
||||
|
||||
It writes `recordings/capture_<timestamp>.pcap` next to itself, flushing after every packet, so
|
||||
the file can be read while it grows. Stop it with Ctrl+C. Use `-o <dir>` to write elsewhere;
|
||||
`recordings/` is gitignored, since captures are large and are data rather than source. Captures
|
||||
taken before 2026-09-14 are still in `its-g5-receiver-firmware/recordings/`; the host tests read
|
||||
both directories.
|
||||
|
||||
### The CR insertion, and why captures used to be corrupt
|
||||
|
||||
ESP-IDF's newlib console converts LF to CRLF on its way out, and that applies to every `0x0a` byte
|
||||
of the **binary** pcap stream, not only to log text. Each inserted CR shifts everything after it,
|
||||
so pcap record headers and captured frames alike come out corrupt, and the file stops being
|
||||
parseable at the first occurrence.
|
||||
|
||||
Measured on 2026-09-14: a 787 KB capture parsed cleanly for only 82 of about 2000 records, and
|
||||
DENMs appeared on nonsense BTP ports because their payloads contain `0x0a` often. `undo_crlf()` in
|
||||
`live_capture.py` reverses it on the raw stream before any framing, which is exact; afterwards a
|
||||
capture parsed to EOF and DENMs read as port 2002 again.
|
||||
|
||||
**Captures taken before 2026-09-14 are truncated at their first corrupted record.** Anything
|
||||
measured from them is worth re-checking. `dump_pcap.py` reads the same console and has not been
|
||||
given the same treatment yet.
|
||||
|
||||
## Checking a capture
|
||||
|
||||
```powershell
|
||||
py -3.11 ..\obu-firmware\test\pcap_gn_tally.py recordings\capture_<timestamp>.pcap
|
||||
```
|
||||
|
||||
One row per station, packet type, BTP port and GN lifetime. For the messages themselves, decode
|
||||
the payloads with `asn1tools` against the modules in `../asn1/` and re-encode them: identical bytes
|
||||
mean the message was read exactly, wrong bytes mean it was not. `obu-firmware/test/check_replay.py`
|
||||
does this over a whole capture.
|
||||
|
||||
## Flashing the sniffer firmware
|
||||
|
||||
From the receiver checkout, with its **pinned** ESP-IDF (not the global 5.5.4 install):
|
||||
|
||||
```powershell
|
||||
cd its-g5-receiver-firmware
|
||||
git submodule update --init --recursive
|
||||
.\esp-idf\install.bat
|
||||
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
|
||||
.\esp-idf\export.ps1
|
||||
idf.py set-target esp32c5
|
||||
idf.py -p COM8 -b 921600 flash
|
||||
```
|
||||
|
||||
Its `sdkconfig` for a bare board (nothing wired) needs SPI Ethernet off, and the pcap destination
|
||||
set to Memory, both under `idf.py menuconfig`. Wired variants have ready-made configs in that
|
||||
checkout: `sdkconfig.proto-w5500`, `sdkconfig.proto-enc28j60`, `sdkconfig.proto-spi-eppp`.
|
||||
|
||||
To reflash a board that already has a built image, without a toolchain terminal:
|
||||
|
||||
```powershell
|
||||
cd its-g5-receiver-firmware\build
|
||||
C:\Espressif\python_env\idf5.5_py3.11_env\Scripts\python.exe -m esptool --chip esp32c5 -p COM8 -b 921600 write_flash --flash_mode dio --flash_freq 80m --flash_size 16MB 0x2000 bootloader/bootloader.bin 0x8000 partition_table/partition-table.bin 0x1e000 ota_data_initial.bin 0x20000 its-g5-receiver-firmware.bin
|
||||
```
|
||||
|
||||
## Pulling a capture after the fact
|
||||
|
||||
Only for the Memory destination, and the buffer is small (`SNIFFER_PCAP_MEMORY_SIZE`, 4096 bytes
|
||||
by default) - a smoke test, not a session. In the device console (`idf.py -p COM8 monitor`, exit
|
||||
with Ctrl+T then Ctrl+X):
|
||||
|
||||
```
|
||||
sniffer -P
|
||||
sniffer --stop
|
||||
```
|
||||
|
||||
Then, with the port free:
|
||||
|
||||
```powershell
|
||||
py -3.11 dump_pcap.py COM8
|
||||
```
|
||||
@@ -0,0 +1,101 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Pulls a capture off the ITS-G5 receiver's in-memory pcap buffer over the existing USB serial
|
||||
connection and saves it as a real .pcap file on this machine.
|
||||
|
||||
Requires the firmware to be built with:
|
||||
Example Configuration -> Select destination to store pcap file -> Memory
|
||||
|
||||
Usage (typical):
|
||||
1. Close idf.py monitor (only one program can hold the COM port at a time).
|
||||
2. Run a capture on the device: `sniffer -P` ... let it run ... `sniffer --stop`
|
||||
3. python dump_pcap.py COM5
|
||||
|
||||
The device has no access to this computer's filesystem, so it can't write here directly. Instead,
|
||||
`pcap --dump` streams the raw pcap bytes back over the same serial link, wrapped in plain-text
|
||||
markers ("===PCAP-DUMP-START:<len>===" ... raw bytes ... "===PCAP-DUMP-END==="). This script finds
|
||||
those markers and writes just the raw bytes out as a .pcap file.
|
||||
|
||||
Install dependency once: pip install pyserial
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import datetime
|
||||
import re
|
||||
import sys
|
||||
|
||||
try:
|
||||
import serial
|
||||
except ImportError:
|
||||
print("Missing dependency. Install it with: pip install pyserial", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
START_RE = re.compile(rb"===PCAP-DUMP-START:(\d+)===\n")
|
||||
END_MARKER = b"\n===PCAP-DUMP-END===\n"
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("port", help="Serial port the device is on, e.g. COM5")
|
||||
parser.add_argument("-b", "--baud", type=int, default=115200, help="Baud rate (default: 115200)")
|
||||
parser.add_argument("-o", "--outdir", default="recordings", help="Output directory (default: ./recordings)")
|
||||
parser.add_argument("-t", "--timeout", type=float, default=15.0, help="Seconds to wait for the dump to start")
|
||||
args = parser.parse_args()
|
||||
|
||||
import os
|
||||
os.makedirs(args.outdir, exist_ok=True)
|
||||
|
||||
print(f"Opening {args.port} @ {args.baud}...")
|
||||
with serial.Serial(args.port, args.baud, timeout=1) as ser:
|
||||
# Nudge the console in case there's stale input, then request the dump.
|
||||
ser.reset_input_buffer()
|
||||
ser.write(b"\r\n")
|
||||
ser.write(b"pcap -f dump --dump\r\n")
|
||||
|
||||
print("Waiting for dump to start...")
|
||||
buf = b""
|
||||
match = None
|
||||
deadline = datetime.datetime.now() + datetime.timedelta(seconds=args.timeout)
|
||||
while datetime.datetime.now() < deadline:
|
||||
chunk = ser.read(256)
|
||||
if chunk:
|
||||
buf += chunk
|
||||
match = START_RE.search(buf)
|
||||
if match:
|
||||
break
|
||||
if not match:
|
||||
print("Timed out waiting for '===PCAP-DUMP-START:...===' marker.\n"
|
||||
"Check that: the firmware is built with the Memory pcap destination, a capture was\n"
|
||||
"actually taken ('sniffer -P' then 'sniffer --stop'), and no other program (like\n"
|
||||
"idf.py monitor) is holding the serial port open.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
length = int(match.group(1))
|
||||
print(f"Dump starting, {length} bytes expected.")
|
||||
|
||||
# Anything after the marker in our buffer is already part of the payload.
|
||||
payload = buf[match.end():]
|
||||
remaining = length - len(payload)
|
||||
while remaining > 0:
|
||||
chunk = ser.read(min(remaining, 4096))
|
||||
if not chunk:
|
||||
print(f"Serial read timed out with {remaining} bytes still missing.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
payload += chunk
|
||||
remaining -= len(chunk)
|
||||
|
||||
# Drain (and sanity-check) the trailing end marker, but don't fail hard if it's not exact.
|
||||
tail = ser.read(len(END_MARKER))
|
||||
if tail != END_MARKER:
|
||||
print("Warning: end marker didn't match exactly - payload may still be fine.", file=sys.stderr)
|
||||
|
||||
timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
outpath = os.path.join(args.outdir, f"capture_{timestamp}.pcap")
|
||||
with open(outpath, "wb") as f:
|
||||
f.write(payload)
|
||||
|
||||
print(f"Saved {len(payload)} bytes to {outpath}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,226 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Continuously listens on the ITS-G5 receiver's serial console and writes every captured packet into a
|
||||
live-growing .pcap file, with no console commands needed on the device side.
|
||||
|
||||
The firmware streams every packet it captures out over the same serial connection the console runs on,
|
||||
automatically, as soon as the sniffer is running (which happens on boot by default). Each packet is framed
|
||||
with plain-text markers so this script can pull the binary pcap bytes out of the stream even though
|
||||
regular log lines are interleaved with it:
|
||||
|
||||
===PCAP-LIVE-HEADER:<len>===\\n<24 raw bytes>\\n (sent once, the pcap global header)
|
||||
===PCAP-LIVE-PKT:<len>===\\n<raw bytes>\\n (sent once per captured packet)
|
||||
|
||||
Usage:
|
||||
python live_capture.py COM5
|
||||
|
||||
Runs until you press Ctrl+C. Writes to recordings/capture_<timestamp>.pcap, flushing after every packet
|
||||
so you can open the file in Wireshark while it's still being written (use "File > Open" again, or
|
||||
Wireshark's own "Follow" won't auto-refresh but re-opening will show the latest packets).
|
||||
|
||||
Install dependency once: pip install pyserial
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import datetime
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
|
||||
try:
|
||||
import serial
|
||||
except ImportError:
|
||||
print("Missing dependency. Install it with: pip install pyserial", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# \r? because the ESP console emits CRLF: on Windows the markers arrive as
|
||||
# "===PCAP-LIVE-PKT:310===\r\n", which never matched a bare \n and left the capture silently
|
||||
# empty while the device was streaming perfectly well.
|
||||
HEADER_RE = re.compile(rb"===PCAP-LIVE-HEADER:(\d+)===\r?\n")
|
||||
PKT_RE = re.compile(rb"===PCAP-LIVE-PKT:(\d+)===\r?\n")
|
||||
|
||||
LINKTYPE_ETHERNET = 1
|
||||
LINKTYPE_IEEE802_11_RADIOTAP = 127
|
||||
|
||||
|
||||
def mac_str(b):
|
||||
return ":".join(f"{x:02x}" for x in b)
|
||||
|
||||
|
||||
def build_default_pcap_header(link_type):
|
||||
"""Synthesizes the same 24-byte global pcap header the firmware would have sent, for when we
|
||||
connect after the device's one-time header already went out (see the race note in main())."""
|
||||
header = bytearray(24)
|
||||
header[0:4] = bytes([0xD4, 0xC3, 0xB2, 0xA1]) # magic (LE bytes of 0xA1B2C3D4)
|
||||
header[4:6] = (2).to_bytes(2, "little") # major version
|
||||
header[6:8] = (4).to_bytes(2, "little") # minor version
|
||||
header[16:20] = (0x40000).to_bytes(4, "little") # snaplen
|
||||
header[20:24] = link_type.to_bytes(4, "little")
|
||||
return bytes(header)
|
||||
|
||||
|
||||
def summarize_packet(link_type, record_bytes, index):
|
||||
"""Best-effort human-readable one-line summary of a captured packet, for live feedback.
|
||||
record_bytes is the raw 16-byte pcap record header followed by the captured frame."""
|
||||
seconds = int.from_bytes(record_bytes[0:4], "little")
|
||||
microseconds = int.from_bytes(record_bytes[4:8], "little")
|
||||
cap_len = int.from_bytes(record_bytes[8:12], "little")
|
||||
frame = record_bytes[16:]
|
||||
ts = f"{seconds}.{microseconds:06d}"
|
||||
|
||||
if link_type == LINKTYPE_IEEE802_11_RADIOTAP and len(frame) >= 24:
|
||||
radiotap_len = int.from_bytes(frame[2:4], "little")
|
||||
rssi = frame[8] - 256 if frame[8] >= 128 else frame[8]
|
||||
station_id = int.from_bytes(frame[16:24], "little")
|
||||
mac_frame = frame[radiotap_len:]
|
||||
if len(mac_frame) >= 16:
|
||||
dst = mac_str(mac_frame[4:10])
|
||||
src = mac_str(mac_frame[10:16])
|
||||
else:
|
||||
dst = src = "?"
|
||||
station = f"{station_id:012x}" if station_id else "unknown"
|
||||
return (f"#{index:<5} [{ts}] len={cap_len:<5} rssi={rssi:>4}dBm "
|
||||
f"station={station} {src} -> {dst}")
|
||||
|
||||
if link_type == LINKTYPE_ETHERNET and len(frame) >= 14:
|
||||
dst = mac_str(frame[0:6])
|
||||
src = mac_str(frame[6:12])
|
||||
ethertype = int.from_bytes(frame[12:14], "big")
|
||||
return f"#{index:<5} [{ts}] len={cap_len:<5} eth {src} -> {dst} type=0x{ethertype:04x}"
|
||||
|
||||
return f"#{index:<5} [{ts}] len={cap_len:<5} (unrecognized frame format)"
|
||||
|
||||
|
||||
def undo_crlf(chunk, state):
|
||||
"""Undo the CR the device console inserts before every LF.
|
||||
|
||||
ESP-IDF's newlib console converts LF to CRLF on its way out, and that happens to every 0x0A
|
||||
byte of the binary pcap stream too, not only to log text. Each inserted CR shifts everything
|
||||
after it, so pcap record headers and captured frames alike come out corrupt. This is the
|
||||
"byte inserted mid-frame" seen in older recordings; with DENM traffic on air it wrecks most
|
||||
of a capture (measured 2026-09-14: a 787 KB file parsed cleanly for only 82 records).
|
||||
|
||||
Dropping one CR immediately before each LF undoes it exactly, provided it is done on the raw
|
||||
stream before any framing and a trailing CR is carried across read boundaries. CR and LF are
|
||||
written as byte values here so the transformation cannot be confused with an escape.
|
||||
"""
|
||||
CR, LF = bytes([13]), bytes([10])
|
||||
if state["pending_cr"]:
|
||||
chunk = CR + chunk
|
||||
state["pending_cr"] = False
|
||||
if chunk.endswith(CR):
|
||||
chunk = chunk[:-1]
|
||||
state["pending_cr"] = True
|
||||
return chunk.replace(CR + LF, LF)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("port", help="Serial port the device is on, e.g. COM5")
|
||||
parser.add_argument("-b", "--baud", type=int, default=115200, help="Baud rate (default: 115200)")
|
||||
parser.add_argument("-o", "--outdir", default="recordings", help="Output directory (default: ./recordings)")
|
||||
args = parser.parse_args()
|
||||
|
||||
os.makedirs(args.outdir, exist_ok=True)
|
||||
timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
outpath = os.path.join(args.outdir, f"capture_{timestamp}.pcap")
|
||||
|
||||
print(f"Opening {args.port} @ {args.baud}...")
|
||||
print(f"Writing live capture to {outpath}")
|
||||
print("Press Ctrl+C to stop.")
|
||||
|
||||
header_written = False
|
||||
link_type = None
|
||||
packet_count = 0
|
||||
buf = b""
|
||||
total_bytes = 0
|
||||
last_status = time.monotonic()
|
||||
printed_raw_preview = False
|
||||
|
||||
crlf_state = {"pending_cr": False}
|
||||
|
||||
with serial.Serial(args.port, args.baud, timeout=1) as ser, open(outpath, "wb") as outfile:
|
||||
def read_bytes(n):
|
||||
return undo_crlf(ser.read(n), crlf_state)
|
||||
|
||||
try:
|
||||
while True:
|
||||
chunk = read_bytes(256)
|
||||
if chunk:
|
||||
buf += chunk
|
||||
total_bytes += len(chunk)
|
||||
|
||||
now = time.monotonic()
|
||||
if now - last_status >= 2:
|
||||
last_status = now
|
||||
print(f"[diagnostic] {total_bytes} raw bytes received so far, "
|
||||
f"{packet_count} packets recognized, header_written={header_written}")
|
||||
if total_bytes > 0 and not printed_raw_preview and not header_written and not PKT_RE.search(buf):
|
||||
# We're getting bytes but none of them look like our markers - show a preview
|
||||
# so we can tell whether this is plain log text (markers just haven't shown up
|
||||
# yet), garbage (baud/port mismatch), or something else entirely.
|
||||
preview = buf[:200]
|
||||
print(f"[diagnostic] no markers matched yet - raw preview: {preview!r}")
|
||||
printed_raw_preview = True
|
||||
elif total_bytes == 0:
|
||||
print("[diagnostic] zero bytes received from the port at all - this points at "
|
||||
"the wrong COM port, another program holding the port, or a port that "
|
||||
"isn't actually wired to the console/sniffer output.")
|
||||
|
||||
# The device only sends the global header once, right when the sniffer first starts
|
||||
# (typically within a second or two of boot). If this script connects even slightly
|
||||
# late - very likely right after a fresh flash, since esptool itself resets the board -
|
||||
# that header is already gone before we ever see it. Rather than blocking forever
|
||||
# waiting for a header that's never coming, look for whichever marker shows up first.
|
||||
header_match = None if header_written else HEADER_RE.search(buf)
|
||||
pkt_match = PKT_RE.search(buf)
|
||||
|
||||
if header_match and (not pkt_match or header_match.start() < pkt_match.start()):
|
||||
length = int(header_match.group(1))
|
||||
buf = buf[header_match.end():]
|
||||
while len(buf) < length:
|
||||
buf += read_bytes(length - len(buf))
|
||||
header_bytes = buf[:length]
|
||||
outfile.write(header_bytes)
|
||||
outfile.flush()
|
||||
buf = buf[length:]
|
||||
header_written = True
|
||||
if length >= 24:
|
||||
link_type = int.from_bytes(header_bytes[20:24], "little")
|
||||
print(f"Got pcap global header (link type {link_type}) - device is streaming.\n")
|
||||
continue
|
||||
|
||||
if not header_written and pkt_match:
|
||||
link_type = LINKTYPE_IEEE802_11_RADIOTAP
|
||||
outfile.write(build_default_pcap_header(link_type))
|
||||
outfile.flush()
|
||||
header_written = True
|
||||
print("Note: missed the device's one-time pcap header (it was likely sent before "
|
||||
"this script connected, e.g. right after a flash/reset) - assuming WLAN "
|
||||
"radiotap capture and writing a default header instead.\n")
|
||||
# fall through and process pkt_match below, don't discard this packet
|
||||
|
||||
if not pkt_match:
|
||||
# Keep the buffer from growing unbounded while waiting for a marker, but don't
|
||||
# discard anything - a marker could be split across reads.
|
||||
if len(buf) > 65536:
|
||||
buf = buf[-4096:]
|
||||
continue
|
||||
|
||||
length = int(pkt_match.group(1))
|
||||
buf = buf[pkt_match.end():]
|
||||
while len(buf) < length:
|
||||
buf += read_bytes(length - len(buf))
|
||||
record_bytes = buf[:length]
|
||||
outfile.write(record_bytes)
|
||||
outfile.flush()
|
||||
buf = buf[length:]
|
||||
packet_count += 1
|
||||
print(summarize_packet(link_type, record_bytes, packet_count))
|
||||
except KeyboardInterrupt:
|
||||
print(f"\nStopped. {packet_count} packets saved to {outpath}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -51,3 +51,49 @@ Known gaps, tracked as TODOs in the source: no real GNSS (lat/long hardcoded
|
||||
0), no real time source (detectionTime/referenceTime hardcoded 0, decodes as
|
||||
2004-01-01), fixed (non-rotating) pseudonym MAC, SHB instead of GeoBroadcast
|
||||
(no multi-hop forwarding), unsecured (no IEEE 1609.2 signing).
|
||||
|
||||
## Running it as a bench beacon
|
||||
|
||||
This firmware needs no phone: it beacons a CAM every second by itself
|
||||
(`TX_INTERVAL_MS`) from station `0x0BADC0DE` (195936478), stationType 5
|
||||
(passengerCar), at the hardcoded bench position, under the fixed MAC
|
||||
`02:00:00:00:00:01`, on 5900 MHz. That makes it the quickest way to put known,
|
||||
repeatable traffic on air, and it is how the 4-bit `yawRateConfidence` encoding
|
||||
was confirmed over the air on 2026-09-14.
|
||||
|
||||
A board with only one USB-C port is fine. This firmware's console is on UART0,
|
||||
so such a board shows no log output, but nothing here needs the console.
|
||||
|
||||
Flash it from the toolchain terminal (ESP-IDF 5.5.4, see the table above):
|
||||
|
||||
```powershell
|
||||
cd C:\Users\Ashin\AndroidStudioProjects\MicrOBU\obu-cam-transmistter
|
||||
idf.py -p COM10 -b 921600 flash
|
||||
```
|
||||
|
||||
Or flash the existing build without any toolchain terminal:
|
||||
|
||||
```powershell
|
||||
cd obu-cam-transmistter\build
|
||||
C:\Espressif\python_env\idf5.5_py3.11_env\Scripts\python.exe -m esptool --chip esp32c5 -p COM10 -b 921600 write_flash --flash_mode dio --flash_freq 80m --flash_size 2MB 0x2000 bootloader/bootloader.bin 0x8000 partition_table/partition-table.bin 0x10000 obu_firmware.bin
|
||||
```
|
||||
|
||||
It starts beaconing as soon as it boots, so there is nothing to start by hand,
|
||||
and unplugging it is how you stop it.
|
||||
|
||||
**It transmits under the same MAC as the phone's CAM pinger**, so on air the two
|
||||
are told apart by station ID (195936478 here, 999999 for the pinger), never by
|
||||
source address.
|
||||
|
||||
To see what it is sending, capture on the sniffer board and decode:
|
||||
|
||||
```powershell
|
||||
cd capture
|
||||
py -3.11 live_capture.py COM8
|
||||
py -3.11 ..\obu-firmware\test\pcap_gn_tally.py recordings\capture_<timestamp>.pcap
|
||||
```
|
||||
|
||||
The tally lists it as SHB / port 2001 / lifetime `0x05`. For the message itself,
|
||||
decode the payload with `asn1tools` against `asn1/cam_1_4_1.asn` +
|
||||
`asn1/cdd_1_3_1_1.asn`; re-encoding must return the identical bytes. On
|
||||
2026-09-14, 72 of 72 frames did.
|
||||
|
||||
@@ -14,7 +14,9 @@ PYTHON_ASN1 = py -3.11
|
||||
FW = ../../main
|
||||
BUILD = build
|
||||
EXE = $(if $(filter Windows_NT,$(OS)),.exe,)
|
||||
RECORDINGS = $(wildcard ../../../its-g5-receiver-firmware/recordings/*.pcap)
|
||||
# Captures live in capture/recordings/ since 2026-09-14; older ones are still in the
|
||||
# receiver checkout beside this repo.
|
||||
RECORDINGS = $(wildcard ../../../capture/recordings/*.pcap ../../../its-g5-receiver-firmware/recordings/*.pcap)
|
||||
FUZZ_ITER = 2000000
|
||||
FUZZ_SEED = 1
|
||||
|
||||
|
||||
Reference in New Issue
Block a user