Ashin Walpola
73d4477f1e
live_capture.py and dump_pcap.py were untracked files inside the third-party its-g5-receiver-firmware checkout, so the tools every on-air measurement depends on were versioned nowhere and would vanish with a fresh clone of that project. They are ours rather than that project's, so they now live in capture/, with the setup notes rewritten as its README: which port the sniffer speaks on and why, how to capture, how to check a capture, and how to flash the sniffer board. live_capture.py carries the fix made on 2026-09-14. The sniffer's console converts LF to CRLF on its way out, and that applies to every 0x0a byte of the binary pcap stream, not only to log text, so every inserted CR shifts the rest of the stream. A 787 KB capture parsed cleanly for only 82 of about 2000 records, and DENMs turned up on nonsense BTP ports because their payloads carry 0x0a often. undo_crlf() reverses it on the raw stream before any framing, which is exact; afterwards a capture parsed to EOF and DENMs read as port 2002. Captures taken before that date are truncated at their first corrupted record, so anything measured from them is worth re-checking. capture/recordings/ is gitignored, since captures are data rather than source. The host tests now read both that directory and the older one in the receiver checkout, so no capture has to be moved while it is being written. dump_pcap.py reads the same console and still needs the same treatment; that is recorded in TODO.md.
103 lines
4.3 KiB
Markdown
103 lines
4.3 KiB
Markdown
# Sniffer board and capture tooling
|
|
|
|
How to put an ESP32-C5 on the ITS-G5 channel as a passive sniffer, pull its captures onto this
|
|
PC, and check what is on air. The sniffer firmware itself is the third-party
|
|
`its-g5-receiver-firmware` checkout beside this repo; only the tooling and these notes are ours.
|
|
|
|
| File | What it does |
|
|
|---|---|
|
|
| `live_capture.py` | Streams the device's captures into a growing `.pcap` while it runs. The usual choice. |
|
|
| `dump_pcap.py` | Pulls one capture out of the device's in-memory buffer after the fact. |
|
|
| `../obu-firmware/test/pcap_gn_tally.py` | Tallies GeoNetworking headers per station over a `.pcap`. |
|
|
|
|
One-time: `pip install pyserial` (present in Python 3.11 on the bench PC, so `py -3.11` works).
|
|
|
|
## Which port
|
|
|
|
The sniffer firmware's console, and with it the pcap stream, goes out **UART0** - the board's
|
|
USB-bridge port (a CH343, its own COM number), not the native USB-C port. A board with only one
|
|
USB-C port cannot be used as a sniffer for this reason. On the bench this has been COM5 and, after
|
|
a re-enumeration, COM8.
|
|
|
|
## Live capture (preferred)
|
|
|
|
```powershell
|
|
cd capture
|
|
py -3.11 live_capture.py COM8
|
|
```
|
|
|
|
It writes `recordings/capture_<timestamp>.pcap` next to itself, flushing after every packet, so
|
|
the file can be read while it grows. Stop it with Ctrl+C. Use `-o <dir>` to write elsewhere;
|
|
`recordings/` is gitignored, since captures are large and are data rather than source. Captures
|
|
taken before 2026-09-14 are still in `its-g5-receiver-firmware/recordings/`; the host tests read
|
|
both directories.
|
|
|
|
### The CR insertion, and why captures used to be corrupt
|
|
|
|
ESP-IDF's newlib console converts LF to CRLF on its way out, and that applies to every `0x0a` byte
|
|
of the **binary** pcap stream, not only to log text. Each inserted CR shifts everything after it,
|
|
so pcap record headers and captured frames alike come out corrupt, and the file stops being
|
|
parseable at the first occurrence.
|
|
|
|
Measured on 2026-09-14: a 787 KB capture parsed cleanly for only 82 of about 2000 records, and
|
|
DENMs appeared on nonsense BTP ports because their payloads contain `0x0a` often. `undo_crlf()` in
|
|
`live_capture.py` reverses it on the raw stream before any framing, which is exact; afterwards a
|
|
capture parsed to EOF and DENMs read as port 2002 again.
|
|
|
|
**Captures taken before 2026-09-14 are truncated at their first corrupted record.** Anything
|
|
measured from them is worth re-checking. `dump_pcap.py` reads the same console and has not been
|
|
given the same treatment yet.
|
|
|
|
## Checking a capture
|
|
|
|
```powershell
|
|
py -3.11 ..\obu-firmware\test\pcap_gn_tally.py recordings\capture_<timestamp>.pcap
|
|
```
|
|
|
|
One row per station, packet type, BTP port and GN lifetime. For the messages themselves, decode
|
|
the payloads with `asn1tools` against the modules in `../asn1/` and re-encode them: identical bytes
|
|
mean the message was read exactly, wrong bytes mean it was not. `obu-firmware/test/check_replay.py`
|
|
does this over a whole capture.
|
|
|
|
## Flashing the sniffer firmware
|
|
|
|
From the receiver checkout, with its **pinned** ESP-IDF (not the global 5.5.4 install):
|
|
|
|
```powershell
|
|
cd its-g5-receiver-firmware
|
|
git submodule update --init --recursive
|
|
.\esp-idf\install.bat
|
|
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
|
|
.\esp-idf\export.ps1
|
|
idf.py set-target esp32c5
|
|
idf.py -p COM8 -b 921600 flash
|
|
```
|
|
|
|
Its `sdkconfig` for a bare board (nothing wired) needs SPI Ethernet off, and the pcap destination
|
|
set to Memory, both under `idf.py menuconfig`. Wired variants have ready-made configs in that
|
|
checkout: `sdkconfig.proto-w5500`, `sdkconfig.proto-enc28j60`, `sdkconfig.proto-spi-eppp`.
|
|
|
|
To reflash a board that already has a built image, without a toolchain terminal:
|
|
|
|
```powershell
|
|
cd its-g5-receiver-firmware\build
|
|
C:\Espressif\python_env\idf5.5_py3.11_env\Scripts\python.exe -m esptool --chip esp32c5 -p COM8 -b 921600 write_flash --flash_mode dio --flash_freq 80m --flash_size 16MB 0x2000 bootloader/bootloader.bin 0x8000 partition_table/partition-table.bin 0x1e000 ota_data_initial.bin 0x20000 its-g5-receiver-firmware.bin
|
|
```
|
|
|
|
## Pulling a capture after the fact
|
|
|
|
Only for the Memory destination, and the buffer is small (`SNIFFER_PCAP_MEMORY_SIZE`, 4096 bytes
|
|
by default) - a smoke test, not a session. In the device console (`idf.py -p COM8 monitor`, exit
|
|
with Ctrl+T then Ctrl+X):
|
|
|
|
```
|
|
sniffer -P
|
|
sniffer --stop
|
|
```
|
|
|
|
Then, with the port free:
|
|
|
|
```powershell
|
|
py -3.11 dump_pcap.py COM8
|
|
```
|