Files
MicrOBU/obu-firmware/test/host
Ashin Walpola 73d4477f1e Move the capture tooling into the repo
live_capture.py and dump_pcap.py were untracked files inside the third-party
its-g5-receiver-firmware checkout, so the tools every on-air measurement depends
on were versioned nowhere and would vanish with a fresh clone of that project.
They are ours rather than that project's, so they now live in capture/, with the
setup notes rewritten as its README: which port the sniffer speaks on and why,
how to capture, how to check a capture, and how to flash the sniffer board.

live_capture.py carries the fix made on 2026-09-14. The sniffer's console
converts LF to CRLF on its way out, and that applies to every 0x0a byte of the
binary pcap stream, not only to log text, so every inserted CR shifts the rest
of the stream. A 787 KB capture parsed cleanly for only 82 of about 2000
records, and DENMs turned up on nonsense BTP ports because their payloads carry
0x0a often. undo_crlf() reverses it on the raw stream before any framing, which
is exact; afterwards a capture parsed to EOF and DENMs read as port 2002.
Captures taken before that date are truncated at their first corrupted record,
so anything measured from them is worth re-checking.

capture/recordings/ is gitignored, since captures are data rather than source.
The host tests now read both that directory and the older one in the receiver
checkout, so no capture has to be moved while it is being written.

dump_pcap.py reads the same console and still needs the same treatment; that is
recorded in TODO.md.
2026-09-14 13:38:38 +02:00
..

Host-side tests for obu-firmware

Status (2026-09-11): the chain test, the capture replay and the fuzzer are written and pass; results under "Running". Toolchain: MSYS2 UCRT64 gcc, Option B below (chosen and installed 2026-09-11).

geonet.c, dot11p.c and gn_unwrap.c include nothing but standard C headers, so they compile unmodified on a PC. That makes three things possible without a board: a TX -> RX round trip through our own code, replaying real captures through the RX parser, and fuzzing the parser that reads untrusted radio bytes. ESP-IDF only builds main/ (and components/), so nothing under test/ ever affects the firmware image.

Layout

obu-firmware/
├── main/                      firmware sources; the tests compile these directly, never copies
└── test/
    ├── pcap_gn_tally.py       GN header fields per station over .pcap captures (Python only)
    └── host/
        ├── README.md          this file
        ├── Makefile           `make`: builds ../../main/{geonet,dot11p,gn_unwrap}.c + tests, runs them
        ├── test_chain.c       geonet_wrap_shb -> dot11p_build_frame -> gn_unwrap_its, byte-checked
        ├── test_util.c/.h     shared: guard page, crash report, pcap read/write
        ├── test_replay.c      every recorded frame through gn_unwrap_its, results to a TSV
        ├── check_replay.py    re-derives each result independently, then asn1tools on the messages
        ├── fuzz_gn_unwrap.c   mutation fuzzer; each input ends against a no-access guard page, so
        │                      an over-read faults (no ASan on MinGW)
        └── build/             compiler output; already ignored by the repo's `build/` rule

The repo's vanetza/ folder is a gitignored reading copy only; it is not built. check_replay.py reads the IEEE 1609.2 ASN.1 modules from it (they carry no licence header, so they are not copied into asn1/). Without it, signed frames are still replayed but not checked independently.

Running

From PowerShell, with MSYS2 on PATH for the session (Option B step 3):

$env:PATH = "C:\msys64\ucrt64\bin;C:\msys64\usr\bin;$env:PATH"
cd C:\Users\Ashin\AndroidStudioProjects\MicrOBU\obu-firmware\test\host
make

make runs three things (make chain, make replay, make fuzz run one). A non-zero exit, or a CRASH ... during: <what> line (from the fuzzer, followed by the input as hex), is a failure.

  • chain (test_chain.c): frames built by the firmware's own TX code, checked byte by byte against EN 302 636-4-1 and parsed back. Covers the CAM layout (non-QoS and QoS), Source Position Vector edges, output-buffer bounds, a 512-byte payload through main.c's buffer sizes, hand-built GeoBroadcast frames in all three shapes, signed frames in all three COER length forms plus a top-level unsecuredData, one-byte mutations that must be rejected or accepted, the Common Header's payload length as the message boundary, the 8-byte RX trailer, and every truncation length of each frame against the guard page. pcap_gn_tally.py then reads the frames back as a second parser; build/test_chain.pcap opens in Wireshark too. 2026-09-11: 1731 checks, 0 failed.
  • replay (test_replay.c + check_replay.py): every record in its-g5-receiver-firmware/recordings/*.pcap through gn_unwrap_its, each cut to main.c's 800-byte capture buffer as on the board. check_replay.py re-derives each result on its own (its own GN/BTP parse; the security envelope decoded by asn1tools from the IEEE 1609.2 modules), compares record by record, then decodes and re-encodes every distinct message with asn1tools - a byte-identical re-encode is only possible when the message was cut at exactly the right byte. Needs py -3.11 with asn1tools. 2026-09-11: 15 145 records, 15 131 accepted (10 831 CAM, 4 300 DENM; 157 of them signed); C and Python agree on every record; 11 043 of the 11 106 distinct messages re-encode byte-identically. The other 63 fail the same way with the old 8 trailing bytes put back, so the boundary is not the cause: 5 are our own CAMs from before the 2026-08-20 yawRateConfidence fix, 56 come from the CiT One and 1 from another station (see TODO.md), and 1 uses an extension asn1tools cannot re-encode.
  • fuzz (fuzz_gn_unwrap.c): random edits of every recorded frame, each run against the guard page; an over-read crashes, an accepted payload outside its input fails. Default 2 000 000 iterations (about 2 s); make fuzz FUZZ_ITER=50000000 FUZZ_SEED=7 for a longer run. 2026-09-11: 50 000 000 iterations, no crash.

Not covered: main.c (serial prefix parsing, queues) and serial_link.c, which need ESP-IDF; and the phone's encoder, whose bytes are opaque here (asn1tools and the app's golden test cover it).

Option A (not used): WSL2 + Ubuntu 24.04

Kept as the fallback if AddressSanitizer or libFuzzer are ever needed; Option B has neither.

  1. In PowerShell as Administrator:

    wsl --install -d Ubuntu-24.04
    

    Reboot if it asks. Ubuntu then opens and asks for a Linux username and password (separate from the Windows account). Checked 2026-09-11: Hyper-V is already running on this PC, so no BIOS change should be needed. If the install says virtualization is disabled, enable Intel VT-x / AMD SVM in the BIOS.

  2. Confirm it is WSL 2: wsl -l -v should list Ubuntu-24.04 with VERSION 2.

  3. Inside Ubuntu, the compilers:

    sudo apt update
    sudo apt install -y build-essential clang cmake ninja-build git pkg-config python3
    
  4. Smoke test: the firmware sources compile on the host (expect no output):

    cd /mnt/c/Users/Ashin/AndroidStudioProjects/MicrOBU/obu-firmware/test/host
    cc -std=c11 -Wall -Wextra -fsyntax-only ../../main/geonet.c ../../main/dot11p.c ../../main/gn_unwrap.c
    

Option B (chosen): native Windows gcc via MSYS2

Enough for the round-trip test, the capture replay and the guard-page fuzzer. No libFuzzer and no AddressSanitizer with MinGW gcc, which is why the fuzzer uses a guard page instead.

  1. In PowerShell: winget install -e --id MSYS2.MSYS2 (installs to C:\msys64).

  2. Open MSYS2 UCRT64 from the Start menu and run pacman -Syu. If the window closes, reopen it and run pacman -Syu again. Then:

    pacman -S --needed mingw-w64-ucrt-x86_64-gcc make
    
  3. C:\msys64\ucrt64\bin must be on PATH. Calling C:\msys64\ucrt64\bin\gcc.exe by its full path alone exits 1 with no message, because gcc's compiler stages load their DLLs from that folder. make lives on the MSYS side, in C:\msys64\usr\bin. Either work inside the MSYS2 UCRT64 shell, which has both, or put them on PATH for the current session:

    $env:PATH = "C:\msys64\ucrt64\bin;C:\msys64\usr\bin;$env:PATH"   # PowerShell
    
    export PATH=/c/msys64/ucrt64/bin:/c/msys64/usr/bin:$PATH          # Git Bash
    

    Adding them to the user PATH permanently also works; it was deliberately not done by setup.

  4. Smoke test (expect no output):

    cd C:\Users\Ashin\AndroidStudioProjects\MicrOBU\obu-firmware\test\host
    gcc -std=c11 -Wall -Wextra -fsyntax-only ../../main/geonet.c ../../main/dot11p.c ../../main/gn_unwrap.c
    

Installed on this PC 2026-09-11: MSYS2 20260611, gcc 16.2.0 (UCRT64), GNU Make 4.4.1. All three firmware sources compile with -std=c11 -O2 -Wall -Wextra -Wpedantic and no warnings.

Line endings

The repo runs with core.autocrlf=true, so Windows checkouts have CRLF line endings. C compilers don't care; shell scripts run from WSL do (bash: $'\r': command not found). When the first .sh file lands here, add *.sh text eol=lf to a root .gitattributes (none exists yet).